Best Practices¶

This guide provides guidance and best practices for users of Actian DataConnect to effectively use the design tools and create efficient integration solutions.

DataConnect is used to create and modify artifacts that contain instructions for the Runtime Engine to execute.

Although this topic focuses on design and development of integration solutions using the Actian DataConnect, various SDKs and APIs are also available to facilitate design and management in a programmatic fashion.

Workspace Best Practices¶

Effective workspace management is essential for maintaining organized, efficient, and reliable data integration projects.

Workspace Organization¶

Effective workspace organization is fundamental to maintaining efficient and scalable data integration projects in Actian DataConnect.

Plan Your Workspace Structure¶

Organize by project or purpose:

• Create separate workspaces for different environments (Development, Test, Production)
• Use dedicated workspaces for different clients or business domains
• Maintain isolated workspaces for experimentation vs. production work

Example workspace strategy:

Development-Workspace/
├── CustomerIntegration/
├── SupplierIntegration/
└── InternalReporting/

Production-Workspace/
├── CustomerIntegration/
└── SupplierIntegration/

Use Meaningful Names¶

Workspace naming:

• Use descriptive names that reflect the workspace purpose
• Examples: "Q4 ETL Development", "Customer Integration - Prod", "Data Quality Testing"
• Avoid generic names like "Workspace1" or "Test"

Project and artifact naming:

• Use clear, consistent naming conventions across all artifacts
• Include version numbers or dates when appropriate
• Example: CustomerValidation_v2 or AccountsETL_2026Q2

Maintain Local Storage for Active Development¶

Storage recommendations:

Organize with Logical Hierarchies¶

Project folder structure:

• Group related artifacts within projects
• Use nested folders to organize by function, data source, or workflow stage
• Keep related designs together for easier maintenance

Example hierarchy:

CustomerIntegration/
├── Schemas/
│   ├── CustomerSchema.xsd
│   └── OrderSchema.xsd
├── Validations/
│   ├── CustomerValidation.dq
│   └── OrderValidation.dq
└── Documentation/
    └── README.md

Workspace Management¶

Backup Regularly¶

Critical practice: Always backup your workspace folders before major operations.

Backup strategy:

• Backup before importing workspaces
• Create copies before bulk operations or refactoring
• Use version control systems (Git) for artifact tracking
• Store backups in safe, redundant locations

Sync with File System Proactively¶

When to sync your workspace:

• After version control operations (git pull, merge, checkout)
• After external file modifications (text editors, file managers)
• Before critical operations (deployments, exports, major edits)
• When troubleshooting missing or out-of-sync files
• After team members modify shared workspace files

How to sync: Click the Refresh (sync) button in the Project Explorer toolbar to scan for file system changes.

Verify File Permissions¶

Ensure proper access:

• Check that DataConnect has read/write permissions on workspace folders
• Avoid system-protected folders (C:/Windows/, C:/Program Files/)
• Use user-accessible locations like Documents/ or dedicated project folders
• Test write access before creating or importing workspaces

Clean Up Obsolete Artifacts¶

Maintain clean workspaces:

• Remove outdated versions of designs before importing workspaces
• Delete unused or deprecated artifacts regularly
• Archive old projects to separate workspaces or backup locations
• Keep only active, relevant artifacts in production workspaces

Version Control and Collaboration¶

Use Version Control Systems¶

Recommended practices:

• Store workspace folders in Git or other version control systems
• Commit both primary files (.dq) and runtime configuration files (.rtc)
• Include the .metadata folder in .gitignore (workspace-specific metadata)
• Document workspace structure and conventions in README files

Example .gitignore:

# DataConnect workspace metadata
.metadata/

# System files
.DS_Store
Thumbs.db
desktop.ini

Document Workspace Purpose¶

For team collaboration:

• Add README files describing workspace purpose and structure
• Document naming conventions and organizational patterns
• Include setup instructions for new team members
• Maintain changelog of major workspace modifications

Pre-Organize Before Import¶

Best practice for workspace imports:

• Create logical project folder structure before importing
• Group related artifacts together
• Remove test files and obsolete artifacts
• Verify all artifact pairings (.dq + .dq.rtc) are intact

Working with Artifacts¶

Understand Composite Artifact Pairing¶

Key concept: DataConnect uses paired files for artifacts:

• Primary file: Contains design definition (e.g., Accounts.dq)
• Runtime configuration: Stores execution settings (e.g., Accounts.dq.rtc)

Important behaviors:

• Both files are treated as a single logical unit
• Deleting one artifact deletes both files
• Renaming one artifact renames both files
• Always commit both files to version control

Use Descriptive Artifact Names¶

Naming best practices:

• Use names that describe the artifact's purpose
• Avoid generic names like "Test1" or "NewDesign"
• Do not include file extensions when creating artifacts (system adds them automatically)
• Use consistent naming patterns across related artifacts

Leverage Folder Organization¶

Organize artifacts efficiently:

• Create folders for different artifact types (Schemas, Validations, Transformations)
• Group artifacts by data source or target system
• Use nested folders for complex projects with many artifacts
• Keep related designs in the same folder for easier maintenance

Workspace Performance¶

Optimize Large Workspaces¶

For workspaces with many artifacts:

• Break large monolithic workspaces into smaller, focused workspaces
• Archive completed or inactive projects to separate workspaces
• Use workspace switching instead of keeping everything in one workspace
• Sync only when necessary (scanning thousands of files takes time)

Monitor Workspace Size¶

Keep workspaces manageable:

• Regularly review and clean up unused artifacts
• Move completed projects to archive workspaces
• Consider workspace size when planning project organization
• Use separate workspaces for different project phases

Safety and Recovery¶

Non-Destructive Operations¶

Understand deletion behavior:

• Workspace deletion: Removes only database registration (files remain on disk)
• Artifact deletion: Permanently removes files from file system
• Project deletion: Removes entire project folder with all contents

Recovery Procedures¶

If you accidentally delete a workspace:

1. Use Import Workspace to re-register the folder
2. Select the same folder path as the deleted workspace
3. All artifacts remain intact and are immediately available

If artifacts go out of sync:

1. Click the Refresh button to scan file system
2. Restore missing files from backup if needed
3. Remove stale database entries for deleted files

Environment Management¶

Separate Development, Test, and Production¶

Workspace isolation strategy:

• Create separate workspaces for each environment
• Never directly edit production artifacts in development workspaces
• Use export/import or version control to promote artifacts between environments
• Maintain clear naming to distinguish environment workspaces

Switch Contexts Efficiently¶

Use workspace switching:

• Switch between client projects or environments seamlessly
• Keep related work in dedicated workspaces
• Switch workspaces instead of mixing unrelated projects
• One workspace active at a time - full isolation between workspaces

Summary¶

Core best practices:

1. Organize thoughtfully - Use logical hierarchies and meaningful names
2. Backup regularly - Protect your work before major operations
3. Sync proactively - Keep Project Explorer in sync with file system
4. Use version control - Track changes and enable team collaboration
5. Clean up regularly - Remove obsolete artifacts and archive completed work
6. Understand isolation - Leverage workspace independence for organization
7. Verify permissions - Ensure proper file system access
8. Document thoroughly - Help team members understand workspace structure

By following these best practices, you'll maintain organized, efficient, and reliable data integration workspaces in Actian DataConnect.

Preference Settings Best Practices¶

Configure Actian DataConnect settings strategically to optimize your workflow, improve performance, and maintain consistency across your data integration projects.

Appearance and User Interface¶

Choose the Right Theme for Your Environment¶

Theme selection strategy:

• Well-lit offices: Use Light theme to maximize readability
• Low-light environments: Use Dark theme to reduce eye strain
• Mixed environments: Use System theme to automatically match OS settings

Data Quality Configuration¶

Set Appropriate Default Sampling Sizes¶

Sampling is configured as a record range (the first N records) or All records. Choose a default based on your use case:

Configure Appropriate Date/Time Patterns¶

Best practices for date/time settings:

• Set patterns matching your data sources: Choose date, timestamp, and time patterns that match your incoming data format
• Document your choices: Record which patterns you've selected in project documentation
• Consider regional formats: Set the default region to match your data's origin (US, UK, EU, etc.)
• Be consistent: Use the same patterns across all data quality designs in a project

Example strategy:

ISO 8601 format for international data:
- Date pattern: `yyyy-MM-dd` (2024-03-06)
- Timestamp pattern: `yyyy-MM-dd'T'HH:mm:ss`
- Time pattern: `HH:mm:ss` (24-hour)

Set Sensible Rule Defaults¶

Configuration recommendations:

• Duplicate Value minimum count: Keep at 2 for most cases; increase for datasets with expected duplicates
• Equal range binning: Use 10 ranges for initial analysis; adjust based on data distribution
• Most frequent values: Set to 25 for general use; increase to 50-100 for detailed frequency analysis

Execution and Logging Settings¶

Choose Appropriate Log Levels by Environment¶

Environment-specific logging strategy:

Manage Log File Retention¶

Best practices:

• Clear messages after run: Enable (check) when running frequent tests to avoid message accumulation
• Clear log file before run:
• Disable (uncheck) in production to maintain audit trail
• Enable (check) in development to keep log files manageable
• Regular cleanup: Archive or delete old log files periodically to manage disk space

Recommended strategy:

Production: Keep logs, append new entries (audit trail)
Development: Clear logs before run (fresh start each time)
Testing: Retain logs, review before clearing (track issues)

Macro Management¶

Organize Macros by Environment¶

Best practices for macro organization:

• Create separate macro sets for each environment (Development, Test, Production)
• Use descriptive names for macro sets: CustomerDB_Dev, CustomerDB_Prod
• Document macro purpose and expected values
• Use encryption for sensitive values (passwords, API keys, credentials)

Example macro structure:

Global Macros:
├── Development_Environment
│   ├── DB_HOST=dev-server.local
│   ├── DB_USER=dev_user
│   └── LOG_LEVEL=DEBUG
├── Test_Environment
│   ├── DB_HOST=test-server.local
│   ├── DB_USER=test_user
│   └── LOG_LEVEL=INFO
└── Production_Environment
    ├── DB_HOST=prod-server.company.com
    ├── DB_USER=prod_user (encrypted)
    └── LOG_LEVEL=ERROR

Version Control Macro Definitions¶

Recommended practices:

• Export macro definitions and store in version control
• Do not commit encrypted credentials to version control
• Document macro dependencies in project README
• Maintain separate macro files for different environments

For detailed macro configuration, see Macro Manager.

Advanced Configuration¶

Backup Your Settings¶

Configuration backup strategy:

• Regular exports: Navigate to Settings > Advanced > Raw JSON and copy configuration
• Before major changes: Export settings before modifying multiple preferences
• Version control: Store exported settings JSON in your project repository
• Team sharing: Export and share settings across team members for consistency

When to export settings:

• After configuring a new installation to your preferences
• Before major application updates
• When establishing team standards
• For disaster recovery documentation

Verify Settings After Changes¶

Post-change verification:

1. Navigate to Settings > Advanced to view all settings in JSON format
2. Use the Formatted View to confirm settings are applied correctly
3. Click Refresh to reload latest settings if needed
4. Test affected functionality to ensure expected behavior

Cross-Category Configuration Strategy¶

Establish Team Standards¶

Create consistent configuration across team:

• Document standard settings in team wiki or README
• Export reference configuration for new team members
• Review settings during onboarding
• Periodically audit settings for compliance with standards

Example team standards document:

## Team DataConnect Settings Standards

### Data Quality
- Sample size: 10,000 records (development), All records (production)
- Date pattern: yyyy-MM-dd
- Default region: United States

### Execution
- Log level: Warnings (test/prod), Informative (dev)
- Clear messages: Enabled (dev), Disabled (prod)
- Clear log files: Enabled (dev), Disabled (prod)

### Appearance
- Theme: System (auto-adapt to OS)

Environment-Specific Configuration Profiles¶

Maintain different settings per environment:

Summary¶

Core settings best practices:

1. Start with defaults - Modify settings incrementally as you understand their impact
2. Document choices - Record why specific settings were chosen for your environment
3. Use appropriate log levels - Balance diagnostic information with performance
4. Configure by environment - Different settings for dev, test, and production
5. Backup configurations - Export settings regularly for recovery and sharing
6. Test after changes - Verify settings produce expected behavior
7. Establish team standards - Ensure consistency across team members
8. Review periodically - Reassess settings as projects and requirements evolve

By following these preference settings best practices, you'll optimize Actian DataConnect for your specific workflows, improve performance, and maintain consistency across your data integration projects.