Optimize your integrations early using the interactive Studio IDE and repeat the process. The Actian DataConnect Studio has an embedded Runtime Engine that must be used as often as possible while you are designing solutions.

Start with an appropriate size and set of sample data that takes less time to run the integration processes and transformation maps from the Studio IDE. As much as possible, make sure it is represents the production systems to which you are integrating. Do not develop in live production systems or databases. Use a sandbox environment or test server. Also, use anonymous data if the integration involves sensitive personal information such as healthcare records. This allows you to use several features to optimize, handle errors, and debug as you progress. First, implement the core functionality, run the integration, and then make enhancements or corrections iteratively.

Often the Log File name property in the runtime configuration is overlooked. If you use the same log file name for every map, the size of the file can grow quickly. Either change the name of the log file to match the name of the integration or make sure to select the Clear log file before run property to truncate the log at the beginning of execution. This applies to both maps and processes. Limit the types of errors logged during runtime to improve performance.

The Engine Execution Profiler is a design time tool that is enabled or disabled during configuration. When enabled, the profiler generates a graphical view of the integration components. It shows the number of times each operation is performed, time taken for each operation, and details of every statistic in a grid view.

The profiler helps to quickly identify outliers so that you can determine if those operations are required or can be achieved in a more efficient manner. It is also a very effective debugging and troubleshooting tool.