Parallelism and Concurrency
Process Editor allows you to create multi-threaded workflows using the drag-and-drop techniques. You can create multiple branches by linking two or more steps from a single step. Processing occurs on a separate thread for each branch. It continues on multiple threads until all the branches merge back onto the critical path.
Make sure to increase the value of the Max Thread Count property for the runtime configuration of the process (Execution Preferences > Error Handling > Max Thread Count property).
The benefits of using a multi-threaded process include reusing script libraries, sharing global and object variables across threads, and shorter overall processing times. However, the drawback can be a fatal error in one thread crashes the entire integration because they are all running in the same engine instance.
If the data and use case dictates that there are no dependencies across these branches, it can be safe to leverage the Orchestration invoker and fire the same integration in separate engine instances. This same concept can also be achieved using the EXE step and calling the Actian DataConnect Runtime Engine directly but you have to manage the command line scripts. Both these techniques must be governed by the engine concurrency count that is allowed by the license.
Last modified date: 12/03/2024