Executing an Application
Executing a DataFlow application represents the physical plan produced when the application is compiled. Each phase of the physical plan is detected and executed. For each execution phase, the threads are created for the operators that are executed, the DataFlow queues are created and attached to the ports, and other activities are performed that are associated with the physical graph.
After all the phases are successful, the execution of the entire application is complete. After the execution of the application is completed, the run time statistics are finalized and it can be accessed for the job performance.
A DataFlow application can be executed in the following ways:
• Invoking the start() method on LogicalGraph.
• Invoking the start() method on LogicalGraphInstance.
When you use the run() or start() method on LogicalGraph directly, it causes the graph to compile before executing. This is an easy method and produces the same result as using the run() or start() method with the results from the compile() method on a LogicalGraphInstance.
When you use the run() method, it invokes the DataFlow execution synchronously. This method does not return the results until the DataFlow application is completed.
The following is a code sample of synchronously executing a DataFlow application.
Synchronously executing a DataFlow application
// Compile an already composed application LogicalGraphInstance instance = graph.compile(EngineConfig.engine().monitored( true));
// Run the application, waiting for it to complete.
instance.run();
To invoke a DataFlow application asynchronously, use the start() method. This method returns the results immediately by executing the DataFlow application in the background. The current thread will accomplish other tasks in parallel. To wait for the application to complete, invoke the join() method on the LogicalGraphInstance.
Asynchronously executing a DataFlow application
// Compile an already composed application LogicalGraphInstance instance = graph.compile(EngineConfig.engine().monitored( true));
// Start the instance. This call is asyncrhonous. It returns right away.
// Use join to wait for graph completion. instance.start();
// ... do other work
// Join with the graph instance to wait for the application to complete
try {
instance.join();
}
catch (InterruptedException e) { e.printStackTrace();
}
Last modified date: 06/14/2024