System Performance
Many tools are available for monitoring CPU, I/O, and memory activity on a machine. These tools are usually platform specific. The most commonly used tools for platforms supporting DataFlow are highlighted below.
Use system monitoring to get an overall idea of what the performance problem may be. Once this is known, it can be used to determine the possible solutions. For example, if a system resource such as CPU or disk is 100% utilized, the system is the constraint, and performance will only increase by making more of the resource available—possibly by distributing the graph execution across more machines or (if possible) making the operators in the graph more efficient. However, if resources are not being fully utilized, then the constraint is within the application, not the system, and the source of the problem should be sought there.
In most cases, to use these tools, execute a DataFlow application while the monitoring tool is running. The tool will collect and display overall system resource usage. Always try to profile applications without excessive background activity on the system being monitored. Failure to do so can yield poor results, because it becomes difficult to isolate the source of performance issues.
Microsoft Windows
The Windows platform comes with some tools for monitoring system performance already installed.
Windows Task Manager
Enables you to see a quick snapshot of overall CPU usage. This is not broken down by application, so may be misleading if multiple applications are being run at a time.
Tabs are also available for viewing individual processes and network activity. When viewing a process, the CPU usage and memory usage per process are also displayed. Clicking on a column will sort by that column, so viewing the process that is using the most CPU resources is possible.
Windows Performance Monitor
Provides a rich set of metrics that can be monitored. This includes memory and I/O activity along with CPU usage. Usage metrics are tracked over time and displayed in a single graph.
UNIX
There are a wide variety of monitoring tools available for Linux and UNIX platforms. Some commonly used tools are:
top
Shows general system statistics and provides a sorted listing of active processes by CPU usage.
vmstat
Provides snapshots of system memory usage including paging and swapping statistics.
nmon