Was this helpful?
Cluster Job Metrics
This section lists the DataFlow cluster Job statistics description.
 
Metric
Description
CPU Time
The amount of time for which CPU was used for processing job.
IO Read Bytes(local)
The number of bytes read from local file system.
IO Write Bytes(local)
The number of bytes written to local file system.
IO Read Bytes(drfs)
The number of bytes read from drfs.
IO Read Bytes(HDFS)
The number of bytes read from HDFS.
IO Write Bytes(HDFS)
The number of bytes written to HDFS.
Row Count(write)
The number of rows staged usually due to repartitioning in a cluster or due to forced staging.
Row Count(sort)
The number of rows implicitly sorted due to metadata mismatch.
Row Count(Read)
The number of rows read by the downstream operator.
IO Write Operations(local)
The number of write operation to local file system.
IO Read Operations(local)
The number of read operation to local file system.
IO Read Operations(drfs)
The number of read operation to drfs.
IO Write Operations(HDFS)
The number of write operation to HDFS.
IO Read Operations(HDFS)
The number of read operation to HDFS.
Memory(heap)
The total Java heap memory used.
Memory(non-heap)
The total Java non-heap memory used.
Threads(current)
The total number of threads executing (in runnable status) in the Java Virtual Machine.
Threads(peak)
The maximum number of threads executed in the Java Virtual Machine.
GC count(PS Scavenge)
The total number of garbage collections completed by the PS Scavenge garbage collector on the Eden and Survivor components of the “Young Generation” memory.
GC time (PS Scavenge)
The time taken in milliseconds by the PS Scavenge garbage collector to free up memory from the Eden and Survivor components of the “Young Generation” memory.
GC count(PS MarkSweep)
The total number of garbage collections completed by the PS MarkSweep garbage collector on the “Old Generation” memory.
GC time(PS MarkSweep)
The time taken in milliseconds by the PS MarkSweep garbage collector to free up memory from the “Old Generation” memory.
Last modified date: 01/06/2023