Setting Up Clusters : Configuring the Cluster : Configuring Resource Fairness
 
Share this page                  
Configuring Resource Fairness
On Linux, DataFlow can be configured to use cgroups to control resource usage by jobs, permitting multiple jobs to execute simultaneously on a node while still providing a fair distribution of system resources. To enable these controls, you must set two parameters.
job.resource.usage.control
Must be changed to CGROUP. This property is managed as an executor setting as described in Controlling Executor Settings.
node.executor.base.cgroup
Must be set to the appropriate cgroup for every node manager. This property should identify the root cgroup created for DataFlow to administer adhoc subgroups as discussed in Enabling Fair Sharing.
Warning!  Using cgroups is only valid if all nodes are running Linux. If the cluster contains machines running other operating systems, setting clustermgr.resource.enforcement to CGROUP will not start the jobs.