Configuring the Cluster
The DataFlow cluster was designed to minimize the amount of required per-node configuration. When each node is started, you can specify the Cluster Manager host and port. This will allow each node to download the configuration from Cluster Manager. When the configuration is modified, the changes are distributed from Cluster Manager to the nodes in the cluster. Configuration for the cluster is managed centrally from the DataFlow Cluster Admin UI.
To access the admin UI using a web browser, browse to the port that you specified when you started clustermgr (1100 in the example above). To start, you should browse to that port and then log in. The login credentials are:
• User: root
• Password: changeit
We recommend that you change the root password after you log in the first time. To change the password, select Security > Change Password.
After login, you can expand the Cluster Configuration group to see the following pages:
Executor Configuration
You can control the per-job settings such as JVM tuning, log level, and so on. These settings can also be specified programmatically using the DR client. If the client does not specify the settings, then the default settings in the admin UI is used.
Cluster Manager Configuration
You can control the configuration for Cluster Manager. The admin can specify the log configuration, recent jobs history size, and so on.
Machine Classes
You can define various settings for the nodes in your cluster. The preferred method of deployment is for all machines in the cluster to have identical specifications, paths, and so on. For most cases, a single global configuration meets the requirement. However, as clusters modify, we can expect them to become more heterogeneous environments and the paths may vary from one machine to another. To deal with this possible variation, we introduce the machine class. A machine class is a named collection of configuration properties. By default, there is a single "default" machine class to which all machines are assigned.
Node Configuration
You can view machine class assignments and assign machines to machine classes. When machines first contact the cluster, they are automatically assigned to the "default" machine class. Until all machines in the cluster are identical, you need not edit the mapping on this page.
Note: Configuring all cluster machines to be identical for both software installed and file system paths, it simplifies administration of the cluster. In this configuration, you can have a single default machine class.
Last modified date: 06/14/2024