Post-Installation Tasks
Configure Linux Settings
Certain Linux configuration settings should be set on each node to run the installation.
We recommend the following settings, which should be performed on each node.
• (Optional) Check that the Linux OS allows overcommitting of virtual memory:
cat /proc/sys/vm/overcommit_memory # Should return 1
ulimit –v # Should return unlimited
• Check the number of open files set by the installer:
ulimit -n
This number should be large, for example, 30,000.
Enable YARN Integration
By default, YARN support is disabled. During installation (depending on the method you used), you are asked if you want to enable it.
To see whether YARN support is enabled
Issue the following command:
iigetres ii.hostname.x100.yarn.enable
where hostname is the name of the machine where VectorH is installed (master node). You can also look in the config.dat file stored in the /ingres/files directory of that installation.
To enable YARN support after installation
Issue the following command:
iisuhdfs yarn
YARN Resources and VectorH Configuration
When YARN integration is enabled, VectorH generates its default configuration based on the resources allocated to the assigned YARN queue rather than those of the host OS. Additionally, memory and CPU parameters in vectorwise.conf are validated against the assigned YARN queue during x100_server startup to avoid failures due to insufficient queue resources.
For details on how VectorH interacts with YARN, see the chapter “Managing Cluster Resources” in the System Administrator Guide.
Understanding YARN Queues
YARN manages cluster resources by using queues. Resources like CPU and memory are partitioned into queues, to which applications can then be assigned to run in. Each YARN queue is allocated a certain amount of CPU and memory (fixed or elastic), which is shared among the applications assigned to that queue.
How VectorH Configures Resources During Installation
During installation, if YARN integration is enabled, VectorH sets the default values for the following vectorwise.conf parameters based on the resources allocated to the specified YARN queue.
The following is a sample setup where the instance is configured to use the default YARN queue, which has been allocated 8 GB of physical memory and 2 cores per host. There are 3 nodes in the cluster.
...
Enable YARN integration? (y/n) [n] y
Enter the YARN queue to be used for the Vector in Hadoop
application master [default]:
Enter the priority to be used for the Vector in Hadoop
application master [0]:
YARN integration enabled.
CHANGE ii.ip-10-234-29-114_us-west-1_compute_internal.x100.yarn.amqueue: (default)...(default)
CHANGE ii.ip-10-234-29-114_us-west-1_compute_internal.x100.yarn.ampriority: (0)...(0)
Regenerate configuration based on available resources? (y/n) [y]
Generating Vector in Hadoop config file...
Setting: max_memory_size=4096M
Setting: bufferpool_size=2048M
Setting: num_cores=2
Setting: max_parallelism_level=6
Backing up current config:
/opt/Actian/VectorVH/ingres/data/vectorwise/vectorwise.conf
Saving file to:
/opt/Actian/VectorVH/ingres/files/vectorwise.conf_bk_27931
...
Reconfigure YARN Integration
To enable, disable, or reconfigure YARN integration, use the following command:
iisuhdfs yarn
Resource Checking at Runtime (x100_server Startup)
To try to avoid startup failures due to lack of YARN queue resources, VectorH checks the vectorwise.conf parameters against the resources of the configured queue during startup. If insufficient YARN queue resources are available, the startup is aborted, and an error message is written to x100errlog.log. For example:
ERROR: DbAgent is configured to request a minimum of 10240MB of RAM
YARN queue 'default' only allows a maximum of 8192MB.
Aborting...
Note: Startup aborts with an error only if the resources requested by VectorH exceed the "maximum" resource allocation for that queue. If the requested amount exceeds the "configured" amount but not the "maximum", a warning is issued but startup continues.
Restore vectorwise.conf Defaults
If VectorH fails to start due to lack of YARN queue resources, you can restore the settings in vectorwise.conf to their default values.
To revert the vectorwise.conf values to their defaults, use the iisuhdfs genconfig command. For example:
iisuhdfs genconfig
Setting up Vector in Hadoop HDFS Support...
Generating Vector in Hadoop config file...
Setting: max_memory_size=4096M
Backing up current config:
/opt/Actian/VectorVH/ingres/data/vectorwise/vectorwise.conf
Saving file to:
/opt/Actian/VectorVH/ingres/files/vectorwise.conf_bk_22774
The Vector in Hadoop HDFS setup program has successfully completed.