Transparent Huge Pages and Defragmentation
Like other Java applications, Hadoop performance can suffer if the Linux feature Transparent Huge Pages is enabled (especially during the defragmentation process). Hadoop vendors generally recommend disabling this feature. (Installers will report warnings if the feature is enabled.)
We strongly advise to follow the Hadoop vendor's recommendations to disable Transparent Huge Pages and Transparent Huge Pages Defragmentation. Both features should be disabled; it is not sufficient to disable only Transparent Huge Pages.
To check the current settings, issue a command similar to this on all nodes (the current setting is shown in brackets []):
cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
always madvise [never]
cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
always madvise [never]
For instructions specific to your installation, consult your Hadoop and Linux documentation.