YARN Concepts
YARN provides this capability by introducing a few new concepts into Hadoop:
Resource Manager
The Resource Manager controls all resources within the Hadoop cluster. It is responsible for negotiating usage of these resources with applications ready to execute on the cluster. The Resource Manager also monitors applications and provides other services. It generally runs on the head node of the Hadoop cluster.
Node Managers
Node managers are responsible for launching and monitoring containers that are launched on worker nodes of the cluster. A node manager runs on every worker node in the cluster.
Application Master
YARN is used to launch an application master for each instance of an application. The application master allocates resources that are used by the application along with the Resource Manager. Once an application manager obtains resources, it then uses node managers to launch containers for the application on worker nodes. An application master is also launched in a container within the cluster. The Resource Manager selects a worker node within the cluster to launch the container for its application master.
Container
A container is the physical instance of a process executed by YARN on a worker node within the cluster. The context of the container defines the executable to run, the environment, arguments to the executable, and other information set by the client. The container is launched in an operating system context that prevents the container from overutilizing resources. YARN monitors containers launched by an application and provides monitoring and restart services.
Last modified date: 12/09/2024