Wednesday, May 11, 2016


Four Components

Client request job to Resource Manager

Resource Manager

  • handles Client request
  • Start/Monitor ApplicationMaster
  • Monitor NodeManager
  • Resource allocation and call


Node Manager
 sends status heartbeat to Resource Manager
  • single node self resource management
  • handle commands from ResourceManager
  • handle commands from ApplicationMaster

Container: (in NodeManager)a unit of Resource(Memory, disk size, cpu, network io, disk io),  there will be a job task in each Container. Some task in Container also will send Task Status to App Master.

App Master: (in NodeManager)Manager(monitor, manage) for one app. Calculate how much Container it needs, then request the Container from Resource Manager. Once get Container, run the app.
  • cut data to blocks
  • request resource from Resource Manager, and allocate resource to jobs
  • monitor jobs and failure compatible
Example: MapReduce App Master, MPI App Master, etc..

MapReduce in Yarn flow

No comments:

Post a Comment