"Hadoop's resource management layer is called YARN (Yet Another Resource Negotiator). It controls how CPU and memory resources are distributed throughout the cluster. YARN increases the system's scalability and efficiency by separating resource management from job execution. Key components: Resource Manager (RM): Allocates cluster-wide resources. Node Manager (NM): Manages resources on each node. Application Master (AM): Manages execution of a single job."
"Phase 1: Online Assessment Both theoretical and coding questions were covered in the first round of the online exam. MCQs on Java, NoSQL, Hadoop, Spark, and Scala. To assess practical coding abilities, there are two Scala programming questions. This test primarily assessed my proficiency in Scala fundamentals and problem-solving. Phase 2: Technical Interview Technical discussion with coding and conceptual questions comprised the second round."
An interview after four years of data engineering experience consisted of two main phases: an online assessment and a technical interview. The online assessment combined multiple-choice questions on Java, NoSQL, Hadoop, Spark, and Scala with two Scala programming problems focused on fundamentals and problem-solving. The technical interview included coding and conceptual questions such as writing a Scala program to check Fibonacci membership and returning -1 if absent. The technical discussion covered Hadoop YARN's components (Resource Manager, Node Manager, Application Master) and Spark dynamic allocation with parameters for enabling and controlling executor counts.
Read at Medium
Unable to calculate read time
Collection
[
|
...
]