SOFTWARE ENGINEER ( HADOOP )

Installation and configuration of the Hadoop platform distributions (Cloudera, Hortonworks, and MAPR) and Hadoop component services, including adding edge nodes and gateway nodes, and assigning master and slave nodes in the cluster.Adding and deleting nodes, connecting the servers through remote secure shell (SSH), and setting up Rack Awareness.Setting up HDFS replication factors for data replicas, configuring log4j properties, and integrating AWS CloudWatch. Upgrading and patching the cluster from one version to another (CDH, HDP, MAPR) and patching the Linux servers.Optimizing and tuning the Hadoop environments to meet performance requirements. Installing and configuring monitoring tools for all critical Hadoop systems and services.Configuring and maintaining high availability (HA) for HDFS, YARN (Yet Another Resource Negotiator) Resource Manager, MapReduce, Hive, HBase, Kafka, and Spark.Managing scalable Hadoop virtual and physical cluster environments. Managing backup and disaster recovery for Hadoop data. Collaborating with big data developers and designers for use case-specific scalable supportable infrastructure.Providing responsive support for day-to-day requests from development, support, and business analyst teams.Conducting performance analysis and debugging of slow-running development and production processes. Performing product/tool upgrades and applying patches for identified defects with root cause analysis (RCA). Performing ongoing capacity management forecasts, including timing and budget considerations.Designing scripts for automation of jobs to run in Hadoop environments for validation checks to monitor cluster health.Requirements:The minimum education requirement is a Bachelor’s degree in Computer Science, Information Technology, or a related technical field.Should have good knowledge of Cloudera, Hadoop, HDFS, Hive, Oozie, Spark, Python, Scala, and Splunk.

#J-18808-Ljbffr