Aloden, Inc.
Hadoop Developer with Python
Aloden, Inc., Chicago, Illinois, United States, 60290
Hadoop Developer with Python Only W2 (Citizen,TN, GC, GC EAD and H4EAD)Location:
Chicago, IL / Denver, CO
Duration:
12+ Months
Must-Have Skills:Hadoop:
Extensive experience with Hadoop ecosystem and related technologiesProgramming:
Proficiency in Python (or Scala)SparkSQL:
Strong skills in Spark SQL for data processing and analysisSQL:
Proficiency in at least one of the following: MySQL, Hive, ImpalaData Ingestion:
Experience with ingesting data from various sources (message queues, file shares, REST APIs, relational databases) and handling different data formats (JSON, CSV, XML).Spark Structured Streaming:
Experience working with Spark Structured Streaming for real-time data processing.Responsibilities:
Design, develop, and maintain data pipelines using Hadoop, Spark, and Python/ScalaIngest data from various sources and transform it into usable formatsOptimize Spark/MapReduce and SQL jobs for performanceWork in an Agile development environment, collaborating with cross-functional teamsUtilize version control systems like SVN, Git, or BitbucketDeploy and manage applications using Jenkins and handle Jar managementQualifications:
Strong understanding of Big Data concepts and technologiesHands-on experience with Hadoop, Spark, Sqoop, Kafka, MapReduce, and NoSQL databases (HBase, Solr, etc.)Proficiency in Python or Scala for data processing and scriptingExperience with Linux operating systemFamiliarity with data visualization tools like Elastic Search and KibanaAbility to work independently and as part of a teamExcellent communication and problem-solving skillsAdaptable and eager to learn new technologies
Chicago, IL / Denver, CO
Duration:
12+ Months
Must-Have Skills:Hadoop:
Extensive experience with Hadoop ecosystem and related technologiesProgramming:
Proficiency in Python (or Scala)SparkSQL:
Strong skills in Spark SQL for data processing and analysisSQL:
Proficiency in at least one of the following: MySQL, Hive, ImpalaData Ingestion:
Experience with ingesting data from various sources (message queues, file shares, REST APIs, relational databases) and handling different data formats (JSON, CSV, XML).Spark Structured Streaming:
Experience working with Spark Structured Streaming for real-time data processing.Responsibilities:
Design, develop, and maintain data pipelines using Hadoop, Spark, and Python/ScalaIngest data from various sources and transform it into usable formatsOptimize Spark/MapReduce and SQL jobs for performanceWork in an Agile development environment, collaborating with cross-functional teamsUtilize version control systems like SVN, Git, or BitbucketDeploy and manage applications using Jenkins and handle Jar managementQualifications:
Strong understanding of Big Data concepts and technologiesHands-on experience with Hadoop, Spark, Sqoop, Kafka, MapReduce, and NoSQL databases (HBase, Solr, etc.)Proficiency in Python or Scala for data processing and scriptingExperience with Linux operating systemFamiliarity with data visualization tools like Elastic Search and KibanaAbility to work independently and as part of a teamExcellent communication and problem-solving skillsAdaptable and eager to learn new technologies