Spark Developer

Position: Spark Developer Location: Chicago, IL (Hybrid) Duration: Contract Job Summary: We are seeking a skilled and motivated Spark Developer to join our team. The ideal candidate will have experience in big data processing and analytics, with a strong background in Apache Spark. You will be responsible for designing, developing, and maintaining scalable and efficient data processing solutions. Key Responsibilities: Develop and maintain large-scale data processing pipelines using Apache Spark. Optimize and tune Spark jobs for performance and scalability. Collaborate with data engineers, data scientists, and other stakeholders to gather requirements and translate them into technical solutions. Implement data integration solutions to extract, transform, and load data from various sources. Ensure data quality and integrity through rigorous testing and validation. Troubleshoot and resolve issues related to data processing and job execution. Stay up-to-date with the latest industry trends and best practices in big data technologies. Required Skills and Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field. Proven experience as a Spark Developer or in a similar role. Strong programming skills in Scala, Java, or Python. Proficiency in using Apache Spark for data processing and analytics. Experience with big data technologies such as Hadoop, Hive, and Kafka. Familiarity with data storage solutions like HDFS, S3, or similar. Knowledge of SQL and experience with relational databases. Ability to work independently and as part of a team. Excellent problem-solving and analytical skills. Strong communication and collaboration abilities. Preferred Qualifications: Experience with cloud platforms such as AWS, Google Cloud, or Azure. Understanding of containerization technologies like Docker and Kubernetes. Knowledge of stream processing frameworks such as Apache Flink or Storm. Experience with machine learning libraries and frameworks. Benefits: Competitive salary and performance bonuses. Comprehensive health, dental, and vision insurance. Flexible working hours and remote work options. Professional development opportunities. A collaborative and inclusive work environment.