Logo
Aloden, Inc.

Lead Data Engineer - Python/Pyspark

Aloden, Inc., Charlotte, North Carolina, United States, 28245


Lead Data Engineer - Python/PysparkLocations:

Charlotte, NC

Work Authorization:

W2 only

Must-Have Skills for Leads (2 positions):Python & PySpark:

Strong proficiency in both Python and PySpark (70% PySpark, 30% Java focus).Java:

Solid understanding and experience in Java development.Must-Have Skills for Other Engineers (3 positions):

Python & PySpark:

Strong proficiency in Python and PySpark.Additional Essential Skills (for all positions):

SQL:

Strong SQL skills for data querying and manipulation.AWS:

Extensive experience with AWS cloud services and its data-related offerings.Data Lake:

Experience working with and building data lakes.Databricks (Plus):

If mentioned on the resume, candidates must be able to articulate their experience with Databricks.Responsibilities:

Design, develop, and maintain data pipelines using PySpark and Java.Migrate data from legacy platforms to a new AWS-based platform.Write and optimize complex Spark transformations and SQL queries.Work collaboratively with data scientists and other stakeholders.Ensure data quality, integrity, and security.Qualifications:

Strong understanding of Big Data concepts and technologies.Hands-on experience with data processing and transformation using PySpark.Proficiency in Python or Java development.Experience with AWS cloud services, especially those related to data storage and processing (e.g., S3, EMR, Redshift).Excellent problem-solving and communication skills.