Aloden, Inc.
Lead Data Engineer - Python/Pyspark
Aloden, Inc., Charlotte, North Carolina, United States, 28245
Lead Data Engineer - Python/PysparkLocations:
Charlotte, NC
Work Authorization:
W2 only
Must-Have Skills for Leads (2 positions):Python & PySpark:
Strong proficiency in both Python and PySpark (70% PySpark, 30% Java focus).Java:
Solid understanding and experience in Java development.Must-Have Skills for Other Engineers (3 positions):
Python & PySpark:
Strong proficiency in Python and PySpark.Additional Essential Skills (for all positions):
SQL:
Strong SQL skills for data querying and manipulation.AWS:
Extensive experience with AWS cloud services and its data-related offerings.Data Lake:
Experience working with and building data lakes.Databricks (Plus):
If mentioned on the resume, candidates must be able to articulate their experience with Databricks.Responsibilities:
Design, develop, and maintain data pipelines using PySpark and Java.Migrate data from legacy platforms to a new AWS-based platform.Write and optimize complex Spark transformations and SQL queries.Work collaboratively with data scientists and other stakeholders.Ensure data quality, integrity, and security.Qualifications:
Strong understanding of Big Data concepts and technologies.Hands-on experience with data processing and transformation using PySpark.Proficiency in Python or Java development.Experience with AWS cloud services, especially those related to data storage and processing (e.g., S3, EMR, Redshift).Excellent problem-solving and communication skills.
Charlotte, NC
Work Authorization:
W2 only
Must-Have Skills for Leads (2 positions):Python & PySpark:
Strong proficiency in both Python and PySpark (70% PySpark, 30% Java focus).Java:
Solid understanding and experience in Java development.Must-Have Skills for Other Engineers (3 positions):
Python & PySpark:
Strong proficiency in Python and PySpark.Additional Essential Skills (for all positions):
SQL:
Strong SQL skills for data querying and manipulation.AWS:
Extensive experience with AWS cloud services and its data-related offerings.Data Lake:
Experience working with and building data lakes.Databricks (Plus):
If mentioned on the resume, candidates must be able to articulate their experience with Databricks.Responsibilities:
Design, develop, and maintain data pipelines using PySpark and Java.Migrate data from legacy platforms to a new AWS-based platform.Write and optimize complex Spark transformations and SQL queries.Work collaboratively with data scientists and other stakeholders.Ensure data quality, integrity, and security.Qualifications:
Strong understanding of Big Data concepts and technologies.Hands-on experience with data processing and transformation using PySpark.Proficiency in Python or Java development.Experience with AWS cloud services, especially those related to data storage and processing (e.g., S3, EMR, Redshift).Excellent problem-solving and communication skills.