Diverse Lynx
Lead ETL Developer with PySpark
Diverse Lynx, Minneapolis, Minnesota, United States, 55400
Role: Lead ETL Developer with PySpark
Location: Minneapolis Downtown, MN (Onsite)
Job Type: Contract
Job Description:
ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark
Responsibilities: Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product Owners. Interact with stakeholders Provide knowledge transfer to other team members. Creating and testing pipeline jobs locally using aws glue interactive session. Performance tuning of PySpark jobs. AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers. Must Haves:
Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives. Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySpark Develop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL Developer SQL expert AWS Glue WITH Python (PySpark) PySpark Data frame APISpark SQL Knowledge in AWS services (e.g., DMS, S3, RDS, Redshift, Step Function) Nice to Haves:
ETL development experience with tools e.g., SAP BODS, Informatica. Good understanding of version control tools like Git, GitHub, TortoiseHg. Financial services experience Agile
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Job Description:
ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark
Responsibilities: Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product Owners. Interact with stakeholders Provide knowledge transfer to other team members. Creating and testing pipeline jobs locally using aws glue interactive session. Performance tuning of PySpark jobs. AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers. Must Haves:
Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives. Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySpark Develop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL Developer SQL expert AWS Glue WITH Python (PySpark) PySpark Data frame APISpark SQL Knowledge in AWS services (e.g., DMS, S3, RDS, Redshift, Step Function) Nice to Haves:
ETL development experience with tools e.g., SAP BODS, Informatica. Good understanding of version control tools like Git, GitHub, TortoiseHg. Financial services experience Agile
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.