Compunnel Inc.
AWS Data Engineer (Technical Lead) - W2 Only
Compunnel Inc., San Francisco, California, United States, 94101
Position: AWS Data Engineer (Technical Lead) - W2 OnlyLocation: SFO, CA (Onsite)Duration: Contract
Job Description:The role involves leading the design and management of data solutions in a cloud environment with a focus on AWS services.
Responsibilities:•Lead the design, implementation, and optimization of scalable data pipelines and architectures utilizing AWS Glue, Elastic MapReduce (EMR), Lambda, Redshift, Athena, DynamoDB, OpenSearch, and S3.•Use Spark on AWS for data transformation and processing across large datasets.•Develop and maintain efficient data workflows with SQS for task queueing and orchestration.•Integrate, transform, and manage data using Mulesoft and Talend for seamless data integration.•Ensure high-performance data storage, retrieval, and analytics across Redshift, DynamoDB, and Athena.•Oversee data consistency, integrity, and compliance through IQVIA MDM solutions.•Apply best practices in data governance, security, and scalability within a collaborative and cross-functional team environment.
Qualifications:•Proven expertise in AWS data engineering, specifically with Glue, EMR, Lambda, Redshift, Athena, DynamoDB, OpenSearch, and S3.•Strong experience with data integration (Mulesoft, Talend)•Working knowledge of master data management (proficiency in IQVIA MDM is a plus).•Demonstrated ability to lead technical projects and mentor data engineering teams.•Exceptional analytical and communication skills.
Job Description:The role involves leading the design and management of data solutions in a cloud environment with a focus on AWS services.
Responsibilities:•Lead the design, implementation, and optimization of scalable data pipelines and architectures utilizing AWS Glue, Elastic MapReduce (EMR), Lambda, Redshift, Athena, DynamoDB, OpenSearch, and S3.•Use Spark on AWS for data transformation and processing across large datasets.•Develop and maintain efficient data workflows with SQS for task queueing and orchestration.•Integrate, transform, and manage data using Mulesoft and Talend for seamless data integration.•Ensure high-performance data storage, retrieval, and analytics across Redshift, DynamoDB, and Athena.•Oversee data consistency, integrity, and compliance through IQVIA MDM solutions.•Apply best practices in data governance, security, and scalability within a collaborative and cross-functional team environment.
Qualifications:•Proven expertise in AWS data engineering, specifically with Glue, EMR, Lambda, Redshift, Athena, DynamoDB, OpenSearch, and S3.•Strong experience with data integration (Mulesoft, Talend)•Working knowledge of master data management (proficiency in IQVIA MDM is a plus).•Demonstrated ability to lead technical projects and mentor data engineering teams.•Exceptional analytical and communication skills.