Veolia
Manager Data Engineering DBT
Veolia, Paramus, New Jersey, us, 07653
Job Description
Position Purpose:
As Manager of Data Engineering, you will lead a team of data engineers in developing and managing data systems. This role requires strategic oversight of data integration, system development, and team management to ensure efficient and effective data solutions.
Primary Duties/Responsibilities:
Develop and operate data management tools, monitoring data flows, data quality, data cleansing and data processing.
Lead strategic planning for key service development of our data lake core stack such as data ingestion, pipeline optimization, data transformation, orchestration etc.
Manage a team of data engineers, providing mentorship and career development.
Collaborate with various stakeholders to align data engineering goals with organizational objectives.
Create and document logical data integration strategies for data flows between disparate systems and the enterprise data warehouse/data lakes.
Ensure data security and compliance with relevant data protection regulations.
Drive innovation in data management and analytics solutions.
Work in an Agile environment that focuses on collaboration and teamwork.
Oversee the development and maintenance of data infrastructure.
Monitor production job schedule and correct job failures in a timely manner.
Qualifications:
Education/Experience/Background:
MS degree in Computer Science or computer related field from an accredited institution.
7+ years hands proven experience as a Data Engineer or similar role.
7+ years of strong experience building, running and maintaining datalake(s) and warehouse(s) in a cloud environment.
Knowledge/Skills/Abilities:
Strong leadership skills with a track record of team development.
More than 4 years of experience developing with Python.
4+ years performing with production environments in a DevOps culture managing code composed of multi-developer teams, following industry best practices.
4+ years SQL development experience.
Experience with data modeling.
4+ years bash scripting experience.
Strong experience with Git, CI/CD (preferably GitLab), Docker and Terraform.
Experience deploying and running services in Cloud Big Data platforms such as BigQuery and Snowflake.
Strong experience with GCP services.
Experience designing and building data pipelines using tools like Apache Beam, CDAP (Data Fusion) or other ETLs.
Knowledge with CDC design patterns and their challenges.
Experience with DAG workflows orchestration such as Apache Airflow.
Experience with NoSQL databases is a plus (i.e Firestore, MongoDB).
Experience designing and developing APIs is a plus (i.e using FastAPI, Flask).
Required Certification/Licenses/Training:
(Nice to have) Google Cloud Data Engineer.
Terraform Associate Certification.
Physical Requirements:
Hybrid mode (2 days a week in the office).
#J-18808-Ljbffr
Position Purpose:
As Manager of Data Engineering, you will lead a team of data engineers in developing and managing data systems. This role requires strategic oversight of data integration, system development, and team management to ensure efficient and effective data solutions.
Primary Duties/Responsibilities:
Develop and operate data management tools, monitoring data flows, data quality, data cleansing and data processing.
Lead strategic planning for key service development of our data lake core stack such as data ingestion, pipeline optimization, data transformation, orchestration etc.
Manage a team of data engineers, providing mentorship and career development.
Collaborate with various stakeholders to align data engineering goals with organizational objectives.
Create and document logical data integration strategies for data flows between disparate systems and the enterprise data warehouse/data lakes.
Ensure data security and compliance with relevant data protection regulations.
Drive innovation in data management and analytics solutions.
Work in an Agile environment that focuses on collaboration and teamwork.
Oversee the development and maintenance of data infrastructure.
Monitor production job schedule and correct job failures in a timely manner.
Qualifications:
Education/Experience/Background:
MS degree in Computer Science or computer related field from an accredited institution.
7+ years hands proven experience as a Data Engineer or similar role.
7+ years of strong experience building, running and maintaining datalake(s) and warehouse(s) in a cloud environment.
Knowledge/Skills/Abilities:
Strong leadership skills with a track record of team development.
More than 4 years of experience developing with Python.
4+ years performing with production environments in a DevOps culture managing code composed of multi-developer teams, following industry best practices.
4+ years SQL development experience.
Experience with data modeling.
4+ years bash scripting experience.
Strong experience with Git, CI/CD (preferably GitLab), Docker and Terraform.
Experience deploying and running services in Cloud Big Data platforms such as BigQuery and Snowflake.
Strong experience with GCP services.
Experience designing and building data pipelines using tools like Apache Beam, CDAP (Data Fusion) or other ETLs.
Knowledge with CDC design patterns and their challenges.
Experience with DAG workflows orchestration such as Apache Airflow.
Experience with NoSQL databases is a plus (i.e Firestore, MongoDB).
Experience designing and developing APIs is a plus (i.e using FastAPI, Flask).
Required Certification/Licenses/Training:
(Nice to have) Google Cloud Data Engineer.
Terraform Associate Certification.
Physical Requirements:
Hybrid mode (2 days a week in the office).
#J-18808-Ljbffr