JobRialto
Senior Data Engineer
JobRialto, San Francisco, California, United States, 94199
Job Summary
The Senior Data Engineer will lead the design and implementation of data pipelines and products supporting the expansion of a Loyalty program. This role involves building scalable data solutions, ensuring seamless data integration, and collaborating with cross-functional teams to meet data requirements. The ideal candidate has extensive experience with cloud platforms, big data technologies, and data architecture, coupled with strong analytical skills and a passion for data-driven decision-making.
Key Responsibilities
• Design and implement data pipelines and solutions to support the Loyalty program expansion.
• Maintain seamless data flow from Loyalty systems to data lakes and warehouses while ensuring data integrity and availability.
• Integrate data from transactional databases, third-party APIs, and external sources into the data lake.
• Develop and maintain ETL processes for analytics and reporting.
• Collaborate with cross-functional teams including Engineering, Business Analytics, Data Science, and Product Management.
• Optimize data storage, retrieval, and pipeline performance for scalability and efficiency.
• Monitor and troubleshoot data pipelines to ensure reliability.
• Implement and enforce data governance policies to ensure security, privacy, and compliance.
• Develop documentation and standards for data processes and procedures.
Required Qualifications
• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
• 7-8 years of experience in data engineering, focusing on data architecture and pipeline development.
• Proficiency with cloud platforms (GCP) and big data technologies such as Airflow, Spark, DBT, Databricks, BigQuery, and other GCP services.
• Strong SQL skills and experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Proficiency in Python for programming and data engineering tasks.
• Experience in data modeling, schema design, and data warehousing concepts.
• Familiarity with data lake architectures and best practices.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.
Preferred Qualifications
• Experience with ecommerce systems and their data integration.
• Knowledge of data visualization tools (e.g., Tableau, Looker).
• Understanding of machine learning and data analytics.
Certifications
• AWS Certified Data Analytics (preferred)
• Google Professional Data Engineer (preferred)
Location:
San Francisco, CA - Remote
Education:
Bachelors Degree
The Senior Data Engineer will lead the design and implementation of data pipelines and products supporting the expansion of a Loyalty program. This role involves building scalable data solutions, ensuring seamless data integration, and collaborating with cross-functional teams to meet data requirements. The ideal candidate has extensive experience with cloud platforms, big data technologies, and data architecture, coupled with strong analytical skills and a passion for data-driven decision-making.
Key Responsibilities
• Design and implement data pipelines and solutions to support the Loyalty program expansion.
• Maintain seamless data flow from Loyalty systems to data lakes and warehouses while ensuring data integrity and availability.
• Integrate data from transactional databases, third-party APIs, and external sources into the data lake.
• Develop and maintain ETL processes for analytics and reporting.
• Collaborate with cross-functional teams including Engineering, Business Analytics, Data Science, and Product Management.
• Optimize data storage, retrieval, and pipeline performance for scalability and efficiency.
• Monitor and troubleshoot data pipelines to ensure reliability.
• Implement and enforce data governance policies to ensure security, privacy, and compliance.
• Develop documentation and standards for data processes and procedures.
Required Qualifications
• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
• 7-8 years of experience in data engineering, focusing on data architecture and pipeline development.
• Proficiency with cloud platforms (GCP) and big data technologies such as Airflow, Spark, DBT, Databricks, BigQuery, and other GCP services.
• Strong SQL skills and experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Proficiency in Python for programming and data engineering tasks.
• Experience in data modeling, schema design, and data warehousing concepts.
• Familiarity with data lake architectures and best practices.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.
Preferred Qualifications
• Experience with ecommerce systems and their data integration.
• Knowledge of data visualization tools (e.g., Tableau, Looker).
• Understanding of machine learning and data analytics.
Certifications
• AWS Certified Data Analytics (preferred)
• Google Professional Data Engineer (preferred)
Location:
San Francisco, CA - Remote
Education:
Bachelors Degree