DATAMAXIS
Data Engineer
DATAMAXIS, Dearborn, Michigan, United States, 48120
Rate: depending on experience
Work location: MI (hybrid)
We are seeking a highly skilled and certified Data Engineer with expertise in Big Data technologies, Kafka, and Google Cloud Platform (GCP). The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines, as well as experience with real-time data processing. You will be responsible for ensuring the reliability, efficiency, and scalability of our data infrastructure.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines and ETL processes using Big Data technologies.
Implement and manage real-time data streaming solutions using Apache Kafka.
Architect and manage data storage solutions on Google Cloud Platform (GCP), ensuring high availability and reliability.
Collaborate with data scientists, analysts, and other engineering teams to integrate data solutions into various applications.
Optimize data systems and pipelines for performance, scalability, and cost-efficiency.
Ensure data quality and integrity across all data pipelines.
Monitor and troubleshoot data pipelines to ensure continuous and reliable data flow.
Document data engineering processes and maintain comprehensive records of all data infrastructure components.
Stay up to date with the latest trends and best practices in data engineering, Big Data, Kafka, and GCP.
Qualifications:
Certification:
Google Cloud Certified - Professional Data Engineer or equivalent certification.
Experience:
3+ years of experience in data engineering with a focus on Big Data technologies.
Technical Skills:
Proficiency in Apache Kafka for real-time data streaming.
Hands-on experience with Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, etc.
Strong knowledge of ETL processes and data pipeline orchestration tools.
Experience with distributed data processing frameworks like Apache Hadoop, Apache Spark, etc.
Proficiency in programming languages such as Python, Java, or Scala.
Experience with SQL and NoSQL databases.
Work location: MI (hybrid)
We are seeking a highly skilled and certified Data Engineer with expertise in Big Data technologies, Kafka, and Google Cloud Platform (GCP). The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines, as well as experience with real-time data processing. You will be responsible for ensuring the reliability, efficiency, and scalability of our data infrastructure.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines and ETL processes using Big Data technologies.
Implement and manage real-time data streaming solutions using Apache Kafka.
Architect and manage data storage solutions on Google Cloud Platform (GCP), ensuring high availability and reliability.
Collaborate with data scientists, analysts, and other engineering teams to integrate data solutions into various applications.
Optimize data systems and pipelines for performance, scalability, and cost-efficiency.
Ensure data quality and integrity across all data pipelines.
Monitor and troubleshoot data pipelines to ensure continuous and reliable data flow.
Document data engineering processes and maintain comprehensive records of all data infrastructure components.
Stay up to date with the latest trends and best practices in data engineering, Big Data, Kafka, and GCP.
Qualifications:
Certification:
Google Cloud Certified - Professional Data Engineer or equivalent certification.
Experience:
3+ years of experience in data engineering with a focus on Big Data technologies.
Technical Skills:
Proficiency in Apache Kafka for real-time data streaming.
Hands-on experience with Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, etc.
Strong knowledge of ETL processes and data pipeline orchestration tools.
Experience with distributed data processing frameworks like Apache Hadoop, Apache Spark, etc.
Proficiency in programming languages such as Python, Java, or Scala.
Experience with SQL and NoSQL databases.