Infovision
Lead Big Data Engineer (GCP Cloud)
Infovision, Irving, Texas, United States, 75084
Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-term
Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools such as BigQuery, Dataflow, Pub/Sub, GCS.Strong experience with the Hadoop Big Data Ecosystem: HDFS, Hive, Pig, HBase, YARN, and others.Proficiency in Python and Scala for data engineering and scripting.In-depth knowledge of SQL for data manipulation, transformation, and analysis.Experience building both batch and real-time streaming data pipelinesSolid understanding of Hadoop fundamental conceptsStrong familiarity with CI/CD pipelinesExcellent problem-solving skills with an ability to troubleshoot complex data systems and performance bottlenecks.Proven ability to lead and mentor teams of engineers, driving the development of cutting-edge data solutions.
Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools such as BigQuery, Dataflow, Pub/Sub, GCS.Strong experience with the Hadoop Big Data Ecosystem: HDFS, Hive, Pig, HBase, YARN, and others.Proficiency in Python and Scala for data engineering and scripting.In-depth knowledge of SQL for data manipulation, transformation, and analysis.Experience building both batch and real-time streaming data pipelinesSolid understanding of Hadoop fundamental conceptsStrong familiarity with CI/CD pipelinesExcellent problem-solving skills with an ability to troubleshoot complex data systems and performance bottlenecks.Proven ability to lead and mentor teams of engineers, driving the development of cutting-edge data solutions.