Motion Recruitment
Data Engineer / Databricks / 100% remote
Motion Recruitment, New York, New York, United States,
We are currently seeking a Senior Data Engineer with expertise in Databricks at a cutting-edge start-up.
The company is focused on leveraging big data and analytics to provide actionable insights to its clients. As a Senior Data Engineer, you will be instrumental in designing, building, and maintaining scalable data pipelines using Databricks and other cloud-based platforms.
The ideal candidate has extensive experience in data engineering, working with large-scale data infrastructures, and optimizing ETL processes.
Required Skills & Experience:
Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.At least 5 years of experience in data engineering, with a focus on building data pipelines.Proven experience with Databricks and Apache Spark.Strong proficiency in Python or Scala.Experience with cloud platforms such as AWS or Azure.Expertise in building and optimizing ETL processes.Desired Skills & Experience:
Experience with real-time data processing.Knowledge of ML model deployment within data pipelines.Familiarity with Delta Lake and data lakes architecture.What You Will Be Doing:
Tech Breakdown:
60% Databricks & Spark Development40% ETL OptimizationDaily Responsibilities:
70% Hands-On Coding20% Data Pipeline Design & Development10% Documentation & Reporting
Posted by:
Julie Bennett
Specialization:
Python Machine Learning/Data Science
The company is focused on leveraging big data and analytics to provide actionable insights to its clients. As a Senior Data Engineer, you will be instrumental in designing, building, and maintaining scalable data pipelines using Databricks and other cloud-based platforms.
The ideal candidate has extensive experience in data engineering, working with large-scale data infrastructures, and optimizing ETL processes.
Required Skills & Experience:
Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.At least 5 years of experience in data engineering, with a focus on building data pipelines.Proven experience with Databricks and Apache Spark.Strong proficiency in Python or Scala.Experience with cloud platforms such as AWS or Azure.Expertise in building and optimizing ETL processes.Desired Skills & Experience:
Experience with real-time data processing.Knowledge of ML model deployment within data pipelines.Familiarity with Delta Lake and data lakes architecture.What You Will Be Doing:
Tech Breakdown:
60% Databricks & Spark Development40% ETL OptimizationDaily Responsibilities:
70% Hands-On Coding20% Data Pipeline Design & Development10% Documentation & Reporting
Posted by:
Julie Bennett
Specialization:
Python Machine Learning/Data Science