Agility Partners LLC
Databricks Lead Company Hidden Retail Cincinnati, OH Architect 1 Opening Posted
Agility Partners LLC, Cincinnati, Ohio, United States, 45208
Agility Partners is seeking a qualified Advanced Data Engineer to fill an open position with a leading grocer in the Cincinnati area.
Key Responsibilities
Design, build, and maintain
ETL/ELT pipelines
in Databricks to process large-scale structured and unstructured datasets.Ensure timely ingestion, transformation, and delivery of data to internal consumers, analytics platforms, and machine learning models.Perform large-scale data transformation, cleaning, and aggregation using Spark and Databricks notebooks.Optimize data processing workflows for efficiency, scalability, and cost-effectiveness by fine-tuning Spark jobs and cluster configurations.
Benefits and Perks
Work within a collaborative team environment where ideas and creativity are welcomed! Family and work-life balance are important to this organization and valued for the employees.Working for an organization that focuses on company culture, inclusion, and diversity.50% medical coverage for you and your entire family, short/long term disability, and life insurance options.401(k).Life Insurance.Disability coverage.
The Ideal Candidate
Qualifications:
Bachelor's degree in Computer Science, Information Technology, or a related field. A Master’s degree is preferred.7+ years of experience in data engineering, with at least 4 years leading teams and projects in Databricks and Apache Spark.Strong hands-on experience with Databricks, including managing clusters, developing pipelines, and optimizing workloads.Proficiency in Python, Scala, or SQL for data processing within Databricks.Expertise in cloud platforms such as AWS, Azure, or GCP and their data services (e.g., S3, ADLS, BigQuery).Deep understanding of ETL/ELT processes, data modeling, and working with large-scale distributed systems.Familiarity with DevOps practices, including CI/CD, containerization, and infrastructure automation (e.g., Jenkins, Docker, Kubernetes).Experience with data governance, security, and compliance best practices.Excellent problem-solving skills and a proactive approach to identifying and resolving performance bottlenecks.Strong communication and interpersonal skills, with experience collaborating with both technical and non-technical stakeholders.
#J-18808-Ljbffr
Key Responsibilities
Design, build, and maintain
ETL/ELT pipelines
in Databricks to process large-scale structured and unstructured datasets.Ensure timely ingestion, transformation, and delivery of data to internal consumers, analytics platforms, and machine learning models.Perform large-scale data transformation, cleaning, and aggregation using Spark and Databricks notebooks.Optimize data processing workflows for efficiency, scalability, and cost-effectiveness by fine-tuning Spark jobs and cluster configurations.
Benefits and Perks
Work within a collaborative team environment where ideas and creativity are welcomed! Family and work-life balance are important to this organization and valued for the employees.Working for an organization that focuses on company culture, inclusion, and diversity.50% medical coverage for you and your entire family, short/long term disability, and life insurance options.401(k).Life Insurance.Disability coverage.
The Ideal Candidate
Qualifications:
Bachelor's degree in Computer Science, Information Technology, or a related field. A Master’s degree is preferred.7+ years of experience in data engineering, with at least 4 years leading teams and projects in Databricks and Apache Spark.Strong hands-on experience with Databricks, including managing clusters, developing pipelines, and optimizing workloads.Proficiency in Python, Scala, or SQL for data processing within Databricks.Expertise in cloud platforms such as AWS, Azure, or GCP and their data services (e.g., S3, ADLS, BigQuery).Deep understanding of ETL/ELT processes, data modeling, and working with large-scale distributed systems.Familiarity with DevOps practices, including CI/CD, containerization, and infrastructure automation (e.g., Jenkins, Docker, Kubernetes).Experience with data governance, security, and compliance best practices.Excellent problem-solving skills and a proactive approach to identifying and resolving performance bottlenecks.Strong communication and interpersonal skills, with experience collaborating with both technical and non-technical stakeholders.
#J-18808-Ljbffr