Tata Consultancy Services
Azure Databricks Lead
Tata Consultancy Services, Atlanta, Georgia, United States, 30383
Job Description: We are seeking a skilled Azure Databricks Tech Lead to join our dynamic team. As the Azure Databricks Tech Lead, you will play a pivotal role in architecting, implementing, and optimizing Azure Databricks solutions to support our data engineering and analytics initiatives. You will collaborate closely with cross-functional teams to understand business requirements, design scalable data solutions, and lead the execution of projects.
Responsibilities
1.Technical Leadership: Lead the design, development, and implementation of Azure Databricks solutions, ensuring alignment with business objectives and best practices.
2.Architecture and Design: Architect robust and scalable data pipelines and analytics solutions on Azure Databricks, leveraging services such as Apache Spark, Delta Lake, and MLflow.
3.Data Engineering: Develop ETL processes, data ingestion pipelines, and data transformations to support data integration and analytics requirements.
4.Performance Optimization: Optimize Azure Databricks workloads for performance, reliability, and cost-efficiency, utilizing techniques such as partitioning, caching, and resource management.
5.Collaboration: Collaborate with data scientists, analysts, and business stakeholders to understand requirements, translate them into technical solutions, and deliver actionable insights.
6.Mentorship and Training: Provide guidance, mentorship, and technical expertise to team members, fostering a culture of continuous learning and development.
7.Documentation and Best Practices: Document architecture, design decisions, and technical guidelines, ensuring adherence to best practices and standards.
Responsibilities
1.Technical Leadership: Lead the design, development, and implementation of Azure Databricks solutions, ensuring alignment with business objectives and best practices.
2.Architecture and Design: Architect robust and scalable data pipelines and analytics solutions on Azure Databricks, leveraging services such as Apache Spark, Delta Lake, and MLflow.
3.Data Engineering: Develop ETL processes, data ingestion pipelines, and data transformations to support data integration and analytics requirements.
4.Performance Optimization: Optimize Azure Databricks workloads for performance, reliability, and cost-efficiency, utilizing techniques such as partitioning, caching, and resource management.
5.Collaboration: Collaborate with data scientists, analysts, and business stakeholders to understand requirements, translate them into technical solutions, and deliver actionable insights.
6.Mentorship and Training: Provide guidance, mentorship, and technical expertise to team members, fostering a culture of continuous learning and development.
7.Documentation and Best Practices: Document architecture, design decisions, and technical guidelines, ensuring adherence to best practices and standards.