Infogain Corp
Azure Data Engineer (Lead)
Infogain Corp, Houston, Texas, United States, 77246
Azure Data Engineer (Lead)
Location: Gurugram, IndiaExperience:
8 to 11 years of overall experience in Cloud Data Engineering, with a minimum of 3 to 5 years experience in Azure Databricks and PySpark.Skills:Proficient in SQL with experience in complex joins, subqueries, functions, and procedures.Experience in SQL tuning and query optimization.Design, develop, and maintain ETL pipelines using Databricks, PySpark, and Azure Data Factory (ADF).Experience in implementing large and complex hosted solutions using Azure (ADF, Azure Data Lake Storage, and Databricks).Experience with Delta tables, deduplication, and merging with terabytes of datasets.Optimize and fine-tune existing ETL workflows for performance and scalability.Experience working with large datasets.Experience working with batch and streaming data processing (good to have).Ability to develop reliable, secure, and compliant data processing systems.Work closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.Self-driven and able to work independently without support.Desirable:
2 to 3 years of experience in ADF (medium expertise required).ABOUT THE COMPANY
Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in various industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. Infogain is also a Microsoft Gold Partner and Azure Expert Managed Services Provider (MSP).
#J-18808-Ljbffr
Location: Gurugram, IndiaExperience:
8 to 11 years of overall experience in Cloud Data Engineering, with a minimum of 3 to 5 years experience in Azure Databricks and PySpark.Skills:Proficient in SQL with experience in complex joins, subqueries, functions, and procedures.Experience in SQL tuning and query optimization.Design, develop, and maintain ETL pipelines using Databricks, PySpark, and Azure Data Factory (ADF).Experience in implementing large and complex hosted solutions using Azure (ADF, Azure Data Lake Storage, and Databricks).Experience with Delta tables, deduplication, and merging with terabytes of datasets.Optimize and fine-tune existing ETL workflows for performance and scalability.Experience working with large datasets.Experience working with batch and streaming data processing (good to have).Ability to develop reliable, secure, and compliant data processing systems.Work closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.Self-driven and able to work independently without support.Desirable:
2 to 3 years of experience in ADF (medium expertise required).ABOUT THE COMPANY
Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in various industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. Infogain is also a Microsoft Gold Partner and Azure Expert Managed Services Provider (MSP).
#J-18808-Ljbffr