Omega Solutions
Lead Azure data engineer
Omega Solutions, Santa Clara, California, us, 95053
Responsibilities:
Design, develop, test, and deploy Data Engineering pipelines on Azure cloud platform.
Provide technical leadership and guide the team.
Analyze and profile the data to understand and resolve the data quality issues.
Participate in design and code reviews and provide guidance.
Create near real-time data pipelines and document the design.
Requirements:
10+ years of overall IT experience in designing and implementing Data Engineering solutions.
6+ years of experience primarily on Azure cloud platform using Azure BLOB Storage / ADLS, Azure Data Factory, Databricks, Synapse, Logic Apps and Key Vault.
Must have worked as a Lead on at least 2 projects.
Experience in coding SQLs and Python.
Must have experience in Data Warehousing.
Informatica Cloud experience is a plus.
Experience in Cosmos, HDInsight, Event Hubs, Azure Functions, APIs is a plus.
Experience in streaming data ingestion and stream data processing is a plus.
Experience in Spark (Python, Scala, or SQL) is a plus.
DataOps, or DevOps experience is a plus.
Azure Cloud certification preferred.
Design, develop, test, and deploy Data Engineering pipelines on Azure cloud platform.
Provide technical leadership and guide the team.
Analyze and profile the data to understand and resolve the data quality issues.
Participate in design and code reviews and provide guidance.
Create near real-time data pipelines and document the design.
Requirements:
10+ years of overall IT experience in designing and implementing Data Engineering solutions.
6+ years of experience primarily on Azure cloud platform using Azure BLOB Storage / ADLS, Azure Data Factory, Databricks, Synapse, Logic Apps and Key Vault.
Must have worked as a Lead on at least 2 projects.
Experience in coding SQLs and Python.
Must have experience in Data Warehousing.
Informatica Cloud experience is a plus.
Experience in Cosmos, HDInsight, Event Hubs, Azure Functions, APIs is a plus.
Experience in streaming data ingestion and stream data processing is a plus.
Experience in Spark (Python, Scala, or SQL) is a plus.
DataOps, or DevOps experience is a plus.
Azure Cloud certification preferred.