Sr Azure Data Engineer

140-150k + Bonus + Benefits / Greater Chicago Area / Onsite

• Provide support in designing and overseeing enterprise-grade data pipelines and data stores, essential for the development of advanced analytics programs, machine learning models, and statistical methods.
• Implement automation and streamline processes to optimize the entire data and analytics platform, ensuring efficient throughput and high-performance outcomes.
• Recognize, devise, and execute internal process enhancements, including automation of manual tasks, optimizing data delivery, and redesigning architecture or infrastructure to enhance scalability.
• Collate large, intricate datasets that align with functional and non-functional business demands.
• Develop processes that facilitate data transformation, manage data structures, metadata, dependencies, and workload management.
• Collaborate with business users to understand functional and data requirements, contributing to the enhancement of data models and pipelines.
• Apply expert-level analytical and problem-solving skills to diagnose and resolve intricate technical issues.
• Create, maintain, and continuously enhance scalable data pipelines, while also developing new data source integrations to accommodate the growing volume and complexity of data.
• Designing, implementing, and managing data extraction, transformation, and loading (ETL) processes.
• Creating comprehensive technical specification documents and application interface designs.
• Creating data processing and integration solutions for both batch and real-time scenarios, proficiently handling structured and unstructured data.
• Leading and participating in design discussions, code reviews, and project-related team meetings.
• Providing mentoring and guidance to junior team members, promoting a culture of knowledge sharing and collaborative problem-solving.

Qualifications

• Exhibit a thorough understanding of Data Lake architectures, including raw, enriched, and curated layer concepts, and ETL/ELT operations.
• Exhibit a solid understanding of database design, data warehousing concepts, big data platforms, and ETL operations.
• Experience working with data integration techniques & self-service data preparation.
• Experience in requirements analysis, design, and prototyping.
• Experience deploying modern data solutions leveraging components like Azure functions, Azure Databricks, Azure Data Factory, Azure Databricks, Data Flows, Azure Data Lake, Azure SQL, Azure Synapse, Streaming Analytics.
• Experience with DevOps tools like Azure DevOps, Jenkins, Maven etc.
• Experience in building/operating/maintaining fault tolerant and scalable data processing integrations.
• Experience with SQL.
• Demonstrated experience of turning business use cases and requirements into technical solutions.