Insight Global
REMOTE Lead Azure Data Engineer
Insight Global, Minneapolis, Minnesota, United States, 55400
Job Description
Apart of the B360 team, giving holistic views of patients and members.
*Take epics from POs, break them down into stories and lead the team in design best practices
Design, develop, and implement end-to-end data solutions using Azure Databricks and other relevant technologies.
Build and maintain scalable data pipelines to extract, transform, and load large volumes of data from various sources.
Collaborate with cross-functional teams to gather and analyze data requirements, then translate business needs into optimized and efficient data models.
Ensure data quality and integrity by implementing data validation and cleansing processes.
Optimize data processing and storage for improved performance and efficiency.
Conduct data profiling, analysis, and modeling to identify opportunities for data-driven insights and recommendations.
Monitor and troubleshoot data pipelines and processes to ensure continuous data flow and availability.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
3+ years of most-recent experience as a Data Engineer in an Azure, not heavy AWS, environment.
Proficient with scripting in Python, as well as with other languages such as PySpark, Scala, and SQL.
Strong experience building pipelines with Databricks and with other Azure services, such as Azure Data Factory and Azure SQL Database.
Experience with data integration, transformation and orchestration frameworks such as Airflow, dbt, Fivetran or similar (dependencies and best practices).
*Rest API experience, specifically GraphQL- a builder not just a consumer connecting to APIs
*Data Lake and/or Data Warehouse management experience
*NoSQL experience, specifically performance modeling for cloud computing
*DevOps experience CI/CD pipelines and containerization
Clear and personable communicator to work closely with the business to problem-solve.
*Has been a lead before and can coach teammates on new skills and help hone them in on the right direction to stay in scope of stories.
*Experience in a large organization, and ability to work in that kind of fast-paced, agile environment. *Security best practices for API development.
*Medical industry knowledge/Healthcare data experience (payor/provider, familiar with HIPPA regulations)
*Relevant certifications in Azure Databricks and Fabric/medallion architecture. null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Apart of the B360 team, giving holistic views of patients and members.
*Take epics from POs, break them down into stories and lead the team in design best practices
Design, develop, and implement end-to-end data solutions using Azure Databricks and other relevant technologies.
Build and maintain scalable data pipelines to extract, transform, and load large volumes of data from various sources.
Collaborate with cross-functional teams to gather and analyze data requirements, then translate business needs into optimized and efficient data models.
Ensure data quality and integrity by implementing data validation and cleansing processes.
Optimize data processing and storage for improved performance and efficiency.
Conduct data profiling, analysis, and modeling to identify opportunities for data-driven insights and recommendations.
Monitor and troubleshoot data pipelines and processes to ensure continuous data flow and availability.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
3+ years of most-recent experience as a Data Engineer in an Azure, not heavy AWS, environment.
Proficient with scripting in Python, as well as with other languages such as PySpark, Scala, and SQL.
Strong experience building pipelines with Databricks and with other Azure services, such as Azure Data Factory and Azure SQL Database.
Experience with data integration, transformation and orchestration frameworks such as Airflow, dbt, Fivetran or similar (dependencies and best practices).
*Rest API experience, specifically GraphQL- a builder not just a consumer connecting to APIs
*Data Lake and/or Data Warehouse management experience
*NoSQL experience, specifically performance modeling for cloud computing
*DevOps experience CI/CD pipelines and containerization
Clear and personable communicator to work closely with the business to problem-solve.
*Has been a lead before and can coach teammates on new skills and help hone them in on the right direction to stay in scope of stories.
*Experience in a large organization, and ability to work in that kind of fast-paced, agile environment. *Security best practices for API development.
*Medical industry knowledge/Healthcare data experience (payor/provider, familiar with HIPPA regulations)
*Relevant certifications in Azure Databricks and Fabric/medallion architecture. null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.