Dynamed Solutions, LLC
Data Engineer
Dynamed Solutions, LLC - Columbia, Maryland, United States, 21046
Work at Dynamed Solutions, LLC
Overview
- View job
Overview
Join to apply for the
Data Engineer
role at
Dynamed Solutions, LLC
Position Title: Data Engineer - CRISP Insights Reports Directly To:
Senior Data Engineer Technical Lead Location:
Columbia, MD Hours Per Week:
40 Risk Designation:
High Job Summary The Data Engineer will support HIT activities in the Baltimore/Washington region by designing, developing, and maintaining a data analytics platform. The current infrastructure includes Azure Data Lake 2, Azure Data Factory, Spark, and Databricks, with around 300 production use cases consolidating healthcare data streams. The role involves creating new use cases via Databricks programming, collaborating effectively, and learning new skills. The position also supports implementing new ingestion pipelines and troubleshooting production issues, working closely with a senior data engineer. This position offers a base salary and requires the candidate to work in the Columbia office one day a week. Essential Duties and Responsibilities
Design, build, and maintain large, complex data pipelines that meet business needs Develop automated controls and perform testing to ensure quality Create scalable frameworks for data ingestion and transformation Troubleshoot data pipeline issues and perform root cause analysis Collaborate with source system and business owners to incorporate changes Document data workflows, processes, and infrastructure Qualifications
Successful candidates will demonstrate proficiency in the following: Advanced programming skills in PySpark, Python, Spark SQL Understanding of Data Lakes, ETL processes within Azure Experience building complex data pipelines using Agile methodologies Experience with Databricks and Azure Data Factory architecture and implementation Required Experience and Education
4+ years of college education 4+ years of data integration experience 2+ years of advanced PySpark programming Experience with Databricks (preferred) Additional Details
Seniority level: Mid-Senior level Employment type: Full-time Job function: Information Technology Industries: IT Services and IT Consulting
#J-18808-Ljbffr
Data Engineer
role at
Dynamed Solutions, LLC
Position Title: Data Engineer - CRISP Insights Reports Directly To:
Senior Data Engineer Technical Lead Location:
Columbia, MD Hours Per Week:
40 Risk Designation:
High Job Summary The Data Engineer will support HIT activities in the Baltimore/Washington region by designing, developing, and maintaining a data analytics platform. The current infrastructure includes Azure Data Lake 2, Azure Data Factory, Spark, and Databricks, with around 300 production use cases consolidating healthcare data streams. The role involves creating new use cases via Databricks programming, collaborating effectively, and learning new skills. The position also supports implementing new ingestion pipelines and troubleshooting production issues, working closely with a senior data engineer. This position offers a base salary and requires the candidate to work in the Columbia office one day a week. Essential Duties and Responsibilities
Design, build, and maintain large, complex data pipelines that meet business needs Develop automated controls and perform testing to ensure quality Create scalable frameworks for data ingestion and transformation Troubleshoot data pipeline issues and perform root cause analysis Collaborate with source system and business owners to incorporate changes Document data workflows, processes, and infrastructure Qualifications
Successful candidates will demonstrate proficiency in the following: Advanced programming skills in PySpark, Python, Spark SQL Understanding of Data Lakes, ETL processes within Azure Experience building complex data pipelines using Agile methodologies Experience with Databricks and Azure Data Factory architecture and implementation Required Experience and Education
4+ years of college education 4+ years of data integration experience 2+ years of advanced PySpark programming Experience with Databricks (preferred) Additional Details
Seniority level: Mid-Senior level Employment type: Full-time Job function: Information Technology Industries: IT Services and IT Consulting
#J-18808-Ljbffr