Health New England
Data Engineer
Health New England, Springfield, Massachusetts, us, 01119
Purpose
Contributes to the overall success of the IT Department ensuring specific individual goals, plans, initiatives are executed / delivered in support of the team's business strategies and objectives. Ensures all activities conducted are in compliance with governing regulations, internal policies and procedures.
The Data Engineer will be responsible for expanding and optimizing data, data pipeline architecture, and data flows. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up in a hybrid, multi-cloud environment. This individual will support our software developers, database architects, data analysts, and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. This individual will be a crucial contributor to data governance activities in ensuring data standards, semantics, structures, and models are employed during data ingestion, transformation, processing, and consumption. This individual will collaborate with data governance stakeholders such as data stewards, data custodians, and data subject area experts to ensure existing data standards are kept up-to-date and new standards are incorporated through governance process.
Accountabilities
Champions a customer focused culture to deepen client relationships and leverage broader Health New England's relationships, systems and knowledge.Create and maintain optimal data pipeline architectureIdentify, design, and implement internal process improvements: automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.Design and develop stored procedures, functions, views and triggers to be used during the ETL/ELT processes using Microsoft SQL Server and Azure data stack.Builds data transformations including importing data from files, moving data from one database platform to another, other data stores including Azure data storage and file systems, using technologies such as Azure Data Factory, MS SSIS, Databricks, Spark, and Synapse Data Flows.Incorporate security and access controls preserving privacy, integrity, and confidentiality of the data in transit and at restAnalyze and develop strategies and approaches to import and transfer data between source, staging, and destinations using medallion architecture (bronze, silver, and gold data levels).Test and prepare ETL processes for deployment to production and non-production environmentsSupport system and acceptance testing including the development or refinement of test plans.Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologiesParticipate and contribute to data governance by collaborating with business data stewards, data custodians, and other SME in ensuring that data policies and standards are enforced and cataloged.Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needsCreate data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leaderWork with data and analytics experts to strive for greater functionality in our data systemsUnderstand how Health New England's culture and the CX Promise should be considered in day-to-day activities and decisions.Actively pursues effective and efficient operations of their respective areas in accordance with Health New England's Values, its Code of Conduct and the Associate Handbook, while ensuring the adequacy, adherence to and effectiveness of day-to-day business controls to meet obligations with respect to operational, compliance, and conductChampions a high performance environment and contributes to an inclusive work environment.Position: Data Engineer/IT Data Management
Reporting Relationships (Job Titles only)
Primary Manager:
(include secondary Manager if applicable)
IT Manager Data ManagementDirect Reports:
NAShared Reports (solid/dotted if applicable):
NAExternal/Internal Contacts
Internal
External
Education / Experience / Other Information (include only those that are specific to the role)
Bachelor's degree in Computer Science, Statistics, Informatics, Information Systems or related field with a minimum of 2 + years' experience in Data Engineer or ETL Engineer role or an equivalent combination of education and experience. Preferred experience using the following software/tools:
2 + years' Experience with relational SQL database such as MS SQL Server, Azure SQL, MySQL.Familiarity with non-relational data stores such as file stores, columnar, key-value, and document stores.Experience with any of the data pipeline and workflow management tools such as MS SSIS or Azure Data Factory.Familiarity with cloud services such as managed service for SQL ServerFamiliarity with batch and stream processingExperience with scripting languages such as Python Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databasesHealth Insurance data domain familiarity would be a plusExperience building and optimizing data pipelines, architectures and data sets using technologies such as MS SSIS, Azure Data Factory, or a similar technologyExperience performing analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvementStrong analysis, processing, and wrangling skills working with structured, semi-structured, and unstructured datasetsBuild processes supporting data transformation, data structures, metadata, dependency and workload managementWorking knowledge of message queuing, stream processing, and highly scalable 'big data' data storesAbility to work both independently and as part of a collaborative teamStrong experience in software engineering best practicesExcellent communication skills, both verbal and writtenCustomer-service mindset with the desire to add value to the companyFast learner who seeks to understand not only the data, but what the data means to the businessComfortable leading meetings and conducting training sessions as neededSelf-driven with ability to manage workload with minimal supervisionStrong time management, planning, and prioritization experience
Contributes to the overall success of the IT Department ensuring specific individual goals, plans, initiatives are executed / delivered in support of the team's business strategies and objectives. Ensures all activities conducted are in compliance with governing regulations, internal policies and procedures.
The Data Engineer will be responsible for expanding and optimizing data, data pipeline architecture, and data flows. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up in a hybrid, multi-cloud environment. This individual will support our software developers, database architects, data analysts, and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. This individual will be a crucial contributor to data governance activities in ensuring data standards, semantics, structures, and models are employed during data ingestion, transformation, processing, and consumption. This individual will collaborate with data governance stakeholders such as data stewards, data custodians, and data subject area experts to ensure existing data standards are kept up-to-date and new standards are incorporated through governance process.
Accountabilities
Champions a customer focused culture to deepen client relationships and leverage broader Health New England's relationships, systems and knowledge.Create and maintain optimal data pipeline architectureIdentify, design, and implement internal process improvements: automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.Design and develop stored procedures, functions, views and triggers to be used during the ETL/ELT processes using Microsoft SQL Server and Azure data stack.Builds data transformations including importing data from files, moving data from one database platform to another, other data stores including Azure data storage and file systems, using technologies such as Azure Data Factory, MS SSIS, Databricks, Spark, and Synapse Data Flows.Incorporate security and access controls preserving privacy, integrity, and confidentiality of the data in transit and at restAnalyze and develop strategies and approaches to import and transfer data between source, staging, and destinations using medallion architecture (bronze, silver, and gold data levels).Test and prepare ETL processes for deployment to production and non-production environmentsSupport system and acceptance testing including the development or refinement of test plans.Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologiesParticipate and contribute to data governance by collaborating with business data stewards, data custodians, and other SME in ensuring that data policies and standards are enforced and cataloged.Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needsCreate data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leaderWork with data and analytics experts to strive for greater functionality in our data systemsUnderstand how Health New England's culture and the CX Promise should be considered in day-to-day activities and decisions.Actively pursues effective and efficient operations of their respective areas in accordance with Health New England's Values, its Code of Conduct and the Associate Handbook, while ensuring the adequacy, adherence to and effectiveness of day-to-day business controls to meet obligations with respect to operational, compliance, and conductChampions a high performance environment and contributes to an inclusive work environment.Position: Data Engineer/IT Data Management
Reporting Relationships (Job Titles only)
Primary Manager:
(include secondary Manager if applicable)
IT Manager Data ManagementDirect Reports:
NAShared Reports (solid/dotted if applicable):
NAExternal/Internal Contacts
Internal
External
Education / Experience / Other Information (include only those that are specific to the role)
Bachelor's degree in Computer Science, Statistics, Informatics, Information Systems or related field with a minimum of 2 + years' experience in Data Engineer or ETL Engineer role or an equivalent combination of education and experience. Preferred experience using the following software/tools:
2 + years' Experience with relational SQL database such as MS SQL Server, Azure SQL, MySQL.Familiarity with non-relational data stores such as file stores, columnar, key-value, and document stores.Experience with any of the data pipeline and workflow management tools such as MS SSIS or Azure Data Factory.Familiarity with cloud services such as managed service for SQL ServerFamiliarity with batch and stream processingExperience with scripting languages such as Python Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databasesHealth Insurance data domain familiarity would be a plusExperience building and optimizing data pipelines, architectures and data sets using technologies such as MS SSIS, Azure Data Factory, or a similar technologyExperience performing analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvementStrong analysis, processing, and wrangling skills working with structured, semi-structured, and unstructured datasetsBuild processes supporting data transformation, data structures, metadata, dependency and workload managementWorking knowledge of message queuing, stream processing, and highly scalable 'big data' data storesAbility to work both independently and as part of a collaborative teamStrong experience in software engineering best practicesExcellent communication skills, both verbal and writtenCustomer-service mindset with the desire to add value to the companyFast learner who seeks to understand not only the data, but what the data means to the businessComfortable leading meetings and conducting training sessions as neededSelf-driven with ability to manage workload with minimal supervisionStrong time management, planning, and prioritization experience