Vital Care Infusion Services
Technical Lead Data Engineering
Vital Care Infusion Services, Nashville, Tennessee, United States,
Job Summary:
The Technical Lead, Data Engineering will be responsible for architect design, develop and implement technology and data solutions to meet current and future data warehousing and reporting needs. The experienced lead engineer will set the technical data direction, establish best practices, and partner with all consumers of data to build and support the company's data warehouse, ETL, and reporting platforms.
Duties/Responsibilities:
Data Architecture and Design:Design, architect, and implement scalable and efficient data pipelines to ingest, process, and transform large volumes of structured and unstructured data from multiple sources.Define and implement data models, schemas, and storage solutions that optimize performance, reliability, and scalability.Evaluate and recommend appropriate data storage technologies, frameworks, and platforms based on the organization's requirements and objectives.Data Pipeline Development and Automation:Develop and maintain robust, fault-tolerant data pipelines using ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes.Implement data validation, error handling, and monitoring mechanisms to ensure data quality and integrity throughout the pipeline lifecycle.Automate data workflows, scheduling, and orchestration tasks using tools such as Apache Airflow, Luigi, or similar frameworks.Data Integration and Transformation:Integrate data from various internal and external sources, including databases, APIs, logs, and streaming platforms, to support business analytics, reporting, and machine learning initiatives.Transform raw data into meaningful insights and actionable information through data cleansing, enrichment, normalization, and aggregation techniques.Collaborate with data scientists and analysts to translate business requirements into technical specifications and data processing workflows.Performance Optimization and Scalability:Optimize the performance and efficiency of data pipelines, storage systems, and processing engines to meet service level agreements (SLAs) and performance targets.Identify and resolve performance bottlenecks, scalability issues, and resource constraints through system tuning, caching strategies, and infrastructure scaling.Data Security and Compliance:Implement and enforce data security controls, encryption mechanisms, and access policies to protect sensitive information and ensure compliance with regulatory requirements (e.g., GDPR, HIPAA, CCPA).Monitor data access patterns, audit trails, and user activities to detect and mitigate potential security threats and data breaches.Documentation and Collaboration:Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing, collaboration, and troubleshooting.Collaborate with cross-functional teams, including data scientists, software engineers, and business stakeholders, to understand data requirements, prioritize initiatives, and deliver solutions that meet business objectives.
Education and Experience:
Bachelor’s or master’s degree in computer science, Engineering, or a related field.Technical Leadership experience in data engineering, database development, or related roles, focusing on designing and building data pipelines and infrastructure.
Required Skills/Abilities:
Proficiency in SQL and other programming languages with data processing frameworks.Strong understanding of distributed systems, data modeling, database design principles, and performance optimization techniques.Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g., S3, Redshift, BigQuery, Dataflow).Familiarity with containerization and orchestration tools like Docker, Kubernetes, and infrastructure-as-code (IaC) concepts.Excellent problem-solving skills, attention to detail, and the ability to work effectively in a fast-paced, collaborative environment.Effective communication and interpersonal skills, with the ability to communicate complex technical concepts to non-technical stakeholders and collaborate effectively with cross-functional teams.
#J-18808-Ljbffr
The Technical Lead, Data Engineering will be responsible for architect design, develop and implement technology and data solutions to meet current and future data warehousing and reporting needs. The experienced lead engineer will set the technical data direction, establish best practices, and partner with all consumers of data to build and support the company's data warehouse, ETL, and reporting platforms.
Duties/Responsibilities:
Data Architecture and Design:Design, architect, and implement scalable and efficient data pipelines to ingest, process, and transform large volumes of structured and unstructured data from multiple sources.Define and implement data models, schemas, and storage solutions that optimize performance, reliability, and scalability.Evaluate and recommend appropriate data storage technologies, frameworks, and platforms based on the organization's requirements and objectives.Data Pipeline Development and Automation:Develop and maintain robust, fault-tolerant data pipelines using ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes.Implement data validation, error handling, and monitoring mechanisms to ensure data quality and integrity throughout the pipeline lifecycle.Automate data workflows, scheduling, and orchestration tasks using tools such as Apache Airflow, Luigi, or similar frameworks.Data Integration and Transformation:Integrate data from various internal and external sources, including databases, APIs, logs, and streaming platforms, to support business analytics, reporting, and machine learning initiatives.Transform raw data into meaningful insights and actionable information through data cleansing, enrichment, normalization, and aggregation techniques.Collaborate with data scientists and analysts to translate business requirements into technical specifications and data processing workflows.Performance Optimization and Scalability:Optimize the performance and efficiency of data pipelines, storage systems, and processing engines to meet service level agreements (SLAs) and performance targets.Identify and resolve performance bottlenecks, scalability issues, and resource constraints through system tuning, caching strategies, and infrastructure scaling.Data Security and Compliance:Implement and enforce data security controls, encryption mechanisms, and access policies to protect sensitive information and ensure compliance with regulatory requirements (e.g., GDPR, HIPAA, CCPA).Monitor data access patterns, audit trails, and user activities to detect and mitigate potential security threats and data breaches.Documentation and Collaboration:Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing, collaboration, and troubleshooting.Collaborate with cross-functional teams, including data scientists, software engineers, and business stakeholders, to understand data requirements, prioritize initiatives, and deliver solutions that meet business objectives.
Education and Experience:
Bachelor’s or master’s degree in computer science, Engineering, or a related field.Technical Leadership experience in data engineering, database development, or related roles, focusing on designing and building data pipelines and infrastructure.
Required Skills/Abilities:
Proficiency in SQL and other programming languages with data processing frameworks.Strong understanding of distributed systems, data modeling, database design principles, and performance optimization techniques.Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g., S3, Redshift, BigQuery, Dataflow).Familiarity with containerization and orchestration tools like Docker, Kubernetes, and infrastructure-as-code (IaC) concepts.Excellent problem-solving skills, attention to detail, and the ability to work effectively in a fast-paced, collaborative environment.Effective communication and interpersonal skills, with the ability to communicate complex technical concepts to non-technical stakeholders and collaborate effectively with cross-functional teams.
#J-18808-Ljbffr