Logo
Robert Half

Data Engineer

Robert Half, The Woodlands, TX, United States

Position Summary

The Data Engineer is a critical role focused on building and optimizing data pipeline architecture, as well as supporting data initiatives across the organization. The ideal candidate will have a strong technical background in data engineering, excellent problem-solving skills, and the ability to work collaboratively with cross-functional teams. The Data Engineer is responsible for designing, implementing, and maintaining scalable data pipelines that ensure the availability and quality of data for analytics and business intelligence purposes.

As the Data Engineer, you will play a key role in developing and maintaining the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from various sources. You will also be responsible for ensuring data integrity and consistency, as well as supporting the data needs of multiple teams, systems, and products. The ideal candidate is passionate about leveraging data to drive business insights and is committed to staying current with emerging technologies and industry trends.

Key Responsibilities:

  • Data Architecture & Modelling: Apply industry-accepted data architecture principles to design and maintain data models, stored procedures, replication, and ensure security and compliance.
  • Data Pipeline Development: Design, build, and maintain scalable data pipelines using Azure Data Factory, Lakehouse, Data Warehouses, Notebooks, and Dataflows within the Microsoft Fabric environment.
  • Data Extraction & Integration: Develop solutions for integrating and processing both structured and unstructured data from various sources to provide meaningful analytics.
  • Cross-Functional Collaboration: Work closely with data scientists, analysts, and business teams to understand data needs and deliver data solutions.
  • Optimization & Troubleshooting: Continuously optimize data pipelines to ensure reliability, efficiency, and performance. Troubleshoot and resolve data pipeline issues.
  • Data Governance & Security: Implement best practices for data governance and security to ensure data integrity and compliance with regulations.
  • Data Infrastructure Maintenance: Monitor and maintain data infrastructure for high availability and scalability.
  • Technology Advancements: Stay current with advancements in data engineering technologies, particularly Microsoft Fabric, and provide guidance on new technology trends.
  • Roadmap Management: Lead the Data Platform roadmap, focusing on capacity planning, future-proofing, and ongoing optimization.
  • Data Quality: Lead efforts to ensure the accessibility, consistency, and quality of data across the organization.
  • Vendor & Stakeholder Collaboration: Collaborate with IT teams, BI vendors, and business stakeholders to resolve issues and implement solutions. Manage relationships with vendors to shape the Data Platform roadmap.
  • Data Literacy Program: Promote and support the launch of a data literacy program across the business, enhancing data-driven decision-making.

Essential Qualifications:

  • Education: BS or MS in Computer Science or related technical field.
  • Experience: At least 8 years of experience in data infrastructure, including Data Warehouses and Data Marts. 8+ years of experience with Power BI, SQL, and ETL tools.
  • Technical Skills:
  • Proficiency with data pipeline tools such as Azure Data FactoryPySparkSpark SQL, and Python
  • Strong understanding of data modeling, ETL processes, and data warehousing concepts.
  • Experience working with cloud platforms, specifically Microsoft Azure
  • Skills:
  • Strong problem-solving, troubleshooting, and communication skills.
  • Ability to manage and communicate data warehouse strategies and work effectively with both technical and business teams.
  • Familiarity with SQLNo-SQLPower BI, and Data Warehousing technologies.

Desirable Qualifications:

  • Data Modeling: 5+ years of experience designing conceptual, logical, and physical data models on Azure.
  • Efficiency Improvement: Proven track record of enhancing system performance and efficiency.
  • Self-Starter: Independent thinker with a strong ability to manage projects and lead technical initiatives autonomously.
  • Advanced Technologies: Experience with Salesforce data structures, Python, SQL, SSRS, SSAS, SSIS, and Power BI.
  • Agile Methodology: Familiarity with Agile development practices and tools.
  • Business Collaboration: Experience working closely with business stakeholders to understand their data needs and develop solutions.
  • Communication Skills: Excellent presentation, oral, and written communication skills.

Work Environment & Travel:

  • The role may require up to 10% travel, including both domestic and international travel, particularly during peak times.