Data Engineer II

Be part of a growing, successful company in an exciting and challenging field. Border States is 100% employee-owned, which means you're empowered to make decisions, find solutions and receive rewards for your hard w This position will be located in: Fargo , ND Job Summary The Data Engineer II designs and develops scalable, metadata-driven ETL/ELT data pipelines within the Azure ecosystem, primarily using Azure Synapse, Spark, SQL, and Python. Integrates data from various sources, including SAP S4, APIs, and other enterprise systems, ensuring efficient data flow and accurate analysis. Works closely with cross-functional teams to optimize performance, enhance data quality, and implement data governance standards. Contributes to building an advanced cloud data engineering platform leveraging Azure Synapse, Azure Purview, Power BI, and SAP Datasphere, while supporting the overall data engineering strategy of the organization. Responsibilities Essential Functions Develops, optimizes, and manages ETL/ELT pipelines. Designs scalable architectures for large-scale data processing within the Azure ecosystem. Implements data integration solutions using Azure Synapse Pipelines, Azure Data Factory, and other Azure-native tools. Integrates data from enterprise data sources (SAP, API, etc.) into cloud-based data solutions. Establishes CI/CD pipelines using Git (Azure DevOps), and related tools for automated deployment and monitoring of data solutions. Troubleshoots, optimizes, and maintains the performance of data pipelines and cloud-based platforms. Collaborates with business and technical teams to architect robust data solutions and drive insights from data. Applies best practices for data governance, security, and data quality using Azure Purview and other governance tools. Documents technical solutions and processes and provides training for internal teams on the data platform. Non-essential Functions Adheres to Border States' System Development Life Cycle (SDLC) best practice guidelines and application change management procedure. Actively tracks and reports on the effectiveness of deployed technologies in supporting the goals of the program/project/initiative. Maintains familiarity with front-end reporting tools, including Power BI, for enhanced data visualization and reporting. Other duties as assigned by supervisor or designate. Qualifications Bachelor's degree in Computer Sciences / Information Technology / MIS / Mathematics, or equivalent experience. 4-7 years of proven experience in developing technical data solutions within cloud environments, with a strong focus on Azure Synapse. Expertise in designing and developing ETL/ELT processes using metadata-driven pipelines. Proficient in SQL, Spark, Python, and Git for data engineering and pipeline automation. Extensive experience with Azure cloud services, including Azure Storage, Azure Data Factory, Azure DevOps, and Azure Purview. Hands-on experience with data quality management and governance. Familiarity with SAP systems, including SAP Datasphere and HANA S/4, specifically in working with Core Data Services (CDS) and HANA views. Solid understanding of Agile methodologies and tools such as Jira. Strong communication skills, able to work effectively with cross-functional teams and explain technical concepts to non-technical stakeholders. Skills and Abilities Ability to troubleshoot and resolve complex data engineering challenges. A team player with a passion for learning and applying new technologies and methodologies. Skilled in working in a fast-paced environment and delivering high-quality results under pressure. Strong problem-solving skills and resilience in overcoming obstacles. Effective communicator who can build relationships and work well with both technical and non-technical teams. Physical Requirements Frequency at which you would carry/lift in a typical day: Occasionally 0 - 3 % Frequently34 - 66% Continuously: 67 - 100% Lift from Waist: