Lorven Technologies
ETL Data Engineer - 12 Months Contract - Washington D.C - Onsite
Lorven Technologies, Washington, District of Columbia 20036
Role: ETL Data Engineer Location: Washington DC Duration: 12 Months Contract Position Responsibilities: Analyzes, designs, develops, implements, replicates, and supports complex enterprise data projects. Interfaces with other agencies, consults with and informs user departments on system requirements, advises on environmental constraints and operating difficulties for the current state, advises and resolves problems using cloud solutions, and develops and replicates future enhancements to the District's data systems. Strong knowledge of Extraction, Transformation, and Loading (ETL) processes using frameworks like Azure Data Factory, Synapse, Databricks, and Informatica by gathering requirements from the stakeholders or analyzing the existing code and performing enhancements or new development. Establishing the cloud and on-premises connectivity in different systems like ADLS, ADF, Synapse, Databricks. Hands-on experience in Azure cloud services like Azure Data Factory or Azure Synapse, MSSQL DB, Azure SQL DB, Azure Data Lake Storage Gen2, Blob Storage, Python, etc. Worked on creating end to end pipelines to load data by reading it from multiple sources or source systems and load to landing layer or SQL tables. Familiarity/experience with data integration and data pipeline tools (e.g., Informatica, Synapse, Apache NiFi, Apache Airflow). Familiarity/experience with various data formats, including database-specific (Oracle, SQL Server, DB2, QuickBase), text formats(CSV, XML), and Binary(Parquet, AVRO). Develops, standardizes, and optimizes existing data workflow/pipelines adhering to best practices. Adhere and contribute to enterprise data governance standards by ensuring data accuracy, consistency, security, and reliability. Automates, monitors, alerts, and manages data pipelines and workflows. Analyzes and evaluates system changes to determine feasibility and provides alternative solutions, backup, and rollback procedures. Works on developing new systems, upgrades, and enhancements to existing systems and ensures systems follow approved standards and consistency after the changes. Develops complex programs and reports in database query language. Familiarity/experience with data visualization tools. Familiarity/experience handling and securing sensitive data based on the level of sensitivity. Position Duties and Expertise: Demonstrates expertise in conveying technical and functional concepts for a specific technical specialty. Identifies improvements to project standards to achieve high-quality services/ products. This is a professional position that may require subject matter expertise consistent with demanding and rare technological skills. May require coordination of programming activities being conducted by the application development team. Confers with other business and technical personnel to resolve problems of intent, inaccuracy, or feasibility of computer processing and project design. Works with necessary personnel to determine if modifications are required with interested personnel to determine the necessity for modifications or enhancements. Leverages excellent written and verbal communication skills to develop new business processes and programming solutions as directed by business and technical stakeholders. May coordinate activities of application developers. Able to identify best practices and standards for using the product. Proven track record of hands-on technical design and code work within large complex systems. Proven hands-on technical work with a variety of technologies. Demonstrated technical expertise in integrating diverse technical environments and cross-platform technologies. Delivers support and design for industry-specific applications that require integration with statewide systems or applications. Interact with executive-level business users or technical experts. Advanced experience in the required technical subject matter. May function as a niche technical SME (Subject Matter Expert). Has proven experience across large and complex implementations and systems.