Data Engineer

Job title: Data Engineer

Location: Remote

Duration: long Term

Corp to Corp

Mandatory Skills: Snowflake, DBT, Python, DWH

Job Description:

• 10+ years' experience with the following:
• Snowflake (Columnar MPP Cloud data warehouse)
• DBT (ETL tool)
• Python
• Experience designing and implementing Data Warehouse

Responsibility:

• Build High level technical design both for Streaming and batch processing systems
• Design and build reusable components, frameworks and libraries at scale to support analytics data products
• Perform POCs on new technology, architecture patterns
• Design and implement product features in collaboration with business and Technology stakeholders
• Anticipate, identify, and solve issues concerning data management to improve data quality
• Clean, prepare and optimize data at scale for ingestion and consumption
• Drive the implementation of new data management projects and re-structure of the current data architecture
• Implement complex automated workflows and routines using workflow scheduling tools
• Build continuous integration, test-driven development and production deployment frameworks
• Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
• Analyze and profile data for the purpose of designing scalable solutions
• Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
• Lead, Mentor and develop offshore Data Engineers in adopting best practices and deliver data products.
• Partner closely with product management to understand business requirements, breakdown Epics,
• Partner with Engineering Managers to define technology roadmaps, align on design, architecture, and enterprise strategy

Accountability:

• Requires minimal daily supervision
• Receives detailed instruction on new assignments and determines next steps with guidance
• Regularly reviews goals and objectives with supervisor
• Demonstrates competence in relevant job responsibilities which allows for increasing level of independence
• Ability to manage and prioritize multiple tasks
• Ability to work under pressure and meet deadlines

Problem Solving:

• Makes logical suggestions of likely causes of problems and independently suggests solutions
• Excellent organizational skills are required to prioritize responsibilities, thus completing work in a timely fashion
• Outstanding ability to multiplex tasks as required
• Excellent project management and/or business analysis skills.
• Attention to detail and concern for impact is essential.

Preferred:

• Azure/AWS cloud technology
• SQL objects (procedures, triggers, views, functions) in SQL Server. SQL query optimizations
• Understanding of T-SQL, indexes, stored procedures, triggers, functions, views, etc.
• Design and development of Azure/AWS Data Factory Pipelines preferred.
• Design and development of data marts in Snowflake preferred
• Working knowledge of Azure/AWS Architecture, Data Lake, Data Factory
• Business analysis experience to analyze data to write code and drive solutions
• Knowledge of: Git, Azure DevOps, Agile, Jira and Confluence