Big Data Engineer

This person is a key member of the TXDOT Enterprise Data Architecture team, and this resource will develop the requisite data pipelines to load the TxDOT data warehouse and other analytical repositories to support advanced analytics by the business teams. They will use a combination of technologies and methods to (1) build a replication framework, (2) curate and standardize data coming from different sources, and (3) load reporting structures. This resource will be using core cloud data warehouse tools and other data management-related technologies.Responsibilities:•Optimally leverage the data integration tool components for developing efficient solutions for data management, data wrangling, data packaging, and integration. Develop overall design and determine the division of labor across various architectural components•Publish data into the TxDOT data marketplace with necessary information to facilitate search, access, and use of the data sets by the consumers and analysts•Support the business IT teams in using the Enterprise Data Platform tools, methods, etc. for customizing any published data set•Build and deploy standard architecture components•Assist in the development of task plans including schedule and effort estimationThe workers responsibilities and skills must include:WORKER SKILLS AND QUALIFICATIONS (Required)Minimum (Required): insert specific minimum skills for this Contract Position)YearsSkills/Experience

2 Experience working with Snowflake DB5 Strong and deep experience building data pipelines using Python, AWS services (Glue, Step Function, Event manager, etc.)

5 Experience developing, deploying, and supporting scalable and high-performance data pipelines (leveraging distributed, data movement technologies and approaches, including but not limited to ETL and streaming ingestion and processing)6 Experience with 1 complete life cycle Data and Business Intelligence implementations (architecting ETL pipelines, building data delivery frameworks, etc.)

Required Skills : SDLCBackground Check :YesDrug Screen :YesNotes :Selling points for candidate :Project Verification Info :Candidate must be your W2 Employee :NoExclusive to Apex :NoFace to face interview required :NoCandidate must be local :NoCandidate must be authorized to work without sponsorship ::NoInterview times set : :NoType of project :Development/EngineeringMaster Job Title :Big Data: DevBranch Code :Austin