Logo
Bespoketechinc

QH - Data Engineer - SME

Bespoketechinc, Mc Lean, Virginia, us, 22107


Data Engineer - Subject Matter ExpertLocation: McLean

** MUST HAVE A POLY CLEARANCE TO APPLY**

Description:The candidate shall develop new tools, code, and services to execute data engineering activities on provided systems. Data engineering activities for each organization shall include the following tasks:Movement of structured and unstructured data (gigabyte to terabyte range) using approved methods.Execute data ingestion activities for storing data in a local or enterprise level (Integrated Data Layer) location.View data in its source format.Develop code to format data that facilitates exploration.Analyze source data formats and work with Data Scientists to determine the formats and transforms that best meet objectives.Develop code and tools to provide one-time and ongoing data formatting and transformations into enterprise or boutique data models.Implement existing ETL code and best practices/standards that are currently in use in the enterprise.Develop an

ETL Code Transition Plan

when a specific project is identified.Develop and deliver

Software Documentation

for each code project that includes ETL mappings, code use guide, code location (generally GitHub) and access instructions, and anomalies encountered.Facilitate

Code Reviews

twice a year for each organization and one for each project. Code Reviews shall identify bugs and areas for code improvement to ensure high quality software. Candidate shall document results.Candidate shall provide consulting services to support needs for data transport, ingestion, conditioning, access, and management.Candidate shall support code review up to two times a year.Required Skills:

AWS (Intermediate)Linux (Intermediate)Python (Intermediate)SQL (Intermediate)HTTP API Usage/Integration (Intermediate)Experience analyzing diverse file types (text, image, video, audio, and binary) (Basic)Experience with geospatial tools/data (Basic)Strongly Desired Skills:

NiFi (Intermediate)Docker (Intermediate)ElasticSearch (Intermediate)Kibana (Basic)Puppet (Intermediate)Solr (Basic)Postgres Admin (Basic)MariaDB Admin (Basic)Hadoop/Spark (Intermediate)Kafka (Beginner)

#J-18808-Ljbffr