Bespoketechinc
QH - Data Engineer - SME
Bespoketechinc, Mc Lean, Virginia, us, 22107
Data Engineer - Subject Matter ExpertLocation: McLean
** MUST HAVE A POLY CLEARANCE TO APPLY**
Description:The candidate shall develop new tools, code, and services to execute data engineering activities on provided systems. Data engineering activities for each organization shall include the following tasks:Movement of structured and unstructured data (gigabyte to terabyte range) using approved methods.Execute data ingestion activities for storing data in a local or enterprise level (Integrated Data Layer) location.View data in its source format.Develop code to format data that facilitates exploration.Analyze source data formats and work with Data Scientists to determine the formats and transforms that best meet objectives.Develop code and tools to provide one-time and ongoing data formatting and transformations into enterprise or boutique data models.Implement existing ETL code and best practices/standards that are currently in use in the enterprise.Develop an
ETL Code Transition Plan
when a specific project is identified.Develop and deliver
Software Documentation
for each code project that includes ETL mappings, code use guide, code location (generally GitHub) and access instructions, and anomalies encountered.Facilitate
Code Reviews
twice a year for each organization and one for each project. Code Reviews shall identify bugs and areas for code improvement to ensure high quality software. Candidate shall document results.Candidate shall provide consulting services to support needs for data transport, ingestion, conditioning, access, and management.Candidate shall support code review up to two times a year.Required Skills:
AWS (Intermediate)Linux (Intermediate)Python (Intermediate)SQL (Intermediate)HTTP API Usage/Integration (Intermediate)Experience analyzing diverse file types (text, image, video, audio, and binary) (Basic)Experience with geospatial tools/data (Basic)Strongly Desired Skills:
NiFi (Intermediate)Docker (Intermediate)ElasticSearch (Intermediate)Kibana (Basic)Puppet (Intermediate)Solr (Basic)Postgres Admin (Basic)MariaDB Admin (Basic)Hadoop/Spark (Intermediate)Kafka (Beginner)
#J-18808-Ljbffr
** MUST HAVE A POLY CLEARANCE TO APPLY**
Description:The candidate shall develop new tools, code, and services to execute data engineering activities on provided systems. Data engineering activities for each organization shall include the following tasks:Movement of structured and unstructured data (gigabyte to terabyte range) using approved methods.Execute data ingestion activities for storing data in a local or enterprise level (Integrated Data Layer) location.View data in its source format.Develop code to format data that facilitates exploration.Analyze source data formats and work with Data Scientists to determine the formats and transforms that best meet objectives.Develop code and tools to provide one-time and ongoing data formatting and transformations into enterprise or boutique data models.Implement existing ETL code and best practices/standards that are currently in use in the enterprise.Develop an
ETL Code Transition Plan
when a specific project is identified.Develop and deliver
Software Documentation
for each code project that includes ETL mappings, code use guide, code location (generally GitHub) and access instructions, and anomalies encountered.Facilitate
Code Reviews
twice a year for each organization and one for each project. Code Reviews shall identify bugs and areas for code improvement to ensure high quality software. Candidate shall document results.Candidate shall provide consulting services to support needs for data transport, ingestion, conditioning, access, and management.Candidate shall support code review up to two times a year.Required Skills:
AWS (Intermediate)Linux (Intermediate)Python (Intermediate)SQL (Intermediate)HTTP API Usage/Integration (Intermediate)Experience analyzing diverse file types (text, image, video, audio, and binary) (Basic)Experience with geospatial tools/data (Basic)Strongly Desired Skills:
NiFi (Intermediate)Docker (Intermediate)ElasticSearch (Intermediate)Kibana (Basic)Puppet (Intermediate)Solr (Basic)Postgres Admin (Basic)MariaDB Admin (Basic)Hadoop/Spark (Intermediate)Kafka (Beginner)
#J-18808-Ljbffr