Bespoke Technologies LLC
QH - Data Engineer - SME
Bespoke Technologies LLC, Mc Lean, Virginia, us, 22107
Data Engineer - Subject Matter ExpertLocation: McLean
** MUST HAVE A POLY CLEARANCE TO APPLY**
Description:The candidate shall develop new tools, code, and services to execute data engineering activities on provided systems. Data engineering activities for each organization shall include the following tasks:Movement of structure and unstructured data (gigabyte to terabyte range) using approved methodsExecute data ingestion activities for storing data in a local or enterprise level (Integrated Data Layer) locationView data in its source formatDevelop code to format data that facilitates explorationAnalyze source data formats and work with Data Scientists to determine the formats and transforms that best meet objectivesDevelop code and tools to provide one-time and on-going data formatting and transformations into enterprise or boutique data modelsImplement existing ETL code and best practices/standards that are currently in use in the enterpriseDevelop an
ETL Code Transition Plan
when a specific project is identifiedDevelop and deliver
Software Documentation
for each code project that includes ETL mappings, code use guide, code location (generally GitHub) and access instructions), and anomalies encountered.Facilitate
Code Reviews
twice a year for each organization and one for each project. Code Reviews shall identify bugs and areas for code improvement to ensure high quality software. Candidate shall document results.Candidate shall provide consulting services to support needs for data transport, ingestion, conditioning, access, and management.Candidate shall support code review up to two times a year.Required Skills:
AWS (Intermediate)Linux (Intermediate)Python (Intermediate)SQL (Intermediate)HTTP API Usage/Integration (Intermediate)Experience analyzing diverse file types (text, image, video, audio, and binary) (Basic)Experience with geospatial tools/data (Basic)Strongly Desired skills:
NiFi (Intermediate)Docker (Intermediate)ElasticSearch (Intermediate)Kibana (Basic)Puppet (Intermediate)Solr (Basic)Postgres Admin(Basic)MariaDB Admin (Basic)Hadoop/Spark (Intermediate)Kafka (Beginner)
** MUST HAVE A POLY CLEARANCE TO APPLY**
Description:The candidate shall develop new tools, code, and services to execute data engineering activities on provided systems. Data engineering activities for each organization shall include the following tasks:Movement of structure and unstructured data (gigabyte to terabyte range) using approved methodsExecute data ingestion activities for storing data in a local or enterprise level (Integrated Data Layer) locationView data in its source formatDevelop code to format data that facilitates explorationAnalyze source data formats and work with Data Scientists to determine the formats and transforms that best meet objectivesDevelop code and tools to provide one-time and on-going data formatting and transformations into enterprise or boutique data modelsImplement existing ETL code and best practices/standards that are currently in use in the enterpriseDevelop an
ETL Code Transition Plan
when a specific project is identifiedDevelop and deliver
Software Documentation
for each code project that includes ETL mappings, code use guide, code location (generally GitHub) and access instructions), and anomalies encountered.Facilitate
Code Reviews
twice a year for each organization and one for each project. Code Reviews shall identify bugs and areas for code improvement to ensure high quality software. Candidate shall document results.Candidate shall provide consulting services to support needs for data transport, ingestion, conditioning, access, and management.Candidate shall support code review up to two times a year.Required Skills:
AWS (Intermediate)Linux (Intermediate)Python (Intermediate)SQL (Intermediate)HTTP API Usage/Integration (Intermediate)Experience analyzing diverse file types (text, image, video, audio, and binary) (Basic)Experience with geospatial tools/data (Basic)Strongly Desired skills:
NiFi (Intermediate)Docker (Intermediate)ElasticSearch (Intermediate)Kibana (Basic)Puppet (Intermediate)Solr (Basic)Postgres Admin(Basic)MariaDB Admin (Basic)Hadoop/Spark (Intermediate)Kafka (Beginner)