CommIT Enterprises
Data Pipeline Engineer
CommIT Enterprises, North Charleston, South Carolina, United States, 29405
Overview:
CommIT Enterprises, Inc. is seeking an
Data Pipeline Engineer
to join the team in
Charleston, SC- *this position can also be remote! We are looking for a talented and forward-facing Data Pipeline Engineer who will be a critical part of a fast-moving Marine Corps product development team. This team is responsible for the design, development, testing, implementation and support for a wide range of cloud-based data integrations, data pipelines, data warehousing solutions, and delivery of Business Intelligence (BI) services to Marines globally.Established in 2001, CommIT is a Certified Veteran-Owned Small Business (CVOSB) providing innovative technical engineering and data science services. Our enterprise systems support includes the Department of Defenses (DoD) GCSS-MC, CAC2S, TBMCS-MC, and the Department of Veterans Affairs (VA) telehealth communications. We offer acquisition management, systems engineering, Agile software development, cloud management, IT modernization, data analytics, cybersecurity, and training, including leading-edge DevSecOps, automated testing, and mobile application development.Responsibilities:Your essential job functions will include but may not be limited to-You will identify and deliver data pipeline solutions every day!In this role you will be working in a cross-functional team to build data pipeline solutions and processes to enable data warehousing, analytics and reporting for the USMC Logistics community.You will be responsible for anything and everything related to existing and future Data Pipelines!Work with the Product Owner team and data analysts to develop efficient ETL processes using Apache Nifi, and other data pipeline tools.Build performant data transformations for analysis and reporting.Utilize modern ETL/ELT practices to move data into our Data Lake (AWS S3) and Warehouse (AWS Redshift).Own the design, code development, code reviews, testing, data quality monitoring, deployment activities and operations support for all Nifi data flows.Work alongside the infrastructure, data ingestion, data warehouse and analytics teams to manage and optimize ETL and data pipeline parameters, scheduling, monitoring and alerting.Support development of data dictionary, connection guides, operations and user support SLAs.Basic SQL operations required to Load data into the Data Warehouse (AWS Redshift)Proven analytical, troubleshooting, and problem-solving skills.Ability to communicate well cross-functionally, including non-technical users.Must have a strong background in building ETL workflows. Apache Nifi is preferred, but other similar tools such as AWS Glue, Talend, DataBricks, etcAbility to be self-starting, responsive and collaborative in a remote environment working with a distributed team.Stay current with the latest design and technology trends and adopt modern techniques as appropriate.Familiarity with operating in Cloud environmentsProficiency in one or more database platforms (e.g., MySQL, Oracle, SQL Server).Strong understanding of SQL and ability to optimize queries for performance.Familiarity with database monitoring tools to track performance metrics.Experience with open-source softwareExperience with AWS GovCloudFamiliarity with AWS Glue and AWS RedshiftFamiliarity with BI tools, e.g. Microsoft Power BI and AWS QuickSightBasic Linux command line experienceExperience with NNCompass functionality
Qualifications:Required Experience and Education:Education:
BS Degree in Electrical/ Electronic/ Computer /Software Engineering, Physics, or Computer Science from an accredited College or University.Experience:
Six (6) years of experience in DoD C4ISR systems, or information technologies to include: Systems Analysis, Systems Architecture, Systems/Equipment Support, Test and Evaluation, and Logistics support of C5ISR requirements. Three (3) years of technical experience in support of DoD C4ISR systems, or information technologies. Experience may be achieved simultaneously
OR-
Education:
Technical training in Computer Science, Information Systems, Engineering, Business, and Supply Chain related scientific or technical discipline or 10 years of Marine Corps experience in Maintenance, Communications, Logistics, or in the Retail Supply (SMU/Reparable Issue Point/RIP) and Organizational Supply field preferably.Experience:
Twelve (12) years of hands-on experience with a USMC supply, maintenance or logistics IT system, GCSS-MC, to includethree (3) of the following five (5) areas: Systems Requirements, Operational Requirements, Data Analysis, Test & Evaluation, and Training. Experience may be achieved simultaneously. Recognized expert who has demonstrated industry and/or public service (e.g., federal, state, or local) leadership in Defense Acquisitions University Level I or II (Life Cycle Logistics or Program Management) and experience with Major Automated Information System (MAIS).
Security Requirements:Secret ClearanceDOD 8570 compliant Security + and OS certificationEqual Opportunity Employer:CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.
CommIT Enterprises, Inc. is seeking an
Data Pipeline Engineer
to join the team in
Charleston, SC- *this position can also be remote! We are looking for a talented and forward-facing Data Pipeline Engineer who will be a critical part of a fast-moving Marine Corps product development team. This team is responsible for the design, development, testing, implementation and support for a wide range of cloud-based data integrations, data pipelines, data warehousing solutions, and delivery of Business Intelligence (BI) services to Marines globally.Established in 2001, CommIT is a Certified Veteran-Owned Small Business (CVOSB) providing innovative technical engineering and data science services. Our enterprise systems support includes the Department of Defenses (DoD) GCSS-MC, CAC2S, TBMCS-MC, and the Department of Veterans Affairs (VA) telehealth communications. We offer acquisition management, systems engineering, Agile software development, cloud management, IT modernization, data analytics, cybersecurity, and training, including leading-edge DevSecOps, automated testing, and mobile application development.Responsibilities:Your essential job functions will include but may not be limited to-You will identify and deliver data pipeline solutions every day!In this role you will be working in a cross-functional team to build data pipeline solutions and processes to enable data warehousing, analytics and reporting for the USMC Logistics community.You will be responsible for anything and everything related to existing and future Data Pipelines!Work with the Product Owner team and data analysts to develop efficient ETL processes using Apache Nifi, and other data pipeline tools.Build performant data transformations for analysis and reporting.Utilize modern ETL/ELT practices to move data into our Data Lake (AWS S3) and Warehouse (AWS Redshift).Own the design, code development, code reviews, testing, data quality monitoring, deployment activities and operations support for all Nifi data flows.Work alongside the infrastructure, data ingestion, data warehouse and analytics teams to manage and optimize ETL and data pipeline parameters, scheduling, monitoring and alerting.Support development of data dictionary, connection guides, operations and user support SLAs.Basic SQL operations required to Load data into the Data Warehouse (AWS Redshift)Proven analytical, troubleshooting, and problem-solving skills.Ability to communicate well cross-functionally, including non-technical users.Must have a strong background in building ETL workflows. Apache Nifi is preferred, but other similar tools such as AWS Glue, Talend, DataBricks, etcAbility to be self-starting, responsive and collaborative in a remote environment working with a distributed team.Stay current with the latest design and technology trends and adopt modern techniques as appropriate.Familiarity with operating in Cloud environmentsProficiency in one or more database platforms (e.g., MySQL, Oracle, SQL Server).Strong understanding of SQL and ability to optimize queries for performance.Familiarity with database monitoring tools to track performance metrics.Experience with open-source softwareExperience with AWS GovCloudFamiliarity with AWS Glue and AWS RedshiftFamiliarity with BI tools, e.g. Microsoft Power BI and AWS QuickSightBasic Linux command line experienceExperience with NNCompass functionality
Qualifications:Required Experience and Education:Education:
BS Degree in Electrical/ Electronic/ Computer /Software Engineering, Physics, or Computer Science from an accredited College or University.Experience:
Six (6) years of experience in DoD C4ISR systems, or information technologies to include: Systems Analysis, Systems Architecture, Systems/Equipment Support, Test and Evaluation, and Logistics support of C5ISR requirements. Three (3) years of technical experience in support of DoD C4ISR systems, or information technologies. Experience may be achieved simultaneously
OR-
Education:
Technical training in Computer Science, Information Systems, Engineering, Business, and Supply Chain related scientific or technical discipline or 10 years of Marine Corps experience in Maintenance, Communications, Logistics, or in the Retail Supply (SMU/Reparable Issue Point/RIP) and Organizational Supply field preferably.Experience:
Twelve (12) years of hands-on experience with a USMC supply, maintenance or logistics IT system, GCSS-MC, to includethree (3) of the following five (5) areas: Systems Requirements, Operational Requirements, Data Analysis, Test & Evaluation, and Training. Experience may be achieved simultaneously. Recognized expert who has demonstrated industry and/or public service (e.g., federal, state, or local) leadership in Defense Acquisitions University Level I or II (Life Cycle Logistics or Program Management) and experience with Major Automated Information System (MAIS).
Security Requirements:Secret ClearanceDOD 8570 compliant Security + and OS certificationEqual Opportunity Employer:CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.