The MITRE Corporation
Senior Data Engineer - AWS
The MITRE Corporation, Mc Lean, Virginia, us, 22107
Why choose between doing meaningful work and having a fulfilling life? At MITRE, you can have both. That's because MITRE people are committed to tackling our nation's toughest challenges—and we're committed to the long-term well-being of our employees. MITRE is different from most technology companies. We are a not-for-profit corporation chartered to work for the public interest, with no commercial conflicts to influence what we do. The R&D centers we operate for the government create lasting impact in fields as diverse as cybersecurity, healthcare, aviation, defense, and enterprise transformation. We're making a difference every day—working for a safer, healthier, and more secure nation and world. Our workplace reflects our values. We offer competitive benefits, exceptional professional development opportunities, and a culture of innovation that embraces diversity, inclusion, flexibility, collaboration, and career growth. If this sounds like the choice you want to make, then choose MITRE—and make a difference with us.
Join our Enterprise Data Warehouse team as we prepare for a major evolution in technology and process roll-out across MITRE. This is an exciting time to join the Enterprise Data Warehouse team as the use and gold source for our data evolves and grows exponentially. Your opportunity to make an impact is great as you help our team evolve to cloud-based hybrid data and real time data access. We are looking for a highly collaborative, team-oriented person to join us! A “can-do” and growth-focused attitude as well as the ability to work across more than one simultaneous project are musts!
Responsibilities include:
Design, develop, and implement robust ETL solutions using Python and PySpark to extract, transform, and load data from various sources into AWS data services.
Optimize ETL processes for performance and scalability utilizing AWS Glue, EMR, Step Functions, and Lambda to ensure efficient data processing and timely delivery.
Ensure data integrity and quality throughout the ETL process by implementing thorough data validation checks and error handling mechanisms.
Manage AWS services such as Glue, EMR, Step Functions, and Lambda, including configuration, monitoring, and troubleshooting to maintain operational excellence.
Collaborate with cross-functional teams including data engineers, data scientists, and business stakeholders to understand data requirements and deliver tailored ETL solutions.
Troubleshoot complex technical issues and provide advanced operational support to internal MITRE customers in AWS
Basic Qualifications:
Bachelor’s degree with eight years’ related experience, or a master’s degree with six years’ related experience, preferably with a technical major such as engineering, computer science, etc.
Self-motivated, curious, and collaborative, with a passion to learn new technologies and develop new skills
Demonstrated experience in developing ETL pipelines using Python and PySpark, with a strong understanding of data processing techniques.
Expertise in SQL for data manipulation, querying, and optimization to work with various database platforms including Postgres, DynamoDB, Oracle, and Redshift.
Hands-on experience with AWS Glue, EMR, Step Functions, and Lambda for building and orchestrating ETL workflows in a cloud environment.
Experience implementing Continuous Integration/Continuous Deployment (CI/CD) pipelines using AWS CDK or similar tools for automating deployment and testing of ETL solutions.
Preferred Qualifications:
Previous experience leading or mentoring a team of developers/engineers in a collaborative environment.
AWS certifications such as AWS Certified Developer or AWS Certified Solutions Architect, demonstrating proficiency in AWS services and best practices.
Familiarity with big data technologies such as Apache Spark, Hadoop, or Kafka for processing large-scale datasets.
Experience in data modeling and schema design for optimizing database performance and scalability.
Experience working in Agile development methodologies, such as Scrum or Kanban, for iterative and collaborative project delivery.
This requisition requires the candidate to have a minimum of the following clearance(s):
None
This requisition requires the hired candidate to have or obtain, within one year from the date of hire, the following clearance(s):
None
Work Location Type:
Hybrid
MITRE is proud to be an equal opportunity employer. MITRE recruits, employs, trains, compensates, and promotes regardless of age; ancestry; color; family medical or genetic information; gender identity and expression; marital, military, or veteran status; national and ethnic origin; physical or mental disability; political affiliation; pregnancy; race; religion; sex; sexual orientation; and any other protected characteristics. For further information please visit the Equal Employment Opportunity Commission website EEO is the Law Poster (https://www.eeoc.gov/sites/default/files/2022-10/22-088_EEOC_KnowYourRights_10_20.pdf) and Pay Transparency (https://www.dol.gov/sites/dolgov/files/OFCCP/pdf/pay-transp_%20English_formattedESQA508c.pdf) .
MITRE intends to maintain a website that is fully accessible to all individuals. If you are unable to search or apply for jobs and would like to request a reasonable accommodation for any part of MITRE’s employment process, please email recruitinghelp@mitre.org .
Copyright © 2024, The MITRE Corporation. All rights reserved. MITRE is a registered trademark of The MITRE Corporation. Material on this site may be copied and distributed with permission only.
Benefits information may be found here (https://careers.mitre.org/us/en/benefits)
Join our Enterprise Data Warehouse team as we prepare for a major evolution in technology and process roll-out across MITRE. This is an exciting time to join the Enterprise Data Warehouse team as the use and gold source for our data evolves and grows exponentially. Your opportunity to make an impact is great as you help our team evolve to cloud-based hybrid data and real time data access. We are looking for a highly collaborative, team-oriented person to join us! A “can-do” and growth-focused attitude as well as the ability to work across more than one simultaneous project are musts!
Responsibilities include:
Design, develop, and implement robust ETL solutions using Python and PySpark to extract, transform, and load data from various sources into AWS data services.
Optimize ETL processes for performance and scalability utilizing AWS Glue, EMR, Step Functions, and Lambda to ensure efficient data processing and timely delivery.
Ensure data integrity and quality throughout the ETL process by implementing thorough data validation checks and error handling mechanisms.
Manage AWS services such as Glue, EMR, Step Functions, and Lambda, including configuration, monitoring, and troubleshooting to maintain operational excellence.
Collaborate with cross-functional teams including data engineers, data scientists, and business stakeholders to understand data requirements and deliver tailored ETL solutions.
Troubleshoot complex technical issues and provide advanced operational support to internal MITRE customers in AWS
Basic Qualifications:
Bachelor’s degree with eight years’ related experience, or a master’s degree with six years’ related experience, preferably with a technical major such as engineering, computer science, etc.
Self-motivated, curious, and collaborative, with a passion to learn new technologies and develop new skills
Demonstrated experience in developing ETL pipelines using Python and PySpark, with a strong understanding of data processing techniques.
Expertise in SQL for data manipulation, querying, and optimization to work with various database platforms including Postgres, DynamoDB, Oracle, and Redshift.
Hands-on experience with AWS Glue, EMR, Step Functions, and Lambda for building and orchestrating ETL workflows in a cloud environment.
Experience implementing Continuous Integration/Continuous Deployment (CI/CD) pipelines using AWS CDK or similar tools for automating deployment and testing of ETL solutions.
Preferred Qualifications:
Previous experience leading or mentoring a team of developers/engineers in a collaborative environment.
AWS certifications such as AWS Certified Developer or AWS Certified Solutions Architect, demonstrating proficiency in AWS services and best practices.
Familiarity with big data technologies such as Apache Spark, Hadoop, or Kafka for processing large-scale datasets.
Experience in data modeling and schema design for optimizing database performance and scalability.
Experience working in Agile development methodologies, such as Scrum or Kanban, for iterative and collaborative project delivery.
This requisition requires the candidate to have a minimum of the following clearance(s):
None
This requisition requires the hired candidate to have or obtain, within one year from the date of hire, the following clearance(s):
None
Work Location Type:
Hybrid
MITRE is proud to be an equal opportunity employer. MITRE recruits, employs, trains, compensates, and promotes regardless of age; ancestry; color; family medical or genetic information; gender identity and expression; marital, military, or veteran status; national and ethnic origin; physical or mental disability; political affiliation; pregnancy; race; religion; sex; sexual orientation; and any other protected characteristics. For further information please visit the Equal Employment Opportunity Commission website EEO is the Law Poster (https://www.eeoc.gov/sites/default/files/2022-10/22-088_EEOC_KnowYourRights_10_20.pdf) and Pay Transparency (https://www.dol.gov/sites/dolgov/files/OFCCP/pdf/pay-transp_%20English_formattedESQA508c.pdf) .
MITRE intends to maintain a website that is fully accessible to all individuals. If you are unable to search or apply for jobs and would like to request a reasonable accommodation for any part of MITRE’s employment process, please email recruitinghelp@mitre.org .
Copyright © 2024, The MITRE Corporation. All rights reserved. MITRE is a registered trademark of The MITRE Corporation. Material on this site may be copied and distributed with permission only.
Benefits information may be found here (https://careers.mitre.org/us/en/benefits)