Amazon
Software Development Engineer, Managed Streaming for Apache Kafka
Amazon, Seattle, Washington, us, 98127
Description
Come build the future of data streaming with the Amazon Managed Streaming for Kafka (MSK) team!
We are seeking builders for our Amazon MSK service, a fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. We are looking for engineers who are enthusiastic about data streaming, and are as passionate about contributing to open source as they are about solving real customers' business needs, at AWS scale.
As a member of the Amazon MSK team, you will be making contributions to the entire stack - the APIs and the workflows that make up the MSK service, the core Kafka platform, and stand-alone tools that make it easier for Kafka community to operate Kafka better. Upstream compatibility is a core tenet of MSK. Your code changes to the Kafka platform will be released back to open source. As a member of an AWS service that builds on top of a popular open source technology, this is a unique opportunity to work on a team that straddles both worlds – open source and Amazon-internal software. You will design and build new features, make performance improvements, identify and investigate new technologies, prototype solutions, build scalable services, and test and review changes, to deliver an exceptional customer experience.
The ideal candidate has experience designing large-scale systems supporting millions of transactions per second, enjoys solving complex software problems, and possesses analytical, design and problem-solving skills. Ideally you have an in-depth understanding of streaming data technologies like Amazon Kinesis or Apache Kafka, and experience with open-source data processing frameworks like Apache Spark, Apache Flink, or Apache Storm. Your responsibilities will include collaborating with other engineers to build a large scale AWS service, and work with senior leaders to define your team's roadmap, including identifying design and code changes needed in the underlying open source platforms.
Learn more about Amazon MSK: - AWS re:Invent 2018: Amazon Managed Streaming for Kafka (Amazon MSK) Launch (https://www.youtube.com/watch?v=zhsVfsykBHc)
AWS re:Invent 2020: How Goldman Sachs uses an Amazon MSK backbone for Transaction Banking Platform (https://www.youtube.com/watch?v=MGaLNSLK3hA)
AWS re:Invent 2020: How New Relic is migrating its Apache Kafka cluster to Amazon MSK (https://www.youtube.com/watch?v=Bod2yn16TXM)
AWS re:Invent 2021: How Coinbase uses Amazon MSK as an event store for applications (https://www.youtube.com/watch?v=5yMzTwumD_g)
MSK Tiered Storage: Optimize cost and improve Kafka scalability (https://www.youtube.com/watch?v=0Ki2CBbN5bA)
AWS re:Invent 2023: How Rivian builds real-time analytics from electric vehicles (https://www.youtube.com/watch?v=io5w08-WKHI)
Utility Computing (UC)
AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for customers who require specialized security solutions for their cloud services.
About AWS
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Basic Qualifications
3+ years of non-internship professional software development experience
2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Experience programming with at least one software programming language
Preferred Qualifications
3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
Bachelor's degree in computer science or equivalent
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $129,300/year in our lowest geographic market up to $223,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.
Come build the future of data streaming with the Amazon Managed Streaming for Kafka (MSK) team!
We are seeking builders for our Amazon MSK service, a fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. We are looking for engineers who are enthusiastic about data streaming, and are as passionate about contributing to open source as they are about solving real customers' business needs, at AWS scale.
As a member of the Amazon MSK team, you will be making contributions to the entire stack - the APIs and the workflows that make up the MSK service, the core Kafka platform, and stand-alone tools that make it easier for Kafka community to operate Kafka better. Upstream compatibility is a core tenet of MSK. Your code changes to the Kafka platform will be released back to open source. As a member of an AWS service that builds on top of a popular open source technology, this is a unique opportunity to work on a team that straddles both worlds – open source and Amazon-internal software. You will design and build new features, make performance improvements, identify and investigate new technologies, prototype solutions, build scalable services, and test and review changes, to deliver an exceptional customer experience.
The ideal candidate has experience designing large-scale systems supporting millions of transactions per second, enjoys solving complex software problems, and possesses analytical, design and problem-solving skills. Ideally you have an in-depth understanding of streaming data technologies like Amazon Kinesis or Apache Kafka, and experience with open-source data processing frameworks like Apache Spark, Apache Flink, or Apache Storm. Your responsibilities will include collaborating with other engineers to build a large scale AWS service, and work with senior leaders to define your team's roadmap, including identifying design and code changes needed in the underlying open source platforms.
Learn more about Amazon MSK: - AWS re:Invent 2018: Amazon Managed Streaming for Kafka (Amazon MSK) Launch (https://www.youtube.com/watch?v=zhsVfsykBHc)
AWS re:Invent 2020: How Goldman Sachs uses an Amazon MSK backbone for Transaction Banking Platform (https://www.youtube.com/watch?v=MGaLNSLK3hA)
AWS re:Invent 2020: How New Relic is migrating its Apache Kafka cluster to Amazon MSK (https://www.youtube.com/watch?v=Bod2yn16TXM)
AWS re:Invent 2021: How Coinbase uses Amazon MSK as an event store for applications (https://www.youtube.com/watch?v=5yMzTwumD_g)
MSK Tiered Storage: Optimize cost and improve Kafka scalability (https://www.youtube.com/watch?v=0Ki2CBbN5bA)
AWS re:Invent 2023: How Rivian builds real-time analytics from electric vehicles (https://www.youtube.com/watch?v=io5w08-WKHI)
Utility Computing (UC)
AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for customers who require specialized security solutions for their cloud services.
About AWS
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Basic Qualifications
3+ years of non-internship professional software development experience
2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Experience programming with at least one software programming language
Preferred Qualifications
3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
Bachelor's degree in computer science or equivalent
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $129,300/year in our lowest geographic market up to $223,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.