Amazon
Senior Data Engineer - Community Discovery ML
Amazon, San Francisco, California, United States, 94199
Senior Data Engineer - Community Discovery ML
If you are interested in this position, please apply on Twitch's Career site
here .
About Us:Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day. We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities.
About the Role:The Community Discovery ML team focuses on providing personalized, relevant experiences for Twitch users through Recommendation and Search. We are looking for a senior data engineer to join us. You will be the first data engineer hired in a hybrid team of ML engineers and scientists, working on data challenges related to ML models and products. You will extend, design, and build new capabilities in our data systems to ensure fast ML model development and productionization. You will impact cross teams by defining expectations for data usage patterns and data quality.
You will report to an Engineering Manager and work in San Francisco / Bay Area.
You Will:Oversee team data architecture to meet ML use cases in production.Design and build scalable data pipelines to support personalization models.Develop and maintain low-latency, large-scale streaming and batch data processing systems.Collaborate with applied scientists and ML engineers to integrate data into production models.Optimize data workflows for performance and cost efficiency.Implement best practices for data governance and security.Troubleshoot and resolve data-related issues, with a focus on identifying and solving data quality problems.Mentor others in the team in data related solutions and skills.
Minimum Requirements:6+ years of experience as a data engineer or in a similar role.Proficiency in SQL, Python, or Scala.Experience with building batch and streaming data pipelines with high throughput and low latency.Strong understanding of data architecture and data modeling principles.Experience analyzing large datasets to identify gaps and inconsistencies, provide data insights, and promote effective product solutions.Hands-on experience with cloud platforms (AWS, GCP, or Azure) and their data services.Familiarity with ETL tools and data warehousing solutions.Experience with distributed data processing technologies such as Apache Spark, Flink, and Kafka.Experience working with cross-functional roles like ML engineers and scientists.Experience with AWS data ecosystems like Redshift, Kinesis, and Glue.Understand data requirements for ML production systems.Extensive experience with mature and large-scale production data systems and capable of defining a strong North Star and making incremental progress towards that.
Perks:- Medical, Dental, Vision & Disability Insurance- 401(k)- Maternity & Parental Leave- Flexible PTO- Amazon Employee Discount
We are an equal opportunity employer and value diversity at Twitch. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status, or other legally protected status.
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Pursuant to the Los Angeles Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $139,100/year in our lowest geographic market up to $240,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit
here . This position will remain posted until filled. Applicants should apply via our internal or external career site.
#J-18808-Ljbffr
If you are interested in this position, please apply on Twitch's Career site
here .
About Us:Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day. We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities.
About the Role:The Community Discovery ML team focuses on providing personalized, relevant experiences for Twitch users through Recommendation and Search. We are looking for a senior data engineer to join us. You will be the first data engineer hired in a hybrid team of ML engineers and scientists, working on data challenges related to ML models and products. You will extend, design, and build new capabilities in our data systems to ensure fast ML model development and productionization. You will impact cross teams by defining expectations for data usage patterns and data quality.
You will report to an Engineering Manager and work in San Francisco / Bay Area.
You Will:Oversee team data architecture to meet ML use cases in production.Design and build scalable data pipelines to support personalization models.Develop and maintain low-latency, large-scale streaming and batch data processing systems.Collaborate with applied scientists and ML engineers to integrate data into production models.Optimize data workflows for performance and cost efficiency.Implement best practices for data governance and security.Troubleshoot and resolve data-related issues, with a focus on identifying and solving data quality problems.Mentor others in the team in data related solutions and skills.
Minimum Requirements:6+ years of experience as a data engineer or in a similar role.Proficiency in SQL, Python, or Scala.Experience with building batch and streaming data pipelines with high throughput and low latency.Strong understanding of data architecture and data modeling principles.Experience analyzing large datasets to identify gaps and inconsistencies, provide data insights, and promote effective product solutions.Hands-on experience with cloud platforms (AWS, GCP, or Azure) and their data services.Familiarity with ETL tools and data warehousing solutions.Experience with distributed data processing technologies such as Apache Spark, Flink, and Kafka.Experience working with cross-functional roles like ML engineers and scientists.Experience with AWS data ecosystems like Redshift, Kinesis, and Glue.Understand data requirements for ML production systems.Extensive experience with mature and large-scale production data systems and capable of defining a strong North Star and making incremental progress towards that.
Perks:- Medical, Dental, Vision & Disability Insurance- 401(k)- Maternity & Parental Leave- Flexible PTO- Amazon Employee Discount
We are an equal opportunity employer and value diversity at Twitch. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status, or other legally protected status.
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Pursuant to the Los Angeles Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $139,100/year in our lowest geographic market up to $240,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit
here . This position will remain posted until filled. Applicants should apply via our internal or external career site.
#J-18808-Ljbffr