Amazon
Applied Scientist, Conversational Assistant Services
Amazon, Boston, Massachusetts, us, 02298
Applied Scientist, Conversational Assistant Services
Job ID: 2742001 | Amazon.com Services LLCAs part of the Alexa CAS team, our mission is to provide scalable and reliable evaluation of state-of-the-art Conversational AI. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), and Natural Language Processing (NLP), to invent and build end-to-end evaluation of how customers perceive context-aware conversational AI assistants.
A successful candidate will have a strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experience in building Generative AI solutions with LLMs, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc.
As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel methods for evaluating conversational assistants. You will analyze and understand user experiences by leveraging Amazon’s heterogeneous data sources and build evaluation models using machine learning methods.
Key job responsibilities
Design, build, test and release predictive ML models using LLMsEnsure data quality throughout all stages of acquisition and processing, including data sourcing/collection, ground truth generation, normalization, and transformation.Collaborate with colleagues from science, engineering, and business backgrounds.Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions.Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use cases.
About the teamCentral Analytics and Research Science (CARS) is an analytics, software, and science team within Amazon's Conversational Assistant Services (CAS) organization. Our mission is to provide an end-to-end understanding of how customers perceive the assistants they interact with – from the metrics themselves to software applications to deep dives on those metrics – allowing assistant developers to improve their services.
BASIC QUALIFICATIONS
PhD, or Master's degree and 4+ years of CS, CE, ML or related field experienceExperience building machine learning models or developing algorithms for business applicationsExperience in patents or publications at top-tier peer-reviewed conferences or journalsExperience with programming languages such as Python, Java, C++Hands-on experience and deep understanding of the Large Language Models architecturesPREFERRED QUALIFICATIONS
Research experience in conversational assistant or LLM evaluationPublications at peer-reviewed NLP/ML conferences (e.g. ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, etc.)Hands-on experience with using RLHF models
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.
Posted:
August 23, 2024
#J-18808-Ljbffr
Job ID: 2742001 | Amazon.com Services LLCAs part of the Alexa CAS team, our mission is to provide scalable and reliable evaluation of state-of-the-art Conversational AI. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), and Natural Language Processing (NLP), to invent and build end-to-end evaluation of how customers perceive context-aware conversational AI assistants.
A successful candidate will have a strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experience in building Generative AI solutions with LLMs, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc.
As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel methods for evaluating conversational assistants. You will analyze and understand user experiences by leveraging Amazon’s heterogeneous data sources and build evaluation models using machine learning methods.
Key job responsibilities
Design, build, test and release predictive ML models using LLMsEnsure data quality throughout all stages of acquisition and processing, including data sourcing/collection, ground truth generation, normalization, and transformation.Collaborate with colleagues from science, engineering, and business backgrounds.Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions.Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use cases.
About the teamCentral Analytics and Research Science (CARS) is an analytics, software, and science team within Amazon's Conversational Assistant Services (CAS) organization. Our mission is to provide an end-to-end understanding of how customers perceive the assistants they interact with – from the metrics themselves to software applications to deep dives on those metrics – allowing assistant developers to improve their services.
BASIC QUALIFICATIONS
PhD, or Master's degree and 4+ years of CS, CE, ML or related field experienceExperience building machine learning models or developing algorithms for business applicationsExperience in patents or publications at top-tier peer-reviewed conferences or journalsExperience with programming languages such as Python, Java, C++Hands-on experience and deep understanding of the Large Language Models architecturesPREFERRED QUALIFICATIONS
Research experience in conversational assistant or LLM evaluationPublications at peer-reviewed NLP/ML conferences (e.g. ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, etc.)Hands-on experience with using RLHF models
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.
Posted:
August 23, 2024
#J-18808-Ljbffr