Salesforce.Com Inc
Director - LLM Operations
Salesforce.Com Inc, Bellevue, Washington, us, 98009
To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.Job CategoryOperations
Job DetailsAbout SalesforceWe're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good - you've come to the right place.We're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good - you've come to the right place.
As the Director of LLM Ops for the Digital Customer Success AI Team, you will be accountable for developing, implementing and strategizing how the DCS AI team evaluates generative response accuracy, and drive improvement efforts across multiple teams, organizations and technologies including content and data improvement, RAG and LLM Model Improvements, and new technologies for evaluating Generative AI Responses.
This role will manage a team of program managers who are accountable for different aspects of the designed strategy. The Director of LLM Ops will also be expected to represent the team's work in monthly and quarterly business reviews against strategic KPIs and organizational business outcomes. In addition, this role is expected to manage roadmaps, intake, and prioritization of initiatives and programs designed to increase Generative AI answer accuracy and quality within Salesforce's customer facing Generative AI solutions.
Your ImpactLead a team of high performing Program Managers with the goal of increasing Generative AI answer quality.
Develop and implement testing and evaluation strategies across multiple teams and organization.
Influence the testing and evaluations framework for all of Salesforce Customer Success Group and other stakeholder organizations.
Maintain and enhance production relationships and influence team roadmaps where needed.
Maintain and develop an Evaluations Roadmap including schedule of resources and delivery, and project prioritization against available capacity.
Support and enhance the evaluation framework and model for both proactive model testing and improvements, as well as reactive customer session evaluations.
Build and support data storytelling motions that help provide clarity to the Evaluations Program and it's impact on Salesforce business outcomes.
Required SkillsProven experience with Generative AI Technology and an understanding of Generative AI functionality principals.
Proven experience with program or product management functions.
Proven experience leading a high performing team and delivering business outcomes.
Executive presence, and experience telling highly complex technology and data stories to a wide range of audiences at different technical levels.
Preferred: Experience and knowledge of Data Cloud or Data Grounding for Generative AI.
Preferred: Experience managing content repositories or knowledge bases.
Preferred: Experience in Prompt Engineering and Implementation
Our Investment In You:World-class enablement and on-demand training - check out Trailhead.com for a sneak peek!
Exposure to executive thought leaders with a passion for living our values
Clear path to promotion with accelerated leadership development programs
Weekly 1:1 coaching with your leadership
Fast Ramp mentorship program
Week-long product bootcamp
Sandler Sales Training
Volunteer OpportunitiesHave you heard of our 1:1:1 model, focused on giving back to the community? The success in giving back have helped us and our Trailblazer community become a leader in philanthropy, culture, and innovation. We are one of the World's Most Innovative Companies, according to
Forbes , we are #1 in
PEOPLE 's Top 15 Companies that Care, and are on Fortune's Change the World list. Values create value. Our values helped drive our revenue number to $13.28 billion dollars in fiscal years in 2019.We have a public-facing website that explains our various benefits for:Health benefits
Financial benefits and perks
Time off and leave policies
Parental benefits
Perks and discounts
Visit https://www.salesforcebenefits.com/ for the full breakdown.
AccommodationsIf you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form.Posting StatementAt Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com.Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce.Salesforce welcomes all.For Washington-based roles, the base salary hiring range for this position is $159,000 to $253,600. Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for incentive compensation, equity, benefits. More details about our company benefits can be found at the following link: https://www.salesforcebenefits.com.
Job DetailsAbout SalesforceWe're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good - you've come to the right place.We're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good - you've come to the right place.
As the Director of LLM Ops for the Digital Customer Success AI Team, you will be accountable for developing, implementing and strategizing how the DCS AI team evaluates generative response accuracy, and drive improvement efforts across multiple teams, organizations and technologies including content and data improvement, RAG and LLM Model Improvements, and new technologies for evaluating Generative AI Responses.
This role will manage a team of program managers who are accountable for different aspects of the designed strategy. The Director of LLM Ops will also be expected to represent the team's work in monthly and quarterly business reviews against strategic KPIs and organizational business outcomes. In addition, this role is expected to manage roadmaps, intake, and prioritization of initiatives and programs designed to increase Generative AI answer accuracy and quality within Salesforce's customer facing Generative AI solutions.
Your ImpactLead a team of high performing Program Managers with the goal of increasing Generative AI answer quality.
Develop and implement testing and evaluation strategies across multiple teams and organization.
Influence the testing and evaluations framework for all of Salesforce Customer Success Group and other stakeholder organizations.
Maintain and enhance production relationships and influence team roadmaps where needed.
Maintain and develop an Evaluations Roadmap including schedule of resources and delivery, and project prioritization against available capacity.
Support and enhance the evaluation framework and model for both proactive model testing and improvements, as well as reactive customer session evaluations.
Build and support data storytelling motions that help provide clarity to the Evaluations Program and it's impact on Salesforce business outcomes.
Required SkillsProven experience with Generative AI Technology and an understanding of Generative AI functionality principals.
Proven experience with program or product management functions.
Proven experience leading a high performing team and delivering business outcomes.
Executive presence, and experience telling highly complex technology and data stories to a wide range of audiences at different technical levels.
Preferred: Experience and knowledge of Data Cloud or Data Grounding for Generative AI.
Preferred: Experience managing content repositories or knowledge bases.
Preferred: Experience in Prompt Engineering and Implementation
Our Investment In You:World-class enablement and on-demand training - check out Trailhead.com for a sneak peek!
Exposure to executive thought leaders with a passion for living our values
Clear path to promotion with accelerated leadership development programs
Weekly 1:1 coaching with your leadership
Fast Ramp mentorship program
Week-long product bootcamp
Sandler Sales Training
Volunteer OpportunitiesHave you heard of our 1:1:1 model, focused on giving back to the community? The success in giving back have helped us and our Trailblazer community become a leader in philanthropy, culture, and innovation. We are one of the World's Most Innovative Companies, according to
Forbes , we are #1 in
PEOPLE 's Top 15 Companies that Care, and are on Fortune's Change the World list. Values create value. Our values helped drive our revenue number to $13.28 billion dollars in fiscal years in 2019.We have a public-facing website that explains our various benefits for:Health benefits
Financial benefits and perks
Time off and leave policies
Parental benefits
Perks and discounts
Visit https://www.salesforcebenefits.com/ for the full breakdown.
AccommodationsIf you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form.Posting StatementAt Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com.Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce.Salesforce welcomes all.For Washington-based roles, the base salary hiring range for this position is $159,000 to $253,600. Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for incentive compensation, equity, benefits. More details about our company benefits can be found at the following link: https://www.salesforcebenefits.com.