Salesforce, Inc.
Software Engineering Architect
Salesforce, Inc., Bellevue, Washington, us, 98009
We are seeking an experienced Salesforce Software Quality Architect with a strong background in SaaS platforms and expertise in evaluating Large Language Models (LLMs). This role requires a deep understanding of Salesforce architecture, software quality assurance, and the ability to assess and optimize LLMs for delivering accurate, safe, and high-quality responses.
As a Quality Architect, you will play a key role in defining, implementing, and maintaining the quality strategy for Salesforce products, with a focus on integrating cutting-edge AI technologies. You will collaborate with cross-functional teams, including product development, data science, and AI teams, to ensure that Salesforce’s AI-driven solutions meet the highest quality standards for enterprise applications.Key Responsibilities:Define Quality Strategies : Develop and implement robust quality strategies for Salesforce’s SaaS products, focusing on AI and LLM integrations.Evaluate LLM Performance : Assess and optimize Large Language Models (LLMs) for quality, accuracy, relevance, and safety in various use cases.Testing and Automation : Design test plans and automation frameworks for both traditional software components and LLM-based features, ensuring seamless integration and high performance.Quality Metrics : Establish and track quality metrics, including accuracy, relevance, engagement, and safety of LLM-generated responses.Collaborate on AI Model Improvements : Work closely with data scientists and machine learning engineers to provide feedback on LLM performance and recommend improvements based on quality testing results.SaaS Expertise : Apply your knowledge of Salesforce architecture and SaaS platforms to guarantee high availability, scalability, and user satisfaction in AI-powered applications.Human Evaluation : Oversee human evaluation processes for subjective quality dimensions, such as response engagement, user safety, and contextual accuracy.Drive Innovation & Adoption : Bring structure to the engineering decision-making process by communicating and evaluating solution options, and facilitating agreement among key stakeholders that helps prioritize high-value solutions, driving business impact.Multiplier - Connect the “Art of the Possible” : Provide technical leadership for critical areas that significantly impact customer success. You will bring new best practices to R&D and actively ensure that they are being used.Cross-Platform Collaboration : Use your understanding of customers’ needs across industries and multiple technology landscapes (CRM, Modern Data Stack, Analytics & BI, CRM and AI) to develop solutions across Salesforce's technology stack.Technical Leadership on Proof of Concepts : Influence the direction of R&D as a whole through your technical, process, or product knowledge leadership.Mentor and Organization Builder : Be a cornerstone in the infrastructure of technical expertise represented by the organization's senior engineers, challenging and engaging them to develop their expertise and leadership contributions.Required Skills and Qualifications:Experience in Salesforce : Understanding of Salesforce’s architecture, platform features, and integration with AI/ML components.SaaS Expertise : Extensive experience with SaaS platforms, cloud architecture, and enterprise-level software quality assurance.LLM Expertise : Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks.Quality Assurance Leadership : Experience leading quality assurance initiatives for complex software systems, including test automation and performance testing.AI & Machine Learning Knowledge : Strong understanding of AI/ML principles, particularly in evaluating and optimizing model performance, response consistency, and safety.Testing Tools & Frameworks : Familiarity with tools like Selenium, TestNG, JUnit, and other automation frameworks, plus experience with AI/ML testing platforms.Collaboration & Communication : Excellent communication skills, with the ability to work cross-functionally with product managers, AI/ML engineers, and data scientists.Problem-Solving & Analytical Skills : Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement.Preferred Qualifications:Experience in ethical AI practices and evaluating models for fairness, bias, and user safety.Knowledge of tools for model interpretability and explainability (e.g., LIME, SHAP).Strong understanding of public cloud infrastructure - AWS/Azure/GCP.Certification in Salesforce.Ability to architect, design, implement, test, and deliver test frameworks for highly scalable products.
#J-18808-Ljbffr
As a Quality Architect, you will play a key role in defining, implementing, and maintaining the quality strategy for Salesforce products, with a focus on integrating cutting-edge AI technologies. You will collaborate with cross-functional teams, including product development, data science, and AI teams, to ensure that Salesforce’s AI-driven solutions meet the highest quality standards for enterprise applications.Key Responsibilities:Define Quality Strategies : Develop and implement robust quality strategies for Salesforce’s SaaS products, focusing on AI and LLM integrations.Evaluate LLM Performance : Assess and optimize Large Language Models (LLMs) for quality, accuracy, relevance, and safety in various use cases.Testing and Automation : Design test plans and automation frameworks for both traditional software components and LLM-based features, ensuring seamless integration and high performance.Quality Metrics : Establish and track quality metrics, including accuracy, relevance, engagement, and safety of LLM-generated responses.Collaborate on AI Model Improvements : Work closely with data scientists and machine learning engineers to provide feedback on LLM performance and recommend improvements based on quality testing results.SaaS Expertise : Apply your knowledge of Salesforce architecture and SaaS platforms to guarantee high availability, scalability, and user satisfaction in AI-powered applications.Human Evaluation : Oversee human evaluation processes for subjective quality dimensions, such as response engagement, user safety, and contextual accuracy.Drive Innovation & Adoption : Bring structure to the engineering decision-making process by communicating and evaluating solution options, and facilitating agreement among key stakeholders that helps prioritize high-value solutions, driving business impact.Multiplier - Connect the “Art of the Possible” : Provide technical leadership for critical areas that significantly impact customer success. You will bring new best practices to R&D and actively ensure that they are being used.Cross-Platform Collaboration : Use your understanding of customers’ needs across industries and multiple technology landscapes (CRM, Modern Data Stack, Analytics & BI, CRM and AI) to develop solutions across Salesforce's technology stack.Technical Leadership on Proof of Concepts : Influence the direction of R&D as a whole through your technical, process, or product knowledge leadership.Mentor and Organization Builder : Be a cornerstone in the infrastructure of technical expertise represented by the organization's senior engineers, challenging and engaging them to develop their expertise and leadership contributions.Required Skills and Qualifications:Experience in Salesforce : Understanding of Salesforce’s architecture, platform features, and integration with AI/ML components.SaaS Expertise : Extensive experience with SaaS platforms, cloud architecture, and enterprise-level software quality assurance.LLM Expertise : Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks.Quality Assurance Leadership : Experience leading quality assurance initiatives for complex software systems, including test automation and performance testing.AI & Machine Learning Knowledge : Strong understanding of AI/ML principles, particularly in evaluating and optimizing model performance, response consistency, and safety.Testing Tools & Frameworks : Familiarity with tools like Selenium, TestNG, JUnit, and other automation frameworks, plus experience with AI/ML testing platforms.Collaboration & Communication : Excellent communication skills, with the ability to work cross-functionally with product managers, AI/ML engineers, and data scientists.Problem-Solving & Analytical Skills : Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement.Preferred Qualifications:Experience in ethical AI practices and evaluating models for fairness, bias, and user safety.Knowledge of tools for model interpretability and explainability (e.g., LIME, SHAP).Strong understanding of public cloud infrastructure - AWS/Azure/GCP.Certification in Salesforce.Ability to architect, design, implement, test, and deliver test frameworks for highly scalable products.
#J-18808-Ljbffr