Karkidi
Global IT QA Software Engineer Senior Manager – Gen AI/ LLM
Karkidi, Boston, Massachusetts, us, 02298
This is a GenAI Quality Engineer Senior Manager position within BCG's Quality Engineering Center of Expertise (QE CoE). The QE CoE resides within BCG’s Cloud & Platform Engineering (CPE) Portfolio. The role of CPE includes developing and supporting new and enhanced applications, as well as integrating and configuring third party software applications used by our Global HR, Finance, Knowledge, Marketing, Legal, etc. functions. The responsibilities of CPE include enabling efficient processes within the Global Functions; leading and implementing system change initiatives; and supporting the functional user community to optimize the management of the business.As a GenAI Quality Engineer Senior Manager, you will oversee the quality engineering processes for AI/ML/LLM models and systems. You will be responsible for developing and implementing testing frameworks to ensure the performance, accuracy, and reliability of generative AI solutions. Key duties include designing comprehensive test plans and utilizing advanced testing methodologies to identify and resolve defects.You will collaborate with cross-functional teams including data scientists, software engineers, and product owners to ensure that AI/ML/LLM models meet specified standards and user requirements. Additionally, you will monitor and analyze model performance, provide feedback for continuous improvement, and stay updated on the latest advancements in AI technology and quality engineering practices.This role requires strong technical expertise in AI/ML/LLMs, programming, and testing tools, along with leadership and project management skills to drive the quality engineering strategy and ensure the successful deployment of AI/ML/LLM solutions.YOU'RE GOOD AT
Developing comprehensive test plansImplementing advanced testing methodologiesIdentifying and resolving defectsCollaborating with cross-functional teamsEnsuring AI model performance and reliabilityAnalyzing model performance metricsProviding feedback for continuous improvementStaying updated on AI technology advancementsApplying strong technical expertise in AI/ML/LLMsUtilizing programming and testing toolsManaging quality engineering strategiesOverseeing successful AI/ML/LLM solution deploymentCommunicating effectively with stakeholdersProblem-solving and critical thinkingProject managementUnderstanding AI/ML/LLM model standards and requirementsConducting risk assessmentsEnhancing testing frameworksMaintaining high standards of AI/ML/LLM model accuracy and performanceYOU BRING (EXPERIENCE & QUALIFICATIONS)
Bachelor’s degree in computer science, Engineering, or a related fieldExtensive experience in quality engineering or software testingProven experience in testing Gen AI, LLMs, Artificial Intelligence, Machine Learning, or Natural Language Processing based systems for scale and robustness.Strong background in AI and machine learningProficiency in programming languages (e.g., Python, Java)Experience with AI/ML/LLM testing tools and frameworks e.g. DeepEval or equivalent.Solid understanding of AI model development and deploymentProven leadership and team management skillsExcellent problem-solving and analytical abilitiesStrong communication and collaboration skillsKnowledge of industry standards and best practices in AI/LLM quality engineeringExperience with performance and reliability testingYOU'LL WORK WITH BCG’s information technology group collaboratively delivers the latest digital technologies that enable our consultants to lead and our business to grow. For our IT jobs, we seek individuals with expertise in the areas of IT infrastructure, application development, business systems, collaborative and social technologies, information security, and project leadership.
#J-18808-Ljbffr
Developing comprehensive test plansImplementing advanced testing methodologiesIdentifying and resolving defectsCollaborating with cross-functional teamsEnsuring AI model performance and reliabilityAnalyzing model performance metricsProviding feedback for continuous improvementStaying updated on AI technology advancementsApplying strong technical expertise in AI/ML/LLMsUtilizing programming and testing toolsManaging quality engineering strategiesOverseeing successful AI/ML/LLM solution deploymentCommunicating effectively with stakeholdersProblem-solving and critical thinkingProject managementUnderstanding AI/ML/LLM model standards and requirementsConducting risk assessmentsEnhancing testing frameworksMaintaining high standards of AI/ML/LLM model accuracy and performanceYOU BRING (EXPERIENCE & QUALIFICATIONS)
Bachelor’s degree in computer science, Engineering, or a related fieldExtensive experience in quality engineering or software testingProven experience in testing Gen AI, LLMs, Artificial Intelligence, Machine Learning, or Natural Language Processing based systems for scale and robustness.Strong background in AI and machine learningProficiency in programming languages (e.g., Python, Java)Experience with AI/ML/LLM testing tools and frameworks e.g. DeepEval or equivalent.Solid understanding of AI model development and deploymentProven leadership and team management skillsExcellent problem-solving and analytical abilitiesStrong communication and collaboration skillsKnowledge of industry standards and best practices in AI/LLM quality engineeringExperience with performance and reliability testingYOU'LL WORK WITH BCG’s information technology group collaboratively delivers the latest digital technologies that enable our consultants to lead and our business to grow. For our IT jobs, we seek individuals with expertise in the areas of IT infrastructure, application development, business systems, collaborative and social technologies, information security, and project leadership.
#J-18808-Ljbffr