Autodesk
AI Research Dataset Creation & Annotation Manager
Autodesk, San Francisco, California, United States, 94199
Job Requisition ID #
24WD82502
Position Overview * *
The work we do at Autodesk touches nearly every person on the planet. By creating software tools for making buildings, machines, and even the latest movies, we influence and empower some of the most creative people in the world. As an AI Research Dataset Creation & Annotation Manager at Autodesk Research, you will work side-by-side with world-class researchers and engineers to build and label datasets to train new ML-powered product features to help our customers imagine, design, and make a better world.
You are passionate about solving problems and building things. You are excited to collaborate with researchers and engineers to bring the ML models they develop into Autodesk products.
You will report to the Senior Director of AI Research in Autodesk Research. We are a global team, located in London, San Francisco, Toronto, and remotely. For this role we support both in-person, hybrid, and remote work.
Responsibilities
Development of specialized datasetsto evaluate, fine-tune, and trainlarge 2D & 3D models for Design & Make
Oversee relationships with labeling vendors and contractors, managing data collection budgets and projects end-to-end
Drive data collection projects from start to finish by gathering requirements, defining success criteria, and adjusting tothe dynamic requirements of AI Research
Manage teams of AI Data Research Scientists, engineers, and labelling contractorsto collect, review, and deliver high-quality data
Apply a high degree of operational rigor and reporting to be excellent cross-functional partners to our legal, finance, and Trusted AI teams
Achieve the highest quality data possible, while doing so in an increasingly efficient manner
Design and write clear labeling and annotation guidelines for human labeling tasks
Spearhead efforts to develop tooling and process improvements, optimizing for quality, throughput, and usability of datasets for AI training
Partner with cross-functional teams to proactively identify data requirements of AI Research projects and products
Are an expert in managing data labeling operations, particularly for dynamic and nondeterministic labels
Are an exceptionally clear communicator who can anticipate the needs of our vendors, researchers, and internal and external partners
Understand the intricacies of collecting data from human annotators/ labelers for training or evaluating AI models
Enjoy problem-solving, both technically and creatively, using your technical and non-technical skills
Enjoy getting into the weeds of data and forming bottom-up assessments on edge cases and quality deficiencies
Are comfortable using a combination of internal & external tools and exhibit exceptional judgment in building repeatable processes
Are deeply adaptable to projects in varying domains and complexities, and effectively reason about systems and processes for domains even beyond your core expertise
Have an action-oriented and deeply curious mind
Are interested in developing new processes and finding new efficiencies by combining VLMs and other automatic labeling techniques with human labeling
Can be an effective thought partner to our researchers, collaborating on how best to achieve their goals
Thrive in dynamic environments and can pivotquickly when faced with changing research requirements or edge cases
Minimum Qualifications
3+ years of experience managing data collection projectsfor industry-scale AI model training, fine-tuning, and evaluation
Experience creating & curating large-scale datasets (10^6+ data points) and solving associated technical challenges
Experience in managing cross team data curation efforts, coordinating/managing internal tool-chains and software instrumentations for annotation the occur
Experience in managing human resources for the annotation efforts (either in-house or 3rd party)
Experience withmanaging budget allocationsfor big-scale annotations
Strong AI Research background with data-efficient ML experience
Experience managing data labeling operations
Familiarity with tools like AWS, SQL, parallel computing, Apache Spark, or similar for data management
Experimental design with human subjects experience
Excellent communication skills and the ability to collaborate with diverse teams
Nice to have: Experience with models fine-tuned on human data (e.g. RLHF)
Nice to have: The ability to debug and contribute to our annotation tools when appropriate, though this will not be themain focus of the role
Nice to have: Knowledge of the design, manufacturing, AEC, or media & entertainment industries
Nice to have: Experience with Autodesk or similar products (CAD, CAE, CAM, etc.), and related datatypes
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.
When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. For U.S.-based roles, we expect a starting base salary between $176,800 and $286,000. Offers are based on the candidate’s experience and geographic location, and may exceed this range. In addition to base salaries, we also have a significant emphasis on annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.
Equal Employment Opportunity
At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.
Diversity & Belonging
We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).
24WD82502
Position Overview * *
The work we do at Autodesk touches nearly every person on the planet. By creating software tools for making buildings, machines, and even the latest movies, we influence and empower some of the most creative people in the world. As an AI Research Dataset Creation & Annotation Manager at Autodesk Research, you will work side-by-side with world-class researchers and engineers to build and label datasets to train new ML-powered product features to help our customers imagine, design, and make a better world.
You are passionate about solving problems and building things. You are excited to collaborate with researchers and engineers to bring the ML models they develop into Autodesk products.
You will report to the Senior Director of AI Research in Autodesk Research. We are a global team, located in London, San Francisco, Toronto, and remotely. For this role we support both in-person, hybrid, and remote work.
Responsibilities
Development of specialized datasetsto evaluate, fine-tune, and trainlarge 2D & 3D models for Design & Make
Oversee relationships with labeling vendors and contractors, managing data collection budgets and projects end-to-end
Drive data collection projects from start to finish by gathering requirements, defining success criteria, and adjusting tothe dynamic requirements of AI Research
Manage teams of AI Data Research Scientists, engineers, and labelling contractorsto collect, review, and deliver high-quality data
Apply a high degree of operational rigor and reporting to be excellent cross-functional partners to our legal, finance, and Trusted AI teams
Achieve the highest quality data possible, while doing so in an increasingly efficient manner
Design and write clear labeling and annotation guidelines for human labeling tasks
Spearhead efforts to develop tooling and process improvements, optimizing for quality, throughput, and usability of datasets for AI training
Partner with cross-functional teams to proactively identify data requirements of AI Research projects and products
Are an expert in managing data labeling operations, particularly for dynamic and nondeterministic labels
Are an exceptionally clear communicator who can anticipate the needs of our vendors, researchers, and internal and external partners
Understand the intricacies of collecting data from human annotators/ labelers for training or evaluating AI models
Enjoy problem-solving, both technically and creatively, using your technical and non-technical skills
Enjoy getting into the weeds of data and forming bottom-up assessments on edge cases and quality deficiencies
Are comfortable using a combination of internal & external tools and exhibit exceptional judgment in building repeatable processes
Are deeply adaptable to projects in varying domains and complexities, and effectively reason about systems and processes for domains even beyond your core expertise
Have an action-oriented and deeply curious mind
Are interested in developing new processes and finding new efficiencies by combining VLMs and other automatic labeling techniques with human labeling
Can be an effective thought partner to our researchers, collaborating on how best to achieve their goals
Thrive in dynamic environments and can pivotquickly when faced with changing research requirements or edge cases
Minimum Qualifications
3+ years of experience managing data collection projectsfor industry-scale AI model training, fine-tuning, and evaluation
Experience creating & curating large-scale datasets (10^6+ data points) and solving associated technical challenges
Experience in managing cross team data curation efforts, coordinating/managing internal tool-chains and software instrumentations for annotation the occur
Experience in managing human resources for the annotation efforts (either in-house or 3rd party)
Experience withmanaging budget allocationsfor big-scale annotations
Strong AI Research background with data-efficient ML experience
Experience managing data labeling operations
Familiarity with tools like AWS, SQL, parallel computing, Apache Spark, or similar for data management
Experimental design with human subjects experience
Excellent communication skills and the ability to collaborate with diverse teams
Nice to have: Experience with models fine-tuned on human data (e.g. RLHF)
Nice to have: The ability to debug and contribute to our annotation tools when appropriate, though this will not be themain focus of the role
Nice to have: Knowledge of the design, manufacturing, AEC, or media & entertainment industries
Nice to have: Experience with Autodesk or similar products (CAD, CAE, CAM, etc.), and related datatypes
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.
When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. For U.S.-based roles, we expect a starting base salary between $176,800 and $286,000. Offers are based on the candidate’s experience and geographic location, and may exceed this range. In addition to base salaries, we also have a significant emphasis on annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.
Equal Employment Opportunity
At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.
Diversity & Belonging
We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).