Stanford University
Senior Data Engineer
Stanford University, Stanford, California, United States, 94305
Senior Data Engineer
School of Medicine, Stanford, California, United States
Academic
Post Date Aug 26, 2024
Requisition # 104382
Stanford University is launching an interdisciplinary Neuro-AI project dedicated to building a foundation model of the brain. This endeavor will involve multiple labs and faculty across the Stanford campus, including the Wu Tsai Neurosciences Institute, Stanford Bio-X, and the Human-Centered Artificial Intelligence Institute. Leveraging cutting-edge advances in electrophysiology and machine learning, this project aims to create a functional "digital twin" — a model that captures both the activity dynamics of the brain at cellular resolution and the intelligent behavior it generates, including perception, motor planning, learning, reasoning, and problem-solving.
This ambitious initiative promises to offer unprecedented insights into the brain's algorithms of perception and cognition while serving as a key resource for aligning artificial intelligence models with human-like neural representations. As part of this project, we are seeking a talented senior data engineer with extensive experience in data infrastructure engineering to lead a team of engineers in building robust data pipelines. The team is responsible for designing, building, and operating the data pipeline infrastructure, which includes the entire flow of data from neurophysiological data acquisition to storage, processing, and preparation for large-scale training of machine learning-based foundation models. Ideal candidates will have practical experience in designing and scaling big data pipelines and proficiency with tools and frameworks such as Apache Spark, Airflow, DeltaLake, or similar technologies.
This position promises a vibrant and cooperative atmosphere within the laboratories of Andreas Tolias (https://toliaslab.org), Tirin Moore (https://www.moorelabstanford.com) and other labs at Stanford University renowned for their expertise in perception, cognition, pioneering neural recording techniques, computational neuroscience, machine learning, and Neuro-AI research.
Role & Responsibilities:
•Lead a team of engineers to design, build, and maintain high-throughput data pipelines.
•Set up and maintain the hardware and software infrastructure to support distributed computing, data orchestration, and distributed data storage.
•Tightly coordinate with experimentalists, research scientists, and machine learning engineers to accelerate and facilitate the workflows for large-scale neuroscientific data analyses and foundation model training.
Key qualifications:
•PhD or Master’s degree in Computer Science or related fields.
•At least 5 years of experience in designing or running big data pipelines with a particular focus on data infrastructure engineering.
•Detailed knowledge and experience in working with state-of-the-art big data tools and frameworks (e.g. Apache Spark, Airflow, Delta Lake, or similar).
•Strong expertise in setting up and managing large-scale data and compute infrastructure to support high-throughput data processing.
•Strong software engineering background for ensuring high-quality code and continuous development of data analysis pipelines in coordination with other teams.
•Excellent communication skills to work effectively within an interdisciplinary team constituting varying degrees of technical skills
Preferred qualifications:
•Experience with machine learning techniques and their associated challenges for data pipeline engineering
•Experience in leading a team in managing software and/or hardware infrastructure for data storage and analyses.
What we offer:
•Work on a collaborative and uniquely positioned project spanning several disciplines, from neuroscience to artificial intelligence and engineering.
•Work jointly with a vibrant team of researchers and scientists in a project dedicated to one mission, rooted in academia but inspired by science in industry.
•Competitive salary and benefits.
•Strong mentoring in career development.
Please complete the basic application through Stanford Careers, and we request that you also send your CV and one page interest statement to: recruiting@enigmaproject.ai
The expected pay range for this position is $132,000 to $165,000 per annum.
Stanford University has provided a pay range representing its good faith estimate of what the university reasonably expects to pay for the position. The pay offered to the selected candidate will be determined based on factors including (but not limited to) the experience and qualifications of the selected candidate including equivalent years since their applicable education, field or discipline; departmental budget availability; internal equity; among other factors.
Additional Information
Schedule: Full-time
Job Code: 6446
Employee Status: Fixed-Term
Grade: R99
Requisition ID: 104382
Work Arrangement : Hybrid Eligible, On Site
School of Medicine, Stanford, California, United States
Academic
Post Date Aug 26, 2024
Requisition # 104382
Stanford University is launching an interdisciplinary Neuro-AI project dedicated to building a foundation model of the brain. This endeavor will involve multiple labs and faculty across the Stanford campus, including the Wu Tsai Neurosciences Institute, Stanford Bio-X, and the Human-Centered Artificial Intelligence Institute. Leveraging cutting-edge advances in electrophysiology and machine learning, this project aims to create a functional "digital twin" — a model that captures both the activity dynamics of the brain at cellular resolution and the intelligent behavior it generates, including perception, motor planning, learning, reasoning, and problem-solving.
This ambitious initiative promises to offer unprecedented insights into the brain's algorithms of perception and cognition while serving as a key resource for aligning artificial intelligence models with human-like neural representations. As part of this project, we are seeking a talented senior data engineer with extensive experience in data infrastructure engineering to lead a team of engineers in building robust data pipelines. The team is responsible for designing, building, and operating the data pipeline infrastructure, which includes the entire flow of data from neurophysiological data acquisition to storage, processing, and preparation for large-scale training of machine learning-based foundation models. Ideal candidates will have practical experience in designing and scaling big data pipelines and proficiency with tools and frameworks such as Apache Spark, Airflow, DeltaLake, or similar technologies.
This position promises a vibrant and cooperative atmosphere within the laboratories of Andreas Tolias (https://toliaslab.org), Tirin Moore (https://www.moorelabstanford.com) and other labs at Stanford University renowned for their expertise in perception, cognition, pioneering neural recording techniques, computational neuroscience, machine learning, and Neuro-AI research.
Role & Responsibilities:
•Lead a team of engineers to design, build, and maintain high-throughput data pipelines.
•Set up and maintain the hardware and software infrastructure to support distributed computing, data orchestration, and distributed data storage.
•Tightly coordinate with experimentalists, research scientists, and machine learning engineers to accelerate and facilitate the workflows for large-scale neuroscientific data analyses and foundation model training.
Key qualifications:
•PhD or Master’s degree in Computer Science or related fields.
•At least 5 years of experience in designing or running big data pipelines with a particular focus on data infrastructure engineering.
•Detailed knowledge and experience in working with state-of-the-art big data tools and frameworks (e.g. Apache Spark, Airflow, Delta Lake, or similar).
•Strong expertise in setting up and managing large-scale data and compute infrastructure to support high-throughput data processing.
•Strong software engineering background for ensuring high-quality code and continuous development of data analysis pipelines in coordination with other teams.
•Excellent communication skills to work effectively within an interdisciplinary team constituting varying degrees of technical skills
Preferred qualifications:
•Experience with machine learning techniques and their associated challenges for data pipeline engineering
•Experience in leading a team in managing software and/or hardware infrastructure for data storage and analyses.
What we offer:
•Work on a collaborative and uniquely positioned project spanning several disciplines, from neuroscience to artificial intelligence and engineering.
•Work jointly with a vibrant team of researchers and scientists in a project dedicated to one mission, rooted in academia but inspired by science in industry.
•Competitive salary and benefits.
•Strong mentoring in career development.
Please complete the basic application through Stanford Careers, and we request that you also send your CV and one page interest statement to: recruiting@enigmaproject.ai
The expected pay range for this position is $132,000 to $165,000 per annum.
Stanford University has provided a pay range representing its good faith estimate of what the university reasonably expects to pay for the position. The pay offered to the selected candidate will be determined based on factors including (but not limited to) the experience and qualifications of the selected candidate including equivalent years since their applicable education, field or discipline; departmental budget availability; internal equity; among other factors.
Additional Information
Schedule: Full-time
Job Code: 6446
Employee Status: Fixed-Term
Grade: R99
Requisition ID: 104382
Work Arrangement : Hybrid Eligible, On Site