Software Engineer (data platform)
Genmo Inc., San Francisco, CA, United States
We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the boundaries of what's possible in video generation.
Role overview:
We're seeking an experienced Senior/Staff AI Infra Engineer to design, build, and scale our petabyte-scale data infrastructure. You'll be responsible for creating robust, scalable systems that manage and process enormous datasets crucial for training our advanced AI models.
Genmo’s team is extremely technical with leaders in distributed systems, GPU programming and large-scale training. Ideal candidates will excel at working on highly technical problems and should have excellent problem solving skills, be detail-oriented and communicate clearly.
Key responsibilities:
- Design highly scalable data infrastructure and systems to process petabyte-scale data stores.
- Manage large-scale distributed processing jobs for ingesting and analyzing large-scale data sets for AI training.
- Optimize our storage systems to maximize performance.
- Build monitoring systems to ensure reliability of data infrastructure.
Qualifications:
- Bachelors, Masters or PhD in Computer Science or related field
- Must have:
- Extremely strong experience with Python
- Strong experience with a large-scale distributed computing frameworks (e.g., Spark, Ray)
- Experience with a systems-level language (Rust, Go, C++)
- Familiarity with a lakehouse format such as Delta Lake
- Past experience working on machine learning is a very strong plus
- Strong past experience working with Kubernetes and cloud environments (AWS, GCP, Azure)
- Past experience designing and then scaling a large-scale system from zero to one
Additional information
The role is based in the Bay Area (San Francisco). Candidates are expected to be located near the Bay Area or open to relocation.
Genmo is an Equal Opportunity Employer. Candidates are evaluated without regard to age, race, color, religion, sex, disability, national origin, sexual orientation, veteran status, or any other characteristic protected by federal or state law. Genmo, Inc. is an E-Verify company.
#J-18808-Ljbffr