Backend & Infrastructure Tech Lead
Recruiting From Scratch, San Francisco, CA, United States
About Us:
Our product is doing for LLMs what Google did for websites. In the early internet, the number of websites was exploding and it was hard to figure out what website you should use for what task. Google fixed that problem by building a search engine that aggregated websites across the internet. A similar problem exists in AI today; the number of models is exploding and it's hard to figure out what model you should use for what task. Our product fixes that problem through a model router: You give us your prompt, we run it on the best model in real time.
We can do this because we've learned how to predict the performance of a model without running it. That lets us find a model which can complete your request with the highest performance and lowest cost. The value proposition is simple: stop worrying about AI, start focusing on product.That idea -- making it so that people can stop worrying about AI -- is the core of what we do. Model-routing is just the first tool we're building to help understand the way in which models behave. By pioneering techniques like this, we want to solve the most fundamental problem in AI: understanding why models behave the way they do, and creating guarantees they'll behave the way we want.
About the role:
We are seeking someone to manage the development of the API and backend infrastructure that clients will use to access the model router API and the microservices around it.
In this role, you will:
-
Design and build the dev and prod platforms for our router and our enterprise product
-
Create excellent tooling, systems, and engineering culture for both the backend and ML Engineering teams
-
Own and execute on features for our core routing product
-
Help create a sharp and open culture — we’re a startup, many things will break, and we can work together to fix them
-
Like all other teams, we are responsible for the reliability of the systems we build. This includes an on-call rotation to respond to critical incidents as needed.
Key Requirements:
-
DevOps, Infra, and K8s experience
-
You might thrive in this role if you:
-
Have 8+ years of engineering experience; 4+ years of infrastructure experience
-
Enjoy building and operating scalable, reliable, and secure systems
-
Have experience with: Kubernetes, Python, FastAPI, MongoDB, and Microservices Architectures.
- Competitive salary ($250k+) and equity packages
- Health, dental, and vision insurance plans
- Unlimited PTO
- Daily lunch
- Team dinners