Software Engineer, Data Platform

About Middesk Our goal is to make it easier for businesses to work together. Since 2018, we have been a leading disruptor in the business identity market. While consumer identity technology has seen significant modernization and digitization over the past decade, business identity and verification largely continues to be a manual process, specifically in the context of onboarding businesses to receive access to financial products and services. Our best-in-class business identity platform provides access to complete, up-to-date information that financial services institutions and fintechs need to make educated decisions about their customers and facilitate rapid onboarding and transacting. Our vision is that every company can instantly gain access to all the data, products, and services they need to establish and grow their business with ease. Middesk came out of Y Combinator, is backed by Sequoia Capital and Accel Partners, and was recently named to Forbes Fintech 50 List and cited as an industry leader in business verification by digital identity strategy firm, Liminal. About Middesk Engineering "Velocity" is the rate at which we effect realized value for our customers, not the rate at which we ship code. We believe that great products result when technical excellence and craft is combined with deep understanding of our customers and the problems we can help them solve. We're a humble, self-motivated team that seeks to ship early and ship often; we pride ourselves with being willing to roll up our sleeves to solve even the messiest problems our customers present to us. Middesk Engineering is customer-first engineering. The Role Middesk is, at its core, a data company. We live and die by the quality of our data asset and the engine that powers it. We're on a mission to build a comprehensive and complete business dataset for every business in the world. As part of the Foundation team at Middesk, you'll collaborate with Data Science, Infrastructure and Data Engineering to build and maintain our own proprietary Entity Resolution system used to power the Middesk business identity platform, scaling our system to resolve millions of business identities across hundreds of data sources and thousands of distinct data sets. You'll often work with and support product engineers looking to launch new products and features. The ideal candidate is ready to learn relentlessly, eager to learn a wide-breadth of technologies and software stacks, build and scale our data infrastructure and data pipelines and connect them into our core business verification products. What You'll Do: Build the batch and streaming data pipelines critical to Middesk's data infrastructure using Airflow, BigQuery, Dataflow, Dataproc and VertexAI. Build reliable, scalable, maintainable, and cost-efficient systems across the stack. Build, scale and extend existing web scraping platforms and capabilities used for real-time data acquisition. Build, update and maintain our interface layer between our application and data platform stack to improve performance and as we continue to scale our platform and datasets. What We're Looking For: 3 years of experience working in a data engineering or backend engineering role Strong programming skills in at least one backend language and web application framework Knowledge of SQL and experience with RDMS like postgresql, mySQL, etc. Nice to Haves: Fluency with Ruby, Python or Go Experience building and designing collections stored on ElasticSearch Experience working with Airflow or other orchestration platforms Experience working on data pipelines and data streaming with tools like Spark or Kafka Experience building machine learning systems Experience with Terraform, Datadog, or Kubernetes Experience with scraper technologies, including agentic AI