Stack AV
Senior Software Engineer, Data Platform
Stack AV, Pittsburgh, Pennsylvania, us, 15289
About Stack
At Stack, we are focused on AI advancements across diverse technical domains, transforming how AI is applied in the physical realm. Our solutions are designed to navigate and understand the world with unprecedented safety, reliability, and efficiency, transforming how modern industries operate. Our decades of expertise spans diverse AI applications reflecting our commitment to exploring new frontiers and delivering excellence in AI technology.
About the Team:
The Data Platform team is part of the Platform Infrastructure Engineering and Reliability group, and is responsible for designing, implementing and maintaining the Stack AV on-premises data platform. The team supports large scale OLAP/OLTP and feature engineering workloads for multiple Product Development groups across the company. The team is tasked with building a highly-available and scalable data infrastructure to power our critical services and pipelines. Members of the team are expected to have a strong focus on requirements gathering and design process to ensure our platform meets the needs of the business and our internal customers.
What Success Looks Like:Data Platform engineering focus, with experience supporting large scale interactive and batch workloads. Experience with on-premises infrastructure, Kubernetes compute, S3 compatible object stores (Vast Data, Minio) managed via Terraform or ArgoCD.Expert level understanding of Data Engineering concepts and open source technologies, with ability to distill concepts and provide training to customers across the company.Experience running a modern data stack on-premises using open source technologies. Familiarity with streaming platforms (Kafka, Pulsar), processing engines (Trino, Apache Spark), table formats (ICEBERG, Delta Lake, Hudi, Xtable) and Postgres.Fluent in Python, and SQL, with experience writing and maintaining highly available data applications using Trino and Apache Spark.Strong focus on requirements gathering and overall design process. Experience architecting and optimizing large-scale systems to facilitate machine learning, simulation, and data processing exercises.Experience running and supporting workflow orchestration systems such as Argo Workflows, Airflow, or Flyte.io.Strong focus on providing production-grade service to internal engineering teams through observability, instrumentation, and well-defined SLOs.Experience working in a diverse and distributed team, with a focus on communication and customer empathy.Desire to work cross-functionally within Stack to meet customer use cases and deliver business value. #LI-SS1
We are proud to be an equal opportunity workplace. We believe that diverse teams produce the best ideas and outcomes. We are committed to building a culture of inclusion, entrepreneurship, and innovation across gender, race, age, sexual orientation, religion, disability, and identity.
Check out our Privacy Policy.
Please Note:
Pursuant to its business activities and use of technology, Stack AV complies with all applicable U.S. national security laws, regulations, and administrative requirements, which can restrict Stack AV's ability to employ certain persons in certain positions pursuant to a range of national security-related requirements. As such, this position may be contingent upon Stack AV verifying a candidate's residence, U.S. person status, and/or citizenship status. This position may also involve working with software and technologies subject to U.S. export control regulations. Under these regulations, it may be necessary for Stack AV to obtain a U.S. government export license prior to releasing its technologies to certain persons. If Stack AV determines that a candidate's residence, U.S. person status, and/or citizenship status will require a license, prohibit the candidate from working in this position, or otherwise be subject to national security-related restrictions, Stack AV expressly reserves the right to either consider the candidate for a different position that is not subject to such restrictions, on whatever terms and conditions Stack AV shall establish in its sole discretion, or, in the alternative, decline to move forward with the candidate's application.
At Stack, we are focused on AI advancements across diverse technical domains, transforming how AI is applied in the physical realm. Our solutions are designed to navigate and understand the world with unprecedented safety, reliability, and efficiency, transforming how modern industries operate. Our decades of expertise spans diverse AI applications reflecting our commitment to exploring new frontiers and delivering excellence in AI technology.
About the Team:
The Data Platform team is part of the Platform Infrastructure Engineering and Reliability group, and is responsible for designing, implementing and maintaining the Stack AV on-premises data platform. The team supports large scale OLAP/OLTP and feature engineering workloads for multiple Product Development groups across the company. The team is tasked with building a highly-available and scalable data infrastructure to power our critical services and pipelines. Members of the team are expected to have a strong focus on requirements gathering and design process to ensure our platform meets the needs of the business and our internal customers.
What Success Looks Like:Data Platform engineering focus, with experience supporting large scale interactive and batch workloads. Experience with on-premises infrastructure, Kubernetes compute, S3 compatible object stores (Vast Data, Minio) managed via Terraform or ArgoCD.Expert level understanding of Data Engineering concepts and open source technologies, with ability to distill concepts and provide training to customers across the company.Experience running a modern data stack on-premises using open source technologies. Familiarity with streaming platforms (Kafka, Pulsar), processing engines (Trino, Apache Spark), table formats (ICEBERG, Delta Lake, Hudi, Xtable) and Postgres.Fluent in Python, and SQL, with experience writing and maintaining highly available data applications using Trino and Apache Spark.Strong focus on requirements gathering and overall design process. Experience architecting and optimizing large-scale systems to facilitate machine learning, simulation, and data processing exercises.Experience running and supporting workflow orchestration systems such as Argo Workflows, Airflow, or Flyte.io.Strong focus on providing production-grade service to internal engineering teams through observability, instrumentation, and well-defined SLOs.Experience working in a diverse and distributed team, with a focus on communication and customer empathy.Desire to work cross-functionally within Stack to meet customer use cases and deliver business value. #LI-SS1
We are proud to be an equal opportunity workplace. We believe that diverse teams produce the best ideas and outcomes. We are committed to building a culture of inclusion, entrepreneurship, and innovation across gender, race, age, sexual orientation, religion, disability, and identity.
Check out our Privacy Policy.
Please Note:
Pursuant to its business activities and use of technology, Stack AV complies with all applicable U.S. national security laws, regulations, and administrative requirements, which can restrict Stack AV's ability to employ certain persons in certain positions pursuant to a range of national security-related requirements. As such, this position may be contingent upon Stack AV verifying a candidate's residence, U.S. person status, and/or citizenship status. This position may also involve working with software and technologies subject to U.S. export control regulations. Under these regulations, it may be necessary for Stack AV to obtain a U.S. government export license prior to releasing its technologies to certain persons. If Stack AV determines that a candidate's residence, U.S. person status, and/or citizenship status will require a license, prohibit the candidate from working in this position, or otherwise be subject to national security-related restrictions, Stack AV expressly reserves the right to either consider the candidate for a different position that is not subject to such restrictions, on whatever terms and conditions Stack AV shall establish in its sole discretion, or, in the alternative, decline to move forward with the candidate's application.