Aitopics
Senior Data Engineer - Compute and Storage Infrastructure
Aitopics, Sunnyvale, California, United States, 94087
Senior Data Engineer - Compute and Storage Infrastructure
CoreWeave
powers the creation and delivery of intelligence that drives innovation. CoreWeave is the AI Hyperscaler, delivering a cloud platform of cutting-edge services powering the next wave of AI. The company’s technology provides enterprises and leading AI labs with the most performant, efficient, and resilient solutions for accelerated computing. Since 2017, CoreWeave has operated a growing footprint of data centers covering every region of the US and across Europe. CoreWeave was ranked as one of the TIME100 most influential companies of 2024.As the leader in the industry, we thrive in an environment where adaptability and resilience are key. Our culture offers career-defining opportunities for those who excel amid change and challenge. If you’re someone who thrives in a dynamic environment, enjoys solving complex problems, and is eager to make a significant impact, CoreWeave is the place for you. Join us, and be part of a team solving some of the most exciting challenges in the industry.About the TeamThe Data Engineering team builds foundational datasets and analytics services that enable BI and data science across CoreWeave. We seek to democratize insights and foster a culture where data-driven decision-making thrives at every level.About the RoleWe’re looking for a seasoned Senior Data Engineer with an SRE’s skillset to build and scale foundational data storage and processing infrastructure that will power BI and data science initiatives across CoreWeave. This engineer will take ownership of our data lake and clustered computing services, creating a robust ecosystem for batch processing and data science applications. You’ll be instrumental in designing, implementing, and maintaining systems that enable efficient and secure use of big data technologies, developing tools and frameworks to enhance usability across the organization.ResponsibilitiesArchitect, deploy, and scale data storage and processing infrastructure to support analytics and data science workloads.Manage and maintain data lake and clustered computing services, ensuring reliability, security, and scalability.Build and optimize frameworks and tools to simplify big data technology usage.Collaborate with cross-functional teams to align data infrastructure with business goals and requirements.Ensure data governance and security best practices across all platforms.Monitor, troubleshoot, and optimize system performance and resource utilization.QualificationsYou thrive in a fast-paced, complex work environment and love tackling hard problems.5+ years
of experience with Kubernetes and Helm, with a deep understanding of container orchestration.Hands-on experience
administering and optimizing clustered computing technologies on Kubernetes, such as Spark, Trino, Flink, Ray, Kafka, StarRocks, or similar.5+ years
of programming experience in C++, C#, Java, or Python.3+ years
of experience scripting in Python or Bash for automation and tooling.Strong understanding of data storage technologies, distributed computing, and big data processing pipelines.Proficiency in data security best practices and managing access in complex systems.Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $175,000-$205,000. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience.What We OfferIn addition to a competitive salary, we offer a variety of benefits to support your needs, including:Medical, dental, and vision insurance - 100% paid for by CoreWeaveCompany-paid Life InsuranceVoluntary supplemental life insuranceShort and long-term disability insuranceFlexible Spending AccountTuition ReimbursementMental Wellness Benefits through Spring HealthFamily-Forming support provided by CarrotFlexible, full-service childcare support with Kinside401(k) with a generous employer matchFlexible PTOCatered lunch each day in our office and data center locationsA casual work environmentA work culture focused on innovative disruptionOur WorkplaceAt CoreWeave, we are committed to operating as a hybrid workplace, offering employees flexibility in how they structure their time between in-office and remote work. We recognize the significance of fostering connections, collaboration, and creativity within our office culture and its positive impact on our business. Our philosophy operating as a hybrid workplace underscores our dedication to enabling employees to tailor work-life balance to their individual preferences.For those who do not live within 30 miles of one of our offices, we are open to considering remote work for candidates whose skills and experience strongly align with the role. While we prioritize a hybrid work environment for most roles, we understand the importance of flexibility and are open to remote work for specific positions and specialized skill sets. Onboarding is essential to your success. New employees not based out of an office will be invited to attend onboarding training at one of our hubs within their first month of employment. We continue to foster a collaborative environment by bringing teams together quarterly.CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information.As part of this commitment and consistent with the Americans with Disabilities Act (ADA), CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: careers@coreweave.com.
#J-18808-Ljbffr
CoreWeave
powers the creation and delivery of intelligence that drives innovation. CoreWeave is the AI Hyperscaler, delivering a cloud platform of cutting-edge services powering the next wave of AI. The company’s technology provides enterprises and leading AI labs with the most performant, efficient, and resilient solutions for accelerated computing. Since 2017, CoreWeave has operated a growing footprint of data centers covering every region of the US and across Europe. CoreWeave was ranked as one of the TIME100 most influential companies of 2024.As the leader in the industry, we thrive in an environment where adaptability and resilience are key. Our culture offers career-defining opportunities for those who excel amid change and challenge. If you’re someone who thrives in a dynamic environment, enjoys solving complex problems, and is eager to make a significant impact, CoreWeave is the place for you. Join us, and be part of a team solving some of the most exciting challenges in the industry.About the TeamThe Data Engineering team builds foundational datasets and analytics services that enable BI and data science across CoreWeave. We seek to democratize insights and foster a culture where data-driven decision-making thrives at every level.About the RoleWe’re looking for a seasoned Senior Data Engineer with an SRE’s skillset to build and scale foundational data storage and processing infrastructure that will power BI and data science initiatives across CoreWeave. This engineer will take ownership of our data lake and clustered computing services, creating a robust ecosystem for batch processing and data science applications. You’ll be instrumental in designing, implementing, and maintaining systems that enable efficient and secure use of big data technologies, developing tools and frameworks to enhance usability across the organization.ResponsibilitiesArchitect, deploy, and scale data storage and processing infrastructure to support analytics and data science workloads.Manage and maintain data lake and clustered computing services, ensuring reliability, security, and scalability.Build and optimize frameworks and tools to simplify big data technology usage.Collaborate with cross-functional teams to align data infrastructure with business goals and requirements.Ensure data governance and security best practices across all platforms.Monitor, troubleshoot, and optimize system performance and resource utilization.QualificationsYou thrive in a fast-paced, complex work environment and love tackling hard problems.5+ years
of experience with Kubernetes and Helm, with a deep understanding of container orchestration.Hands-on experience
administering and optimizing clustered computing technologies on Kubernetes, such as Spark, Trino, Flink, Ray, Kafka, StarRocks, or similar.5+ years
of programming experience in C++, C#, Java, or Python.3+ years
of experience scripting in Python or Bash for automation and tooling.Strong understanding of data storage technologies, distributed computing, and big data processing pipelines.Proficiency in data security best practices and managing access in complex systems.Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $175,000-$205,000. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience.What We OfferIn addition to a competitive salary, we offer a variety of benefits to support your needs, including:Medical, dental, and vision insurance - 100% paid for by CoreWeaveCompany-paid Life InsuranceVoluntary supplemental life insuranceShort and long-term disability insuranceFlexible Spending AccountTuition ReimbursementMental Wellness Benefits through Spring HealthFamily-Forming support provided by CarrotFlexible, full-service childcare support with Kinside401(k) with a generous employer matchFlexible PTOCatered lunch each day in our office and data center locationsA casual work environmentA work culture focused on innovative disruptionOur WorkplaceAt CoreWeave, we are committed to operating as a hybrid workplace, offering employees flexibility in how they structure their time between in-office and remote work. We recognize the significance of fostering connections, collaboration, and creativity within our office culture and its positive impact on our business. Our philosophy operating as a hybrid workplace underscores our dedication to enabling employees to tailor work-life balance to their individual preferences.For those who do not live within 30 miles of one of our offices, we are open to considering remote work for candidates whose skills and experience strongly align with the role. While we prioritize a hybrid work environment for most roles, we understand the importance of flexibility and are open to remote work for specific positions and specialized skill sets. Onboarding is essential to your success. New employees not based out of an office will be invited to attend onboarding training at one of our hubs within their first month of employment. We continue to foster a collaborative environment by bringing teams together quarterly.CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information.As part of this commitment and consistent with the Americans with Disabilities Act (ADA), CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: careers@coreweave.com.
#J-18808-Ljbffr