DataDirect Networks
Staff Software Engineer - Infinia Data Engineer
DataDirect Networks, Chicago, Illinois, United States
Staff Software Engineer - Infinia Data Engineer
Job Locations:
IL-Remote Overview
This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing. DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management. Job Description
Position Summary:
We are looking for a highly skilled Staff Software Engineer to join our Data Engine team. In this pivotal role, you will design, develop, and optimize the core components of Infinia, DDN's advanced distributed intelligent data platform. Your work will involve creating the systems for processing, storing, and executing complex distributed data queries using state-of-the-art open-source technologies such as Apache Iceberg, Delta Lake, and Apache Spark. Key Responsibilities:
Design and enhance the core components of Infinia to run open-source technologies with high performance and scalability. Create and implement optimized execution plans to run on Infinia, leveraging its high-performance and scalability capabilities. Optimize data storage and retrieval using Parquet, ORC, HDFS, and other formats. Contribute to the open-source community when appropriate by collaborating on projects and integrating new features. Solve complex data processing challenges at scale, pushing the boundaries of what's possible with distributed data platforms. Work closely with the wider Infinia organization, including data scientists and other engineers, to deliver high-quality solutions. Provide technical leadership and mentorship to junior engineers. Required Qualifications:
Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field. 8+ years of software engineering experience, with 5+ years in big data technologies. Strong expertise in Scala, Java, and Python. Proven expertise with Apache Spark, Apache Iceberg, and Delta Lake. Hands-on experience with Parquet, ORC, HDFS, Avro, and other data formats. Preferred Skills:
Experience with real-time data processing and streaming frameworks. Previous experience as Apache committer to relevant open-source projects. Proven ability to lead technical projects and mentor other engineers. This role offers an exceptional opportunity to participate in a high-impact engineering organization at the core of DDN's cutting-edge storage solutions. If you are passionate about solving complex technical challenges and driving innovation in high-performance systems, we encourage you to apply. Interview Process:
After submitting your application, one of our recruiters will review your resume. If your application passes this stage, you will be invited to a 30-minute interview during which a member of our team will ask some basic questions. If you clear the interview, you will enter the main process, which can consist of up to four interviews in total: Coding assessment: Often in a language of your choice. Systems design: Translate high-level requirements into a scalable, fault-tolerant service (depending on role). Real-time problem-solving: Demonstrate practical skills in a live problem-solving session. Meet and greet with the wider team. Our goal is to finish the main process in 2-3 weeks at most. DataDirect Networks (DDN) is an Equal Opportunity/Affirmative Action employer.
All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, gender expression, transgender, sex stereotyping, sexual orientation, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.
#J-18808-Ljbffr
Job Locations:
IL-Remote Overview
This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing. DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management. Job Description
Position Summary:
We are looking for a highly skilled Staff Software Engineer to join our Data Engine team. In this pivotal role, you will design, develop, and optimize the core components of Infinia, DDN's advanced distributed intelligent data platform. Your work will involve creating the systems for processing, storing, and executing complex distributed data queries using state-of-the-art open-source technologies such as Apache Iceberg, Delta Lake, and Apache Spark. Key Responsibilities:
Design and enhance the core components of Infinia to run open-source technologies with high performance and scalability. Create and implement optimized execution plans to run on Infinia, leveraging its high-performance and scalability capabilities. Optimize data storage and retrieval using Parquet, ORC, HDFS, and other formats. Contribute to the open-source community when appropriate by collaborating on projects and integrating new features. Solve complex data processing challenges at scale, pushing the boundaries of what's possible with distributed data platforms. Work closely with the wider Infinia organization, including data scientists and other engineers, to deliver high-quality solutions. Provide technical leadership and mentorship to junior engineers. Required Qualifications:
Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field. 8+ years of software engineering experience, with 5+ years in big data technologies. Strong expertise in Scala, Java, and Python. Proven expertise with Apache Spark, Apache Iceberg, and Delta Lake. Hands-on experience with Parquet, ORC, HDFS, Avro, and other data formats. Preferred Skills:
Experience with real-time data processing and streaming frameworks. Previous experience as Apache committer to relevant open-source projects. Proven ability to lead technical projects and mentor other engineers. This role offers an exceptional opportunity to participate in a high-impact engineering organization at the core of DDN's cutting-edge storage solutions. If you are passionate about solving complex technical challenges and driving innovation in high-performance systems, we encourage you to apply. Interview Process:
After submitting your application, one of our recruiters will review your resume. If your application passes this stage, you will be invited to a 30-minute interview during which a member of our team will ask some basic questions. If you clear the interview, you will enter the main process, which can consist of up to four interviews in total: Coding assessment: Often in a language of your choice. Systems design: Translate high-level requirements into a scalable, fault-tolerant service (depending on role). Real-time problem-solving: Demonstrate practical skills in a live problem-solving session. Meet and greet with the wider team. Our goal is to finish the main process in 2-3 weeks at most. DataDirect Networks (DDN) is an Equal Opportunity/Affirmative Action employer.
All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, gender expression, transgender, sex stereotyping, sexual orientation, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.
#J-18808-Ljbffr