NVIDIA
Staff Data Engineer
NVIDIA, Santa Clara, California, us, 95053
NVIDIA is hiring senior distributed systems with a data engineering emphasis to develop and scale its AI and deep learning platforms. Our team is building a software 2.0 developer platform with a focus on datasets for AI application development. Together, we will advance NVIDIA's capacity to build and deploy leading solutions for a broad range of AI-based applications such as autonomous vehicles, healthcare, virtual reality, graphics engines, and visual computing. Together, with NVIDIA partners, we will bring autonomous vehicles to life!What You'll Be Doing
Architect and build scalable and distributed commitment to improvement, compute, and data pipelines that will help power the IT Data Lake as a centralized data platform.Design and build PB sized scalable data lake and structured/unstructured data query interfaces and microservices to ingest, index, mine, transform, and compose large datasets.Build Cloud Cost and Usages data patterns to crawl, collect and transform TBs of data on a daily basis.Enabling data models/views across TBs of data which can be consumed by analytical tools such as PowerBI. Build PBI analytics for finance reporting.Build and implement support for versioned, traceable, and immutable datasets in a data lake in a distributed and scalable manner.Enable efficient and thoughtful data selection - one of the key ingredients for successful machine learning!Hands-on writing code of high quality, good design & architecture, fully tested and peer reviews.Collaborate with multiple product/engineering teams to understand their data and compute requirements (SW, HW, Automobile, AI) to integrate amazing innovations and algorithms into our production systems.Automate everything for measuring, testing, updating, monitoring, and alerting the data platform.What We Need To See
Bachelors (or equivalent experience) or Masters in Computer Architecture, Computer Science, or related data-intensive Engineering Degree.8+ years of proven experience in Data Engineering, worked on designing and developing software with Big Data, Data Lake/ Lake House ecosystem, Data Analytics, backend microservices architecture, and heterogeneous data types at scale.Proven in-depth experience in creating ETL pipelines using Databricks, Spark, Python, SQL, Scala, Kafka, Presto, Parquet, Streaming, events, bots, AWS/cloud ecosystem.Proficient in developing Micro Services and using AWS frameworks such as SQS, Stream, Kubernetes, EC2, S3, Lambda, etc.Experience with data pipelines/analysis/visualization tooling such as Elastic stack, Logstash, Kibana, Kafka, Grafana, Splunk, Pandas, Message brokers, Data modeling.Expertise in Data Lakehouse architecture and end-to-end Databricks techniques including Data Science components.Worked on end-to-end data lifecycle from Data Ingestion, Data Transformation, and Data Consumption layer. Versed with API and its usability.Knowledge of Cloud solutions like Kendra, SageMaker, Auto-ML, Big Query, RedShift, Glue, Athena.Ways To Stand Out From The Crowd
Understanding and experience around Cost and Usages analytics is a plus.Expert in Spark, Parquet, streaming, events, Kafka, telemetry, MapReduce, Hadoop, Hive, Presto, Spark, data query approaches, and dashboarding.The one who has implemented Enterprise use cases like CMDB, Governance, time series classification, telemetry anomaly detection, logs, and real-time data ingestion through APIs.Experience with structured data such as Avro, Parquet, Protobuf, Thrift, and concepts like schema evolution.Working knowledge of Amazon Web Services, Kubernetes, Docker is a plus.NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people on the planet working for us. If you're creative and autonomous, we want to hear from you!The base salary range is $160,000 - $304,750. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits.NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
#J-18808-Ljbffr
Architect and build scalable and distributed commitment to improvement, compute, and data pipelines that will help power the IT Data Lake as a centralized data platform.Design and build PB sized scalable data lake and structured/unstructured data query interfaces and microservices to ingest, index, mine, transform, and compose large datasets.Build Cloud Cost and Usages data patterns to crawl, collect and transform TBs of data on a daily basis.Enabling data models/views across TBs of data which can be consumed by analytical tools such as PowerBI. Build PBI analytics for finance reporting.Build and implement support for versioned, traceable, and immutable datasets in a data lake in a distributed and scalable manner.Enable efficient and thoughtful data selection - one of the key ingredients for successful machine learning!Hands-on writing code of high quality, good design & architecture, fully tested and peer reviews.Collaborate with multiple product/engineering teams to understand their data and compute requirements (SW, HW, Automobile, AI) to integrate amazing innovations and algorithms into our production systems.Automate everything for measuring, testing, updating, monitoring, and alerting the data platform.What We Need To See
Bachelors (or equivalent experience) or Masters in Computer Architecture, Computer Science, or related data-intensive Engineering Degree.8+ years of proven experience in Data Engineering, worked on designing and developing software with Big Data, Data Lake/ Lake House ecosystem, Data Analytics, backend microservices architecture, and heterogeneous data types at scale.Proven in-depth experience in creating ETL pipelines using Databricks, Spark, Python, SQL, Scala, Kafka, Presto, Parquet, Streaming, events, bots, AWS/cloud ecosystem.Proficient in developing Micro Services and using AWS frameworks such as SQS, Stream, Kubernetes, EC2, S3, Lambda, etc.Experience with data pipelines/analysis/visualization tooling such as Elastic stack, Logstash, Kibana, Kafka, Grafana, Splunk, Pandas, Message brokers, Data modeling.Expertise in Data Lakehouse architecture and end-to-end Databricks techniques including Data Science components.Worked on end-to-end data lifecycle from Data Ingestion, Data Transformation, and Data Consumption layer. Versed with API and its usability.Knowledge of Cloud solutions like Kendra, SageMaker, Auto-ML, Big Query, RedShift, Glue, Athena.Ways To Stand Out From The Crowd
Understanding and experience around Cost and Usages analytics is a plus.Expert in Spark, Parquet, streaming, events, Kafka, telemetry, MapReduce, Hadoop, Hive, Presto, Spark, data query approaches, and dashboarding.The one who has implemented Enterprise use cases like CMDB, Governance, time series classification, telemetry anomaly detection, logs, and real-time data ingestion through APIs.Experience with structured data such as Avro, Parquet, Protobuf, Thrift, and concepts like schema evolution.Working knowledge of Amazon Web Services, Kubernetes, Docker is a plus.NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people on the planet working for us. If you're creative and autonomous, we want to hear from you!The base salary range is $160,000 - $304,750. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits.NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
#J-18808-Ljbffr