palo_alto_networks
Big Data Engineer
palo_alto_networks, Santa Clara, California, us, 95053
THE MISSION:
Our daily fight with cyber bad guys requires us to collect and analyze a lot of data…. a LOT of data! And, as our customer base continues its rapid growth, we must look at faster and more robust tools to help us and our customers make the best decisions possible.
With your knowledge of Hadoop and Big Data technologies, you will add your tools-building superpowers to a small team tasked with building out a DevOps automation environment, one that will step up our Business Intelligence game and help us protect our customers from cyber intruders.
We offer the chance to be part of an important mission: ending breaches and protecting our way of digital life. If you are a motivated, intelligent, creative, and hardworking individual, then this job is for you!
THE JOB:
As a Big Data Engineer, you will be an integral member of our Big Data & Analytics team responsible for design and development.
Partner with data analysts, product owners, and data scientists to better understand requirements, finding bottlenecks, resolutions, etc.
You will be an SME for all things ‘Big Data’ as well as mentor other team members.
Design and develop different architectural models for our scalable data processing as well as scalable data storage.
Build data pipelines and ETL using heterogeneous sources.
You will build data ingestion from various source systems to Hadoop using Kafka, Flume, Sqoop, Spark Streaming, etc.
You will transform data using data mapping and data processing capabilities like MapReduce, Spark SQL.
You will be responsible for ensuring that the platform goes through Continuous Integration (CI) and Continuous Deployment (CD) with DevOps automation.
Expand and grow data platform capabilities to solve new data problems and challenges.
Support Big Data and batch/real-time analytical solutions leveraging transformational technologies like Apache Beam.
You will have the ability to research and assess open-source technologies and components to recommend and integrate into the design and implementation.
You will work with development and QA teams to design Ingestion Pipelines, Integration APIs, and provide Hadoop ecosystem services.
THE SKILLS:
8+ years of experience with the Hadoop ecosystem and Big Data technologies.
Ability to dynamically adapt to conventional big-data frameworks and tools with the use-cases required by the project.
Hands-on experience with the Hadoop ecosystem (HDFS, MapReduce, HBase, Hive, Impala, Spark, Kafka, Kudu, Solr).
Experience with building stream-processing systems using solutions such as Spark Streaming, Storm, or Flink, etc.
Experience in other open-sources like Druid, Elastic Search, Logstash, etc. is a plus.
Knowledge of design strategies for developing scalable, resilient, always-on data lakes.
Some knowledge of agile (scrum) development methodology is a plus.
Strong development/automation skills. Must be very comfortable with reading and writing Scala, Python, or Java code.
Excellent inter-personal and teamwork skills.
Can-do attitude on problem solving, quality, and ability to execute.
Degree in Bachelor of Science in Computer Science or equivalent.
#J-18808-Ljbffr
Our daily fight with cyber bad guys requires us to collect and analyze a lot of data…. a LOT of data! And, as our customer base continues its rapid growth, we must look at faster and more robust tools to help us and our customers make the best decisions possible.
With your knowledge of Hadoop and Big Data technologies, you will add your tools-building superpowers to a small team tasked with building out a DevOps automation environment, one that will step up our Business Intelligence game and help us protect our customers from cyber intruders.
We offer the chance to be part of an important mission: ending breaches and protecting our way of digital life. If you are a motivated, intelligent, creative, and hardworking individual, then this job is for you!
THE JOB:
As a Big Data Engineer, you will be an integral member of our Big Data & Analytics team responsible for design and development.
Partner with data analysts, product owners, and data scientists to better understand requirements, finding bottlenecks, resolutions, etc.
You will be an SME for all things ‘Big Data’ as well as mentor other team members.
Design and develop different architectural models for our scalable data processing as well as scalable data storage.
Build data pipelines and ETL using heterogeneous sources.
You will build data ingestion from various source systems to Hadoop using Kafka, Flume, Sqoop, Spark Streaming, etc.
You will transform data using data mapping and data processing capabilities like MapReduce, Spark SQL.
You will be responsible for ensuring that the platform goes through Continuous Integration (CI) and Continuous Deployment (CD) with DevOps automation.
Expand and grow data platform capabilities to solve new data problems and challenges.
Support Big Data and batch/real-time analytical solutions leveraging transformational technologies like Apache Beam.
You will have the ability to research and assess open-source technologies and components to recommend and integrate into the design and implementation.
You will work with development and QA teams to design Ingestion Pipelines, Integration APIs, and provide Hadoop ecosystem services.
THE SKILLS:
8+ years of experience with the Hadoop ecosystem and Big Data technologies.
Ability to dynamically adapt to conventional big-data frameworks and tools with the use-cases required by the project.
Hands-on experience with the Hadoop ecosystem (HDFS, MapReduce, HBase, Hive, Impala, Spark, Kafka, Kudu, Solr).
Experience with building stream-processing systems using solutions such as Spark Streaming, Storm, or Flink, etc.
Experience in other open-sources like Druid, Elastic Search, Logstash, etc. is a plus.
Knowledge of design strategies for developing scalable, resilient, always-on data lakes.
Some knowledge of agile (scrum) development methodology is a plus.
Strong development/automation skills. Must be very comfortable with reading and writing Scala, Python, or Java code.
Excellent inter-personal and teamwork skills.
Can-do attitude on problem solving, quality, and ability to execute.
Degree in Bachelor of Science in Computer Science or equivalent.
#J-18808-Ljbffr