Mavens Guild
Senior Spark & Python Developer
Mavens Guild, Mc Lean, Virginia, us, 22107
What we would like to see:In a senior developer role, you will design and build data flow and data integration processes to enhance loss prevention technologies for a leading financing firm. Drawing from your vast hands-on experience in Python (PySpark), Spark, REST, Java, and Scala, you will develop, test, and deploy end-to-end solutions using full-stack development tools within AWS (EMR, S3) cloud-based infrastructure.
A typical day as a Senior Spark Programmer:
Develop fault-tolerant, streaming as well as batch data integration processes using Spark/PySpark, Java and performance enhanced SQL.
Develop and lead development efforts of application programming interfaces to enable integration of fraud detection systems with a host of new reporting and data mining tools.
Design and develop automation of data flow tasks and end-to-end process testing.
Design and develop scalable frameworks to ingest, transform, store, and present loss prevention information to downstream systems.
Implement and lead implementation of Agile best practices and a continuous integration ecosystem.
Implement and lead implementation efforts of Spark/Python based solution architecture, scalable process frameworks, advanced analytics, and responsive RESTful services.
What you will need to bring to the table:
5+ years experience with processing of structured, unstructured and semi-structured data using in-memory cluster computing technologies, specifically with Spark.
5+ years experience as a Java programmer.
3+ years experience programming in Python (PySpark API).
3+ years experience with cloud services offered through AWS, like EMR, Redshift, or S3.
3+ years experience in developing, testing and deploying RESTful APIs for high volume data streams.
Experience with continuous integration tools like Jenkins.
Experience working in a distributed computing infrastructure like Hadoop and/or Spark.
Critical and analytical approach to solving technical problems.
Excellent interpersonal skills and ability to clearly communicate highly technical concepts to business stakeholders and technical developers alike.
#J-18808-Ljbffr
A typical day as a Senior Spark Programmer:
Develop fault-tolerant, streaming as well as batch data integration processes using Spark/PySpark, Java and performance enhanced SQL.
Develop and lead development efforts of application programming interfaces to enable integration of fraud detection systems with a host of new reporting and data mining tools.
Design and develop automation of data flow tasks and end-to-end process testing.
Design and develop scalable frameworks to ingest, transform, store, and present loss prevention information to downstream systems.
Implement and lead implementation of Agile best practices and a continuous integration ecosystem.
Implement and lead implementation efforts of Spark/Python based solution architecture, scalable process frameworks, advanced analytics, and responsive RESTful services.
What you will need to bring to the table:
5+ years experience with processing of structured, unstructured and semi-structured data using in-memory cluster computing technologies, specifically with Spark.
5+ years experience as a Java programmer.
3+ years experience programming in Python (PySpark API).
3+ years experience with cloud services offered through AWS, like EMR, Redshift, or S3.
3+ years experience in developing, testing and deploying RESTful APIs for high volume data streams.
Experience with continuous integration tools like Jenkins.
Experience working in a distributed computing infrastructure like Hadoop and/or Spark.
Critical and analytical approach to solving technical problems.
Excellent interpersonal skills and ability to clearly communicate highly technical concepts to business stakeholders and technical developers alike.
#J-18808-Ljbffr