Accord Technologies Inc
Senior PySpark Developer
Accord Technologies Inc - Woodbridge, New Jersey, United States
Work at Accord Technologies Inc
Overview
- View job
Overview
5 days ago Be among the first 25 applicants
Get AI-powered advice on this job and more exclusive features.
Job Type: W2 contract.
We are seeking an experienced Job Title: Senior PySpark Developer
Location: Iselin, NJ (onsite)
Duration: 12 months
Job Type: W2 contract.
We are seeking an experienced
Senior PySpark Developer
to join our Data Engineering team in Iselin, NJ. In this role, you will design, develop, and optimize large-scale
data processing pipelines
using
PySpark, Databricks, and AWS/Azure cloud services . You will collaborate with cross-functional teams to deliver high-performance data solutions that drive business intelligence, analytics, and reporting for our financial services platforms.
Key Responsibilities
Design & Development: Develop and optimize PySpark applications for batch and real-time data processing. Build scalable ETL/ELT pipelines using Databricks, Spark SQL, and Delta Lake. Integrate data from multiple sources (databases, APIs, streaming platforms) into cloud-based data lakes/warehouses. Implement data transformations, aggregations, and joins efficiently in distributed environments. Performance Tuning & Optimization: Optimize Spark jobs for performance (partitioning, caching, broadcast joins). Troubleshoot and resolve data skew, memory issues, and job failures. Monitor and fine-tune Databricks clusters for cost efficiency. Cloud & Big Data Technologies: Work with AWS (EMR, Glue, S3) or Azure (Synapse, Data Lake, ADF) for data storage and processing. Implement data governance, security, and compliance best practices. Collaboration & Leadership: Partner with data scientists, analysts, and business teams to deliver actionable insights. Mentor junior developers and enforce coding standards, testing, and CI/CD practices.
Required Skills & Experience
5+ years of hands-on PySpark development experience. Strong expertise in Apache Spark (SQL, DataFrames, RDDs) and Databricks. Proficiency in Python, SQL, and shell scripting. Experience with cloud platforms (AWS/Azure) and big data tools (Hive, Kafka, Snowflake). Knowledge of data modeling, partitioning, and performance optimization. Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
Preferred Qualifications
Financial services or banking industry experience. Certifications in AWS/Azure, Databricks, or Spark. Familiarity with CI/CD (Jenkins, GitLab) and infrastructure-as-code (Terraform).
Seniority level
Seniority level Mid-Senior level Employment type
Employment type Full-time Job function
Job function Engineering and Information Technology Referrals increase your chances of interviewing at Accord Technologies Inc by 2x Get notified about new Senior Software Engineer jobs in
Iselin, NJ . Lead Java Software Engineer - Vice President
New York, NY $125,000.00-$203,000.00 4 months ago Senior Software Engineer - BLAW Contracts Drafting
New York, NY $160,000.00-$240,000.00 4 days ago VP - Lead Software Engineer, Fixed Income - Parametric
New York, NY $150,000.00-$210,000.00 1 week ago Whippany, NJ $209,639.00-$225,161.00 5 days ago Whippany, NJ $209,639.00-$225,161.00 5 days ago New York, NY $175,000.00-$270,000.00 1 week ago Senior / Staff Software Engineer - Computational Chemistry / Molecular Dynamics
New York, NY $150,000.00-$250,000.00 3 days ago Senior Director of Software Engineering - Software Engineering Team Lead
Senior Software Engineer - Buy-Side Violation Manager
New York, NY $160,000.00-$240,000.00 3 days ago Lead .NET Software Engineer - Vice President
Jersey City, NJ $134,000.00-$182,000.00 6 months ago React.JS Software Engineer (Senior Full Stack React.JS Developer)
Principal Software Engineer - Computational Chemistry / Molecular Dynamics
New York, NY $175,000.00-$270,000.00 1 week ago Senior Staff Software Engineer - Core HR Product
Senior Software Engineering Lead - Private Equity & Real Estate Technology
New York, NY $175,000.00-$235,000.00 6 days ago Software Engineer III | Lead Full Stack Developer
Holmdel, NJ $125,000.00-$155,000.00 5 days ago Senior ETL Developer Lead with Guidewire Experience - Remote
Vice President, Senior Software Engineer - Capital Markets
New York, NY $166,000.00-$205,000.00 1 week ago New York, NY $190,000.00-$280,000.00 1 week ago Software Engineer, Data Solutions Lead - Senior Vice President
Lead SDET || Atlanta, GA or Frisco, TX || Remote work
Front-End Designer & Animator, Senior Associate
Senior Engineering Manager - Backend Developer User Experience
Senior Python / Counterparty Credit Risk Senior Application Developer - VP - NEW YORK
New York, NY $142,320.00-$213,480.00 3 days ago New York City Metropolitan Area $125,000.00-$350,000.00 1 month ago Senior Mainframe Developer Role || Onsite for Naperville, IL
Software Engineer III | Lead Full Stack Developer
New York County, NY $150,000.00-$300,000.00 1 week ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
We are seeking an experienced Job Title: Senior PySpark Developer
Location: Iselin, NJ (onsite)
Duration: 12 months
Job Type: W2 contract.
We are seeking an experienced
Senior PySpark Developer
to join our Data Engineering team in Iselin, NJ. In this role, you will design, develop, and optimize large-scale
data processing pipelines
using
PySpark, Databricks, and AWS/Azure cloud services . You will collaborate with cross-functional teams to deliver high-performance data solutions that drive business intelligence, analytics, and reporting for our financial services platforms.
Key Responsibilities
Design & Development: Develop and optimize PySpark applications for batch and real-time data processing. Build scalable ETL/ELT pipelines using Databricks, Spark SQL, and Delta Lake. Integrate data from multiple sources (databases, APIs, streaming platforms) into cloud-based data lakes/warehouses. Implement data transformations, aggregations, and joins efficiently in distributed environments. Performance Tuning & Optimization: Optimize Spark jobs for performance (partitioning, caching, broadcast joins). Troubleshoot and resolve data skew, memory issues, and job failures. Monitor and fine-tune Databricks clusters for cost efficiency. Cloud & Big Data Technologies: Work with AWS (EMR, Glue, S3) or Azure (Synapse, Data Lake, ADF) for data storage and processing. Implement data governance, security, and compliance best practices. Collaboration & Leadership: Partner with data scientists, analysts, and business teams to deliver actionable insights. Mentor junior developers and enforce coding standards, testing, and CI/CD practices.
Required Skills & Experience
5+ years of hands-on PySpark development experience. Strong expertise in Apache Spark (SQL, DataFrames, RDDs) and Databricks. Proficiency in Python, SQL, and shell scripting. Experience with cloud platforms (AWS/Azure) and big data tools (Hive, Kafka, Snowflake). Knowledge of data modeling, partitioning, and performance optimization. Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
Preferred Qualifications
Financial services or banking industry experience. Certifications in AWS/Azure, Databricks, or Spark. Familiarity with CI/CD (Jenkins, GitLab) and infrastructure-as-code (Terraform).
Seniority level
Seniority level Mid-Senior level Employment type
Employment type Full-time Job function
Job function Engineering and Information Technology Referrals increase your chances of interviewing at Accord Technologies Inc by 2x Get notified about new Senior Software Engineer jobs in
Iselin, NJ . Lead Java Software Engineer - Vice President
New York, NY $125,000.00-$203,000.00 4 months ago Senior Software Engineer - BLAW Contracts Drafting
New York, NY $160,000.00-$240,000.00 4 days ago VP - Lead Software Engineer, Fixed Income - Parametric
New York, NY $150,000.00-$210,000.00 1 week ago Whippany, NJ $209,639.00-$225,161.00 5 days ago Whippany, NJ $209,639.00-$225,161.00 5 days ago New York, NY $175,000.00-$270,000.00 1 week ago Senior / Staff Software Engineer - Computational Chemistry / Molecular Dynamics
New York, NY $150,000.00-$250,000.00 3 days ago Senior Director of Software Engineering - Software Engineering Team Lead
Senior Software Engineer - Buy-Side Violation Manager
New York, NY $160,000.00-$240,000.00 3 days ago Lead .NET Software Engineer - Vice President
Jersey City, NJ $134,000.00-$182,000.00 6 months ago React.JS Software Engineer (Senior Full Stack React.JS Developer)
Principal Software Engineer - Computational Chemistry / Molecular Dynamics
New York, NY $175,000.00-$270,000.00 1 week ago Senior Staff Software Engineer - Core HR Product
Senior Software Engineering Lead - Private Equity & Real Estate Technology
New York, NY $175,000.00-$235,000.00 6 days ago Software Engineer III | Lead Full Stack Developer
Holmdel, NJ $125,000.00-$155,000.00 5 days ago Senior ETL Developer Lead with Guidewire Experience - Remote
Vice President, Senior Software Engineer - Capital Markets
New York, NY $166,000.00-$205,000.00 1 week ago New York, NY $190,000.00-$280,000.00 1 week ago Software Engineer, Data Solutions Lead - Senior Vice President
Lead SDET || Atlanta, GA or Frisco, TX || Remote work
Front-End Designer & Animator, Senior Associate
Senior Engineering Manager - Backend Developer User Experience
Senior Python / Counterparty Credit Risk Senior Application Developer - VP - NEW YORK
New York, NY $142,320.00-$213,480.00 3 days ago New York City Metropolitan Area $125,000.00-$350,000.00 1 month ago Senior Mainframe Developer Role || Onsite for Naperville, IL
Software Engineer III | Lead Full Stack Developer
New York County, NY $150,000.00-$300,000.00 1 week ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr