Indotronix UK
Databricks Engineer with Pyspark & Python Exp- US
Indotronix UK, New York, New York, us, 10261
Role:
Lead Databricks Engineer with Pyspark & Python ExpLocation:
NYC, NY OR Iselin, NJ (Hybrid – 3 days’ work from office)Duration:
6+ MonthsClient is okay to pay the expenses for the in-person interview.Job Description:In-person interview is a must; expenses will be paid for in-person if not local. (Financial background is a must).Major Responsibilities:Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo.Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.Optimize Databricks jobs for performance and scalability to handle big data workloads.Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.Implement best practices for data management, security, and governance within the Databricks environment.Experience designing and developing Enterprise Data Warehouse solutions.Demonstrated proficiency with Data Analytics, Data Insights.Proficient writing SQL queries and programming including stored procedures and reverse engineering existing processes.Azure Synapse/Bigquery/Redshift is good to have.Perform code reviews to ensure fit to requirements, optimal execution patterns, and adherence to established standards.Skills:5+ years’ – Strong experience in Finance / Banking industry – Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo.10+ years - Enterprise Data Management.10+ years - SQL Server based development of large datasets.5+ years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.Experience with Cloud based data architectures, messaging, and analytics.Any experience with Regulatory Reporting is a Plus.Education:Minimally a BA degree within an engineering and/or computer science discipline.
#J-18808-Ljbffr
Lead Databricks Engineer with Pyspark & Python ExpLocation:
NYC, NY OR Iselin, NJ (Hybrid – 3 days’ work from office)Duration:
6+ MonthsClient is okay to pay the expenses for the in-person interview.Job Description:In-person interview is a must; expenses will be paid for in-person if not local. (Financial background is a must).Major Responsibilities:Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo.Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.Optimize Databricks jobs for performance and scalability to handle big data workloads.Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.Implement best practices for data management, security, and governance within the Databricks environment.Experience designing and developing Enterprise Data Warehouse solutions.Demonstrated proficiency with Data Analytics, Data Insights.Proficient writing SQL queries and programming including stored procedures and reverse engineering existing processes.Azure Synapse/Bigquery/Redshift is good to have.Perform code reviews to ensure fit to requirements, optimal execution patterns, and adherence to established standards.Skills:5+ years’ – Strong experience in Finance / Banking industry – Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo.10+ years - Enterprise Data Management.10+ years - SQL Server based development of large datasets.5+ years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.Experience with Cloud based data architectures, messaging, and analytics.Any experience with Regulatory Reporting is a Plus.Education:Minimally a BA degree within an engineering and/or computer science discipline.
#J-18808-Ljbffr