JobRialto
.Azure Data Engineer
JobRialto, Jersey City, New Jersey, United States, 07390
Job Summary
The Risk Technology Team is seeking a talented and motivated senior Azure Data Engineer. Responsible for developing and deploying solutions for risk, regulatory, and compliance projects on the Azure platform. Be at the forefront of the on-prem to cloud migration initiative.
Scope
• Strategizes, designs, and develops automated cloud infrastructure and DevOps solutions.
• Works with a team of dynamic and passionate data engineers.
• Supports the growth and scale of our Azure cloud-based technology infrastructure.
• Responsible for complex interface process development, troubleshooting issues, supporting weekend maintenance, and production support.
Primary Responsibilities
• Administration, configuration, and optimization of the Azure Databricks platform.
• Provide technical expertise to deploy applications on the platform.
• Work with data engineers, data scientists, and other stakeholders to ensure the platform meets their requirements.
• Manage the infrastructure and collaborate with teams to ensure integration with all components, including network, security, and remote access.
• Databricks Cluster configuration and Unity catalog repository configuration.
• Seamless integration with on-prem databases and Power BI reports.
• Build large-scale batch and real-time data pipelines with data processing frameworks in the Azure cloud platform.
• Partner with architects, engineers, information analysts, business, and technology stakeholders for developing and deploying enterprise-grade platforms.
• Provide technical guidance on technology implementation and architectural changes.
• Coordinate with SMBC platforms, APIM, Azure DevOps, and Support.
• Analyze and design technical solutions on Azure to address business needs.
• Develop, test, and modify software to improve the efficiency of data platforms and applications.
• Proficient in Python coding and frameworks knowledge.
• Own projects end-to-end, working with business stakeholders, IT teams, data stewards, and 3rd party companies to ensure accurate and insightful outcomes.
• Integrate data from a variety of sources, ensuring adherence to data quality and accessibility standards.
• Enable streaming data use-cases and standard design patterns in a big data environment.
• Collaborate with data scientists to integrate algorithms and models into automated processes.
• Apply expertise, judgment, and precedents to contribute to the resolution of moderately complex problems.
• Review and provide feedback on core code changes and support production deployment.
• Ability to migrate Data Science models written in native Python language to the multi-node Databricks compute is desirable.
Core Technologies
• Azure Databricks, Azure Data Factory, Azure Functions, Azure Data Lake Storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, Azure Service Bus, R, Python, SQL Server, Oracle / PL-SQL, T-SQL and SQL stored procedures, Azure DevOps.
Qualifications
• IT professional experience in Azure Cloud with over 5+ years of Azure Development.
• Ability to recreate existing legacy application logic and functionality into Azure Databricks/Data Lake, SQL Database, and SQL Datawarehouse environment.
• Excellent interpersonal and communication skills.
• Experience with software configuration management tools such as Git/Gitlab.
• Experience with continuous integration and deployment (CI/CD) tools such as Azure DevOps.
• Experience working on Risk Management projects within Equities or Fixed Income space is highly desirable.
\n
Education:
Bachelors Degree
The Risk Technology Team is seeking a talented and motivated senior Azure Data Engineer. Responsible for developing and deploying solutions for risk, regulatory, and compliance projects on the Azure platform. Be at the forefront of the on-prem to cloud migration initiative.
Scope
• Strategizes, designs, and develops automated cloud infrastructure and DevOps solutions.
• Works with a team of dynamic and passionate data engineers.
• Supports the growth and scale of our Azure cloud-based technology infrastructure.
• Responsible for complex interface process development, troubleshooting issues, supporting weekend maintenance, and production support.
Primary Responsibilities
• Administration, configuration, and optimization of the Azure Databricks platform.
• Provide technical expertise to deploy applications on the platform.
• Work with data engineers, data scientists, and other stakeholders to ensure the platform meets their requirements.
• Manage the infrastructure and collaborate with teams to ensure integration with all components, including network, security, and remote access.
• Databricks Cluster configuration and Unity catalog repository configuration.
• Seamless integration with on-prem databases and Power BI reports.
• Build large-scale batch and real-time data pipelines with data processing frameworks in the Azure cloud platform.
• Partner with architects, engineers, information analysts, business, and technology stakeholders for developing and deploying enterprise-grade platforms.
• Provide technical guidance on technology implementation and architectural changes.
• Coordinate with SMBC platforms, APIM, Azure DevOps, and Support.
• Analyze and design technical solutions on Azure to address business needs.
• Develop, test, and modify software to improve the efficiency of data platforms and applications.
• Proficient in Python coding and frameworks knowledge.
• Own projects end-to-end, working with business stakeholders, IT teams, data stewards, and 3rd party companies to ensure accurate and insightful outcomes.
• Integrate data from a variety of sources, ensuring adherence to data quality and accessibility standards.
• Enable streaming data use-cases and standard design patterns in a big data environment.
• Collaborate with data scientists to integrate algorithms and models into automated processes.
• Apply expertise, judgment, and precedents to contribute to the resolution of moderately complex problems.
• Review and provide feedback on core code changes and support production deployment.
• Ability to migrate Data Science models written in native Python language to the multi-node Databricks compute is desirable.
Core Technologies
• Azure Databricks, Azure Data Factory, Azure Functions, Azure Data Lake Storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, Azure Service Bus, R, Python, SQL Server, Oracle / PL-SQL, T-SQL and SQL stored procedures, Azure DevOps.
Qualifications
• IT professional experience in Azure Cloud with over 5+ years of Azure Development.
• Ability to recreate existing legacy application logic and functionality into Azure Databricks/Data Lake, SQL Database, and SQL Datawarehouse environment.
• Excellent interpersonal and communication skills.
• Experience with software configuration management tools such as Git/Gitlab.
• Experience with continuous integration and deployment (CI/CD) tools such as Azure DevOps.
• Experience working on Risk Management projects within Equities or Fixed Income space is highly desirable.
\n
Education:
Bachelors Degree