Diverse Lynx
Data Engineer (Python + Azure Databricks + SQL
Diverse Lynx, Hartford, Connecticut, us, 06112
Position:
Data Engineer (
Python + Azure Databricks + SQL)Location:
Hartford, CT/Stamford, CT- Day 1 OnsitePosition Type: Contract
This position will see you responsible for liaising with the business to gather requirements, working with the team to identify potential tools and through the migration on to a centralized platform consolidate data sources.
Roles and Responsibilities:•Analyses current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services.•Engage and collaborate with customers to understand business requirements/use cases and translate them into detailed technical specifications.•Experience with Python/Pyspark frameworks/libraries.•Expertise of Azure Databricks.•Experienced of SQL.•Experience in designing and implementing complex ETL and process flows•Expertise across data transformation frameworks.•Good to have Azure/Azure DevOps knowledge.•Configuration Management, CI/CD and Cloud•Develop best practices including reusable code, libraries, patterns, and consumable frameworks for cloud-based data warehousing and ETL.•Maintain best practice standards for the development or cloud-based data warehouse solutioning including naming standard
Experience:•Good understanding of the platform and the code/integration behind.•Want to upgrade the system to be the most efficient version.•5-7 years of experience in a Data Engineer role.•Strong knowledge in data modeling, data warehousing, and ETL processes.•Strong knowledge of database management systems (e.g., SQL Server, Oracle, MySQL).•Experience with Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Synapse Analytics, and other Azure data services.•Familiarity with data governance, data quality, and data security best practices.•Experience with DevOps practices, agile methodologies, and collaboration tools.
Key Skills Required:•Python + Azure Databricks + SQL•Experience in designing and hands-on development in cloud-based analytics solutions.•Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required.•Designing and building of data pipelines using API ingestion and Streaming ingestion methods.•Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.•Experience in developing NO SQL solutions using Azure Cosmos DB is essential.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Data Engineer (
Python + Azure Databricks + SQL)Location:
Hartford, CT/Stamford, CT- Day 1 OnsitePosition Type: Contract
This position will see you responsible for liaising with the business to gather requirements, working with the team to identify potential tools and through the migration on to a centralized platform consolidate data sources.
Roles and Responsibilities:•Analyses current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services.•Engage and collaborate with customers to understand business requirements/use cases and translate them into detailed technical specifications.•Experience with Python/Pyspark frameworks/libraries.•Expertise of Azure Databricks.•Experienced of SQL.•Experience in designing and implementing complex ETL and process flows•Expertise across data transformation frameworks.•Good to have Azure/Azure DevOps knowledge.•Configuration Management, CI/CD and Cloud•Develop best practices including reusable code, libraries, patterns, and consumable frameworks for cloud-based data warehousing and ETL.•Maintain best practice standards for the development or cloud-based data warehouse solutioning including naming standard
Experience:•Good understanding of the platform and the code/integration behind.•Want to upgrade the system to be the most efficient version.•5-7 years of experience in a Data Engineer role.•Strong knowledge in data modeling, data warehousing, and ETL processes.•Strong knowledge of database management systems (e.g., SQL Server, Oracle, MySQL).•Experience with Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Synapse Analytics, and other Azure data services.•Familiarity with data governance, data quality, and data security best practices.•Experience with DevOps practices, agile methodologies, and collaboration tools.
Key Skills Required:•Python + Azure Databricks + SQL•Experience in designing and hands-on development in cloud-based analytics solutions.•Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required.•Designing and building of data pipelines using API ingestion and Streaming ingestion methods.•Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.•Experience in developing NO SQL solutions using Azure Cosmos DB is essential.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.