Anblicks
Senior Data Engineer
Anblicks, Dallas, Texas, United States, 75215
Job Title:
Senior Data Engineer
JOB DUTIES:
Participate in daily agile and scrum processes to understand changing business requirements, including examining system configurations and operating procedures, as well as functional requirements gathering.
Ingest and prepare business-ready data in the cloud using Azure Data Factory (ADF) to build ELT/ETL data pipelines, then move the data into a data warehouse (Dedicated SQL Pool) and create data lake zones for data analytics and visualization.
Work with a combination of Azure Data Factory and Azure Databricks, extract, load, and transform data from cloud sources and on-premises databases such as Oracle, SAP, and SQL Server to Data Lake Storage and Azure Synapse Analytics.
Create an Azure Data Factory Pipeline Template to migrate an on-premises data platform to the Azure Cloud using batch processing methods by incremental or full load.
Analyze, design, and build modern data solutions using Azure PaaS services to support data visualization.
Enable private end point, firewall setting and Azure Key Vault for robust data security.
Analyze the existing SSIS packages and integrate it with Azure Data Factory.
Create a JSON structure for data storage in Azure Cosmos DB (SQL API) and work with the API team to create Cosmos DB queries.
Data Model in Snowflake and ELT using Snowflake SQL, implementing complex stored procedures and best practices.
Design and customize dimension data models for data warehouse supporting data using Azure Synapse Analytics.
Build a distributed in-memory application using spark applications and perform analytics efficiently on large datasets using Python and Spark SQL.
Create data visualization reports and dashboards in Power BI using data from the data warehouse, flat files, and Azure SQL.
Responsible for fixing problems and conducting investigations into SQL queries, Stored Procedures related to long running jobs and Azure service performance.
Utilize Azure Monitor and Alert services, create monitors, alerts, and notifications for Data Factory, Synapse Analytics, and Data Lake Storage.
Perform the required daily GIT support for various projects and maintain GIT repositories.
Create CI&CD using Azure DevOps pipeline to deploy Azure Services using ARM Templates.
JOB REQUIREMENTS:
Master's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 2 years of experience. In lieu of the above, we will also accept a Bachelor's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 5 years of progressively responsible post-baccalaureate experience.
Experience to include working on Azure Data Factory (ADF), Oracle, SQL Server, Azure Databricks, Azure Synapse Analytics, Data Lake Storage, Azure PaaS services, SSIS packages, Azure Cosmos DB (SQL API), Python, Spark applications, Azure SQL, SQL queries, Azure Monitor and Alert services, GIT Support, ARM Templates.
HOURS:
M-F, 8:00 a.m. – 5:00 p.m.
JOB LOCATION:
Dallas, Texas. Travel is not required, but candidates must be willing to relocate to unanticipated work locations across the country per contract demand.
CONTACT:
Email resume referencing job code# SDE01302023ANB to Maruthi Technologies INC. DBA Anblicks at [email protected]
#J-18808-Ljbffr
Senior Data Engineer
JOB DUTIES:
Participate in daily agile and scrum processes to understand changing business requirements, including examining system configurations and operating procedures, as well as functional requirements gathering.
Ingest and prepare business-ready data in the cloud using Azure Data Factory (ADF) to build ELT/ETL data pipelines, then move the data into a data warehouse (Dedicated SQL Pool) and create data lake zones for data analytics and visualization.
Work with a combination of Azure Data Factory and Azure Databricks, extract, load, and transform data from cloud sources and on-premises databases such as Oracle, SAP, and SQL Server to Data Lake Storage and Azure Synapse Analytics.
Create an Azure Data Factory Pipeline Template to migrate an on-premises data platform to the Azure Cloud using batch processing methods by incremental or full load.
Analyze, design, and build modern data solutions using Azure PaaS services to support data visualization.
Enable private end point, firewall setting and Azure Key Vault for robust data security.
Analyze the existing SSIS packages and integrate it with Azure Data Factory.
Create a JSON structure for data storage in Azure Cosmos DB (SQL API) and work with the API team to create Cosmos DB queries.
Data Model in Snowflake and ELT using Snowflake SQL, implementing complex stored procedures and best practices.
Design and customize dimension data models for data warehouse supporting data using Azure Synapse Analytics.
Build a distributed in-memory application using spark applications and perform analytics efficiently on large datasets using Python and Spark SQL.
Create data visualization reports and dashboards in Power BI using data from the data warehouse, flat files, and Azure SQL.
Responsible for fixing problems and conducting investigations into SQL queries, Stored Procedures related to long running jobs and Azure service performance.
Utilize Azure Monitor and Alert services, create monitors, alerts, and notifications for Data Factory, Synapse Analytics, and Data Lake Storage.
Perform the required daily GIT support for various projects and maintain GIT repositories.
Create CI&CD using Azure DevOps pipeline to deploy Azure Services using ARM Templates.
JOB REQUIREMENTS:
Master's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 2 years of experience. In lieu of the above, we will also accept a Bachelor's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 5 years of progressively responsible post-baccalaureate experience.
Experience to include working on Azure Data Factory (ADF), Oracle, SQL Server, Azure Databricks, Azure Synapse Analytics, Data Lake Storage, Azure PaaS services, SSIS packages, Azure Cosmos DB (SQL API), Python, Spark applications, Azure SQL, SQL queries, Azure Monitor and Alert services, GIT Support, ARM Templates.
HOURS:
M-F, 8:00 a.m. – 5:00 p.m.
JOB LOCATION:
Dallas, Texas. Travel is not required, but candidates must be willing to relocate to unanticipated work locations across the country per contract demand.
CONTACT:
Email resume referencing job code# SDE01302023ANB to Maruthi Technologies INC. DBA Anblicks at [email protected]
#J-18808-Ljbffr