Insight Global
A fortune 100 client is seeking to add 2 experienced Data Architects to their team for Project: MDIP. These architects will assist in to drive the end-to-end migration and modernization of legacy Teradata workloads to Azure-based solutions, leveraging Azure Data Factory (ADF), Databricks, and Azure DevOps (ADO). Architect and implement scalable data pipelines aligned with the Bronze, Silver, and Gold data architecture, ensuring data quality, governance, and performance optimization across all layers.
Key Responsibilities
Architect and execute the migration of Teradata workloads to Azure Databricks and Synapse Analytics, designing an optimized Lakehouse approach.
Design and manage Bronze, Silver, and Gold data layers to ensure structured data transformation:
o Bronze Layer: Raw ingestion of structured and unstructured data from Teradata and other sources.
o Silver Layer: Data cleaning, deduplication, and standardization for analytical processing.
o Gold Layer: Curated, business-ready datasets optimized for reporting, AI/ML, and self-service analytics.
Build and scale ETL/ELT pipelines using ADF, Databricks (PySpark), and SQL for batch and real-time processing.
Implement CI/CD pipelines in Azure DevOps (ADO) to streamline deployments and version control.
Define and enforce data governance, security, and compliance best practices in Azure.
Develop monitoring and automation frameworks to track key operational KPIs and pipeline performance.
Collaborate with Data Science, Product, and Business teams to align data solutions with business objectives.
Optimize query performance and cost efficiency within Azure Synapse Analytics and Databricks.
Implement Infrastructure-as-Code (IaC) using Terraform or Bicep for cloud resource management.
Support real-time data streaming using Tibco, Kafka or Azure Event Hubs.
Maintain and enhance technical documentation for solution knowledge sharing.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Required Skills & Experience
10+ years in data engineering or architecture. 4+ years working with Azure cloud technologies (Databricks, ADF, Synapse, Event Hubs). Strong hands-on experience with Teradata (Legacy RDBMS) migration to Azure. Expertise with Teradata Tools and Utilities (TPT,BTEQ,QG,NOS) Expertise in Bronze, Silver, and Gold data architecture for efficient data processing. Proficiency in Informatica, ADO, SQL, Python, and Spark (PySpark) for ETL/ELT and transformation. Experience designing and optimizing Azure DevOps (ADO) CI/CD pipelines. Knowledge of data governance, security, and compliance best practices. Familiarity with Terraform or Infrastructure-as-Code (IaC) for cloud provisioning. Background in real-time data streaming (Kafka,Tibco, Event Hubs) and supporting AI/ML workloads in production. Strong understanding of Agile development methodologies.
Nice to Have Skills & Experience
Local to Dallas, TX
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Required Skills & Experience
10+ years in data engineering or architecture. 4+ years working with Azure cloud technologies (Databricks, ADF, Synapse, Event Hubs). Strong hands-on experience with Teradata (Legacy RDBMS) migration to Azure. Expertise with Teradata Tools and Utilities (TPT,BTEQ,QG,NOS) Expertise in Bronze, Silver, and Gold data architecture for efficient data processing. Proficiency in Informatica, ADO, SQL, Python, and Spark (PySpark) for ETL/ELT and transformation. Experience designing and optimizing Azure DevOps (ADO) CI/CD pipelines. Knowledge of data governance, security, and compliance best practices. Familiarity with Terraform or Infrastructure-as-Code (IaC) for cloud provisioning. Background in real-time data streaming (Kafka,Tibco, Event Hubs) and supporting AI/ML workloads in production. Strong understanding of Agile development methodologies.
Nice to Have Skills & Experience
Local to Dallas, TX
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.