TechnoGen
Sr. Data Engineer
TechnoGen, Atlanta, Georgia, United States, 30383
Title:: Sr.
Data Engineer (Teradata ETL with AWS exp.)
Location::
Atlanta, GA (Hybrid role, 3x days onsite/week)
Type of Hire::
Contract (c2c/w2)
Duration::
12 months
Job ID: 38955
Qualifications:
Required Skillsets:
Teradata Expertise:• In-depth knowledge of Teradata architecture and components (e.g., AMP, Nodes, Parsing Engines, etc.).• Proficiency in Teradata SQL and Teradata utilities.
SQL and Scripting:• Strong SQL skills for querying and managing relational databases.• Scripting experience (e.g., Shell, Python) for automating tasks.• Know tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).
ETL Tools:• Experience with ETL tools and technologies used in Teradata environments.
Performance Optimization:• Knowledge of query optimization techniques, indexing, partitioning, and workload management in Teradata.
Data Warehousing Concepts:• Strong understanding of data warehousing principles, including star/snowflake schemas, OLAP, and ETL processes.
Cloud Platform / AWS Expertise:• Proficiency with core AWS services: EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, etc.• Experience with AWS networking (VPC, subnets, security groups, NACLs) and security management.• Familiarity with AWS storage options, such as S3, EFS, and Glacier, in the context of large-scale data storage and retrieval.
Teradata and AWS Integration:• Experience deploying and managing Teradata in AWS cloud environments.• Knowledge of AWS services that enhance Teradata performance, such as using S3 for staging data or integrating with Redshift for analytics.
Automation and Infrastructure as Code (IaC):• Hands-on experience with AWS CloudFormation, Terraform, or similar tools for automating infrastructure deployment.• Proficiency in automating ETL pipelines and data processing workflows using AWS Lambda, Step Functions, or AWS Glue.
Data Security in AWS:• Understanding of encryption and security protocols for data stored and processed in AWS.• Experience implementing compliance standards such as GDPR, HIPAA, or other data protection regulations in cloud environments.
Key Responsibilities:
Database Management:• Design, implement, and maintain Teradata databases.• Optimize and tune queries for performance.• Monitor and troubleshoot database issues.
ETL Development:• Develop and maintain ETL (Extract, Transform, Load) processes for data integration and transformation.• Use tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).
Data Modeling:• Create and maintain data models to support business processes.• Work with stakeholders to understand data requirements and design solutions.
Performance Tuning:• Identify and resolve performance bottlenecks in SQL queries, system configuration, or data design.• Regularly monitor and optimize system performance.
Data Security & Governance:• Implement data security best practices.• Ensure compliance with data governance policies and regulations.
Cloud Integration and Management:• Implement and manage Teradata on AWS infrastructure, leveraging AWS services for scalability and performance.• Manage data migration from on-premises Teradata environments to AWS cloud-based Teradata instances.• Ensure seamless integration between Teradata and AWS services like S3, Redshift, RDS, and Lambda.
Cloud Infrastructure:• Design and deploy cloud infrastructure (VPCs, EC2 instances, S3 buckets) to support Teradata environments.• Implement networking and security best practices in AWS to ensure secure data access and storage.
Automation and DevOps:• Automate infrastructure provisioning, scaling, and monitoring using AWS tools like CloudFormation, Terraform, or AWS CLI.• Use AWS Lambda for automating ETL processes or handling specific workloads.
Data Warehousing and Analytics:• Leverage AWS data services (e.g., Redshift, Athena, Glue) to complement Teradata workloads.• Implement data pipelines using AWS services for ETL processes, integrating data from various sources into Teradata.
Security and Compliance:• Implement AWS security best practices, such as IAM roles and policies, to secure Teradata environments in the cloud.• Ensure compliance with relevant data governance policies, including encryption at rest and in transit.
Collaboration:• Work closely with data architects, analysts, and other stakeholders to understand data needs and deliver solutions.
Data Engineer (Teradata ETL with AWS exp.)
Location::
Atlanta, GA (Hybrid role, 3x days onsite/week)
Type of Hire::
Contract (c2c/w2)
Duration::
12 months
Job ID: 38955
Qualifications:
Required Skillsets:
Teradata Expertise:• In-depth knowledge of Teradata architecture and components (e.g., AMP, Nodes, Parsing Engines, etc.).• Proficiency in Teradata SQL and Teradata utilities.
SQL and Scripting:• Strong SQL skills for querying and managing relational databases.• Scripting experience (e.g., Shell, Python) for automating tasks.• Know tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).
ETL Tools:• Experience with ETL tools and technologies used in Teradata environments.
Performance Optimization:• Knowledge of query optimization techniques, indexing, partitioning, and workload management in Teradata.
Data Warehousing Concepts:• Strong understanding of data warehousing principles, including star/snowflake schemas, OLAP, and ETL processes.
Cloud Platform / AWS Expertise:• Proficiency with core AWS services: EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, etc.• Experience with AWS networking (VPC, subnets, security groups, NACLs) and security management.• Familiarity with AWS storage options, such as S3, EFS, and Glacier, in the context of large-scale data storage and retrieval.
Teradata and AWS Integration:• Experience deploying and managing Teradata in AWS cloud environments.• Knowledge of AWS services that enhance Teradata performance, such as using S3 for staging data or integrating with Redshift for analytics.
Automation and Infrastructure as Code (IaC):• Hands-on experience with AWS CloudFormation, Terraform, or similar tools for automating infrastructure deployment.• Proficiency in automating ETL pipelines and data processing workflows using AWS Lambda, Step Functions, or AWS Glue.
Data Security in AWS:• Understanding of encryption and security protocols for data stored and processed in AWS.• Experience implementing compliance standards such as GDPR, HIPAA, or other data protection regulations in cloud environments.
Key Responsibilities:
Database Management:• Design, implement, and maintain Teradata databases.• Optimize and tune queries for performance.• Monitor and troubleshoot database issues.
ETL Development:• Develop and maintain ETL (Extract, Transform, Load) processes for data integration and transformation.• Use tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).
Data Modeling:• Create and maintain data models to support business processes.• Work with stakeholders to understand data requirements and design solutions.
Performance Tuning:• Identify and resolve performance bottlenecks in SQL queries, system configuration, or data design.• Regularly monitor and optimize system performance.
Data Security & Governance:• Implement data security best practices.• Ensure compliance with data governance policies and regulations.
Cloud Integration and Management:• Implement and manage Teradata on AWS infrastructure, leveraging AWS services for scalability and performance.• Manage data migration from on-premises Teradata environments to AWS cloud-based Teradata instances.• Ensure seamless integration between Teradata and AWS services like S3, Redshift, RDS, and Lambda.
Cloud Infrastructure:• Design and deploy cloud infrastructure (VPCs, EC2 instances, S3 buckets) to support Teradata environments.• Implement networking and security best practices in AWS to ensure secure data access and storage.
Automation and DevOps:• Automate infrastructure provisioning, scaling, and monitoring using AWS tools like CloudFormation, Terraform, or AWS CLI.• Use AWS Lambda for automating ETL processes or handling specific workloads.
Data Warehousing and Analytics:• Leverage AWS data services (e.g., Redshift, Athena, Glue) to complement Teradata workloads.• Implement data pipelines using AWS services for ETL processes, integrating data from various sources into Teradata.
Security and Compliance:• Implement AWS security best practices, such as IAM roles and policies, to secure Teradata environments in the cloud.• Ensure compliance with relevant data governance policies, including encryption at rest and in transit.
Collaboration:• Work closely with data architects, analysts, and other stakeholders to understand data needs and deliver solutions.