Insight Global
Senior ETL Developer
Insight Global, Atlanta, Georgia, United States, 30383
Job Description
We are seeking a highly skilled and experienced Senior ETL Developer with expertise in Snowflake, Databricks Synapse to join our dynamic team. The ideal candidate will have a strong background in ETL processes, data integration, and data warehousing, with a focus on Snowflakes cloud data platform. The Data Engineer will be responsible for building and managing data pipelines to support the PoC and pilot phase. This role involves extracting, transforming, and loading data from diverse sources into the proposed architecture while ensuring high-quality and reliable data flows. The individual will also optimize pipeline performance to validate scalability.
Key Responsibilities:
Design, develop, and maintain ETL processes to support data integration and data warehousing solutions.
Implement and optimize ETL workflows using Snowflake and other relevant technologies.
Collaborate with data architects, data engineers, and business stakeholders to understand data requirements and translate them into effective ETL solutions.
Ensure data quality, integrity, and consistency across all ETL processes.
Automate data ingestion workflows using tools like Talend or Apache NiFi.
Test pipeline scalability under simulated workloads to validate platform performance.
Collaborate with the Data Architect to align pipelines with overall architecture design.
Troubleshoot pipeline issues during testing phases and provide solutions for optimization.
Monitor and troubleshoot ETL processes to ensure optimal performance and reliability.
Provide technical leadership and mentorship to junior ETL developers.
Stay up to date with the latest trends and best practices in ETL development and Snowflake, Databricks, Azure technologies.
Set up and configure cloud infrastructure required for hosting the data platform on hyper-scalers.
Provision compute resources and storage resources to support data ingestion, transformation, and querying.
Ensure scalability by configuring auto-scaling features to handle variable workloads during testing.
Document pipeline configurations for future reference during full-scale implementation.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
8+ years of experience in ETL development, data integration, and data warehousing.
Extensive experience with Snowflake, including design, implementation, and optimization.
Proficiency in ETL tools and technologies (e.g., Alteryx , Informatica, Talend, SSIS).
Strong understanding of SQL and other data manipulation languages.
Experience working with diverse datasets from SQL Server, Oracle, DOMO, etc.
Experience with cloud platforms such as AWS, Azure, or Google Cloud.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills. Bachelors or Masters degree in Computer Science, Information Technology, or a related field.
Snowflake, Databricks and Azure Datalake certification.
Experience with data governance and data security best practices.
Familiarity with other data platforms and technologies
Experience securing hybrid environments during transitions between legacy systems and cloud platforms.
Knowledge of compliance requirements relevant to financial services
Proficiency in cost optimization strategies for cloud resources. null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
We are seeking a highly skilled and experienced Senior ETL Developer with expertise in Snowflake, Databricks Synapse to join our dynamic team. The ideal candidate will have a strong background in ETL processes, data integration, and data warehousing, with a focus on Snowflakes cloud data platform. The Data Engineer will be responsible for building and managing data pipelines to support the PoC and pilot phase. This role involves extracting, transforming, and loading data from diverse sources into the proposed architecture while ensuring high-quality and reliable data flows. The individual will also optimize pipeline performance to validate scalability.
Key Responsibilities:
Design, develop, and maintain ETL processes to support data integration and data warehousing solutions.
Implement and optimize ETL workflows using Snowflake and other relevant technologies.
Collaborate with data architects, data engineers, and business stakeholders to understand data requirements and translate them into effective ETL solutions.
Ensure data quality, integrity, and consistency across all ETL processes.
Automate data ingestion workflows using tools like Talend or Apache NiFi.
Test pipeline scalability under simulated workloads to validate platform performance.
Collaborate with the Data Architect to align pipelines with overall architecture design.
Troubleshoot pipeline issues during testing phases and provide solutions for optimization.
Monitor and troubleshoot ETL processes to ensure optimal performance and reliability.
Provide technical leadership and mentorship to junior ETL developers.
Stay up to date with the latest trends and best practices in ETL development and Snowflake, Databricks, Azure technologies.
Set up and configure cloud infrastructure required for hosting the data platform on hyper-scalers.
Provision compute resources and storage resources to support data ingestion, transformation, and querying.
Ensure scalability by configuring auto-scaling features to handle variable workloads during testing.
Document pipeline configurations for future reference during full-scale implementation.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
8+ years of experience in ETL development, data integration, and data warehousing.
Extensive experience with Snowflake, including design, implementation, and optimization.
Proficiency in ETL tools and technologies (e.g., Alteryx , Informatica, Talend, SSIS).
Strong understanding of SQL and other data manipulation languages.
Experience working with diverse datasets from SQL Server, Oracle, DOMO, etc.
Experience with cloud platforms such as AWS, Azure, or Google Cloud.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills. Bachelors or Masters degree in Computer Science, Information Technology, or a related field.
Snowflake, Databricks and Azure Datalake certification.
Experience with data governance and data security best practices.
Familiarity with other data platforms and technologies
Experience securing hybrid environments during transitions between legacy systems and cloud platforms.
Knowledge of compliance requirements relevant to financial services
Proficiency in cost optimization strategies for cloud resources. null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.