Insight Global
Data Engineer (India - Hybrid/Remote)
Insight Global, Rahway, New Jersey, us, 07065
Job Description
Design, develop and maintain data pipelines to extract data from a variety of sources and populate data lake and data warehouse
Develop the various data transformation rules and data modelling capabilities
Collaborate with Product Analyst, Data Scientists, and Machine Learning Engineers to identify and transform data to make data understandable.
Experience in implementing data quality checks, maintain data catalogues, use orchestration, logging, and monitoring tools, data analysis, information architecture concept and implementation, Git for version control and understand various branching strategies
Use test driven development methodology when building ELT/ETL pipelines
Good SDLC documentation skills required for this position.
Good communication and collaboration skills, and ability to collaborate with other teams to develop solutions, demonstrate growth mindset and ability to work with enterprise teams.
Experience with DevSecOps practices like continuous integration (CI) and continuous delivery (CD), source code version control (e.g., Git), infrastructure-as-code (e.g., CloudFormation, Terraform), and containers (e.g., Docker).
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
5-8+ years of Data Engineering experience based out of India
Solid experience with Amazon Web Services (AWS): S3, IAM, Redshift, Sagemaker, Glue, Lambda, Step Functions, CloudWatch), particularly the serverless services (e.g., Lambda, Glue, Step Functions)
Experience with platforms like Databricks, Dataiku
Proficient in Python programming / Java, SQL Redshift preferred, Jenkins, CloudFormation, Terraform, Git, Docker, 2-3 years of Spark PySpark Cheminformatics and familiarity with the pharmaceutical research
Experience with Atlassian stack of tools for agile software development (e.g., Jira, Confluence). null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Design, develop and maintain data pipelines to extract data from a variety of sources and populate data lake and data warehouse
Develop the various data transformation rules and data modelling capabilities
Collaborate with Product Analyst, Data Scientists, and Machine Learning Engineers to identify and transform data to make data understandable.
Experience in implementing data quality checks, maintain data catalogues, use orchestration, logging, and monitoring tools, data analysis, information architecture concept and implementation, Git for version control and understand various branching strategies
Use test driven development methodology when building ELT/ETL pipelines
Good SDLC documentation skills required for this position.
Good communication and collaboration skills, and ability to collaborate with other teams to develop solutions, demonstrate growth mindset and ability to work with enterprise teams.
Experience with DevSecOps practices like continuous integration (CI) and continuous delivery (CD), source code version control (e.g., Git), infrastructure-as-code (e.g., CloudFormation, Terraform), and containers (e.g., Docker).
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
5-8+ years of Data Engineering experience based out of India
Solid experience with Amazon Web Services (AWS): S3, IAM, Redshift, Sagemaker, Glue, Lambda, Step Functions, CloudWatch), particularly the serverless services (e.g., Lambda, Glue, Step Functions)
Experience with platforms like Databricks, Dataiku
Proficient in Python programming / Java, SQL Redshift preferred, Jenkins, CloudFormation, Terraform, Git, Docker, 2-3 years of Spark PySpark Cheminformatics and familiarity with the pharmaceutical research
Experience with Atlassian stack of tools for agile software development (e.g., Jira, Confluence). null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.