Tata Consultancy Services
Staff Data Engineer
Tata Consultancy Services, Waltham, Massachusetts, United States, 02254
Technical/Functional SkillsStaff Data Engineering role - someone with hands-on experience with cloud infrastructure, PySpark/Spark (strong), CICD, basic software engineering, data engineering, etc. This role will be responsible for handling multiple products and multiple stakeholders, so we are looking for a candidate with lots of hands-on experience who enjoys learning and sharing the latest trends in the tech space related to DE.Tech Stack:
Snowflake, Databricks, Python, PySpark, SQL, and Azure.Experience RequiredAs a Staff Data Engineer, the candidate will be part of a Data Engineering team that is focused on making critical data available to our business teams. Using the agile framework, the candidate will build end-to-end pipelines based on rigorous engineering standards and coding practices to deliver data that is accessible and of the highest quality. A Staff Data Engineer will also contribute to the modernization of our architecture and tools to help increase our output, scalability, and speed.Roles & ResponsibilitiesLead the Data Engineering Team to develop, test, document, and support scalable data pipelines.Build out new data integrations including APIs to support continuing increases in data volume and complexity.Establish and follow data governance processes and guidelines to ensure data availability, usability, consistency, integrity, and security.Build and implement scalable solutions that align to our data governance standards and architectural road maps for data integrations, data storage, reporting, and analytic solutions.Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision-making across the organization.Design and develop data integrations and a data quality framework. Write unit/integration/functional tests and document work.Design, implement, and automate deployment of our distributed system for collecting and processing streaming events from multiple sources.Perform data analysis needed to troubleshoot data-related issues and aid in the resolution of data issues.Guide and mentor junior engineers on coding best practices and optimization.QualificationsEducation: 4-year college degree or equivalent combination of education and experience. Prefer an academic background in Computer Science, Mathematics, Statistics, or related technical field.8 years of relevant work experience in analytics, data engineering, business intelligence, or related field.Skilled in object-oriented programming (Python in particular).Strong experience in Python, PySpark, and SQL.Strong experience in Databricks & Snowflake.Experience developing integrations across multiple systems and APIs.Experience with or knowledge of Agile software development methodologies.Experience with cloud-based databases, specifically Azure technologies (e.g., Azure Data Lake, ADF, Azure DevOps, and Azure Functions).Experience using SQL queries as well as writing and perfecting SQL queries in a business environment with large-scale, complex datasets.Experience with data warehouse technologies and creating ETL and/or ELT jobs.Excellent problem-solving and troubleshooting skills.Process-oriented with great documentation skills.Experience designing data schemas and operating SQL/NoSQL database systems is a plus.
#J-18808-Ljbffr
Snowflake, Databricks, Python, PySpark, SQL, and Azure.Experience RequiredAs a Staff Data Engineer, the candidate will be part of a Data Engineering team that is focused on making critical data available to our business teams. Using the agile framework, the candidate will build end-to-end pipelines based on rigorous engineering standards and coding practices to deliver data that is accessible and of the highest quality. A Staff Data Engineer will also contribute to the modernization of our architecture and tools to help increase our output, scalability, and speed.Roles & ResponsibilitiesLead the Data Engineering Team to develop, test, document, and support scalable data pipelines.Build out new data integrations including APIs to support continuing increases in data volume and complexity.Establish and follow data governance processes and guidelines to ensure data availability, usability, consistency, integrity, and security.Build and implement scalable solutions that align to our data governance standards and architectural road maps for data integrations, data storage, reporting, and analytic solutions.Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision-making across the organization.Design and develop data integrations and a data quality framework. Write unit/integration/functional tests and document work.Design, implement, and automate deployment of our distributed system for collecting and processing streaming events from multiple sources.Perform data analysis needed to troubleshoot data-related issues and aid in the resolution of data issues.Guide and mentor junior engineers on coding best practices and optimization.QualificationsEducation: 4-year college degree or equivalent combination of education and experience. Prefer an academic background in Computer Science, Mathematics, Statistics, or related technical field.8 years of relevant work experience in analytics, data engineering, business intelligence, or related field.Skilled in object-oriented programming (Python in particular).Strong experience in Python, PySpark, and SQL.Strong experience in Databricks & Snowflake.Experience developing integrations across multiple systems and APIs.Experience with or knowledge of Agile software development methodologies.Experience with cloud-based databases, specifically Azure technologies (e.g., Azure Data Lake, ADF, Azure DevOps, and Azure Functions).Experience using SQL queries as well as writing and perfecting SQL queries in a business environment with large-scale, complex datasets.Experience with data warehouse technologies and creating ETL and/or ELT jobs.Excellent problem-solving and troubleshooting skills.Process-oriented with great documentation skills.Experience designing data schemas and operating SQL/NoSQL database systems is a plus.
#J-18808-Ljbffr