SoHo Dragon
Senior Data Engineer
SoHo Dragon, New York, New York, us, 10261
SoHo Dragon represents a large non-profit client that needs to hire a Senior Data Engineer. This role is 100% remote.
Typical Responsibilities and Activities for this Role:
Data Architecture and Integration - Integrate diverse data sources by designing and implementing scalable high performance data pipelines and infrastructure.Build ETL processes using a variety of methods to facilitate efficient movementof data between systemsIncorporate best practices to ensure data pipelines are robust and include errorhandling and alerting.Data Model: Design data models that turn complex organizational data into easy to use data structures ensuring seamless flow for Client's operational, analytics and reporting needs.Use DBT or Snowflake native capabilities to transform data and ensure data is accurate and up to date.Optimize data structures and queries to ensure efficient data storage and retrieval.Data Quality: Implement and enforce data quality standards to ensure the accuracy and reliability of educational data used in academic assessments globally.Collaboration: Collaborate with cross-functional teams, including analysts and business stakeholders, to understand unique data requirements for data processing or analytics needs.Security: Implement and maintain data security measures to protect sensitive information, ensuring compliance with international data privacy regulations.Snowflake Expertise: Possess expertise in Snowflake for effective data warehousing, transformations, data sharing, and structured management in a cloud-based environment.Documentation: Create and maintain comprehensive documentation for data engineering processes, systems, and workflowsTeam Leadership: Provide technical leadership and mentorship to junior members of the data engineering team, fostering a collaborative environment dedicated to excellence in handling educational data.Tableau: Utilize Tableau to deliver data insights using impactful visualizations in a meaningful and actionable manner.
Requirements:• Education: Bachelor's or Master's Degree in Computer Science, Information Technology,or a related field, with a strong emphasis on Data related technologies.• Experience: Significant experience (usually 5+ years) in data engineering roles, with afocus on designing and implementing scalable data solutions.• Programming Skills: Proficiency in programming languages such as Python, Java, orScala.• Database Knowledge: Expertise in working with both SQL and NoSQL databases, withhands-on experience in database design and optimization. Proficiency in Snowflake isrequired.• Knowledgeable of Machine Learning concepts and an interest to grow in the area.• Cloud Platforms: Experience with cloud platforms like Azure, AWS, or Google CloudPlatform, and proficiency in leveraging cloud-based.• Version Control: Knowledge of version control systems like Git or Azure DevOps formanaging the codebase.• Problem-Solving Skills: Strong analytical and problem-solving skills to address complexdata engineering challenges specific to educational data.• Communication: Excellent communication skills to effectively collaborate with crossfunctional teams and non-technical stakeholders.• Adaptability: Ability to adapt to evolving technologies and industry best practices in thefield of data engineering.Nice to Have:• Experience using TIBCO EBX• Experience using Boomi or other low code ETL tools• Experience using Azure for Data Engineering tasks i.e. function apps and other Azurefunctionality that can be used to build data pipelines
Typical Responsibilities and Activities for this Role:
Data Architecture and Integration - Integrate diverse data sources by designing and implementing scalable high performance data pipelines and infrastructure.Build ETL processes using a variety of methods to facilitate efficient movementof data between systemsIncorporate best practices to ensure data pipelines are robust and include errorhandling and alerting.Data Model: Design data models that turn complex organizational data into easy to use data structures ensuring seamless flow for Client's operational, analytics and reporting needs.Use DBT or Snowflake native capabilities to transform data and ensure data is accurate and up to date.Optimize data structures and queries to ensure efficient data storage and retrieval.Data Quality: Implement and enforce data quality standards to ensure the accuracy and reliability of educational data used in academic assessments globally.Collaboration: Collaborate with cross-functional teams, including analysts and business stakeholders, to understand unique data requirements for data processing or analytics needs.Security: Implement and maintain data security measures to protect sensitive information, ensuring compliance with international data privacy regulations.Snowflake Expertise: Possess expertise in Snowflake for effective data warehousing, transformations, data sharing, and structured management in a cloud-based environment.Documentation: Create and maintain comprehensive documentation for data engineering processes, systems, and workflowsTeam Leadership: Provide technical leadership and mentorship to junior members of the data engineering team, fostering a collaborative environment dedicated to excellence in handling educational data.Tableau: Utilize Tableau to deliver data insights using impactful visualizations in a meaningful and actionable manner.
Requirements:• Education: Bachelor's or Master's Degree in Computer Science, Information Technology,or a related field, with a strong emphasis on Data related technologies.• Experience: Significant experience (usually 5+ years) in data engineering roles, with afocus on designing and implementing scalable data solutions.• Programming Skills: Proficiency in programming languages such as Python, Java, orScala.• Database Knowledge: Expertise in working with both SQL and NoSQL databases, withhands-on experience in database design and optimization. Proficiency in Snowflake isrequired.• Knowledgeable of Machine Learning concepts and an interest to grow in the area.• Cloud Platforms: Experience with cloud platforms like Azure, AWS, or Google CloudPlatform, and proficiency in leveraging cloud-based.• Version Control: Knowledge of version control systems like Git or Azure DevOps formanaging the codebase.• Problem-Solving Skills: Strong analytical and problem-solving skills to address complexdata engineering challenges specific to educational data.• Communication: Excellent communication skills to effectively collaborate with crossfunctional teams and non-technical stakeholders.• Adaptability: Ability to adapt to evolving technologies and industry best practices in thefield of data engineering.Nice to Have:• Experience using TIBCO EBX• Experience using Boomi or other low code ETL tools• Experience using Azure for Data Engineering tasks i.e. function apps and other Azurefunctionality that can be used to build data pipelines