TechnoGen
Lead Data Engineer
TechnoGen, San Jose, California, United States, 95199
Lead Data Engineer
REQUIRED SKILLS• Banking industry experience
Technical Lead specific to Data• Data Operations
Informatica / IICSSnowflakeOracle
We are looking for candidates with strong Data Logic experience. Please see below Job description for this position.
Required Azure Senior Data Engineer, who'll play a crucial role in designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your expertise will enable efficient data processing, storage, and integration, empowering data-driven decision-making within the organization. Here are the key responsibilities:
Data Pipeline Design and Implementation:
1. Create robust data pipelines using Azure Data Factory to ingest, transform, and move data across various sources and destinations.
2. Optimize data workflows, ensuring scalability, reliability, and performance.
Data Storage Solutions:
1. Design and maintain data storage solutions, including:
Azure SQL Database: Develop and optimize database schemas, tables, and indexes.
Azure Data Lake: Organize and manage large-scale data lakes.
Azure Blob Storage: Store unstructured data efficiently.
Data Flow Design:
1. Collaborate with stakeholders to understand data requirements.
2. Design efficient data flows, ensuring data quality, consistency, and compliance.
Azure Logic Apps and Function Apps:
1. Leverage Azure Logic Apps for workflow automation, integrating services seamlessly.
2. Utilize Azure Function Apps for serverless compute, event-driven processing, and custom logic.
Performance Monitoring and Optimization:
1. Monitor data pipelines and storage solutions for performance bottlenecks.
2. Optimize query performance, indexing, and data partitioning.
Collaboration and Documentation:
1. Work closely with data scientists, analysts, and other teams.
2. Document data architecture, design decisions, and best practices.
REQUIRED SKILLS• Banking industry experience
Technical Lead specific to Data• Data Operations
Informatica / IICSSnowflakeOracle
We are looking for candidates with strong Data Logic experience. Please see below Job description for this position.
Required Azure Senior Data Engineer, who'll play a crucial role in designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your expertise will enable efficient data processing, storage, and integration, empowering data-driven decision-making within the organization. Here are the key responsibilities:
Data Pipeline Design and Implementation:
1. Create robust data pipelines using Azure Data Factory to ingest, transform, and move data across various sources and destinations.
2. Optimize data workflows, ensuring scalability, reliability, and performance.
Data Storage Solutions:
1. Design and maintain data storage solutions, including:
Azure SQL Database: Develop and optimize database schemas, tables, and indexes.
Azure Data Lake: Organize and manage large-scale data lakes.
Azure Blob Storage: Store unstructured data efficiently.
Data Flow Design:
1. Collaborate with stakeholders to understand data requirements.
2. Design efficient data flows, ensuring data quality, consistency, and compliance.
Azure Logic Apps and Function Apps:
1. Leverage Azure Logic Apps for workflow automation, integrating services seamlessly.
2. Utilize Azure Function Apps for serverless compute, event-driven processing, and custom logic.
Performance Monitoring and Optimization:
1. Monitor data pipelines and storage solutions for performance bottlenecks.
2. Optimize query performance, indexing, and data partitioning.
Collaboration and Documentation:
1. Work closely with data scientists, analysts, and other teams.
2. Document data architecture, design decisions, and best practices.