Wfnen
Data Sc Engineer
Wfnen, Snowflake, Arizona, United States, 85937
This is an exciting opportunity to join a dynamic and growing organization, working at the forefront of technology trends and developments in the social impact sector. Wadhwani Center for Government Digital Transformation (WGDT) works with the government ministries and state departments in India with a mission of “ Enabling digital transformation to enhance the impact of government policy, initiatives and programs .”We are seeking a highly motivated and detail-oriented individual to join our team as a
Data Scientist
with experience in data analysis, machine learning, and statistical modeling to uncover insights, inform decision-making, and contribute to the successful implementation of digital government policies and programs. You will play a key role in driving innovation and optimizing operations across various government ministries and state departments in India.Key Responsibilities:Data Pipeline Development:
Design, build, and optimize data pipelines to extract, transform, and load (ETL) data from different sources, such as databases, APIs, and streaming platforms. Ensure data pipelines are reliable, scalable, and performant.Data Modeling:
Design and implement data models that facilitate efficient storage and retrieval of data. Ensure data integrity and consistency across databases and data systems.Data Warehousing:
Set up and maintain data warehousing solutions to provide a centralized repository of structured and unstructured data for analytics and reporting purposes.Data Quality and Governance:
Implement data quality checks and data governance processes to ensure data accuracy, completeness, and compliance with relevant regulations.Database Management:
Administer and optimize databases, both relational and NoSQL, to manage large volumes of data effectively.Data Security:
Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.Data Integration:
Integrate data from various sources and systems to create a unified view of the Ministry's data landscape.Data Migration:
Plan and execute data migration projects to transfer data between systems while ensuring data consistency and minimal downtime.Collaboration:
Collaborate with cross-functional teams and other stakeholders to understand data requirements and deliver appropriate data solutions.Documentation:
Maintain comprehensive documentation of data pipelines, data models, and data-related processes for reference and knowledge sharing.Desired Skills/Competencies:Education:
A Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science, or equivalent with at least 5 years of experience.Database Management:
Strong expertise in working with databases, such as SQL databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).Big Data Technologies:
Familiarity with big data technologies, such as Apache Hadoop, Spark, and related ecosystem components, for processing and analyzing large-scale datasets.ETL Tools:
Experience with ETL tools (e.g., Apache NiFi, Talend, Apache Airflow) for designing and orchestrating data workflows.Data Modeling and Warehousing:
Knowledge of data modeling techniques and experience with data warehousing solutions (e.g., Amazon Redshift, Google BigQuery, Snowflake).Data Governance and Security:
Understanding of data governance principles and best practices for ensuring data quality and security.Cloud Computing:
Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services for scalable and cost-effective data storage and processing.Streaming Data Processing:
Familiarity with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink) for handling streaming data.KPIs:Data Pipeline Efficiency:
Creation of efficient data pipelines measured through tracking the time taken to extract, transform, and load data from various sources into the data warehouse or data lake.Data Quality:
Quality of data created measured by tracking data accuracy, completeness, consistency, and timeliness.Data Security and Compliance:
Ensure data security and compliance with data privacy regulations by monitoring access controls, data encryption, and adherence to data governance policies.Data Integration Success Rate:
Measure the success rate of data integration projects to assess how well data from different sources is integrated into a unified view.Data Documentation:
Evaluate the completeness and accuracy of data documentation, including data models, data dictionaries, and data flow diagrams.
#J-18808-Ljbffr
Data Scientist
with experience in data analysis, machine learning, and statistical modeling to uncover insights, inform decision-making, and contribute to the successful implementation of digital government policies and programs. You will play a key role in driving innovation and optimizing operations across various government ministries and state departments in India.Key Responsibilities:Data Pipeline Development:
Design, build, and optimize data pipelines to extract, transform, and load (ETL) data from different sources, such as databases, APIs, and streaming platforms. Ensure data pipelines are reliable, scalable, and performant.Data Modeling:
Design and implement data models that facilitate efficient storage and retrieval of data. Ensure data integrity and consistency across databases and data systems.Data Warehousing:
Set up and maintain data warehousing solutions to provide a centralized repository of structured and unstructured data for analytics and reporting purposes.Data Quality and Governance:
Implement data quality checks and data governance processes to ensure data accuracy, completeness, and compliance with relevant regulations.Database Management:
Administer and optimize databases, both relational and NoSQL, to manage large volumes of data effectively.Data Security:
Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.Data Integration:
Integrate data from various sources and systems to create a unified view of the Ministry's data landscape.Data Migration:
Plan and execute data migration projects to transfer data between systems while ensuring data consistency and minimal downtime.Collaboration:
Collaborate with cross-functional teams and other stakeholders to understand data requirements and deliver appropriate data solutions.Documentation:
Maintain comprehensive documentation of data pipelines, data models, and data-related processes for reference and knowledge sharing.Desired Skills/Competencies:Education:
A Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science, or equivalent with at least 5 years of experience.Database Management:
Strong expertise in working with databases, such as SQL databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).Big Data Technologies:
Familiarity with big data technologies, such as Apache Hadoop, Spark, and related ecosystem components, for processing and analyzing large-scale datasets.ETL Tools:
Experience with ETL tools (e.g., Apache NiFi, Talend, Apache Airflow) for designing and orchestrating data workflows.Data Modeling and Warehousing:
Knowledge of data modeling techniques and experience with data warehousing solutions (e.g., Amazon Redshift, Google BigQuery, Snowflake).Data Governance and Security:
Understanding of data governance principles and best practices for ensuring data quality and security.Cloud Computing:
Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services for scalable and cost-effective data storage and processing.Streaming Data Processing:
Familiarity with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink) for handling streaming data.KPIs:Data Pipeline Efficiency:
Creation of efficient data pipelines measured through tracking the time taken to extract, transform, and load data from various sources into the data warehouse or data lake.Data Quality:
Quality of data created measured by tracking data accuracy, completeness, consistency, and timeliness.Data Security and Compliance:
Ensure data security and compliance with data privacy regulations by monitoring access controls, data encryption, and adherence to data governance policies.Data Integration Success Rate:
Measure the success rate of data integration projects to assess how well data from different sources is integrated into a unified view.Data Documentation:
Evaluate the completeness and accuracy of data documentation, including data models, data dictionaries, and data flow diagrams.
#J-18808-Ljbffr