Dice
Sr. Data Engineer
Dice, Princeton, New Jersey, us, 08543
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Global Data Management Inc, is seeking the following. Apply via Dice today!
Sr. Data Platform EngineerLocation:
Princeton, NJ (prefer onsite)
Duration:
6 months
Responsibilities:Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm.Integrate technical functionality, ensuring data accessibility, accuracy, and security.Architect the Unity Catalog to provide centralized access control, auditing, lineage, and data discovery capabilities across Databricks workspaces.Define and organize data assets (structured and unstructured) within the Unity Catalog.Enable data analysts and ETL engineers to discover and classify data, notebooks, dashboards, and files across clouds and platforms.Implement a single permission model for data and AI assets.Define access policies at a granular level (rows, columns, features) to ensure secure and consistent access management across workspaces and platforms.Leverage Delta Sharing to enable easy data sharing across regions and platforms.Ensure that data and AI assets can be securely shared with minimal replication, maintaining a unified experience for users.Utilize AI to automate monitoring, diagnose errors, and maintain data quality.Set up alerts for personally identifiable information (PII) detection and operational intelligence.Work closely with data scientists, analysts, and engineers to promote adoption of the Unity Catalog.Provide training and documentation to ensure effective usage and compliance with governance policies.
Skills:Designed data warehouse and data lake solutions along with data processing pipelines using PySpark in Databricks.Performed Data Modelling on Databricks [Delta Table] for transactional and analytical needs.Designed and developed pipelines to load data to Data Lake.Proficiency in Databricks Platform, including components like Databricks SQL, Delta Live Tables, Databricks Repos, and Task Orchestration.Deep understanding of data governance principles, especially related to data cataloging, access control, lineage, and metadata management.Strong SQL skills for querying and managing data.Ability to design and optimize data models for structured and unstructured data.Understanding of managing compute resources, including clusters and workspaces.Ability to adapt to changes and emerging trends in data engineering and governance.Hands-on development and configuration of Unity Catalog.
#J-18808-Ljbffr
Sr. Data Platform EngineerLocation:
Princeton, NJ (prefer onsite)
Duration:
6 months
Responsibilities:Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm.Integrate technical functionality, ensuring data accessibility, accuracy, and security.Architect the Unity Catalog to provide centralized access control, auditing, lineage, and data discovery capabilities across Databricks workspaces.Define and organize data assets (structured and unstructured) within the Unity Catalog.Enable data analysts and ETL engineers to discover and classify data, notebooks, dashboards, and files across clouds and platforms.Implement a single permission model for data and AI assets.Define access policies at a granular level (rows, columns, features) to ensure secure and consistent access management across workspaces and platforms.Leverage Delta Sharing to enable easy data sharing across regions and platforms.Ensure that data and AI assets can be securely shared with minimal replication, maintaining a unified experience for users.Utilize AI to automate monitoring, diagnose errors, and maintain data quality.Set up alerts for personally identifiable information (PII) detection and operational intelligence.Work closely with data scientists, analysts, and engineers to promote adoption of the Unity Catalog.Provide training and documentation to ensure effective usage and compliance with governance policies.
Skills:Designed data warehouse and data lake solutions along with data processing pipelines using PySpark in Databricks.Performed Data Modelling on Databricks [Delta Table] for transactional and analytical needs.Designed and developed pipelines to load data to Data Lake.Proficiency in Databricks Platform, including components like Databricks SQL, Delta Live Tables, Databricks Repos, and Task Orchestration.Deep understanding of data governance principles, especially related to data cataloging, access control, lineage, and metadata management.Strong SQL skills for querying and managing data.Ability to design and optimize data models for structured and unstructured data.Understanding of managing compute resources, including clusters and workspaces.Ability to adapt to changes and emerging trends in data engineering and governance.Hands-on development and configuration of Unity Catalog.
#J-18808-Ljbffr