Databricks ADMIN

Role : Databricks ADMIN (AWS)

Location : Remote
Detail JD:

Position Overview: The AWS Databricks Platform administrator is a technical role to design, implement and maintain process used to manage the organization's Databricks platform. The administrator will be responsible for facilitating the work of data analysts, data engineers and data scientists while maintaining best practices for security and compliance. Unity catalog will be used to implement access and identity policies. Infrastructure will be maintained using Terraform. The administrator will be responsible for assisting with compute issues, monitoring and alerting, and assuring the platform functions smoothly. This role requires great organization and communication skills.
Job responsibilities:

Manage and maintain role-based access to data and features in the Databricks Platform using Unity Catalog

Implement external access controls for outside teams using service principles, SQL warehouses and Delta Sharing

Work with platform users to solve problems and facilitate work

Create infrastructure for AWS and Databricks using Terraform: S3, IAM roles, Instance Profiles, KMS Keys

Improve processes and systems used to manage infrastructure, users and external access

Keep track of unused assets for pruning

Understand and implement best practices for security and compliance

Implement service principles and access tokens for external users

Understand how to utilize Databricks APIs to automate administrative tasks

Create queries and dashboards to monitor critical systems and processes

Create documentation for users and admins

Automate common admin tasks using Databricks notebooks

Manage workflow tags, cluster tags, workflow naming convention enforcement

Clusters health check and best practices implementation

Regular back up & recovery

Privilege reviews on users and resources