Job Description

Databricks Platform Administrator

Location : Remote

Databricks Platform Administration

• Deploy, configure, and maintain Azure Databricks workspaces
• Manage clusters , pools , and jobs for optimal performance.
• Handle Databricks Runtime upgrades and compatibility testing.
• Configure Databricks notebooks, Repos, and Delta Lake for efficient workflows.

Security & Access Control:

• Implement Identity and Access Management (IAM) using Azure AD / Azure IAM / SCIM .
• Set up and enforce Role-Based Access Control (RBAC) and network security .
• Ensure compliance with data governance policies (Unity Catalog, creation of Catalogs, schema etc.).
• Applying grants of catalogs and schemas
• Manage personal access tokens (PATs) and OAuth-based authentication .

Performance Monitoring & Optimization:

• Monitor cluster performance, auto-scaling, and cost optimization .
• Analyze and fine-tune Spark jobs to improve execution efficiency.
• Use Databricks SQL for querying and performance diagnostics.

Integration & Automation:

• Integrate Databricks with Azure Data Factory, Synapse, Snowflake, Kafka, and other data sources .
• Automate cluster management using Terraform.
• Implement CI/CD pipelines for Databricks notebooks using GitHub or Azure DevOps .

Troubleshooting & Support:

• Investigate and resolve job failures, cluster crashes, and query performance issues .
• Provide L2/L3 support for Databricks infrastructure issues.
• Collaborate with Databricks support for bug fixes and escalations.

Job Tags

Similar Jobs