Hiring.Camp

Senior Azure Databricks Engineer (ESS-424)

Environmentalandsafetysolutionsllc

·

Today

Salary
$51 – $66/hr
Location
Glendale, Arizona
Type
Contract
Department
Engineering
Seniority
Senior
Experience
8+ years
Clearance
Required
Source
BetterTeam

Description

**U.S. Citizenship required and meet security-clearance requirements.

Roles & Responsibilities:
  • Collaborate with Data Platforms and Integration team in architecting and implementing scalable enterprise data platform solutions utilizing Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, and related Azure services.
  • Design and develop enterprise data engineering frameworks supporting batch, streaming, ELT/ETL, and real-time analytics workloads.
  • Administer and maintain Azure Databricks workspaces including cluster configuration, job scheduling, workspace administration, Unity Catalog implementation, secret management, compute policies, cluster policies, and workspace governance.
  • Define and implement enterprise security architecture including identity integration with Microsoft Entra ID (Azure AD), Role-Based Access Control (RBAC), Privileged Identity Management (PIM), data encryption, private networking, managed identities, Key Vault integration, and secure access patterns.
  • Establish enterprise data governance standards utilizing Unity Catalog including data lineage, data quality, metadata management, catalog organization, access policies, and audit capabilities.
  • Optimize Databricks workloads through performance tuning, Spark optimization, cluster sizing, workload isolation, caching strategies, partitioning, Delta optimization, Photon, and cost optimization best practices.
  • Design enterprise monitoring, logging, operational dashboards, alerting, and platform health monitoring utilizing Azure Monitor, Log Analytics, Application Insights, and Databricks system tables.
  • Develop reusable architecture patterns, reference implementations, development standards, naming conventions, coding standards, operational procedures, and platform documentation.
  • Support migration of legacy enterprise data warehouses, ETL solutions, and reporting platforms to Azure Databricks and modern Lakehouse architecture.
  • Provide technical support and troubleshooting for production incidents related to Azure Databricks, Spark workloads, data pipelines, and platform infrastructure.

Requirements:
  • At least 8-10 years of experience in designing and implementing enterprise data platforms.
  • At least 5 years of hands-on experience hands-on Azure Databricks experience in enterprise environments.
  • Experience with architecting modern Lakehouse solutions using Azure Databricks and Delta Lake.
  • Experience in administering Azure Databricks workspaces, clusters, Unity Catalog, compute policies, job orchestration, and workspace security.
  • Experience in developing data engineering solutions using Apache Spark (PySpark and/or Spark SQL).
  • Experience in implementing enterprise data governance utilizing Unity Catalog, metadata management, data lineage, and fine-grained access controls.
  • Experience designing and working with enterprise security architectures including RBAC, Managed Identities, Private Endpoints, Key Vault integration, encryption, and network security.
  • Experience implementing enterprise data quality frameworks, monitoring, observability, and operational dashboards.
  • Strong understanding of data modelling techniques including dimensional modelling, Data Vault, medallion architecture, and enterprise data warehouse concepts.
  • Strong knowledge of Azure Data Lake Storage Gen2, Azure Key Vault, Azure Monitor, Microsoft Entra ID (Azure AD), and Azure networking.
  • Strong SQL programming skills and proficiency in Python and PySpark development.
  • Strong experience implementing CI/CD pipelines utilizing Azure DevOps, automated testing, release management, and deployment automation.

Skills

PythonAzureCI/CDSQLSparkDatabricksData EngineeringETLDevOps

Similar Jobs

30

Senior Data Engineer – Azure & Databricks

Inetum·Brussels, Brussels Capital·Hybrid

Today

Developer / Senior Data Developer - Databricks | Azure | SQL | Power BI

Next·Leicester, England·Hybrid

1d ago

Senior Cloud Platform & DataOps Engineer (Azure / Databricks / Terraform)

Professional Kyndryl·INDLFCHE CHENNAI - DLF IT PARK, India·Remote

2d ago

Azure Databricks - Senior Manager

Crm Experienced Careers Site·Bengaluru, Karnataka

2d ago

Senior Data Engineer – Azure Databricks & DevOps 60–100%

Eraneos·Bern, Switzerland +2

4d ago

Senior Azure Cloud & Network Engineer, Databricks Enablement

49 Solutions·Ottawa, Ontario

Senior Azure Data Engineer (Databricks)

Capco·Edinburgh, Scotland·Remote, Hybrid

1w ago

Senior Azure Data Engineer (Databricks)

Capco·Glasgow, Scotland·Remote, Hybrid

1w ago

Senior Cloud & Platform Engineer (Azure Databricks, Azure Data Lake Storage) F/M/X

Ascent LLC.

Senior Associate Data Engineering Azure/Databricks Houston Hybrid

Referral Publicisgroupe·Houston, Texas·Hybrid

2w ago

Azure +Databricks- Senior Associate

Crm Experienced Careers Site·Bengaluru, Karnataka

2w ago

Senior Manager Information Technology (Enterprise Data Migration _ Azure Cloud _ Databricks _ Data Governance)

Diebold Nixdorf·Hyderabad, Telangana

2w ago

#5 - Senior Backend Engineer - Python & Azure Databricks

Muttdata·Ciudad de Mexico·Remote

3w ago

Databricks + Azure AI - Senior Associate

Crm Experienced Careers Site·Bengaluru, Karnataka

1mo ago

Senior Data Engineer/Analyst - Azure Databricks

Euroclear·Poland

1mo ago

Azure Databricks-Senior Technical Architect

Perficient·Mumbai, Maharashtra

1mo ago

Azure Databricks, IICS,AI - Senior Associate

Crm Experienced Careers Site·Hyderabad, Telangana

1mo ago

Senior Data Engineer - Azure Databricks Platform

EnerMech·Mumbai, Maharashtra

Azure Databricks - Banking - Senior Manager

Crm Experienced Careers Site·Bengaluru, Karnataka

1mo ago

Senior Data Engineer - Azure, AWS, Databricks

ELCA Global·Pully, Vaud

1mo ago

AWS /Azure/ Databricks + AI + Agentic AI+ Python- Senior Associate

Crm Experienced Careers Site·Bengaluru, Karnataka·Remote, Onsite

1mo ago

Azure Databricks design - Senior Manager

Crm Experienced Careers Site·Bengaluru, Karnataka

1mo ago

Senior Associate Data Engineering (Azure/Databricks) Hybrid

Referral Publicisgroupe·Toronto, Ontario·Hybrid

1mo ago

Senior Data Platform Engineer (Azure, Databricks, Terraform)

Accesa·Romania·Remote

2mo ago

Ingeniero de Datos Senior – Microsoft Azure, Fabric & Databricks

Professional Kyndryl·KMX51582 Mexico Puerta Polanco·Remote

2mo ago

Senior Data Platform Engineer (Azure & Databricks) 80-100%

Kantonsspitalbaden·Switzerland

2mo ago

Senior Data Engineer – Microsoft Azure, Fabric & Databricks (m/w/d)

otamo GmbH·Köln, North Rhine-Westphalia·Hybrid

2mo ago

Azure Databricks-Senior Technical Architect

Perficient·Hyderabad, Telangana

2mo ago

Azure Databricks-Senior Technical Architect

Perficient·Pune, Maharashtra

3mo ago

ETL Azure Databricks Engineer - Senior Consultant

Winfo Solutions·Hyderabad, Telangana

3mo ago