Data & Databricks Practice Lead (India - Remote)

We are looking for a dynamic and visionary Practice Lead to build and scale our Data & AI capabilities with a strong focus on Databricks, Apache Spark, Iceberg, Azure Data Fabric, Amazon EMR, and Redshift. This role is pivotal in driving Lakehouse architecture adoption, cloud-native modernization, and ML engineering across industries such as Oil & Gas, Medtech, Retail CPG, and Fintech.

You will lead a cross-functional team of architects, engineers, and ML specialists to deliver high-impact solutions that combine technical excellence, cost efficiency, and governance maturity.

Strategic Leadership & Practice Growth

•            Define and execute the strategic roadmap for the Data & AI practice, aligning with evolving trends in Lakehouse, cloud-native platforms, and AI engineering.

•            Build reusable solution accelerators (e.g., Iceberg ingestion templates, Spark ETL frameworks, MLflow pipelines).

•            Establish and manage strategic partnerships with Databricks, AWS, Azure, and other ecosystem players.

•            Represent the practice in industry forums, client workshops, and thought leadership initiatives.

Technical Architecture & Delivery Oversight

•            Architect scalable Lakehouse solutions using Delta Lake or Iceberg, optimized for performance and cost.

•            Lead the design of unified batch + streaming pipelines using Spark Structured Streaming and Scala.

•            Oversee multi-cloud data platform modernization using Azure Data Fabric, EMR, and Redshift.

•            Guide implementation of data governance frameworks using Unity Catalog, Azure Purview, and Iceberg metadata.

Team Leadership & Capability Building

•            Mentor and grow a high-performing team of data engineers, ML engineers, and platform specialists.

•            Define career paths, training plans, and certification goals aligned with Databricks, AWS, and Azure.

•            Foster a culture of innovation, experimentation, and continuous improvement.

Client Engagement & Business Impact

•            Engage with clients to understand business challenges and translate them into scalable data solutions.

•            Lead solutioning, proposal development, and delivery for strategic engagements.

•            Ensure delivery excellence through performance optimization (Photon, EMR Autoscaling) and cost engineering.

•            10+ years of experience in data engineering, architecture, and platform modernization.

•            3+ years of hands-on experience with Databricks, Apache Spark, Iceberg, and Scala.

•            Deep expertise in cloud-native data platforms (Azure Data Fabric, Amazon EMR, Redshift).

•            Proven track record in building ML pipelines, implementing MLOps (MLflow), and integrating Redshift ML.

•            Strong understanding of data governance, observability, and schema evolution.

•            Excellent leadership, communication, and stakeholder management skills.

•            Certifications in Databricks, Azure Data Engineering, or AWS Big Data.

•            Experience in vertical-specific data solutions (e.g., predictive maintenance in Oil & Gas, real-time analytics in Retail).

•            Familiarity with data mesh architectures, time travel, and audit compliance frameworks.

•            Be at the forefront of Lakehouse innovation and AI-driven transformation.

•            Work with cutting-edge technologies and shape the future of data platforms.

•            Lead a high-impact practice with global reach and industry relevance.