- Location
- Kuala Lumpur, Exchange 106, Malaysia
- Type
- Full-time
- Department
- IT
- Seniority
- Lead
- Experience
- 8+ years
- Source
- Workday
Description
Summary
Our talented Data & AI Practice is made up of globally recognized experts - and there’s room for more analytical and ambitious data professionals. If you’re passionate about helping clients make better data-driven decisions to tackle their most complex business issues, let’s talk. Take your skills to a new level and launch a career where you can truly do what matters.
The Databricks M/L Technical Lead is a senior, hands-on role responsible for the design, development, and delivery of highly scalable, secure, and performant data solutions on the Databricks Lakehouse Platform. It is expected to provide technical leadership to a team of engineers, defining coding standards, implementing architectural patterns, and ensuring the delivery of high-quality data products.
Key Responsibilities
Lead the Design: Define and implement robust data architectures utilizing the Databricks ecosystem, including Delta Lake, Unity Catalog, Machine Learning Models and Databricks Workflows.
Hands-on Development: Serve as the most senior developer, writing high-quality, production-grade code in PySpark/Scala and SQL for complex batch and streaming ETL/ELT pipelines.
Performance Optimization: Lead performance tuning and optimization efforts for large-scale Spark jobs, ensuring efficient cluster utilization and cost management
Standards & Best Practices: Define and enforce technical standards, code quality, testing frameworks (unit, integration), and DataOps/CI/CD pipelines for the engineering team.
.
Skills & Experiences
8+ years of experience in machine learning, data science, or MLOps, with at least 4+ years focused specifically on the Databricks Lakehouse Platform.
Expert proficiency in Python and PySpark/Scala for large-scale data processing and machine learning.
Deep understanding and practical experience with Delta Lake architecture and optimization techniques.
Proven expertise implementing MLOps principles using MLflow (Tracking, Registry, Projects, and Deployment).
.
