- Location
- Pune, India
- Department
- Engineering
- Closing date
- Today
- Source
- Workday
Description
- - - - - - - - - - - -
Job Description:
Own architectural decisions (ADRs), reference blueprints, and target-state designs; arbitrate trade-offs on latency, cost, and scalability.
Architect end-to-end data and AI solutions: medallion Lakehouse (Bronze/Silver/Gold) on ADLS Gen2 + Delta Lake, streaming ingestion, ML deployment pipelines.
Optimize large-scale PySpark/Databricks workloads — partitioning, AQE, skew handling, Photon, cluster tuning.
Design data models across PostgreSQL, nowflake (Kimball, Data Vault 2.0, denormalized serving).
Productionize ML/DL models via Azure ML, AKS, Functions; expose FastAPI endpoints for batch and real-time inference.
Enforce security and compliance — Entra ID, Key Vault, PII, Unity Catalog / Purview, GDPR.
Build CI/CD and MLOps pipelines (Azure DevOps, Terraform/Bicep, MLflow); instrument observability.
Data Modeling Architecture/Design must also be done by Data and AI Solution Architect
This typically includes STAR SCHEMA Design, Normalization/Denormalization Data Patterns, etc.
Ability to evaluate and recommend which platform component would be best fit for given requirement.
Example: Decide between Postgres Db/Mongo Db, Decide between ODAP/One System Platform, etc.
Implementation of Continuous Architecture
Know how about BI domain + AI domains (NLP, GEN A, RAG, Forecasting, Computer Vision, Agentic AI, LLM) average or above average knowledge of concepts and internal technicalities.
Coach delivery teams, align with Data Science, Product, QA, and Platform leads, and contribute to the CDO's data policy.
Operate hands-on when needed — ingestion, cleansing, validation, repository design, root-cause analysis.
Must-Have
Advanced Python, PySpark, SQL
Expert in Databricks, Delta Lake, Spark Structured Streaming
Strong Azure stack: ADLS Gen2, ADF, Synapse, Azure ML, AKS, Event Hubs, Key Vault
Proven design of medallion Lakehouse architectures at TB–PB scale
Deep DB expertise: PostgreSQL, Synapse, Snowflake, ADLS
ML/DL fluency: scikit-learn, TensorFlow, PyTorch, MLflow
Hands-on PII/GDPR data handling and cloud security
CI/CD & MLOps: Azure DevOps or GitHub Actions, Terraform/Bicep, Docker, Kubernetes
