Sr. Data Engineer - Databricks
Anblicks
·Sep 15, 2026
- Location
- Hyderabad, India
- Workplace
- Remote, Hybrid
- Department
- Engineering
- Experience
- 6+ years
- Source
- Keka
Description
- Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and Apache Spark.
- Build robust data ingestion, transformation, processing, and validation frameworks for large-scale datasets.
- Develop and optimize data processing workloads using Python, Java, Spark, and SQL.
- Work extensively with Databricks to develop scalable data engineering solutions and production-grade pipelines.
- Implement efficient data transformation and storage solutions using modern data engineering practices.
- Write complex and optimized SQL queries for large-volume data processing and analytics.
- Perform performance tuning and optimization of Spark jobs, SQL queries, pipelines, and data processing workloads.
- Implement data quality checks, validation frameworks, monitoring, logging, and error-handling mechanisms.
- Work with stakeholders, data analysts, architects, and engineering teams to understand requirements and deliver effective data solutions.
- Contribute to the design and implementation of cloud-based data platforms and modern data architectures.
- Troubleshoot production issues and ensure the reliability, scalability, and performance of data pipelines.
- Follow data engineering best practices related to security, governance, data quality, and operational excellence.
- Mentor junior engineers and contribute to technical standards, reusable frameworks, and engineering best practices.
- 6 - 10 years of professional experience in software development and/or data engineering, with proven experience delivering production-grade data solutions.
- Strong hands-on experience in Data Engineering and ETL/ELT development.
- Strong experience with Databricks and Apache Spark.
- Strong programming skills in Python and Java.
- Advanced proficiency in SQL, including query optimization and working with large datasets.
- Experience designing and developing scalable data pipelines and distributed data processing solutions.
- Strong understanding of data warehousing, data modeling, and large-scale data processing.
- Hands-on experience with data ingestion, transformation, data quality, and pipeline optimization.
- Strong troubleshooting, debugging, and performance tuning capabilities.
- Experience working in Agile development environments and collaborating with cross-functional teams.
- Experience working with AWS cloud services and cloud-based data platforms.
- Experience with Delta Lake and modern Databricks data engineering capabilities.
- Knowledge of Databricks Workflows, Jobs, notebooks, clusters, and deployment practices.
- Experience with CI/CD, Git, and DevOps practices for data engineering.
- Exposure to AdTech, MarTech, Customer Data Platforms (CDP), or customer analytics environments.
- Experience working with high-volume, complex, and diverse datasets.