- Location
- Bangalore, IN
- Department
- Engineering
- Seniority
- Senior
- Closing date
- Today
- Source
- iCIMS
Description
Overview
The Senior Data Engineer (ETL) will lead the design and development of scalable ETL pipelines for Waters’ Enterprise Data Lakehouse. This role requires a strong technical foundation, hands-on experience with Azure databricks, Pyspark, SQL, Python & GITHUB alongwith a strong understanding of standard data engineering practices, and the ability to work cross-functionally with business and technical teams
Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
- Develop and optimize data processing solutions using Databricks and Apache Spark.
- Build data pipelines using PySpark, Python, and SQL.
- Work with Databricks notebooks, workflows/jobs, clusters, and Delta Lake.
- Implement incremental data processing and performance optimization techniques.
- Manage source code using GitHub, including branching, pull requests, code reviews, and repository management.
- Implement and maintain CI/CD pipelines for data engineering workloads using GitHub-based development practices.
Qualifications
- Bachelor’s degree in computer science, Software Engineering, Information Systems, or a related field.
- Typically, 4-7 years of relevant experience in Data Engineering, ETL, DWH, or data analytics decision support role, with some senior roles requiring total experience between 4 to 7 years.
- Strong handson experience with Databricks platform.
- Expert proficiency in SQL and Python (advanced querying, stored procedures, performance tuning).
- Extensive experience with ETL tools and platforms (e.g., SSIS, Azure Data Factory, Azure DataBricks).
- Deep understanding of data warehousing concepts (dimensional modelling, Kimball/Inmon methodologies).
- Familiarity with Big Data technologies (e.g., Spark, Hadoop, Kafka) for processing large
- Experience with version control systems (e.g., Git).
- Strong technical & analytical skills required, including a thorough understanding of how to interpret customer business needs and translate them into operational requirements
Skills
PythonAzureCI/CDSQLSparkHadoopDatabricksData EngineeringETLGitGitHub