Senior Data Engineer (PySpark, SQL, Databricks)

Company:

Marsh Risk

Description:

We are seeking a talented individual to join our Marsh Risk Tech team, a Marsh business. This role will be based in Pune/Gurugram. This is a hybrid role that has a requirement of working at least three days a week in the office.

Senior Data Engineer (PySpark, SQL, Databricks)

We are seeking an experienced Data Engineer with strong expertise in Databricks, PySpark, and SQL to design, build, and optimize scalable cloud-native data solutions. The ideal candidate will play a key role in developing enterprise-grade data pipelines, implementing modern Lakehouse architectures, and delivering high-quality data products that support analytics, reporting, and advanced data initiatives.

This role requires hands-on experience with Databricks, Delta Lake, Medallion Architecture, data modeling, and large-scale distributed data processing. The successful candidate will work closely with data architects, business stakeholders, analysts, and platform teams to build reliable, scalable, and high-performing data solutions.

We will count on you to:

Modern Data Platform & Lakehouse Development

• Design and implement scalable data pipelines using Databricks Lakehouse architecture.

• Build and maintain data products aligned with Medallion Architecture (Bronze, Silver, and Gold layers).

• Develop ingestion, transformation, and consumption pipelines following modern data engineering best practices.

• Design business-ready Gold layer datasets to support reporting, analytics, and downstream consumption.

Delta Lake Development

• Build and manage Delta Lake tables and data assets.

• Implement incremental processing using Merge, Upsert, and Change Data Capture (CDC) patterns.

• Leverage Delta Lake features such as: ACID transactions, Time Travel, Schema Enforcement, Schema Evolution, Data Versioning

• Optimize Delta tables using partitioning, compaction, and performance tuning techniques.

Data Modernization & Migration

• Analyze existing data flows and SQLs and translate legacy workflows into optimized PySpark-based data pipelines.

• Document transformation logic and migration approaches.

Data Engineering & Development

• Build scalable PySpark applications on Databricks.

• Develop reusable frameworks and components to accelerate migration efforts.

• Tune Spark jobs for performance and resource efficiency.

• Analyze Spark execution plans and implement optimization recommendations.

• Implement data quality checks, reconciliation processes, and validation frameworks.

SQL Development

• Write complex SQL queries, stored procedures, and performance-optimized data transformations.

• Perform data analysis, profiling, and troubleshooting using SQL.

• Optimize queries and data models for large-scale datasets.

Governance & Best Practices

• Unity Catalog and governance concepts

• Implement data quality, monitoring, and reconciliation frameworks.

• Support data lineage, metadata management, and governance initiatives.

• Contribute to CI/CD and deployment automation practices.

What you need to have:

• Strong hands-on experience with Databricks and Spark-based data engineering.

• Experience implementing Medallion Architecture (Bronze, Silver, Gold layers).

• Experience designing and managing Delta Lake solutions and a good understanding of Lakehouse Architecture principles.

• Experience developing scalable data pipelines for enterprise analytics platforms.

• Knowledge of data quality, lineage, governance, and observability practices.

What makes you stand out?

• Prior experience with Informatica.

• Cloud experience (AWS preferably).

Why join our team:

• We help you be your best through professional development opportunities, interesting work and supportive leaders.

• We foster a vibrant and inclusive culture where you can work with talented colleagues to create new solutions and have impact for colleagues, clients and communities.

• Our scale enables us to provide a range of career opportunities, as well as benefits and rewards to enhance your well-being.

Marsh (NYSE: MRSH) is a global leader in risk, reinsurance and capital, people and investments, and management consulting, advising clients in 130 countries. With annual revenue of over $27 billion and more than 95,000 colleagues, Marsh helps build the confidence to thrive through the power of perspective. For more information, visit corporate.marsh.com, or follow us on LinkedIn and X.

Marsh is committed to embracing a diverse, inclusive and flexible work environment. We aim to attract and retain the best people and embrace diversity of age, background, caste, disability, ethnic origin, family duties, gender orientation or expression, gender reassignment, marital status, nationality, parental status, personal or social status, political affiliation, race, religion and beliefs, sex/gender, sexual orientation or expression, skin color, or any other characteristic protected by applicable law.

Marsh is committed to hybrid work, which includes the flexibility of working remotely and the collaboration, connections and professional development benefits of working together in the office. All Marsh colleagues are expected to be in their local office or working onsite with clients at least three days per week. Office-based teams will identify at least one “anchor day” per week on which their full team will be together in person.