Databricks Data Engineering Lead

Databricks Data Engineering Lead

Would you like to lead digital transformation?

Would you like to be involved in bringing innovative solutions to our clients?

Join our dynamic Digital Technology Team

We operate at the heart of the digital transformation of our business. Our team is responsible for digitization initiatives, from defining digital strategy to enabling employee success, the Digital Technology team provides world class products and services to our business partners..

Partner with the best

The Databricks Data Engineering Lead – Quality is a technical role responsible for the development, coordination, and operational oversight of Quality Data Products and analytics solutions on the Databricks platform.

As a Databricks Data Engineering Lead, you will be responsible for

• As part of a newly established Data Organization, the role will contribute to the implementation of a Data Mesh operating model, supporting the creation of domain-owned Data Products and helping establish engineering practices, technical standards, and governance processes across the Quality domain.
• The role combines active development responsibilities with technical coordination activities. The individual will design and develop data pipelines, transformations, and analytical datasets while coordinating technical delivery across Data Engineers, Data Architects, Data Governance teams, and external development partners.
• Acting as the primary technical focal point for the Quality domain, the role will translate business requirements into technical solutions, support backlog refinement, coordinate development activities, perform design reviews, manage technical dependencies, and ensure alignment with architecture, governance, and platform standards.
• The position is also responsible for monitoring data pipeline reliability, supporting incident resolution, maintaining technical documentation and metadata, and driving continuous improvements in data quality, performance, and operational stability.
• Within the Quality domain, the role will help deliver and evolve Data Products by ensuring effective collaboration between business stakeholders and engineering teams, from requirements definition through implementation, testing, deployment, and ongoing support.

Fuel your passion

To be successful in this role you will:

Data Engineering

• Lead and coordinate Data Engineering activities, ensuring timely and high-quality delivery across teams.
• Develop and maintain data pipelines and transformations when required.
• Provide technical guidance, design reviews, and implementation support to Data Engineers and vendors.
• Implement and oversee data quality, testing, and monitoring controls.
• Support analytics, reporting, and AI use cases.
• Contribute to data modeling, architecture alignment, and technical standards.
• Coordinate workspace operational requests, configurations, and platform support activities.

Standards & Governance

• Apply engineering standards and development best practices.
• Ensure compliance with testing, documentation, deployment, and operational requirements.
• Contribute to continuous improvement initiatives.

Service Management

• Support ServiceNow incidents, changes, and service requests.
• Participate in root cause analysis and operational reporting.
• Ensure operational processes are consistently followed.

Data Modeling & Business Knowledge

• Quickly understand business processes, source systems, and data structures.
• Collaborate with stakeholders to clarify requirements and data definitions.
• Support logical and physical data modeling activities.

Data Governance & Data Custodian

• Support Data Governance activities for assigned datasets and data products.
• Maintain metadata, lineage, and catalog information.
• Implement data quality controls and validation rules.
• Support data classification, access controls, and audit requirements.
• Participate in data quality monitoring and resolution activities.

AI-Driven Engineering

• Use AI assistants to accelerate development, testing, troubleshooting, and documentation.
• Contribute to AI-enabled software engineering practices.
• Improve productivity, quality, and delivery efficiency through AI adoption.

 

Required Qualifications

• Bachelor's Degree in Computer Science or STEM disciplines.
• 2+ years of Databricks experience.
• Have atleast 3+ years of experience.
• Experience with Python, PySpark, SQL, and AWS-based data platforms.
• Familiarity with Git, CI/CD pipelines, ServiceNow processes, and Agile delivery methodologies.
• Basic experience with Rally (Agile Central) for backlog management, user story tracking, and sprint execution.
• Experience supporting production environments.

Desired Characteristics

• Strong analytical and problem-solving skills.
• Ability to quickly understand new business domains and datasets.
• Delivery-oriented and collaborative mindset with the ability to work across business and technical teams.
• Basic understanding of Agile/Scrum practices, including user stories, backlog refinement, and sprint ceremonies.
• Interest in AI technologies and modern engineering practices.
• Strong focus on quality, automation, governance, and continuous improvement.

Work in a way that works for you

• We recognize that everyone is different and that the way in which people want to work and deliver at their best is different for everyone too. In this role, we can offer the following flexible working patterns:
• Working flexible hours - flexing the times when you work in the day to help you fit everything in and work when you are the most productive

Working with us

Our people are at the heart of what we do at Baker Hughes. We know we are better when all of our people are developed, engaged and able to bring their whole authentic selves to work. We invest in the health and well-being of our workforce, train and reward talent and develop leaders at all levels to bring out the best in each other.

Working for you

Our inventions have revolutionized energy for over a century. But to keep going forward tomorrow, we know we have to push the boundaries today. We prioritize rewarding those who embrace change with a package that reflects how much we value their input. Join us, and you can expect:

• Contemporary work-life balance policies and wellbeing activities
• Comprehensive private medical care options
• Safety net of life insurance and disability programs
• Tailored financial programs
• Additional elected or voluntary benefits

The Baker Hughes internal title for this role is: Digital Technology Specialist - Project Management