Lead Data Engineer

Job description

A leading organization is seeking a Lead Data Engineer to provide hands-on technical leadership for data solutions supporting an investment-focused platform. The Lead Data Engineer will combine senior-level engineering expertise with direct stakeholder collaboration, using Python, Microsoft Azure, and Snowflake to build and enhance scalable data solutions.

The position also supports work involving Large Language Models (LLMs) and artificial intelligence technologies. The ideal candidate can remain deeply involved in development while providing technical direction and translating business needs into effective technical solutions.

Responsibilities

• Design, develop, and enhance data engineering solutions for a technology platform.
• Build and optimize scalable data pipelines and data processing workflows.
• Develop Python solutions for data processing, integration, and analytics.
• Use Microsoft Azure and Snowflake to manage, transform, and optimize data.
• Provide hands-on technical leadership across data engineering initiatives.
• Work directly with business stakeholders to understand needs and translate requirements into technical solutions.
• Support initiatives involving Large Language Models and other AI technologies.
• Collaborate with engineering and business teams throughout solution development and delivery.

Required experience and skills

• Senior-level data engineering experience with hands-on technical leadership responsibilities.
• Strong Python development skills.
• Experience with Microsoft Azure cloud technologies.
• Strong Snowflake experience.
• Experience building and optimizing data pipelines and data workflows.
• Experience developing or supporting solutions that use LLMs or other AI technologies.
• Strong communication and stakeholder management skills.
• Ability to combine hands-on engineering responsibilities with technical leadership.
• Experience collaborating directly with business stakeholders and technical teams in an onsite environment.