Senior ML Ops / LLM Ops Engineer

About Welvaart

At Welvaart, we create technology solutions that put people at the center.
Our close leadership style and flexible culture of growth empower our teams and elevate the quality of our delivery. We combine rigor, innovation, and empathy to drive projects that transform businesses and build lasting relationships of trust.

We complement this vision with a performance‑driven Digital Marketing offering, helping companies strengthen their visibility, enhance their online presence, and accelerate growth through smart, measurable strategies.

Project

This role focuses onbuilding and operating the ML Ops / LLM Ops pipeline that closes it: ingest production signal, redact it, store it, slice it, classify it, surface the failures, mine new eval cases, and alert on regressions. You drive the toolchain decisions, the data-governance posture, and the day-to-day reliability of the pipeline itself. The Head of AI sets vision and priorities and you own the technical execution end-to-end

Role

• Design and build a source-agnostic ingestion pipeline for production ML / LLM traffic
• Design storage tiering based on automotive and company requirements, policy-driven retention windows, and privacy requirements
• Build slicing dashboards and the query path engineers use to debug production at 11p.m.
• Enable autoraters and lightweight LLM classifiers across production traffic
• Build the rule-based triage layer for obvious failures
• Stand up the eval-mining workflow and wire regression alerts to model and prompt deploys
• Implement PII redaction at the ingestion boundary and safety / abuse classification on inbound content
• Define dashboard architecture, wipeout mechanisms, tool and hosting selection, and operate the pipeline end-to-end

We are looking for

• Proven experience building and operating data or ML platform systems in production, covering ingest, schema, storage, access control, alerts, and on-call
• Hands-on experience building and running ML / LLM evaluation systems in production (offline regression sets, online autoraters, LLM-as-judge pipelines, golden datasets)
• Hands-on experience with LLM tracing and observability tooling
• Experience shipping PII redaction or comparable data-handling controls in a regulated or multi-tenant environment, with a pragmatic approach to data governance
• Strong understanding of how ML and LLM-based systems fail in production: hallucination, retrieval failures, agent loops that don't terminate, ASR / TTS degradation, and prompt or model regressions across deploys
• Production Python proficiency; hands-on engineer, not advisory. Comfortable leveraging AI in everything you build
• Preferable multi-tenant or white-label SaaS experience with per-tenant data isolation
• Azure experience and ability to make self-host vs managed SaaS calls on tradeoffs
• Experience with autorater methodology and contamination defenses
• Knowledge of vector databases, embedding-based clustering or unsupervised failure-mode discovery
• Experience with data-versioning tooling (LakeFS, DVC, Delta Lake)
• GDPR / right-to-erasure work
• Embedded, automotive, or another constrained environment context
• Working knowledge of a language beyond English sufficient to validate non-English failure modes
• Prior experience using Cloud (Microsoft Azure and AWS);
• Prior experience with Claude Code;
• Prior experience with GitHub;
• Languages: Python primary, SQL, and some TypeScript for dashboards;
• LLM APIs: Claude (Anthropic), OpenAI, open-source models as needed
• Android/AAOS ecosystem as clients

What you can discover with us?

• Be part of a tech start-up
• Different scopes of project in different sectors
• Structure of fairness and equity salary (Consultant Profile)
• Training & Certification
• Career Path management
• More than 30 Partnerships

UNLEASH THE POWER OF YOUR CAREER