Hiring.Camp

Senior Machine Learning Engineer, Evaluation & Reliability

Bask Health

·
Location
United States · published
Type
Full-time
Department
Engineering
Seniority
Senior
Source
Workable

Description

About Bask Health

Bask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses.

Our platform brings together patient experiences, clinical workflows, pharmacy operations, and commerce tools in one place.

We're an entrepreneurial, product-focused engineering team that values ownership, technical excellence, and building software that solves real problems. Engineers work closely with product teams and company leadership, take features from idea to production, and have a direct role in shaping the platform.

Engineering at Bask is AI-first and builder-led. We use tools like Cursor and Claude Code throughout development while maintaining high standards for reliability, testing, and software quality.

About the Role

We're looking for a Senior Machine Learning Engineer, Evaluation & Reliability to design, build, and maintain the systems that measure and improve the quality of Bask Health's AI-powered applications.

As Bask introduces more AI capabilities across patient experiences, merchant tools, clinical workflows, and internal operations, we need reliable ways to understand how these systems perform. You'll help establish the engineering infrastructure, evaluation methods, and monitoring practices that allow teams to ship AI features with confidence.

This role sits at the intersection of machine learning engineering, software infrastructure, automated testing, and production reliability. You'll work on evaluation pipelines, benchmark datasets, regression testing, model monitoring, and tools that help engineers identify failures before they affect users.

You'll develop systems that measure more than whether an AI model produces a response. You'll help determine whether that response is accurate, relevant, grounded in appropriate information, consistent with application requirements, and safe for its intended use.

You'll also investigate production issues, identify patterns in model failures, and collaborate with applied AI engineers to improve prompts, retrieval systems, model configurations, and application behavior.

Because Bask operates in healthcare-related workflows, reliability is especially important. You'll help establish safeguards for sensitive use cases, support human review where appropriate, and make sure AI systems are evaluated against clear quality expectations before deployment.

This is a hands-on senior engineering position with ownership over evaluation infrastructure and reliability practices. You'll contribute production code, design scalable testing systems, and help establish standards that other engineers can use across multiple AI applications.

We're looking for someone who enjoys turning complex machine learning behavior into measurable engineering systems and who understands that successful AI products depend on consistent performance, not just impressive demonstrations.

What You'll Do

  • Design and build evaluation infrastructure for AI and machine learning applications across Bask Health.
  • Develop automated testing systems for large language model applications and intelligent workflows.
  • Establish measurable quality standards for production AI features.
  • Create evaluation pipelines that assess accuracy, relevance, completeness, consistency, and reliability.
  • Build and maintain representative datasets for testing AI application behavior.
  • Develop regression testing systems to detect quality changes when models, prompts, or application logic are updated.
  • Design evaluation methods for retrieval-augmented generation and knowledge-grounded AI systems.
  • Measure retrieval quality, grounding accuracy, and unsupported model outputs.
  • Build tools for identifying hallucinations, incorrect reasoning, and unreliable responses.
  • Develop testing approaches for AI agents, tool calling, and multi-step workflows.
  • Evaluate whether AI systems follow application instructions and respect workflow boundaries.
  • Implement automated checks for structured outputs, API interactions, and tool execution.
  • Build monitoring systems that track AI performance in production.
  • Develop dashboards and reporting tools for model quality, latency, errors, and operational reliability.
  • Investigate production AI failures and identify their underlying technical causes.
  • Collaborate with applied AI engineers to improve prompts, retrieval pipelines, and model configurations.
  • Establish testing procedures for model upgrades and provider changes.
  • Develop benchmarking tools to compare models based on quality, performance, and cost.
  • Implement evaluation processes for sensitive healthcare-related AI workflows.
  • Work with product and clinical stakeholders to define appropriate acceptance criteria and escalation requirements.
  • Support human review and feedback collection for AI outputs that require additional validation.
  • Design reproducible experiments for evaluating AI system improvements.
  • Build reusable evaluation libraries, services, and internal engineering tools.
  • Integrate AI evaluation checks into development and deployment workflows.
  • Improve observability through structured logging, tracing, and production diagnostics.
  • Mentor engineers on AI testing methodologies and reliability practices.
  • Contribute to architecture discussions involving AI infrastructure, testing, and monitoring.
  • Own evaluation and reliability improvements from technical design through deployment and ongoing maintenance.

Similar Jobs

30

Senior Machine Learning Engineer

Encora·Bogotá, Bogota D.C.·Remote

Today

Senior Machine Learning Engineer

ServiceNow·Hyderabad, Telangana·Onsite

1d ago

Senior Machine Learning Engineer (W/M/X)

Ubisoft·Paris, Île-de-France·Onsite

1d ago

Sr. ASIC Design Engineer, Cloud-Scale Machine Learning Acceleration - Annapurna Labs

Amazon Company

Senior Applied Machine Learning Scientist

Td·TD Centre - TD Tower - 66 Wellington Street West, Toronto·Onsite

1d ago

Senior Manager, Applied AI & Machine Learning

Virtahealth·New York City, New York +1·Remote

1d ago

Sr. Machine Learning Engineer

PayPal·San Jose, California·Hybrid

1d ago

Sr. Machine Learning Engineer

Workday·Boulder, Colorado +1·Hybrid

1d ago

Sr. Machine Learning Engineer

PayPal·US·Hybrid

1d ago

Senior Machine Learning Engineer, Decision Intelligence

Humana·US·Remote

1d ago

Senior Machine Learning Developer

Clera·Montréal, Quebec·Remote

1d ago

Senior Machine Learning Engineer, Shield

Boxinc·Redwood City, California·Hybrid

1d ago

Senior Staff Machine Learning Engineer, Ads Ranking

Reddit·San Francisco, California +1·Remote

1d ago

Senior Machine Learning Engineer

ServiceNow·Santa Clara, California·Onsite

2d ago

Senior Manager, Machine Learning Engineering Mgmt

ServiceNow·Santa Clara, California·Onsite

2d ago

Senior Machine Learning Engineer

ASOS·London, England·Hybrid

2d ago

Senior Machine Learning Engineer (MLOps)

ASOS·London, England·Hybrid

2d ago

Senior Machine Learning Engineer (Recommendations/Search)

ASOS·London, England·Hybrid

2d ago

Senior Machine Learning Engineer (Customer/Marketing)

ASOS·London, England·Hybrid

2d ago

Sr MLA Design Verification Engineer, Cloud-Scale Machine Learning Acceleration - Annapurna Labs

Amazon Company

Senior Artificial Intelligence Machine Learning Scientist

General Motors·Mountain View, California +2·Remote

2d ago

Senior Machine Learning Engineer

Haystack News·Fort Lauderdale, Florida·Hybrid

2d ago

Senior Manager, Machine Learning Science

Expedia·London, England

2d ago

Senior Machine Learning Developer

BMO is·Canada

2d ago

Senior Machine Learning Engineer - Search

THE NEXT DOOR INC·US Remote +1·Remote

Senior Machine Learning Engineer - Search

THE NEXT DOOR INC·Dallas, Texas·Hybrid

Senior Machine Learning Engineer

Floqast·San Jose, California·Hybrid

2d ago

Senior Machine Learning Scientist, Applied Causal Inference

Doordash USA·San Francisco, California +2

3d ago

Senior Machine Learning Engineer

Redbee·Santiago, Santiago Metropolitan·Hybrid

3d ago

Senior Machine Learning Engineer for Tactile Sensors

Melexis·Bevaix, Neuchâtel

3d ago