Centaur.ai ads

🚀 Introducing: Centaur’s CLEAR-Bench (Clinical & Lay Evaluation of AI Readiness)!! There’s a disconnect between what users prefer and what is safe and reliable. We’ve seen this movie before with tech built to maximize engagement. Especially in domains like health, technology shouldn’t just optimize for what users like most. If we only optimize for user preference, we'll build AI that's reassuring, agreeable, but sometimes dangerous. If we only optimize for complete risk elimination, we’ll build an overly sensitive model that users would find frustrating to engage with. So we built CLEAR-Bench. The key idea: evaluate models separately on what consumers value (empathy, helpfulness) and what clinicians require (safety, accuracy, urgency), then look at where they diverge. They diverge a lot. No model was best at both. The model (GPT 5.5 Instant) that drew the fewest clinical flags had the lowest consumer ratings, while the model (Gemini 3.5 Flash) that had the highest consumer ratings drew more clinical flags. The answer people prefer isn't necessarily the most clinically-sound. Consumer ratings don’t track clinical quality. How we did it: across model responses to health queries grounded in real-world usage, we collected multiple consumer, clinician, and LLM judgments across the dimensions that matter most. Consumers rewarded answers that felt warm and clear (and longer!). The LLM judge mostly flagged incorrect facts. Clinicians mostly flagged what was missing: the warning sign, the escalation, the follow-up question. That's why building health AI needs strategic human involvement and multi-dimensional evaluation. It's the Centaur approach: know where experts are essential, where everyday users are the right judges, and where to rely on models. Health AI evaluation needs to measure what matters, not just what people click. Identifying the tradeoffs and realities before and during deployment lets you see where and how to fill the gaps from a robust, grounded standpoint. Building in health? We'll run CLEAR-Bench on your model, be your continuous eval machine, and help bridge any gaps.

The most clinicially-sound model responses for consumer health questions aren't the ones ordinary people like the most. Gemini 3.5 Flash was the most preferred model at 73%, but it was not the strongest on clinical accuracy. The percentage of answers with no clinical inaccuracies was 74.7%, behind both GPT 5.5 Instant at 94.1% and Sonnet 4.6 at 79.3%. GPT 5.5 Instant had the fewest accuracy flags overall. Llama 4 Maverick 17B had the highest flag rate across accuracy, urgency, and safety. And the biggest gap came from how clinicians and an LLM judge reviewed the exact same responses. Clinicians flagged 155 answers for safety that the LLM judge didn’t, with many of their concerns centered on missing warning signs, weak escalation, or questions the model should have asked. The LLM judge, meanwhile, surfaced more factual accuracy issues. Same prompts. Same model responses. Very different picture depending on what you measure and who is doing the evaluation. Full results are live today: https://lnkd.in/efeSrCua

Three new datasets just landed in Centaur’s Arena: 🔬 Lymph node tissue slices 🩻 Pediatric chest X-rays 🧠 EEG recordings Across all three, Centaur’s human consensus outperformed every frontier model tested -- with gaps ranging from 7.8 to 35.1 percentage points. Take a look at the Arena and see how today’s leading models stack up against human expertise! Full story in the comments 👇

Radiology is a domain where the stakes of AI quality immediately impact patient outcomes. When quality matters this much, accuracy must be engineered. This is exactly why radiology is such a clear use case for collective intelligence annotation. https://lnkd.in/eRT7ez4H #RadiologyAI #MedicalImaging #LLMEvaluation #AIQuality #CollectiveIntelligence #DataAnnotation #ModelEvaluation #MedTech #DigitalPathology #HealthcareAI #CentaurAI

Drones and satellites can reveal hidden emissions, but the real breakthrough comes from expert annotation. High-quality labeled data transforms raw images into actionable intelligence—and trains the AI models that will scale this work. Centaur.ai delivers the accuracy-first, expert-validated annotation that ensures your data is trusted. https://lnkd.in/eZVrwjcg #AI #ClimateTech #DataAnnotation #LLM #ModelEvaluation #Sustainability #CentaurAI

We’re thrilled to share that our mobile clinical-reasoning platform, DiagnosUs, has been awarded Gold by HealthAwards.com for Mobile Digital Health Resources. The award underscores what we’ve believed since our earliest research days: high-quality, expert-validated annotation is the foundation of trustworthy AI. https://lnkd.in/em3AFqA5 Thank you to all the clinicians, specialists, and contributors in our network whose expertise makes this possible. #HealthcareAI #DigitalHealth #DataAnnotation #MedAI #ClinicalAI #CentaurAI