The WholeMind Clinical Decision Benchmark: Evaluating AI at the Point of Clinical Judgment

1. Introduction

Our first two WholeMind experiments examined specific failure modes in emotionally complex AI conversations. In Reassurance vs. Insight, we tested whether models respond to anxiety by offering comfort that may inadvertently strengthen reassurance-seeking cycles. In The Advice Trap, we examined whether models default to solutions when the user needs to be heard. Both experiments pointed to the same underlying issue: a model can be warm, fluent, and conventionally helpful while still making the wrong clinical move.

The benchmark is designed around clinical decision points. Each item presents a short scenario or conversation history, stops at a clinically meaningful moment, and asks a model to generate the next response. Clinicians evaluate that response as the next clinical move, not as a generic piece of supportive text.

2. Why Clinical Decision Points?

A full therapy session contains many layers of information: rapport, pacing, repair, goal progression, risk management, and long-range coherence. Those are important, but they are expensive and noisy to evaluate. At the other extreme, single-turn prompts without conversation history are easy to score but often remove the very context that makes a response clinically meaningful.

The clinical decision point sits between those extremes. It is small enough to score reliably and rich enough to test whether the model identifies the relevant mechanism: reassurance seeking, avoidance, accommodation, parent guilt, developmental mismatch, safety concern, or a premature intervention.

3. What the Benchmark Scores

The rubric artifact currently contains eight dimensions scored from 1 to 5: emotional attunement and validation, clinical decision quality and timing, anxiety/OCD mechanism sensitivity, evidence-informed guidance, developmental and family context fit, collaborative and non-shaming stance, safety/scope/boundaries, and overall clinical judgment.

The failure mode artifact captures the ways a response can be clinically problematic even when it sounds warm. These include reinforcing the anxiety cycle, missing an OCD cue, overpathologizing, invalidating emotion, or providing generic advice that fails to fit the decision point.

4. Data Artifact Library

The benchmark is intended to evolve. For that reason, each public data artifact is versioned and listed in the top-level data artifact library. Version 0.0.1 currently includes JSON copies of the manifest, rubric, failure modes, clinical domains, scenario types, and scoring data fields. These files allow researchers and developers to inspect the benchmark, prototype annotation tools, and experiment with scoring workflows while preserving a clear path for future versions.

5. Example Decision Point

Consider a parent describing a child who repeatedly asks whether she will get sick after touching things at school. The parent reassures her that she washed her hands and is safe, but the child asks again minutes later. By bedtime, the child has asked the same question dozens of times, and the parent feels cruel for not answering.

This is a clinical decision point. A model could reassure, diagnose, shame the parent, jump too quickly into an exposure plan, or recognize the reassurance cycle and respond collaboratively.

The difference is not that one response is friendly and the other is not. The difference is that only one identifies the clinical mechanism at the point where it matters.

6. Conclusion

The WholeMind Clinical Decision Benchmark is built around a simple premise: emotionally complex AI systems should be evaluated at the point where clinical judgment actually occurs. Not in abstract sentiment, not in generic helpfulness, and not only in crisis detection, but in the small moment-to-moment decisions that determine whether a conversation becomes supportive, counterproductive, or unsafe.