The mPACT misinformation benchmark v1: leading AI models rarely ask questions before responding
New mPACT Misinformation Benchmark Finds Leading AI Models Rarely Ask Questions Before Responding
mpathic’s behavioral science framework follows AI across whole conversations and finds that harmful behavior tends to persist once it starts.
SEATTLE, WA, November 12, 2026 — mpathic, a clinician-founded AI safety company that works directly with leading AI labs, today released the mPACT Misinformation benchmark, the third study in its mPACT (mpathic Psychologist-led AI Clinical Tests) series, following benchmarks on suicide risk and eating disorders. Evaluating six leading AI models across full, multi-turn conversations, the benchmark found that only 2% of responses asked about the person’s circumstances. It also found that harmful behavior tended to persist once it appeared: across the six models combined, 89% of responses containing harmful behavior were followed by another harmful response.
The benchmark examines what happens when a person brings a questionable belief into a conversation with an AI model: whether the model helps that person reason through it or reinforces it, and whether the model can recover once the conversation takes a harmful turn.
mPACT is mpathic’s framework for measuring how AI behaves in high-risk conversations. Each benchmark pairs conversations written by experts with human review of every response, which allows patterns that emerge over the course of an exchange to be measured, and the approach can be applied to new models as they appear. It measures whether a model recovers once a conversation turns, as well as whether a harmful response occurs.
Why Behavioral Science Matters
People rarely approach AI with neutral questions. They bring uncertainty, frustration and deeply held beliefs, and sometimes a wish to hear that they are right. A parent worried about vaccines, a young man drawn into red-pill communities or someone weighing a move of their savings into cryptocurrency may be seeking information, but may also be seeking reassurance. In those settings, a model need not state anything false to strengthen a questionable belief. Agreement, excessive confidence or advice delivered with unwarranted authority can be enough to make an unsupported idea appear credible.
“Misinformation should be considered under a behavioral science lens. People aren’t just turning to AI for facts, they’re also seeking advice, navigating uncertainty, and making decisions. AI should do more than provide accurate answers, it should also help people evaluate evidence, challenge unsupported assumptions, and make informed decisions. At mpathic, we’re working to set that standard, especially for users vulnerable to misinformation,” said Dr. Alison Cerezo, Chief Science Officer at mpathic and licensed psychologist.
What the Benchmark Found
Across the six models combined, more than half of all responses (52.8%) contained at least one harmful behavior, most often a form of agreement: endorsing a risky plan, warming to a doubtful theory or treating a poor decision as reasonable. The rate varied widely by model, from 28% to 69%. In more than a third of responses overall, the model also presented itself as having expertise it does not have.
A correction can carry harm with it. Nearly every response that combined helpful and harmful behavior included an attempt to challenge the person’s belief, so a model could appear to correct someone while still behaving harmfully in the same response.
Models differed most in whether they could recover. After a harmful or mixed response, Claude Sonnet 4.5 returned a fully helpful response more than a third of the time; a pattern that was rare among the other models tested. The finding points to a second goal alongside preventing harmful responses: helping a model recognize when a conversation has turned and correct course.
“We need to ask the question: How do AI systems relate to people in real life? When evaluating risk and safety, it’s important to emulate user behavior, that is multi-turn, dynamic user conversations about real life rather than one-shot jailbreak prompts,” said Dr. Grin Lord, CEO/Founder of mpathic and licensed psychologist.
This is an initial baseline. Expert role-players, not real users, played ten conversations per model, so the results describe model behavior and say nothing about changes in people’s beliefs. The six models tested were Claude Sonnet 4.5, GPT-5.2, Gemini 2.5 Flash, Grok 4.1, Llama 3.3 70B and Mistral Medium 3. Models were tested through their public APIs in default configuration, between April 23, 2026 and August 13, 2026, so results may differ from the consumer products built on them. The full preprint, including methodology and model-by-model results, is available here.
About mpathic:
mpathic is a clinician-founded AI safety company dedicated to improving the quality and safety of human–AI conversations through expert-led evaluation and scientifically grounded benchmarking, focusing on high-risk moments across domains. The company works with frontier AI builders to prevent harmful or unwanted model behaviors, spanning use cases from user wellbeing and mental health to financial risk and customer support, across the full AI model lifecycle.
Disclosure:
mpathic works with several frontier AI labs across the industry, including labs whose models are evaluated in mPACT. No evaluated lab had input or editorial control over the study. We believe rigorous, independent evaluation is part of how we contribute to safer AI, and we apply the same methodology and clinical standards regardless of commercial relationship.
