Evidence-grounded research synthesis, benchmarked
Can LLM agents turn biological literature into claims you can actually check?
This is not “ChatGPT for biology.” It is a reproducible benchmark that scores agent architectures on whether their claims are traceable to real evidence — and just as importantly, on how often they aren’t.
The headline finding so far
Adding a query-planning step (Agent 1) is not a free upgrade over plain retrieve-then-synthesize (Baseline 1). Once one run’s planner failure is fairly charged against every metric — not silently dropped from the denominator — Agent 1 loses to Baseline 1 on 6 of 8 shared metrics, including a clean 0% vs. 6% unsupported-claim rate.
Baseline1 unsupported-claim rate: 0% · Agent1 (worst-case for its one failure): 6%
See the full architecture comparison →Failure explorer
Real, representative failures: a planner's malformed JSON, a retrieval miss mistaken for a bad claim, a hallucination trap.
Citation audit
Every citation checked for three things: does the source exist, was it retrieved, does the passage actually support the claim.
Methods & limitations
What's frozen, what's live, what's simulated, and exactly what this benchmark does not yet license you to conclude.