OpenAI's BioBasicsBench tests AI on messy wet-lab evidence: GPT-6 Astra leads at 66.7%, but still misreads the bench
OpenAI has a new way to find out whether its models can actually do science — not answer quiz questions about it, but read the messy evidence a working laboratory produces every day. BioBasicsBench, introduced on November 6 with enzyme-design company Ligo Biosciences, hands AI agents 100 tasks drawn from real experiments: gel images, colony plates, microscope stacks, sequencing reads, and instrument exports, and asks them to reach conclusions a bench scientist would accept.
- The test: 100 tasks from real Ligo experiments across five areas — protein methods, imaging and cytometry, assays and quantification, nucleic acids, and molecular cloning. Evidence includes gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photographs, and protocol videos.
- The scores (xhigh effort): GPT-6 Astra 66.7% (95% CI 60.3–73.1%); GPT-6.1 Sol 62.0%; GPT-5.6 Sol 43.8% (CI 37.6–50.0%); GPT-6 Luna 38.3%; GPT-5.5 35.4%; GPT-5.4 33.2%; GPT-5.2 31.2%. Astra used 4,704 output tokens per task, against 9,099 for Sol.
- Scale: 7 models across 37 model-and-reasoning-effort settings, five attempts per task — 18,500 graded results. Scores are mean rubric reward (partial credit), not the share of tasks fully solved.
- Where it helps: Astra read a handwritten “LB Kan50” label and rejected a kanamycin plate as a control for an ampicillin experiment; it spotted dye interference in a cell-imaging task and identified a focus problem in a z-stack.
- Where it fails: In a colony-PCR task Astra picked lanes 4, 6, 7 and 8 — the old plasmid — instead of lanes 5, 9 and 10; in colony counting it overcounted by about 12%. OpenAI says higher scores do not yet amount to reliable laboratory judgment.
OpenAI has a new way to find out whether its models can actually do science — not answer quiz questions about it, but read the messy evidence a working laboratory produces every day. BioBasicsBench, introduced on November 6 with enzyme-design company Ligo Biosciences, hands AI agents 100 tasks drawn from real experiments: gel images, colony plates, microscope stacks, sequencing reads, and instrument exports, and asks them to reach conclusions a bench scientist would accept.
The headline result flatters the newest model. GPT-6 Astra earned 66.7% of the available rubric credit at the highest reasoning setting, well ahead of GPT-5.6 Sol at 43.8%. But the benchmark was built to expose failure, not to crown a winner — and OpenAI’s own write-up is candid about the ways Astra still gets routine lab judgment wrong. The score is a measuring stick, not a milestone.
A benchmark built from real experiments#
Most science benchmarks hand models the clean figures from published papers — the chart that made it into the article, the result that worked. BioBasicsBench starts from the opposite end. Its tasks come from Ligo scientists’ own bench work, and they include the parts of laboratory life that never reach a publication: failed experiments, ambiguous images, conflicting observations, and controls that turn out to be wrong. The job is not to recall biology facts but to interpret evidence, spot errors, and recognize when the evidence cannot justify a conclusion.
Each task pairs raw material with an expert-written grading rubric that awards partial credit for findings, measurements, reasoning, and knowing when to hold back. The tasks went through three rounds of review: a Ligo scientist who had not written the task, an external PhD-level wet-lab scientist, and a final internal pass over the prompt, the evidence, the rubric, and a graded model attempt.
The scores, and what they actually mean#
OpenAI ran seven of its own models — four from the GPT-5 line and three from the GPT-6 line — across 37 combinations of model and reasoning effort, five attempts on every task. That is 18,500 graded answers, and the headline table is the share of available rubric credit each model earned at “xhigh”, the highest reasoning setting run on all seven.
| Model | Mean reward at xhigh effort |
|---|---|
| GPT-6 Astra | 66.7% |
| GPT-6.1 Sol | 62.0% |
| GPT-5.6 Sol | 43.8% |
| GPT-6 Luna | 38.3% |
| GPT-5.5 | 35.4% |
| GPT-5.4 | 33.2% |
| GPT-5.2 | 31.2% |
Two things matter more than the ranking. First, the number is partial credit, not a pass rate: 66.7% does not mean Astra solved two-thirds of the tasks, and a model can collect most of a task’s credit while still reaching the wrong conclusion. Second, Astra did it more cheaply in compute terms — about 4,704 output tokens per task against 9,099 for GPT-5.6 Sol — so the newer model is both stronger and more economical with its reasoning. The confidence intervals (60.3–73.1% for Astra, 37.6–50.0% for Sol) do not overlap, so the lead is not a rounding artifact. But there is no human baseline in the results, and no non-OpenAI model was tested — the benchmark, as reported, measures OpenAI against OpenAI.
Where Astra got it right — and where it went wrong#
The worked examples are the most revealing part of the release, because they show what better multimodal reasoning looks like in practice — and where it still breaks. In a cell-imaging task, GPT-5.6 Sol classified cells as dead from a dye signal; Astra recognized that overlapping detection windows could let a second dye interfere, and called for controls that could separate the signals. In a cloning task, Sol accepted an empty negative-control plate at face value; Astra read the handwritten “LB Kan50” label on the plate’s rim, noticed the plate contained kanamycin while the experiment used ampicillin, and rejected it as a control. In another task, cells seemed to vanish between images; Sol looked for a biological explanation, while Astra identified a z-stack — images taken at different depths — as the cause.
Then the failures. In a colony-PCR task, Astra sorted the colony lanes into the right two groups and then picked the wrong one: it assumed the larger DNA product meant successful cloning and chose lanes 4, 6, 7 and 8, which still carried the original plasmid. The reference answer was lanes 5, 9 and 10. Following that pick would mean purifying the wrong plasmid. In a colony-counting task it failed to separate close neighbours, missed visible colonies, and marked empty agar — an overcount of about 12%, which would inflate the estimated efficiency of the DNA transfer by the same amount. In a third case it generalized that cell vacuoles are darker than their surroundings despite lighter vacuoles in the same image, even though its own intermediate notes recorded that contrast varies with focus. The pattern OpenAI draws out is consistent: the model can observe the evidence correctly and still let the conclusion drift away from it.
What to watch#
Three things decide whether BioBasicsBench becomes a useful instrument or just another press-release number. First, whether OpenAI publishes the dataset, the rubrics, and the scoring code — independent checks on November 6 found no public link to any of them in the announcement, which means outside labs cannot yet verify the scores or run their own models against the same tasks. Second, whether other labs do run it: a benchmark that only ever compares one company’s models against each other tells you about that company’s progress, not the field’s. Third, whether the failure modes actually get fixed — OpenAI says tracking these capabilities will also inform its assessment of their biosafety and biosecurity consequences, so the benchmark doubles as an early-warning instrument for AI in biology.
The honest read is that 66.7% is both a real lead and a long way from the bench. A model that earns two-thirds of the available credit on routine lab evidence is genuinely useful as an assistant — and genuinely unsafe as a judge. BioBasicsBench is valuable precisely because it makes that gap visible, task by task, instead of hiding it behind a single number.
Sources#
- Unite.AI — “OpenAI Introduces BioBasicsBench to Test AI Agents on Wet-Lab Data” (November 6, 2026)
- Kingy — “BioBasicsBench: OpenAI’s Wet-Lab AI Results Explained” (November 6, 2026)
- R&D World — “OpenAI’s BioBasicsBench tests AI on everyday wet-lab data” (November 6, 2026)
- BigGo Finance — “OpenAI Debuts BioBasicsBench, a 100-Task Test of AI in Real Labs” (November 6, 2026)