OpenAI and Ligo Introduce BioBasicsBench to Test AI Agents on Wet-Lab Tasks
Frontier AI models · AI assistants
OpenAI and biotech company Ligo introduced BioBasicsBench, a benchmark of 100 routine wet-lab tasks evaluating AI agents on raw experimental biological data. Tested across seven models, GPT-6 Astra achieved a 66.7% mean reward at the highest reasoning effort ('xhigh'), exceeding GPT-5.6 Sol (43.8%). Despite improved multimodal reasoning, models still exhibited failures such as miscounting colonies and letting assumptions override controls.
OpenAI announced BioBasicsBench, an evaluation benchmark developed with enzyme-design company Ligo to assess how AI agents interpret routine wet-lab experimental evidence. The benchmark includes 100 tasks across five routine domains: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. It tests whether agents can parse raw inputs—including gel images, blot photographs, microscopy z-stacks, and plate readouts—detect errors, and recognize when evidence is insufficient to justify conclusions. OpenAI evaluated seven models across 37 reasoning-effort settings, conducting five attempts per task for a total of 18,500 evaluations. At the highest reported reasoning-effort setting ('xhigh'), GPT-6 Astra achieved a mean reward of 66.7% of available credit, compared with 62.0% for GPT-6.1 Sol and 43.8% for GPT-5.6 Sol. While GPT-6 Astra showed gains in multimodal inspection—such as detecting an incorrect antibiotic label on a negative control plate and recognizing a microscope z-stack—evaluations highlighted persistent shortcomings. Astra allowed prior assumptions to override control interpretations during colony PCR lane selection, discounted contradictory lighter vacuoles in microscopy images, and produced an overcount of approximately 12% in routine colony counting.
Key takeaways
- BioBasicsBench includes 100 routine wet-lab tasks across five biological domains, evaluating agents against expert-written rubrics using raw experimental data.
- At the highest reasoning-effort setting ('xhigh'), GPT-6 Astra scored a 66.7% mean reward, outperforming GPT-6.1 Sol (62.0%) and GPT-5.6 Sol (43.8%).
- GPT-6 Astra demonstrated enhanced multimodal inspection, including identifying mislabeled control plates and recognizing microscope focus depth scans.
- Persistent failure modes remain in wet-lab tasks, including letting prior assumptions override controls, discounting contradictory evidence, and colony miscounting.
Source：OpenAI News Read the original ↗
Prepared, checked and automatically published by AI; not individually reviewed by a human.