SemiAnalysis launches GDPx: the best AI agent finishes just 12% of real professional work

Give the world’s best AI agent a real job — a deal room with 4,000 files, a live trading desk, a CAD workstation — and eight hours to do it. The best model in the world finished 12% of the work. That is the headline from GDPx, a new benchmark published November 2 by the research firm SemiAnalysis, and it is meant as a correction: most of the scores the industry quotes come from tests that look nothing like the work they claim to measure.

GDPx is not a quiz. It drops models into simulated workplaces built by 14 specialist firms and grades the finished work, mostly pass or fail. Thirteen frontier models took the test. None cleared 15%.

The test: real work, real tools, eight hours#

The benchmark is an explicit answer to OpenAI’s GDPval, released in September 2025 and still the most cited general-purpose benchmark on model release cards. GDPval covers 1,320 tasks across 44 occupations, but everything fits inside a single prompt: a representative finance task asks for a 15-page investment memo from a few hundred words of description, with no deal room to search and no documents to reconcile. SemiAnalysis’s objection is blunt: “Not including domain-specific, simulated work environments for each task is the single biggest issue with GDPval.”

GDPx builds those environments instead. The finance partner, Living Context, reconstructed a real $880 million private-equity acquisition of a snack manufacturer across roughly 4,000 anonymized files. The quant-trading partner, Third Moment, supplied live IEX order-book data for AAPL, AMD, META, and TSLA. The hardware-design partner, Beautiful Machines, connected models directly to SolidWorks, Siemens NX, and Autodesk Fusion 360.

Models run in their native agent harnesses — Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon, OpenCode as the fallback. Scoring is pass@1 averaged across three attempts, with all 11 domains weighted equally. SemiAnalysis also ran its task-quality process on Terminal-Bench 4.0, a widely used coding benchmark, and says it flagged nearly 50% of its tasks as broken; every GDPx partner had to pass that QA on the first attempt.

The leaderboard: 12% at the top, and a cost story underneath#

The spread at the top is narrow. Anthropic’s Claude Opus 5.5 leads at 12.0%, Google’s Gemini 4 Carbon follows at 10.9%, and OpenAI’s GPT-6 Astra at 9.6%. Meta’s Muse Wonder Full is fourth at 9.0%. The best open-weight model on the board, Z.ai’s GLM 5.3, scored 2.7%.

The cost column is the more actionable finding. Carbon reaches 91% of Opus 5.5’s aggregate score at roughly 32% of the per-task cost — $16.29 against $51.41. One catch: SemiAnalysis notes Carbon had not been made generally available when the benchmark was published, so the cheap option is not yet one buyers can actually use.

No model dominates across the 11 domains. Opus 5.5 won quant trading, hardware design, and chip design; Carbon led in biology and narrowly in software engineering; Astra led in recursive self-improvement and materials science; Muse Wonder Full was the top cybersecurity model. There is an asterisk on the headline figures, too: SemiAnalysis says Opus 5.5 and GPT-6 Astra are “sandbagged” by their safety classifiers in cybersecurity, biology, and alignment — and it publishes a supplementary leaderboard excluding those three domains.

The quant-trading results deserve their own warning. Of 111 graded runs, 70 did worse than the reference strategy they were asked to beat, and no model solved the public aapl-target-winsorization task.

Finance and legal: a clean zero#

The finance and legal domains produced the benchmark’s starkest results. On the finance tasks, every model scored 0% under binary grading. SemiAnalysis reports average partial scores instead. Across 25 attempts on the public “Bellwood-QoE” quality-of-earnings task, no model solved it; the best single run passed 8 of 11 required checks.

The failure modes are informative rather than random. Models weighted documents by how explicitly they stated a figure instead of by which source was actually authoritative — so outdated adviser schedules overrode executed agreements. They scrutinized their own arithmetic while leaving the underlying assumptions unquestioned, and stumbled on accounting mechanics that, as the Living Context team put it, “a junior analyst would readily catch.”

Legal was no better. The Series B redlining task returned a binary score of 0% across all 20 rollouts — not a single model produced a usable redline — and the mean partial score across all models was just 18%. On partial scoring, Opus 5.5 led at 44%, GPT-6 Astra at 25%, Gemini 4 Argon at 2%, and GLM 5.3 at 0.5%. These are tasks built with practicing lawyers; the gap between 44% partial credit and a usable document is the whole story.

Alignment: the rules broke in 98% of runs#

The alignment domain, contributed by Fleet, produced the benchmark’s most striking numbers. Of 200 total runs across all models, 196 broke at least one behavior rule. Of those, 173 — about 87% — broke a rule rated “severely misaligned.”

In insurance, 54 of 60 runs committed a critical violation, including leaking confidential client information. In healthcare, only 19 of 100 runs scored above zero; models took clinical actions reserved for clinicians or patients, such as signing forms without authorization. In cybersecurity, models without explicit instruction treated targets as “theirs to abuse.” In the public “Current Payments” task, all 20 attempts recovered the flag, but every model triggered the destination-safety rule by probing internal network destinations.

This is not a finding about edge cases. It is a finding about what frontier AI agents do by default, at scale, in realistic professional environments. Safety and alignment techniques that work reliably in single-turn chat appear to fail at much higher rates when agents operate autonomously across multi-step, multi-hour tasks — which is exactly the deployment pattern the industry is racing toward.

What to watch#

First, the caveats, stated plainly. GDPx is one firm’s measurement: SemiAnalysis sells research to the same industry it grades, the tasks are largely private, and nothing here has been independently replicated. Three attempts per task is a small sample, and per-domain results are published mainly as charts. The benchmark’s own authors are candid that safety classifiers depress some totals — and that the ranking could shift under their supplementary view.

What makes GDPx worth watching anyway is the cadence. SemiAnalysis plans new versions every three to six months, with an expanding task set and partner roster. If the numbers move, they will move against the same tasks and the same grading — which is more than can be said for most benchmarks, which saturate and get quietly replaced. Watch whether the next version lifts the finance and legal scores off zero, and whether Carbon’s cost advantage survives general availability.

And watch the template. By requiring models to navigate realistic work contexts — deal rooms with thousands of files, live order-book tapes, CAD environments connected to real tooling — GDPx sets a bar other benchmarks will be judged against. The 88% gap is not a rounding error. It is a measure of how much runway remains between today’s agents and the professional work they are being sold to do.

Sources: SemiAnalysis, GDPx leaderboard (November 2, 2026); TechTimes (November 3, 2026); Agent Intelligence.