Mind2Act World: Diagnostic Benchmark Finds Robots Cannot Yet Link Reasoning and Execution
56,700 simulated episodes find the best agent solves just 12% of tasks requiring both reasoning and execution

A new diagnostic benchmark for robotic manipulation has produced results that cut against the prevailing narrative in embodied AI: the best-performing general-purpose AI agent is not struggling because it cannot think clearly — it is struggling because it cannot reliably grasp a jar.
Researchers from Tsinghua University's mmlab@SIGS laboratory, in collaboration with international institutions including Cambridge, Oxford, Stanford, and Carnegie Mellon, published Mind2Act World on November 19, 2026 as a preprint on arXiv. The benchmark evaluated nine distinct AI systems across 56,700 simulated episodes on a bimanual robotic platform, grading them separately on cognitive tasks, physical execution tasks, and tasks requiring both capabilities working in tandem. The results expose a structural gap that individual success-rate scores had largely concealed: even the highest-performing system collapses when thinking and doing must happen simultaneously.
The benchmark is now openly available, with the leaderboard accepting new entries at the project website and GitHub repository under an MIT license.
Why Separating Cognition from Execution Changes the Diagnostic Picture
The core design insight of Mind2Act World is deceptively simple: before measuring how well a robot integrates reasoning and physical action, measure each capability in isolation. The benchmark's 30 tasks divide into three equal groups of ten. Mind tasks require memory and decision-making but keep physical movements as simple as possible — a robot might need to recall the original stacking order of a steamer basket's layers and restore them. Act tasks provide explicit goals and test whether the robot can execute precise movements under time pressure, such as catching an object from a moving conveyor belt. Mind2Act tasks compose both demands into a closed loop, where what the robot remembers determines what it does next, and the result of one physical action constrains the next decision.
Each task runs at three difficulty levels and in two scene configurations — a clean baseline and a randomized setting with additional objects, varied materials, and altered lighting. The benchmark's scoring is uncompromising: success is determined by physical outcome alone. An object either arrives in the correct position or it does not. A button is either pressed or it is not.
This structure allows the benchmark to generate what it calls "diagnosis cards" — per-model profiles that attribute every failure to its earliest point of collapse, categorized as a Mind error, an Act error, or a Composition error occurring after both capabilities were individually exercised. Across all 56,700 scored episodes, approximately 41% of failures were traced to reasoning errors, 37% to execution errors, and roughly 10% to genuine composition failures where both individual capabilities had been demonstrated but broke down when combined.
Read more: GPT-6 Astra Outperforms Specialized Robot Systems in Simulation
GPT-6 Astra Leads the Agent Category but Cannot Carry a Tray
The benchmark evaluated three large-language-model agents — GPT-6 Astra, DeepSeek-V4.1-Flash, and Gemini-4-Argon — that control the robot through tool calls rather than direct action prediction. These systems access perception and motion-planning modules as external APIs and must issue explicit commands to manipulate the environment.
GPT-6 Astra achieved the best agent-track scores: 52.7% on Mind tasks, 39.0% on Act tasks, and 12.0% on Mind2Act tasks, for an overall score of 34.6 out of 100. Its memory tasks were genuinely impressive: in a hidden-hazard navigation task, it succeeded on every single episode. Its card-matching success rate reached 93%. When scene randomization changed the table surface and lighting conditions, its Mind success rate shifted only from 54.7% to 50.7%, suggesting that its cognitive performance is relatively robust to visual perturbation.
Its physical execution tells a different story. Across all 30 episodes of a tray-carrying task requiring precise bimanual coordination, GPT-6 Astra lost control of the tray on every lift. Its diagnosis card reveals that 55.3% of its 589 failed episodes involved a physical execution error as the first point of failure. Critically, 459 episodes ended when GPT-6 Astra chose to stop voluntarily — and 264 of those self-terminations occurred with more than one-third of its decision budget remaining. The pattern recorded in the paper: GPT-6 Astra's motion-planning module rejected the target pose, and the agent did not retry. The benchmark researchers note that more granular motion-planning tools and automatic retry logic before stopping would address this failure more directly than simply increasing the action budget.
The cost and latency profile of agent-based approaches carries its own implications. GPT-6 Astra required an average of 11.0 seconds per API call, with an average cost of $25.13 per episode and $72.70 per successful episode. While benchmark simulations paused the clock during inference, real-time deployment cannot — and several Act tasks impose time windows measured in seconds.
DeepSeek-V4.1-Flash and Gemini-4-Argon scored considerably lower overall: 6.6% and 4.7% respectively. DeepSeek's cost per episode was $1.24, Gemini's was $1.12 — roughly one-twentieth the cost of GPT-6 Astra per episode, though with dramatically lower task completion rates. Neither agent achieved any success on Mind2Act tasks.
End-to-End Models Excel at Execution but Cannot Remember What They Just Saw
The benchmark's end-to-end policy track evaluated four Vision-Language-Action models — KinRT, π0.5, G0.5, and X-VLA — and two World-Action Models, InternW0 and FastWAM. All six were fine-tuned using the same set of 4,500 expert demonstrations comprising 75.8 hours of robotic manipulation data, allowing direct comparison.
VLAs and WAMs map visual observations and language instructions directly to robot actions without calling external APIs. Their inference latency reflects this architecture: π0.5 generated action predictions in 77.1 milliseconds, while InternW0 required 488.1 milliseconds — both are orders of magnitude faster than the multi-second API round-trips required by agent systems.
The end-to-end models showed their strength on individual execution tasks. KinRT, π0.5, and X-VLA achieved Act success rates between 12.0% and 19.6%. On a timed gate-opening task, three VLAs scored between 76% and 79% — competitive with GPT-6 Astra's 80%. X-VLA reached 64% on a cloth-folding task. These results reflect physical execution competence: fast inference, smooth trajectories, and good fine-motor coordination on well-scoped tasks.
The story reverses sharply on Mind tasks. All four VLAs scored below 5.4% on tasks requiring memory and decision-making. When scene randomization was applied, the pattern deepened: VLA models retained 86%–94% of their Act success count but only 28%–41% of their Mind success count. The researchers note that VLA models appear to represent scenes as "what they look like" rather than "what they contain" — their internal representations do not reliably encode the semantic state information needed for history-dependent decisions.
InternW0, the top-ranked end-to-end system with an overall score of 8.68 out of 100, provides the clearest diagnosis. In a cooking task requiring the robot to place all ingredients before activating heat, 271 of InternW0's 300 episodes involved activating the heat source before all ingredients had been placed. The model's 8,220 failed episodes included 761 classified as "precondition unmet" errors — cases where it proceeded to the next step without confirming that the previous step was complete. The training data contained only successful episodes; the model had no exposure to error-and-recovery sequences, leaving it without a mechanism to check its own work before continuing.
FastWAM produced the lowest overall score at 2.9 out of 100, with Act performance of 7.4% and Mind performance of 0.9%.
Read more: Survey: Why Getting Motion Right Is No Longer Enough to Ship a Robot
Eight Tasks No System Could Complete
One of the benchmark's most direct findings: eight tasks were not completed by any of the nine systems across the entire evaluation. These include tray-carrying (DR4), domino placement (PR5 and PR6), and four Mind2Act composition tasks (CP03, CP04, CP07, CP09). On two of those tasks — a bridge-building task and a whiteboard restoration task — no system earned even first-stage partial credit.
The pattern across the nine systems is complementary rather than convergent. The capabilities that make agents strong (robust memory, language-guided reasoning, scene-invariant cognition) are precisely what end-to-end models lack. The capabilities that make end-to-end models competitive (fast inference, smooth execution, fine-motor coordination) remain out of reach for general-purpose agents whose motion planning fails at precision tasks. This complementarity suggests that the current framing of "agents versus VLAs" as competing paradigms may be obscuring the more important question: what does each approach need to close its specific gap?
Diagnosis Cards as Infrastructure for Targeted Improvement
Beyond ranking systems, Mind2Act World frames its diagnosis card methodology as an infrastructure proposal for the field. The logic is straightforward: if a robot consistently fails at the first step of a 10-step task, additional training data covering steps 6 through 10 is wasted. The diagnosis card identifies exactly where in the task sequence episodes break down, which the benchmark records as the "first irreversible error."
For G0.5, the card shows that 1,324 failed episodes had a first error of missed or failed grasp. In a block-assembly task, 202 of 300 episodes ended without the robot ever gripping a single block. For the whiteboard restoration task, 299 of G0.5's 300 episodes terminated before the robot removed the whiteboard from its stand — meaning nearly all demonstration data for the later steps of that task would be inaccessible during training, since the robot never reached them. The researchers' proposed intervention: collect rollout data starting from the failure state, not from the beginning of the task.
The paper also identifies behavioral patterns in its diagnosis cards using informal descriptors. InternW0 and G0.5 exhibit what the authors characterize as impulsive execution — proceeding confidently into the next step regardless of whether the prior step succeeded. GPT-6 Astra exhibits premature giving-up — terminating with substantial budget remaining when a tool call fails, rather than attempting an alternative approach.
These are not merely rhetorical characterizations. Each pattern points to a specific, tractable intervention: state-confirmation loops for execution-first models; retry-and-reroute mechanisms for agent-based models. The benchmark describes a "self-evolving" loop in which diagnosis outputs feed directly into the next training cycle, so that benchmark performance translates into concrete model improvements rather than a static leaderboard position.
Simulation Scores Still Carry Caveats
Mind2Act World is a simulation benchmark, evaluated on the ARX X5 bimanual robotic platform in a physics simulator rather than on physical hardware. The ARX X5's simulation environment does not yet capture all real-world failure modes: surface friction variability, sensor noise, actuator wear, and environmental unpredictability each introduce failure pathways that do not appear in simulation data. The benchmark's own cost figures — $25.13 per episode for GPT-6 Astra, $72.70 per successful episode — apply only to the API costs and do not include real-world hardware expenses, human safety monitoring, or the latency penalties incurred when inference cannot pause the clock.
The benchmark's decision to use 4,500 demonstrations for fine-tuning end-to-end models is also a methodological limitation the paper acknowledges: a larger demonstration set might raise baselines substantially and change the relative ordering of systems. The evaluation is intended as a diagnostic instrument rather than a final ranking, which is why the leaderboard remains open to new submissions and new systems.
The benchmark's academic authorship spans Tsinghua, Cambridge, Oxford, Stanford, CMU, and ten additional institutions. The affiliated commercial organizations represented among individual authors include Velabot AI, Simple AI, and Baidu AI Cloud. The Chinese technology company 远行光年 (Yuan Xing Guang Nian) is credited in the benchmark's public announcement materials but does not appear in the academic paper's institutional affiliation list.
What Mind2Act World surfaces is not a verdict on which paradigm will prevail. It is a clinical record of where nine current systems break down, granular enough to inform what data to collect and what tools to build next. The gap between GPT-6 Astra's 52.7% on memory tasks and its 12.0% on tasks requiring memory and manipulation together is not primarily a reasoning failure — it is an execution failure that reasoning cannot compensate for. Closing that gap requires progress in physical manipulation, and Mind2Act World is now the available instrument for measuring whether that progress is happening.



