Agentic Robots Survey: Why Getting Motion Right Is No Longer Enough to Ship a Robot

Robots are increasingly able to pick up objects, navigate rooms, and follow natural language instructions. What they cannot reliably do is finish a complex task when something goes slightly wrong. A new 79-page survey submitted to arXiv as a preprint by researchers at CUHK MMLab and Fudan University argues that solving this problem requires rethinking which part of the system deserves engineering attention — and that the answer is not better motion prediction.

The paper, submitted on October 14, 2026, introduces the concept of "agentic robots" — systems capable of planning and coordinating execution using environmental observations and execution feedback. The authors are careful to distinguish this from full robot autonomy. "Agentic" refers to how decisions are internally organized: which model to invoke, when to call it, how to detect that an action did not produce the expected state change, and how to recover before the next step.

That framing matters because the robotics industry is in the middle of an unprecedented commercialization push with a widening gap between simulation scores and real deployments. China produced more than 40,000 humanoid robot units in the first half of 2026 — 97 percent of global production, according to China's Ministry of Industry and Information Technology — while financing for the sector reached 93.5 billion yuan in the same period, roughly five times the equivalent figure a year earlier. At the 2026 World Robot Conference, staff were observed calibrating model parameters on-site before demonstrations. The gap between benchmark numbers and working products is not theoretical.

Four Evolutionary Stages Define the Gap Between Benchmarks and Deployment

The survey maps four evolutionary stages of robotic intelligence. Task-specific policies, trained through reinforcement learning or behavioral cloning on demonstrations, can execute a single skill reliably in controlled conditions. Vision-Language-Action (VLA) models extend this by incorporating pre-trained visual and language knowledge, enabling more flexible instruction following without task-specific retraining from scratch. World models and world-action models go further, predicting the environmental consequences of actions before executing them — a capability that can reduce costly real-world training data by running simulations internally.

The fourth stage — agentic robots — does not replace any of these. It organizes them. A robot navigating a kitchen with conflicting instructions, mid-task interruptions, and changing object positions needs to decide which model handles which sub-problem, when to pass control from a high-level planner to a visuomotor policy, and when execution has quietly failed and a recovery step is needed. None of the first three stages includes that decision logic.

The paper frames this coordination layer using a concept it calls a Harness: the external software structure that specifies how models are called, what state is maintained between steps, and how failure is detected and handled.

The Harness Layer: What Orchestration Actually Requires

The Harness domain is the survey's most distinctive contribution. Prior work on VLAs, world models, and robot learning treated the system layer as an implementation detail rather than a research area. This paper gives it six functional categories: Orchestration, Memory, Monitoring and Recovery, Skills, Infrastructure, and Layered Systems.

Orchestration handles handoffs between model components. A slow-reasoning planner (what the paper calls System 2) breaks the task into sub-goals; a visuomotor policy (System 1) executes each one using learned sensorimotor mappings; a low-level controller (System 0) translates outputs into joint torques. Each boundary is a potential failure point. Memory stores information across steps without requiring weights to be retrained between tasks. Monitoring and Recovery detects silent failures — cases where a skill reports success while the underlying task state did not change — and re-plans accordingly.

Skills are reusable, callable procedures that can be added to the robot's repertoire without end-to-end retraining. Infrastructure defines how software communicates with hardware: message formats, latency budgets, and sensor interfaces. Layered Systems describes the full three-tier architecture and how it handles conflicts between abstraction levels.

The practical significance of this structure can be seen in RoboClaw, an agentic framework described in a separate preprint from AgiBot and Shanghai Jiao Tong University. According to that preprint, RoboClaw reported a 25 percent improvement in success rate on long-horizon tasks and a 53.7 percent reduction in human operator intervention time compared with non-agentic baselines. The gains came primarily from failure detection and recovery logic, not from improvements to the underlying motion policy.

Current Benchmarks Measure the Wrong Thing

The survey's most practically significant argument concerns evaluation. VLA models currently report success rates above 90 percent on the LIBERO simulation benchmark. Those numbers are misleading in a specific way, as a 2026 survey of VLA evaluation methods covering 582 papers documents.

LIBERO measures performance under fixed conditions: the same object positions, the same instructions, the same lighting. Under LIBERO-PRO — a variant that introduces perturbations to positions, lighting, and instruction phrasing — the same models' performance collapses to 0.0 percent, according to its authors. The standard benchmark is measuring robustness within a training distribution, not robustness to the variation that real deployments involve.

The problem is structural: when the Harness layer improves task success, benchmark scores improve alongside model improvements, and the two effects cannot be separated by measuring success rate alone. A robot team that improves its recovery logic will appear to have improved its model. A team that improves its model will appear to have improved its recovery logic. Neither the benchmark nor the success metric can distinguish them.

The survey identifies cross-embodiment generalization as an open challenge with similar structure. Skills learned on a wheeled robot do not transfer directly to a robotic arm or a humanoid. Benchmark scores for one platform say little about another, yet investment decisions and product timelines regularly treat platform-specific success rates as general capability claims.

Four Open Challenges the Survey Identifies

The survey organizes all 17 functional categories across four domains — Model, Data, Environment, and Harness — and identifies four open challenges where current investment is misaligned with deployment blockers.

Data collection relies heavily on human teleoperation demonstrations, which are expensive and produce few failure examples. The survey notes that failed trajectories contain useful sub-trajectory segments — the successful drawer-opening phase of a failed grasp-and-place task — that binary success/failure filtering discards. Agent-guided annotation and refinement can recover that signal.

Simulation environments are mature for static evaluation but weak for dynamic perturbation and cross-embodiment transfer. The survey argues that evaluation needs reconstruction metrics assessing whether the robot's internal state representation matches the actual world, not only whether a final success state was reached.

The Model domain needs tighter integration between VLAs and world-action models. Current systems treat prediction and execution as separate phases. Coupling the action policy to the world model's prediction — and updating that prediction after execution — could reduce recovery steps when unexpected states arise.

What the Field Needs to Measure Next

The survey arrives at a moment when the robotics industry's standard metrics are losing explanatory power. Simulation success rates are saturating. Hardware production is scaling. The gap between them is not being closed by larger models alone.

The 17-category framework the CUHK and Fudan researchers propose gives the field vocabulary for decomposing that gap. Rather than asking only whether a robot achieved a task, it becomes possible to ask which functional category limited performance: was it the planning module's failure to decompose a goal, the monitoring layer's failure to detect an incomplete state transition, or the skill library's failure to generalize to an unseen object orientation?

That diagnostic resolution is what the next generation of robot evaluation needs — and what current benchmark design, concentrated in simulation and measured at success rate alone, cannot provide.