Stanford-Led Team Maps LLM Agents in Computer Graphics With 300-Paper Taxonomy
A 14-author survey defines four axes for agentic computer graphics and finds a critical RL gap

A group of 14 researchers from Stanford University, the University of Texas at Austin, UC Berkeley, Simon Fraser University, and the University of Cambridge has published a comprehensive survey defining the emerging discipline of agentic computer graphics — systems in which large language model agents write, execute, and iteratively refine graphics code to produce 2D images, 3D scenes, and physics-based simulations.
The survey paper, posted to arXiv on November 1, 2026, reviews more than 300 papers and proposes a four-dimensional taxonomy that organizes the field around what an agentic graphics system is asked to do, what model it relies on, how that model is scaffolded, and how it interacts with graphics software and APIs.
A Field Taking Shape
The taxonomy introduces four axes for characterizing any agentic computer graphics system. The Specification dimension describes the input — natural language prompts, reference images, scene graphs, physics constraints, or combinations. The Base Model dimension covers the underlying AI model: vision-language models for tasks involving image interpretation, code-generating LLMs, or specialized domain models. The Harness dimension captures the agent framework — retrieval-augmented pipelines, multi-agent orchestration loops, tool-use scaffolding, and self-correction mechanisms. The Graphics Interface and Tooling dimension addresses how agents interact with professional software environments such as Blender, graphics APIs, or physics simulation engines.
Taken together, the authors argue these four axes offer a systematic way to position any existing or future system within the landscape of agentic computer graphics, and to identify which combinations have been well explored and which remain largely untested.
From 2D to Physics Simulation
The survey covers three major application areas. In 2D graphics, agents are evaluated on tasks including layout generation, vector illustration, texture synthesis, and style-consistent image editing. In 3D scene generation, the challenges span object placement, scene composition, material assignment, and consistent multi-view representation. Physics simulation adds a further layer of temporal reasoning: agents must generate code that produces physically plausible behavior, not just visually coherent output.
The authors use existing benchmarks to ground their review. BlenderGym, presented at CVPR 2025, provides 245 handcrafted task pairs spanning five categories of 3D editing inside Blender and serves as the primary point of reference for evaluating how well current vision-language models handle professional 3D workflows. 4DCodeBench, a newer benchmark introduced in October 2026, extends evaluation to dynamic 3D scenes, requiring agents to generate code that correctly handles temporal evolution — object movement, deformation, and simulation over time.
Where Current Models Struggle
The survey's analysis of model capabilities identifies a consistent pattern across all three application areas: current vision-language models have significant difficulty with spatial reasoning tasks that require understanding how objects relate to one another in 3D space.
Research published earlier in 2026 on spatial reasoning limitations in VLMs found that models trained primarily on 2D image-text pairs often fail to maintain coherent spatial relationships when asked to reason about depth, occlusion, or three-dimensional layout — a limitation that directly constrains their usefulness for 3D graphics tasks.
One recent system, VIGA (Vision-as-Inverse-Graphics Agent), was designed specifically to address this gap. VIGA frames graphics generation as an inverse-graphics problem, using a closed-loop architecture in which the agent renders intermediate outputs and uses visual feedback to correct errors in its generated code. Its authors report a 35.32% improvement on the BlenderGym benchmark compared to prior baselines, though this figure comes from the VIGA team's own evaluation and has not yet been subject to independent reproduction.
The RL Gap
The survey's most pointed observation concerns reinforcement learning. The authors find that while RL-based post-training has dramatically improved general-purpose language models on reasoning tasks, almost no work has applied this approach specifically to graphics domains. The survey frames this as a central open problem: without domain-specific reward signals derived from rendering quality, physics validity, or geometric accuracy, base models cannot develop the fine-grained iterative correction behaviors that professional graphics work demands.
This is not simply a data problem, the authors note. Even with large amounts of graphics code available, standard supervised fine-tuning does not teach a model to reason about why a rendered output is incorrect or how to fix it in a principled way. Reward-driven training against verifiable graphics outputs would, in the survey's framing, represent a qualitatively different kind of learning signal — one the field has not yet invested in building.
Open Problems
The survey closes by identifying several open problems for the field. Evaluation methodology is inconsistent across the literature, with many systems reporting results on custom or proprietary benchmarks that cannot be directly compared. Hallucination in graphics contexts is poorly characterized: agents confidently produce code that renders incorrect geometry or physically implausible behavior in ways that differ from language hallucination and have not been systematically studied.
The authors also raise questions about how agentic graphics systems should handle professional software environments that were not designed for programmatic control. Most current work either constrains agents to Python-scriptable interfaces or uses screenshot-based feedback, both of which impose significant limitations on the range of tasks an agent can perform reliably.
The survey represents a systematic attempt to bring definitional clarity to a research area that has grown quickly but informally. By proposing a shared vocabulary and mapping the landscape of existing work, the authors aim to give the field a common frame of reference for identifying what has been built, what is missing, and what the next generation of agentic graphics systems needs to do.
The paper, "Advances in Agentic Computer Graphics," is available on arXiv.



