IBM Encodes Instruction Boundaries in Granite 4.1 Tokenizer to Block Injection
Experimental 3B and 8B previews report prompt injection success falling from 53–100% to under 6%

IBM released two experimental language models on November 12 — granite-4.1-3b-llmon-preview and granite-4.1-8b-llmon-preview — built on a reformulated prompt structure that separates trusted instructions from untrusted data at the tokenizer level, before any safety classifier runs. The approach, called LLMON (LLM Object Notation, pronounced "lemon"), reportedly reduced indirect prompt injection success rates from a range of 53–100 percent on standard benchmarks to between 0.02 and 5.6 percent in IBM's own evaluations. Both models carry an Apache 2.0 license and are marked as experimental previews.
Why Agent Pipelines Are Structurally Vulnerable to Injection
Indirect prompt injection — OWASP's top-ranked risk for LLM-integrated applications — occurs when malicious instructions hidden in retrieved documents, email threads, or tool responses override a model's system prompt. Because standard chat-format models process every token in their context window with equal authority, a hostile string buried in a retrieved document can function as a command if phrased to resemble legitimate instructions. This ambiguity is not a patching problem: it reflects how chat-format instruction tuning trains models to find and execute instructions wherever they appear, with no structural distinction between authoritative system-level directives and content that happens to look like them.
Prior defenses include StruQ (UC Berkeley, USENIX Security 2025), which reserved special tokens as instruction/data delimiters and reduced attack success below 2 percent on static injection patterns, and CaMeL (Google DeepMind, 2025), which routes planning to a privileged LLM and data processing to a quarantined model that cannot call tools. LLMON extends the reserved-token approach from a two-channel separator into a full structured markup language that is both tokenizer-level and post-training learned, applied to a small-to-mid-scale open-weights model family.
How LLMON Uses Six Reserved Tokens to Encode Privilege
LLMON installs six delimiter tokens directly into the Granite 4.1 tokenizer vocabulary — entries that never appear in ordinary free text and therefore carry no semantic ambiguity with prose content. The six tokens — open, close, open_end, self_close, a period token, and a colon token — open and close typed blocks: system policy, instruction, data, document, tool definitions, and tool results. Block IDs are two-character identifiers encoded as single tokens, so addressing any block always costs exactly two tokens regardless of which ID is chosen.
An exec block at the end of each formatted prompt explicitly names which instruction to execute, which data inputs it operates on, and where output goes. Content marked exec:False remains in context but is not referenced by the exec block, treating it as material rather than directive. IBM describes this boundary as "a learned behavior, not an enforced guarantee" — a statistical training outcome rather than a cryptographic enforcement. A malicious instruction embedded in a retrieved document lands in a data block and, under the trained privilege rule, is treated as text to read rather than a command to execute.
Training Methodology: GRPO Reinforcement and Decoy Injection Data
IBM trained the LLMON previews from Granite 4.1 base checkpoints rather than the released Instruct variants, so the LLMON format is learned from scratch during post-training. The pipeline re-serializes a conventional instruction-tuning corpus into LLMON blocks while adding decoy unbound instructions and data blocks throughout the training data — with target outputs held unchanged — explicitly teaching the model to ignore unreferenced spans rather than merely avoid obvious injection patterns.
Post-training uses five stages of GRPO reinforcement learning covering reasoning, alignment, knowledge calibration, and mathematical competence, followed by a DPO alignment stage using 1,000 on-policy preference pairs drawn from the model's own responses to injection attempts. That final stage sharpens the learned boundary between executable and non-executable content based on what the model itself produces under injection pressure, rather than from a manually curated preference dataset. The full training methodology is described in IBM's HuggingFace blog post accompanying the release.
IBM's Benchmark Claims and Their Explicit Limits
IBM reports results on OPI (Open-Prompt-Injection), ASB (Agent Security Bench), and the StruQ evaluation suite — all static benchmarks testing models against fixed attack templates. On OPI the company claims LLMON models avoid 47 percent more attacks than the best frontier model it tested and 95 percent more than the best available open-source model, with attack success falling to approximately 0.1 percent at 3B scale and 0.02 percent at 8B from baselines of 53–100 percent on undefended models. General capability benchmarks show parity with Granite 4.1 Instruct — LLMON's post-training does not trade measurable general performance for security behavior.
The caveat IBM includes explicitly: OPI, ASB, and StruQ test only static injection patterns. Robustness against adaptive multi-turn attacks — where an adversary iterates on the model's responses to build authority gradually — is ongoing work, and IBM makes no robustness claims for that attack class. Static benchmarks set a floor on what LLMON can resist, not a ceiling on what a sophisticated attacker can construct.
Deployment as Agent Infrastructure, Not Chat Replacement
The LLMON previews are not drop-in replacements for Granite 4.1 Instruct. Conventional chat-formatted prompts are explicitly out of distribution for these models; feeding a standard messages array without the LLMON chat template produces unreliable behavior. IBM ships a chat_template.jinja file that handles LLMON serialization automatically via Hugging Face's apply_chat_template call, translating structured message objects — with named instruction, data, and requirement parts carrying exec flags — into the reserved-token markup. Developers do not write the markup directly.
LLMON also supports extensible custom tag vocabularies via LoRA adapters, enabling organizations to define domain-specific structural schemas. IBM demonstrates this with a ScienceWorld example using custom tags alongside a toolkit on GitHub and notebooks covering tool-calling agent workflows. The models are explicitly positioned as agent infrastructure — appropriate for retrieval-augmented generation pipelines, tool-calling systems, and multi-turn workflows — not as general-purpose chat assistants.
Two Papers, One Working Example, Many Open Questions
IBM links two preprints — arXiv:2610.38120 ("Post-Training Large Language Models with LLMON") and arXiv:2603.22519 ("LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface") — alongside the release, signaling that the work is intended to seed external evaluation rather than settle the question. How the structural boundary holds against gradient-based adversarial prompting and black-box probe sequences remains uncharacterized in published results, as IBM explicitly notes. What the November 12 release establishes is a compact but complete working example: two Apache-licensed models in which indirect prompt injection has been addressed simultaneously at the tokenizer vocabulary, the post-training objective, and the execution-block format — rather than as a classifier grafted onto the output side. Whether that combination proves robust enough for production agent deployments, and whether it generalizes across model families beyond the Granite 4.1 lineage, is what adaptive red-teaming will determine over the coming months.



