Laya, the Open Rival to TypeSafe's Jev: Fast, but Near Chance on Unseen Decision Tasks

Laya, the Apache 2.0 decision model that Convai Innovations published on Hugging Face three days after TypeSafe AI launched its closed Jev API, has become one of the most visible open alternatives in the new "System One" category. An independent evaluation posted on September 22 shows how narrow its advantage really is. The tester confirmed Laya's speed but found the base model answering at roughly chance on held-out phishing emails, while the fine-tuned checkpoint that beats Jev scores above the ceiling that the benchmark's own labels can support.

That result matters beyond one repository. Laya demonstrates that the interface TypeSafe pitched as a new class of model — program state and typed questions in, calibrated probabilities out, all in one forward pass — can be rebuilt on an off-the-shelf encoder within days. What it has not demonstrated is the capability that makes a decision API worth paying for: answering questions defined at runtime without first training on them.

Laya Arrived Three Days After Jev and Borrowed Its Vocabulary

TypeSafe AI came out of stealth on September 15 with Jev, a hosted model that never generates text and instead returns a choice, a score, or a yes/no probability for each question a developer defines. The company reportedly raised $40 million in seed funding led by DCVC, and founder Diogo Almeida says he helped build the instruction-following methods behind ChatGPT while at OpenAI. Jev costs $0.042 per million input tokens with free output, TypeSafe reports 70 to 500 millisecond response times, and the company has published neither weights nor architecture details.

Laya's GitHub repository appeared on September 18. It copies Jev's three primitives — choice, score, and noul — and adopts the name TypeSafe coined for its training method, Reinforcement Learning for Calibrated Decisions (RLCD). At the time of writing, the repository showed about 4,500 stars and 403 forks, and the Hugging Face page listed 22 community fine-tunes and 21 quantized versions.

Developer Nandakishor M framed the release as vindication, arguing in a Dev.to post that his earlier research anticipated TypeSafe's idea. The papers he cites are related rather than equivalent: a March 2025 arXiv preprint (arXiv:2503.23303) predicted sales-conversion probabilities from Azure OpenAI embeddings trained on GPT-4o synthetic conversations, and a September 2025 preprint proposed routing LLM queries by pre-generation confidence. Both are legitimately related to the decision-probability space, though neither constitutes the same interface design.

How Laya Scores Options Without Generating Text

The English checkpoint pairs ModernBERT-large, a 395-million-parameter bidirectional encoder, with a decision head trained from scratch: two transformer layers, an option scorer, and an "act or escalate" head, for 421 million parameters in total. A multilingual sibling, available at convaiinnovations/laya-multilingual, uses the 322-million-parameter mmBERT-base encoder to cover more than 100 languages.

The key mechanism is how options enter the model. Each question is packed into one sequence, with a [MASK] token placed in front of every allowed answer, followed by the state being judged — such as an email or JSON ticket. After the encoder and head process the sequence, the hidden state at each [MASK] position is projected to a single score, and a softmax over those scores produces the probability of each option. Because answers are defined inside the prompt, a developer can invent a new schema without retraining, and malformed output is impossible by construction.

This is why Laya is fast. An autoregressive LLM produces an answer one token at a time, with each token requiring another pass through the network. An encoder reads the entire sequence once, and Laya batches every question about the same state into that single pass. Convai reports 39.5 milliseconds for one question on an Nvidia T4 and 7.2 milliseconds per question when ten are batched on the multilingual checkpoint. The English weights occupy about 808 megabytes, CPU inference takes roughly 193 to 464 milliseconds, and switching languages without preloading forces a 7-to-10-second model reload.

Training is where Laya leans on TypeSafe's framing. The policy outputs a probability distribution, exploration adds zero-mean Gaussian noise to the option scores, and the reward is a strictly proper scoring rule — a scoring function whose expected value is maximized only when the reported probabilities match the true ones. Laya combines logarithmic and spherical scores and subtracts a ranked probability score for ordinal questions, then updates with REINFORCE against a group-mean baseline, the same variance-reduction idea used in GRPO. The fine-tuned typed-decisions checkpoint also adds soft cross-entropy against the teacher's label distributions, so its training is not purely reinforcement learning.

Laya's Benchmark Win Sits Above Its Own Noise Ceiling

Laya's headline comparison uses typed-decisions, a synthetic benchmark on Hugging Face covering invoice processing, security incidents, customer service, and agent-trace observability. Its gold labels come from sampling a large language model teacher three times, and the teacher agrees with itself only 73.5 percent of the time. That figure acts as a ceiling on recoverable signal, and the dataset's authors warn that scores much above 0.75 mean a model has learned the teacher's quirks rather than the task.

Jev, which never saw the dataset, scored 0.727 zero-shot according to the benchmark's published results. Laya's base checkpoint scored 0.362, barely above a 0.318 random baseline and below the 0.461 achieved by always guessing each question's most common answer. Only after fine-tuning on the benchmark's 1,200-case training split did Laya reach 0.766 — a figure the independent tester reproduced at 0.767 before cautioning that above the ceiling the model "is memorizing noise." Convai's own card now states that Laya "is a fast base to specialise, not a zero-shot decision engine."

Even inside that benchmark, the win depends on which number is read. Laya's fine-tuned checkpoint picks the teacher's top answer more often, but Jev matches the teacher's full probability spread better, with soft accuracy of 0.580 against Laya's 0.471. For software that branches on thresholds rather than on the single most likely label, that distribution is the output that actually drives behavior.

The claims have also shifted since launch. The September 18 Dev.to post compared an 83.8 percent Laya score with a 67.8 percent Jev figure — which the independent tester noted came from two different benchmarks. On its own card, Convai concedes that Jev leads on Banking77, 0.870 to 0.425, because Laya's options share a fixed 256-token budget, leaving three or four tokens per label when a question has 77 answers. Jev supports up to 255 options.

Calibration, the property RLCD is meant to guarantee, also needs outside help. The card's header advertises "mathematically calibrated probabilities," yet the same page states that the checkpoints ship overconfident, with mean expected calibration error falling from 0.466 to 0.081 only after refitting a temperature per question type on held-out data. Proper scoring rules reward honesty on the training distribution; they do not guarantee it elsewhere. The English checkpoint scored zero accuracy on Khmer while reporting 95.2 percent confidence.

Read more: TypeSafe's Jev delivers structured AI decisions without generating a single word

Independent Probe Finds Laya Breaks on Negation and Renamed Options

The September 22 evaluation, run on a single RTX 5090, found Laya's speed claims hold, with a median of 10.7 milliseconds for one question. Accuracy was a different story. On 2,000 balanced emails from the PhishNChips dataset, raw Laya reached 0.505 accuracy because it labeled almost everything legitimate, catching 1.2 percent of phishing attempts.

The underlying ranking was not broken. Laya's AUROC — which measures how well scores separate the two classes regardless of threshold — was 0.678, close to Jev's published 0.689. Temperature scaling cannot move a decision threshold, so the tester fitted a Platt bias term on half the emails and lifted accuracy to 0.611, versus Jev's published raw 0.626 and Claude Haiku 4.5's published 0.813.

A probe of 11 assertions exposed deeper problems. Asked whether a ticket needed a human and whether a bot could resolve it, Laya's probabilities for those two options summed to 0.09, and questions paired with their negations were both answered "no" at 94 and 97 percent confidence. Renaming routing options without changing their meaning flipped the verdict. Laya failed seven of the 11 checks, though the tester stresses the suite is a self-written probe, not a benchmark.

Fine-tuning on 180,000 public items drawn from BoolQ, SQuAD v2, and natural-language-inference sets cut probe failures to one or two and raised typed-decisions accuracy to 0.676 — still below Jev's zero-shot score. It also worsened phishing detection, left renaming instability unresolved, and produced a checkpoint restricted to non-commercial use because two training datasets carry that license.

Open Weights Change the Cost Math, Not the Generalization Problem

For teams with a fixed schema and labeled examples, Laya offers something Jev cannot: weights that run inside a private network, with no per-token fee and no customer data leaving the premises. Convai's notebook fine-tunes a checkpoint in four to five hours on Kaggle's free pair of T4 GPUs. That makes it a credible replacement for a routing or moderation classifier that currently calls a frontier LLM. Self-hosting is not free, however, since GPU capacity, preloading every checkpoint, and on-call maintenance carry real costs that the "$0" comparison omits.

Read more: Local LLM inference costs more than you think: hardware, not electricity, drives the bill

Laya is also not alone. Within a week of Jev's launch, developers posted OpenJev, a 151-million-parameter model built on a ModernBERT-based GLiClass classifier, and an open reproduction that scored 0.697 on typed-decisions after in-domain training. Both descend from a long line of encoder-based zero-shot classifiers, which suggests the System One interface itself will not remain proprietary for long. Jev's case rests on generalization it has not yet let outsiders fully test: its benchmarks remain vendor-run, the typed-decisions reference labels are model-generated, and the service is still in early access.

The practical dividing line is therefore not open versus closed. If questions and options change at runtime, exceed about 20 choices, or arrive in untested phrasings, Laya's own documentation points developers elsewhere. If a workflow is stable enough to label, a fine-tuned encoder can now deliver sub-40-millisecond decisions on commodity hardware.

The next milestone to watch is a held-out-schema test: new questions and option sets, never seen in training, scored against human rather than LLM-teacher labels. TypeSafe has not published such results for Jev, and Laya's base model has not passed one. Whichever System One model clears that bar first will have shown that it makes decisions — not merely that it has learned to reproduce a labeler.