Widening a Recommendation Model Hurts AUC: What Meta Kunlun and 5,700 Teams Found

When Meta engineers wrote in a blog post in October 2026 that their Generative Ads Model had achieved "LLM-like scaling curves at the largest pre-training scale," they were describing something that had eluded the recommendation industry for years. The problem was not a lack of compute — Meta buys GPUs by the thousands — but a fundamental architectural mismatch that made adding more resources to recommendation models roughly as effective as pouring water into a cracked bucket. Meta's fix: redesign the bucket entirely, with a new architecture called Kunlun. Meanwhile, five thousand miles away, 5,746 teams from 52 countries were independently racing to solve the same problem at Tencent's invitation, and converging on strikingly similar conclusions.

The result is an unusually credible piece of AI research. The same structural insight — that recommendation models need both a unified stackable architecture and genuinely new informational inputs to scale — emerged simultaneously from Meta's closed engineering organization running thousands of GPUs and from two senior undergraduates running nearly a thousand experiments on seven GPUs in a university lab.

Why Recommendation Systems Cannot Stack Like Transformers

To understand why scaling recommendation models is harder than scaling language models, it helps to understand what they are actually computing.

A language model processes one type of input: a sequence of tokens. Every element of the sequence is treated with the same structural machinery — a transformer block — and those blocks can be stacked arbitrarily. Want a stronger model? Add more blocks, widen the layers, train on more data. The approach is almost embarrassingly simple in principle, and it works.

A recommendation model processes two structurally different kinds of input simultaneously: behavior sequences (your history of clicks, purchases, video watches, ad interactions) and feature fields (static attributes like your age cohort, the advertised product category, the time of day, your device type). These two input types have almost nothing in common at the data level, and the industry's standard response has been to handle them with separate specialized modules — a sequence model for the history, a feature interaction module for the attribute fields — which are run independently and concatenated at the end for final scoring.

The segregated design works well at small scale. The problem emerges when engineers try to make the model bigger. The sequence module and the feature interaction module use different architectures, which means there is no obvious unified block that can be stacked the way transformer blocks can be stacked. Dumping more compute into the connection point between the two separate modules does not help much, because by the time the outputs are combined, whatever fine-grained detail existed in the raw sequence has already been compressed away by the sequence module. The additional capacity has nowhere useful to go.

Meta described this dynamic precisely in its GEM blog post: scaling laws "don't come naturally" for recommendation systems, which is a pointed statement from a company that has spent billions establishing that they do come naturally for everything else.

Meta's Kunlun Architecture Integrates Both Operations Into One Stackable Block

Meta's answer, deployed in production as the Kunlun architecture (arXiv preprint), is to stop treating the two operations as separate. Kunlun interleaves sequence modeling and feature interaction inside a single unified block that can be stacked repeatedly, producing depth the way transformer blocks produce depth in language models.

The technical components include Generalized Dot-Product Attention (GDPA), which unifies three previously separate attention operations — self-attention, pooling multi-head attention, and cross-attention — into one efficient kernel that achieves roughly twice the forward pass speed of prior implementations at 97 percent Tensor Core utilization on NVIDIA B200 hardware, according to a separate Meta technical blog post. A mixture-of-experts cross-interaction layer handles feature relationships. Component Skip, a pruning technique specific to recommendation's structural properties, cuts floating-point operations by 43 percent with negligible accuracy impact.

The results Meta reported are company-claimed rather than independently verified, but the scale of deployment makes them difficult to dismiss entirely. Across Meta's ad surfaces on Facebook, Instagram, and partner sites, the company reports an 18.9 percent conversion improvement and 12 percent click-through rate gain. Total training compute for GEM increased four times in twelve months, while model FLOPs utilization — the fraction of peak hardware performance the training actually achieves — doubled to between 20 and 25 percent. At the largest pretraining scale, Meta says GEM now exhibits scaling behavior consistent with language model scaling laws, which is the technical claim the rest of the AI field has been waiting to see demonstrated in production recommendation.

One additional finding from Meta's experiments is instructive: prediction quality follows a power law with context length, and larger models benefit more from longer sequences than smaller models do. Meta doubled GEM's context length in production and tested a 25-times extension in capability experiments. This suggests that the new bottleneck is not simply model capacity, but how much historical information the model has access to.

An Open Competition Crowdsourced the Same Answer

Tencent's TAAC × KDD Cup 2026, held at the ACM KDD conference in Jeju Island, South Korea in August 2026, posed a precise problem statement: build a single unified module that handles both behavior sequences and feature fields for predicting whether a user will convert after seeing an ad, subject to strict latency constraints, using anonymized logs from WeChat Video Accounts with 35 million samples and 142 feature fields. No model ensembling permitted. Train and test sets split by time with non-overlapping users — conditions identical to production deployment.

The 5,746 teams that participated approached the unification problem from several different angles.

Read more: NUS open-sources JEVA: a decision-native ranking model that never flips on candidate reordering

QueryFormer (arXiv preprint), the industrial track champion from team sunshot, focused on the query generation step: before the model retrieves relevant fragments from the behavior history, it must decide what it is looking for. The baseline approach generates these queries by projecting user and ad features through a simple MLP. QueryFormer replaced that with a two-stage process: first, a self-attention pass within user tokens and within ad tokens, then a cross-attention pass between user and ad tokens. Only then are the enriched representations used to query the behavior sequence. Making query generation dynamic and sample-dependent — so the same architecture asks a different question for each user-ad pair — was identified in the competition paper as the largest single AUC gain of any individual component change in the competition.

CRAFT (arXiv preprint), the academic track winner from team lozyyeah, approached the problem from the perspective of how information travels through stacked layers. Its authors, including Zichen Luo, Jiachen Guo, Keming Gu, and Jie Zhang, framed recommendation as a "feature transport" problem: rather than having the current user-ad context participate only at the end, CRAFT's design makes the non-sequential context (the feature fields) act as an active controller of how sequence representations evolve at every layer. Each block generates two types of signals from the contextual field — a residual displacement that updates the representation, and a memory-preserving signal that governs what is retained. This prevents useful information from being diluted as it travels through depth. CRAFT's implementation is publicly available on GitHub. The model achieved a test AUC of 0.838148 with six stacked blocks, compared to the previous leaderboard leader's 0.83798.

UniDot (arXiv preprint) took the opposite philosophical approach: reviving a method from 2010. Factorization machines — a classical technique that computes inner products between feature vectors to measure their interaction strength — were considered superseded by deep neural networks. UniDot runs explicit inner product computations at every layer and routes them through a dedicated FM Highway directly to the final classifier, bypassing the deep network layers that would otherwise dilute or transform the signal. The result: an approach that was ancient by deep learning standards outperformed many architecturally novel systems when given the right position in the computation graph.

MaskRec, the Unified Block Innovation Award winner, challenged the assumption that unified architecture means fully connected attention — that every token must interact with every other token. Its design specifies attention access patterns by token role and scene, using global and scene-level memory tokens to aggregate cross-sequence information without requiring all-to-all attention. Unification, its authors argued, does not require mixing all signals indiscriminately.

Widening a Recommendation Model Without New Information Hurts Accuracy

The clearest and most counterintuitive result of the competition came from CoSFormer, the team that won the Scaling Law Original Breakthrough Award despite finishing only tenth in the academic track standings. Team gg — two senior undergraduates competing in a recommendation systems contest for the first time — ran approximately one thousand experiments across seven GPUs to test three different approaches to scaling their model.

In the competition itself, increasing the model's hidden dimension width while keeping the same input token types had caused AUC to drop rather than rise. Post-competition experiments confirmed the pattern more precisely. The first test — increasing width while keeping the same token inputs — produced AUC that plateaued rather than improved. The second test — adding tokens carrying no genuinely new information, inputs that expanded model capacity without introducing any additional signal — likewise plateaued. The third test: expand both model capacity and input information together, introducing new token types carrying genuinely new signals (pre-trained user and item embeddings, cross-interaction terms, sequence summaries at multiple time scales) while scaling model width in proportion. AUC improved monotonically until the available new information was exhausted.

CoSFormer's authors called this the "idle width" hypothesis: model capacity that has nothing new to learn does not learn anything useful and may actively interfere with existing representations. The practical implication is that recommendation AI scaling is not primarily a function of GPU budgets. It is a function of information architecture — specifically, how many meaningfully distinct input signals the model has access to.

This finding resonates with independent observations from Tencent's own RankUp research (arXiv preprint), which found that enriching the diversity of the model's latent representation space — through randomized feature splitting, multi-embedding tables, and cross-integration of pre-trained retrieval embeddings — produced consistent online gains in GMV across WeChat ad surfaces. The convergence across competitor, academic team, and competition setting strengthens the case that this is a structural constraint rather than an artifact of one experimental setup.

What Recommendation Scaling Actually Requires

The combined evidence from Meta's production deployment and the open competition points to a two-part requirement that differs substantially from language model scaling.

First, the architecture must unify sequence modeling and feature interaction into a single stackable block. Without this, additional layers have no consistent structure to exploit, and the compute is wasted. Every top-performing team in TAAC × KDD Cup 2026 abandoned the classical segregated approach.

Second, capacity growth must be accompanied by new informative inputs. Additional parameters without new data signals produce idle computation. This suggests that the research agenda for recommendation scaling is less about finding the right block depth and more about identifying what new categories of information — richer interaction histories, cross-surface behavioral signals, pre-trained semantic embeddings, real-time contextual data — can meaningfully occupy expanded model capacity.

Rui Li of Meta AI, speaking at the TAAC × KDD Cup 2026 workshop, noted that scaling recommendation models requires coordinating model growth with data pipelines, inference infrastructure, and training systems simultaneously. This is a more complex coordination problem than language model scaling, where the data (text from the internet) is largely pre-existing and the infrastructure (GPU clusters) is the main variable.

Whether a formal recommendation scaling law — a precise, reproducible relationship between compute and performance of the kind that has guided frontier LLM development — can be established from this foundation remains an open question. The TAAC 2026 paper's authors describe their finding as a concrete lead rather than a completed law. The difference matters: a lead tells engineers what direction to look; a law tells them exactly how much improvement to expect from a given investment.

The competition's training and evaluation datasets have been released on Hugging Face for non-commercial research use.