DiffusionNFT on MiniMax H3: Open-Source RL Training for Joint Audio-Video Generation

The VeRL-Omni team has published a complete open-source pipeline for applying online reinforcement learning post-training to MiniMax H3 — the 33-billion-parameter model that jointly generates video and stereo audio from a single transformer and is currently the only open-weight video generation system of its kind. The release, implemented in VeRL-Omni's multimodal training framework and backed by the DiffusionNFT algorithm accepted as an ICLR 2026 Oral, covers both text-to-audio-video and image-conditioned generation modes. What makes it technically significant is not the reward curves — those are still early — but the detailed catalogue of silent integration failures that cause RL training to appear to succeed while running in the wrong direction.

The central lesson of the work applies beyond MiniMax H3: every joint multimodal diffusion model that diverges from standard framework assumptions in its timestep convention, weight layout, or latent space packing will produce similar failure modes when developers attempt to apply standard RL training pipelines. Getting training metrics to move is not evidence that training is working correctly.

MiniMax H3 and Why Joint Audio-Video Generation Complicates RL Training

MiniMax H3 is architecturally unusual in ways that matter specifically for post-training. Its H3-Omni-Transformer is a 33-billion-parameter dense single-stream transformer that jointly predicts video and audio latents from a unified packed sequence. The visual modality is processed through H3-VisualVAE — a temporally causal autoencoder with 16× spatial compression, 4× temporal compression, and 24 latent channels — and further patchified with a 1×2×2 patch size before entering the transformer, producing an effective 32× spatial downsampling. The audio modality runs through a separate H3-AudioVAE that compresses 32kHz stereo audio to latent tokens at 40Hz per channel, with independent processing of the left and right channels before recombination.

The transformer uses three-dimensional multimodal rotary position embeddings across time, height, and width dimensions, and applies modality-specific AdaLN branches — not modality-specific attention or feedforward layers. This design choice has an inference consequence: the AdaLN modulation outputs can be precomputed and cached, so roughly 13 billion of the model's parameters do not need to be loaded for inference-only deployment even though they are included in the released weights. The text encoder is Qwen3-VL-32B, with hidden states extracted from its 50th layer passed to the Omni Transformer.

Two additional design decisions distinguish H3 from standard flow-matching models. First, H3 was released as a CFG-distilled checkpoint — inference requires no negative prompt and no classifier-free guidance scale. Second, H3's denoising timestep is defined as a data fraction (the interpolation proportion from clean data to pure noise) rather than the sigma convention used in most public flow-matching frameworks. The velocity vectors H3 expects are also sign-reversed relative to the standard convention. Neither of these differences causes a runtime error when mismatched — they silently invert or distort gradient flow.

DiffusionNFT: Skipping the Reverse Chain Entirely

Standard approaches to RL fine-tuning of diffusion models — DDPO, FlowGRPO, and similar methods — work by discretizing the reverse denoising chain and estimating the log-probability of generated outputs under the current policy, analogous to how language model RL computes log-probs over token sequences. This creates three structural problems for flow-matching models: the algorithm becomes bound to a specific ODE/SDE solver, the forward and reverse processes can become inconsistent during training, and classifier-free guidance must be incorporated into the training loop, adding significant complexity.

DiffusionNFT (Diffusion Negative-aware FineTuning) takes a different route. Rather than estimating likelihoods over the reverse denoising trajectory, it injects reward signals into the forward diffusion process — the direction from clean data toward noise — via flow-matching supervision. The mechanism compares positive and negative generation samples to define an implicit policy improvement direction. High-reward outputs are pulled toward a conventional positive flow-matching target; low-reward outputs receive a negative constraint. The training objective is then a modified flow-matching loss that does not require log-probability estimation and works with any black-box sampler.

The practical consequence of this design, according to the paper's head-to-head comparison, is up to 25× greater efficiency than FlowGRPO. On the GenEval benchmark applied to Stable Diffusion 3.5-Medium, DiffusionNFT raised the score from 0.24 to 0.98 within 1,000 training steps; FlowGRPO reached 0.95 only after more than 5,000 steps and with CFG enabled throughout. These numbers apply to a different model (SD3.5-Medium) and a different benchmark than the H3 experiments described here, and should not be treated as H3 performance projections — but they establish the baseline rationale for choosing DiffusionNFT as the RL algorithm for H3 integration. The VeRL-Omni team selected it first because its shorter training signal chain — requiring only the clean latent, the prompt embedding, and a training timestep, without storing every intermediate denoising step — makes the integration failure modes easier to isolate and verify individually.

How VeRL-Omni's Adapter Architecture Keeps H3-Specific Logic Contained

The implementation in VeRL-Omni follows a three-layer separation. vLLM-Omni handles rollout generation, preserving H3's native denoising loop and returning the clean latent, prompt embedding, and timestep alongside joint video and audio outputs. FSDP2 with Diffusers handles the actor training. VeRL-Omni itself orchestrates data movement, reward collection, DiffusionNFT loss computation, and LoRA synchronization back to the rollout policy. The H3-specific adapter — a separate module — handles all the model-specific translation: splitting the packed latent into video and audio segments, converting between timestep conventions, correcting velocity signs, and mapping Diffusers LoRA layer names to the fused attention and feedforward structures that vLLM-Omni uses during rollout.

This containment is operationally important. Without it, H3's non-standard conventions would need to be addressed individually in the trainer, the reward manager, the rollout system, and the synchronization code. Any one of those sites could silently absorb the convention mismatch without propagating a recoverable error. By isolating the translation layer in the adapter, the team ensured that failures manifested at the adapter boundary — still silently, but at least in a single predictable place.

The LoRA configuration uses rank 64 and alpha 128, with target modules explicitly specified as to_q, to_k, to_v, to_out.0, ff.net.0.proj, and ff.net.2. The explicit specification is not a stylistic choice — it is a requirement. H3's rollout transformer, as vLLM-Omni exposes it, fuses the attention projections and feedforward weights differently from how Diffusers represents the training checkpoint. Without explicitly mapping each projection to the corresponding fused slice in the rollout network, the LoRA adapter registers successfully but hits zero actual layers during rollout — meaning the actor updates never propagate to the sampling policy. Training loss moves, reward curves shift, and nothing changes in generation.

Read more: LAION's 80-million-video archive for multimodal training data

The Six Silent Failures: A Systematic Catalogue

The VeRL-Omni team identifies six distinct failure modes encountered during H3 integration, each characterized by the same property: the training run continues normally, loss remains finite, and reward curves show plausible dynamics, but the model is not actually being trained in the intended direction.

The first is the timestep convention mismatch. H3's DiT accepts data-fraction timesteps running from 0 (clean) to 1 (noisy), with velocity vectors sign-reversed from the standard flow-matching convention. A framework that passes standard sigma values without conversion inverts the gradient direction for every parameter update. No assertion fails; the training log shows nothing unusual.

The second is the rollout-to-actor weight mismatch. The training-side checkpoint (Diffusers format) and the rollout-side checkpoint (vLLM-Omni's fused DiT format) are structurally different representations of the same model. Capturing clean latents from the rollout side is straightforward, but verifying that LoRA updates applied to the Diffusers training checkpoint correctly propagate back to the fused rollout checkpoint requires explicit per-projection mapping. Without it, the adapter registers on the training side, the synchronization step completes without error, and rollout behavior is unchanged — because zero layers were actually updated.

The third is audio dropout in the reward pipeline. Standard video training frameworks handle single-modality latent spaces. H3 outputs a packed joint latent containing both video and audio segments. A reward pipeline that does not explicitly unpack and route the audio segment to CLAP and ImageBind audio-video scorers will compute rewards from video alone. Combined reward values will still change as video quality shifts, making it appear that audio is being optimized when it is not.

The fourth is tensor-parallelism LoRA skipping. At tensor parallelism degree 2, the first feedforward projection layer (ff.net.0.proj) is sharded across devices in a way that causes the LoRA update for that layer to be silently skipped. The fix requires explicitly splitting the LoRA for that layer to match the tensor-parallel sharding. This failure only manifests at TP=2 or higher, meaning single-GPU smoke tests pass while multi-GPU production runs silently under-apply updates.

The fifth is prompt drift from tokenizer re-encoding. H3's text encoder accepts pre-tokenized input from its native tokenizer. Frameworks that decode token IDs to strings and then re-tokenize them using a different tokenizer can introduce subtle token sequence differences that shift the conditioning signal. The H3-specific agent loop in vLLM-Omni sends token IDs directly to the text encoder to avoid this path.

The sixth is the LoRA rollout registration false positive. An adapter that registers successfully against the training-side model does not guarantee it maps to the rollout-side model, because the layer naming conventions differ between the Diffusers and fused DiT representations. The H3 adapter includes a preflight check that validates the target module list against the actual fused structure and raises an explicit error rather than allowing silent mis-registration.

Reward Design for a Model That Covers More Than Reward Models Measure

T2VA and FL2VA training both use the same two-component reward: CLAP (laion/larger_clap_general) for text-to-audio semantic alignment, and ImageBind (imagebind_huge) for audio-to-video alignment. The CLAP score answers whether the generated sound matches what the prompt describes. The ImageBind audio-video score answers whether the audio and video belong to the same scene.

The two scorers are deliberately complementary. A model optimized only on CLAP can satisfy the text-audio relationship while generating video that is entirely misaligned with the audio track. A model optimized only on ImageBind can generate audio and video that are mutually consistent but bear no semantic relationship to the prompt. Using both constrains the generation on both dimensions simultaneously.

The reward hacking risk with a model as capable as H3 is significant. The team notes that H3's base generation quality is already high enough that initial rollouts are usable — which makes RL effective but also makes reward hacking fast. Because CLAP and ImageBind together do not cover visual aesthetics, motion naturalness, long-context temporal consistency, or fine-grained semantic detail, a model that locates the reward model's preferences can raise the combined score while deteriorating on unmeasured dimensions. This is why the team recommends against interpreting rising combined reward as evidence of overall quality improvement without cross-referencing the decomposed CLAP and ImageBind sub-scores against fixed-prompt video samples.

On the T2VA run with 8 GPUs at TP=2, training reward rose steadily from approximately 0.27 to above 0.4 over the course of the run. On the FL2VA validation, reward at checkpoint recovery was 0.41 (CLAP 0.16, ImageBind 0.25), rising to 0.51 over three or more training steps with gradient norms in the 0.06–0.12 range. These numbers are author-reported and represent a single training run on the tested hardware configuration.

FL2VA: First-Frame Conditioning With Shared Infrastructure

The FL2VA (first/last-frame-to-audio-video) mode uses the same reward, actor, and LoRA synchronization infrastructure as T2VA but adds image conditioning through vLLM-Omni's official first/last-frame protocol. The rollout data contract carries the clean latent alongside condition-frame segment indices; the actor reads these metadata to fix the condition-frame latents before computing the DiffusionNFT loss. The optimization target is applied only to the generated video and audio latents — the given keyframe latents are not modified.

This constraint is load-bearing for the training task. An FL2VA model that also optimizes condition-frame latents is not learning to complete motion and audio from a given starting image — it is learning to rewrite the keyframe to fit its generated content. The distinction matters for every practical use case that motivated FL2VA: maintaining character identity across shots, bridging between defined scene endpoints, extending advertising footage from a given brand frame. The VeRL-Omni adapter enforces the constraint structurally by masking condition-frame positions from the DiffusionNFT loss computation.

The FL2VA dataset used in published experiments consists of 27,815 English video captions from ConsisID-preview-Data, filtered by DanceGRPO and available at that project's repository. Reference images are generated from FLUX.1-dev at 400×640 resolution with deterministic per-prompt seeds, producing 27,687 training samples and 128 test samples after a seed-42 shuffle. Datasets produced with this pipeline inherit the non-commercial restriction from FLUX.1-dev's license.

The rollout tensor parallelism for FL2VA is set to TP=4 rather than the TP=2 default for T2VA. The increase is required because the FL2VA rollout checkpoint includes conditioning image processing that occupies additional GPU memory alongside the DiT weights; at TP=2, the actor-to-rollout weight synchronization step encounters insufficient room on 96GB GPUs for the combined load.

Limitations of the Current Pipeline and What Remains Unresolved

The VeRL-Omni team is explicit about what the published pipeline validates and what it does not. The training runs demonstrate that a verifiable end-to-end online RL loop for joint audio-video generation is achievable — that the signal flows correctly from rollout through reward to actor update and back to rollout. They do not demonstrate that the current reward combination produces consistent improvements in perceptual quality across arbitrary prompts, or that the pipeline handles reward hacking robustly at scale.

The reward model coverage gap is the largest open problem. CLAP and ImageBind together cover text-audio alignment and audio-video coherence — two of the dimensions that matter for H3 outputs. Motion consistency, visual aesthetics, long-temporal coherence across the full 121-frame clip, fine-grained prompt adherence, audio quality independent of semantic alignment, and stereo field behavior are all outside the current reward scope. A model that optimizes aggressively on CLAP and ImageBind may degrade on any of these dimensions without the training signal detecting the change.

The training-side rollout uses only 10 diffusion steps for throughput, while validation uses 40. This is a practical tradeoff — full-quality rollouts at 40 steps would make the training loop roughly four times slower — but it means the latent distribution during training differs from the distribution during evaluation. The team notes that 2–4 steps are only suitable for verifying the data contract between components, not for assessing generation quality.

H3's sparse-attention implementation, which the model card describes as having been introduced in the final training stage, is not included in the open-source release. The published weights run with full attention. When MiniMax releases the sparse-attention implementation, inference cost for H3-Base is expected to decrease significantly for long sequences, which will also improve the economics of RL rollout.

Finally, the MiniMax H3 Community License restricts local weight deployment in the United States, European Union, United Kingdom, and South Korea. Developers in those regions working with the released VeRL-Omni training recipe would need to obtain an individual license from MiniMax — a process the company has said is available but has not published a timeline for.

Where H3 Sits in the Open-Weight Video Landscape

MiniMax H3's combination of open weights and native stereo audio generation currently has no direct equivalent. ByteDance's Seedance 2.5 generates clips up to 30 seconds — twice H3's 15-second ceiling — and accepts up to 50 reference assets, but it operates only through a closed API, has no fine-tuning pathway, and does not produce native audio alongside video. Google's Veo 3, which also generates synchronized audio, is likewise closed-API only and is not available for local deployment or post-training. Open-weight video models such as Wan (Alibaba) and CogVideoX offer fine-tuning pathways and have active RL research communities, but neither generates audio — their latent spaces are purely visual, which means all the joint audio-video training infrastructure built in this work would need to be redesigned from scratch rather than adapted.

The practical significance of H3 having open weights that include audio is therefore twofold. First, it is the only model where RL post-training can improve both audio-video alignment and domain-specific audio character simultaneously. A fine-tuning run for advertising video can, in principle, teach the model that sounds in a given brand's content should conform to a particular acoustic profile, that music tempo should align with product appearance pace, and that sound effects should match on-screen action — all within a single training objective applied to a jointly optimized latent space. Second, with dozens of adapter variants and fine-tunes already indexed on Hugging Face, H3's community adoption is already substantial enough that any methodology for RL fine-tuning reaches a ready audience.

The competitive moat is not absolute. MiniMax's copyright litigation with Disney, Universal, and Warner Bros. Discovery remains active in US federal court, which introduces legal risk for commercial workflows built on H3. The territorial license exclusions mean that the largest developer markets — the US, EU, UK, and South Korea — cannot legally self-host H3-Base under the Community License without a separately negotiated individual authorization, limiting RL fine-tuning to practitioners in other regions or those who have obtained individual licenses. These constraints do not affect the technical validity of the training methodology, but they do affect the population of developers who can legally deploy fine-tuned H3 checkpoints in production.

What the Pipeline Opens for Vertical Fine-Tuning

The core question the VeRL-Omni team set out to answer was whether online RL post-training for a joint audio-video diffusion model could be made to run correctly at all. The answer, after resolving the six categories of silent failure, is yes — and the modular adapter architecture that makes the answer possible is also what makes the result reproducible on future models.

For practitioners, the immediate practical value is a verified starting configuration. Rather than rediscovering each failure mode independently, teams working on H3 fine-tuning for advertising, game asset production, or character-consistent clip generation can begin from a recipe that has already identified the timestep convention correction, the LoRA mapping requirements, the audio routing constraint, and the tensor-parallelism LoRA fix. The published recipe documentation has not been independently reproduced outside the VeRL-Omni team as of the time of this reporting, so treating it as a starting point rather than a guaranteed result is appropriate.

The larger implication is methodological. As video generation models increasingly generate audio natively — and as the field moves toward models that also synthesize 3D structure, haptics, or other non-visual outputs — every new modality will bring its own set of framework assumption violations. The failure mode catalogue from this work provides a diagnostic template: check timestep conventions; check rollout-to-actor weight mapping for every LoRA target; explicitly route every modality to its reward scorer; validate with fixed-prompt samples rather than aggregate reward alone. That template will need to be re-applied for each new model architecture, because the silent failures are model-specific even when the categories are not.

The next technical milestone for this line of work is extending the reward scope. Audio quality, motion naturalness, and long-context temporal consistency are all measurable with existing models — they simply are not yet integrated. When they are, the constraint on reward hacking tightens, and fine-tuning for specific vertical use cases becomes more reliable. The infrastructure to run that extended reward loop now exists in open source.