Trida

Diffusion drafts. AR verifies.

Trida models draft a block of tokens with diffusion and check it with their own autoregressive path, in one forward pass. You get the AR answer, several tokens at a time.

Get Trida2.0-4B SGLang or vLLM · OpenAI-compatible server

More tokens
per forward pass.

Trida2.0-4B served self-speculatively, against the same checkpoint decoding one token at a time.

01 / Parallel commit

Tokens per forward pass

2.59tokens

In the serving benchmark, up to 3.10 on GSM8K. AR decoding commits 1.00.

AR decoding 1.00
MMLU-Pro 2.33
BFCL 2.33
GSM8K 3.10

02 / Throughput

Tokens per second, per request

243tok/s

Versus 216 tok/s for the AR path, with verify-guaranteed AR quality.

AR decoding 216 tok/s
Self-speculative 243 tok/s

Concurrency 1, threshold 0.9, reasoning on, HybridDiffusion SGLang integration. The full-mask recipe makes Trida2.0-4B a better drafter, so it is fastest at low concurrency.

The Trida models.

Trillion Labs' block-diffusion language models. Trida2.0-4B and Trida2.1-4B are built on Qwen3.5 with a trained AR path in the same weights; Trida-7B, the first generation, comes from Tri-7B.

Open weights

Qwen3.5-4B base

Trida2.0-4B

The reference model. Block size 8, served self-speculatively or as plain AR. Tuned from an agent-aligned base for tool use and Korean agent tasks.

65.2BFCL non-live AST, self-speculative
Hugging Face ↗ Benchmarks
Coming soon

Writes and decides

Trida2.1-4B

A self-speculative agent model with a typed-decision head and the Qwen3.5 vision tower. One set of weights generates text and answers typed questions with calibrated probabilities.

Until then, use Trida2.0-4B ↗ Decision results
Open weights

First generation · since January 2026

Trida-7B

The first-generation Trida model, where the line started: to our knowledge, the first block-diffusion language model openly released in Korea, continually pre-trained from Tri-7B, for Korean, English and Japanese.

Hugging Face ↗

How one model
does both.

Each training row holds the prompt, a noised copy of the response and a clean copy, tied together by one attention mask.

Training row layout: prompt S, then the noised response x_t, then the clean response x_0. S · prompt x_t · noised response x_0 · clean response causal, visible to everything after it bidirectional inside its block → diffusion causal → the AR path that verifies
Loss = masked cross-entropy on x_t (diffusion) plus next-token cross-entropy on x_0 (AR), weighted. Keeping the AR weight above zero is what makes self-speculative decoding and plain AR serving possible from one checkpoint.
01

AR across blocks

Each noised block attends to the prompt and to the clean copies of earlier blocks, so generation stays left-to-right block by block.

02

Diffusion within a block

Inside a block, positions see each other in both directions, so the whole block can be predicted in a single pass.

03

Lossless verification

Every committed token is an accepted draft equal to the AR token, or the AR correction. The gain is tokens per pass, never a different answer.

Trida2.0-4B
benchmarks.

Concurrency 1, threshold 0.9, reasoning on. A vanilla Qwen3.5-4B baseline is being added.

BenchmarkSelf-speculativeAR
GSM8K84.289.3
MMLU-Pro63.0—
IFEvalprompt-strict69.7—
BFCLnon-live AST65.2—
Ko-AgentBenchL1 / L20.909 / 1.00—
FunctionChatcall / completion0.85 / 0.88—
tau2retail / airline / telecom0.40 / 0.50 / 0.80—

From the Trida2.0-4B model card. Ko-AgentBench and tau2 are small suites, so differences under about 0.1 are within run-to-run noise.

Trida Stack.
Bring your own model.

The open training and serving code behind the Trida models. Start from a pretrained AR checkpoint; no diffusion pre-training needed.

01 / TRAIN

AR → diffusion SFT

Native-torch FSDP2 trainer with packing, bucketing and multi-turn supervision. The output is a stock Hugging Face causal LM plus a mask token.

02 / SERVE

One checkpoint, three modes

Causal, block-diffusion or self-speculative decoding behind an OpenAI-compatible server, on SGLang or a vLLM plugin.

03 / EVALUATE

Benchmarks included

Quick GSM8K, MMLU-Pro and IFEval checks, plus agentic harnesses: BFCL, tau2, FunctionChat, Ko-AgentBench, SWE-bench.

Train: AR → block diffusion
PYTHONPATH=. torchrun --nproc_per_node=8 train.py \
  --model_id Qwen/Qwen3-4B \
  --bd_size 32 --ar_loss_weight 0.2 \
  --max_length 32768 --pack \
  --save_dir checkpoints/my-bd-run
Serve: self-speculative by default
# diffusion draft → AR verify (default)
python inference/serve.py --port 30000

# the same checkpoint as plain AR
python inference/serve.py --mode causal --port 30000

Validated model families

FamilyAttentionNotes
Qwen3Full attention (dense)Baseline path: dense block masks, standard KV cache
Qwen3.5Hybrid: gated-delta linear + periodic fullHybrid-aware masking; rollback by state snapshot and restore

The trainer works with any Hugging Face causal LM whose decoder is reachable via model.get_decoder(). serve.py also fronts other block-diffusion families: SDAR and LLaDA 2.

GitHub · Apache-2.0 trillion-labs/trida-stackTraining, serving and benchmarks in one repo. Notebook First inference runServe Trida2.0-4B and compare decode modes. Notebook First training runConvert an AR checkpoint to block diffusion.

A few details

Can Trida2.0-4B decode with pure diffusion?

Not with this recipe. It is trained for single-shot full-block drafts, which is exactly what self-speculative decoding needs. Serve it self-speculatively or as plain AR.

Which serving engines are supported?

SGLang, through the HybridDiffusion integration, and a self-contained vLLM plugin. Both expose an OpenAI-compatible endpoint.

What does a checkpoint need to be servable?

A stock causal LM, an added mask token, and a block_diffusion.json sidecar with the block size, mask id and AR loss weight. The repo's model contract lists the details.

What are the licenses?

Trida Stack and Trida2.0-4B's weights are Apache-2.0; each model's card states its license.