Tokens and rendering
The training and sampling primitives take token IDs, not text. Here is how to produce them correctly.
The low-level API works in token IDs. The forward pass and forward-backward pass inputs on training runs, and Generate samples and Compute prompt logprobs on samplers, all take arrays of integers rather than strings. Rendering, chat templates and loss masking sit with you, the caller.
This is deliberate. RL, DPO and custom-loss training need exact control over which tokens the model sees and which tokens contribute to the loss, and that control only exists below the level of text. The tokenization routes are how you move between the two worlds.
Tokenize and detokenize
Tokenize text converts a string to token IDs
for a specific model. Tokenizers differ between model families, so always tokenize with the
same model you are training or sampling:
{
"token_ids": [3838, 374, 279, 6722],
"count": 4,
"tokenizer": "qwen3"
}Detokenize token IDs is the reverse mapping, and you will use it more than you expect: Generate samples returns token IDs, so detokenizing is how you read your model's own output.
Render a conversation
Chat models are trained on conversations laid out in a model-specific chat template: special
tokens, role markers and end-of-turn tokens arranged exactly the way the base model saw them
in training. Render a conversation
applies the correct template for a model to a messages array and returns two aligned
arrays: token_ids, and weights, a per-token loss mask.
{
"token_ids": [151644, 872, 198, ...],
"weights": [0, 0, 0, 1, 1, 1],
"count": 96
}Feed token_ids to the training primitives as-is and use weights as the loss mask: a
token with weight 0 is context the model conditions on, a token with weight 1 contributes to
the loss. The train_on parameter controls the mask:
train_on | Trains on |
|---|---|
all | Every token, prompts included. Useful when the whole document is target text. |
assistant | Every assistant message. The standard choice for supervised fine-tuning on conversations. |
last-assistant | Only the final assistant message. Earlier turns are treated purely as context, which suits multi-turn data where only the last reply is the behaviour you want. |
Resist the temptation to hand-roll the template with string concatenation. A hand-rolled template that gets one detail wrong still trains and still converges, so the mistake survives until inference. The classic failure is dropping or mis-masking the end-of-turn token: the model never learns to emit it and will not stop talking. Rendering server-side with Render a conversation removes the whole class of bug.
The compatibility exception
The OpenAI-compatible and Anthropic-compatible routes take plain text and messages and render server-side. That makes them the easy on-ramp: point an existing SDK at your fine-tuned weights and go, with no tokenization code at all.
It also makes them the wrong tool for RL. Server-side rendering means you never see the token IDs that were actually sampled, cannot align logprobs to a loss mask, and cannot control the template. For a training loop, stay in token space: render your conversations, train and sample with token IDs, and detokenize only at the edges. The training loop guide walks through the full cycle.