Samplers
Open inference sessions against base models or your own fine-tuned weights, and generate from raw token IDs.
A sampler is an open inference session. Create one against a base model from the
model catalogue, or against your own fine-tuned weights by pointing
model_path at the sampler weights of a checkpoint. Once open,
the sampler serves generation and logprob requests until you close it.
Sampler routes work in token IDs, not text, and no chat template is applied on the server. Build prompts with Render a conversation or Tokenize text, and decode output with Detokenize token IDs. That exact-token contract is what makes samplers usable inside RL loops, where the tokens you score must be the tokens the model saw.
Samplers are grouped, together with training runs, under a session. Each open sampler holds inference capacity, so close samplers you are done with.
Create a sampler
/v1/samplersscope: inferOpens an inference session against a base model or your own weights. Provide exactly one of
base_model or model_path.
Body parameters
base_modelstringServe a base model from the catalogue, for example Qwen/Qwen3-8B. Provide this or
model_path, not both.
model_pathstringServe your own fine-tuned weights. A nucleus:// path ending in /sampler_weights/NNNNNN, for
example nucleus://run_3Bz:train:0/sampler_weights/000240. Provide this or base_model, not
both.
The response includes the sampler_id you pass to every other sampler route.
curl https://api.nucleus-ai.io/v1/samplers \
-H "Authorization: Bearer $NUCLEUS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model_path": "nucleus://run_3Bz:train:0/sampler_weights/000240"}'{
"sampler_id": "sess_A1:sample:0",
"base_model": "Qwen/Qwen3-8B",
"model_path": "nucleus://run_3Bz:train:0/sampler_weights/000240"
}Retrieve a sampler
/v1/samplers/{sampler_id}scope: readReturns the base model and weights path behind the session, along with its status.
Path parameters
sampler_idstringrequiredID of the sampler, from the create response.
curl https://api.nucleus-ai.io/v1/samplers/sess_A1:sample:0 \
-H "Authorization: Bearer $NUCLEUS_API_KEY"{
"sampler_id": "sess_A1:sample:0",
"base_model": "Qwen/Qwen3-8B",
"status": "ready"
}Generate samples
/v1/samplers/{sampler_id}/samplescope: inferGenerates completions from exact token IDs. No chat template is applied: the model continues
from precisely the tokens you send, which is what an RL loop needs when the sampled tokens
must line up with the tokens you later train on. Decode each sample's token_ids with
Detokenize token IDs.
Path parameters
sampler_idstringrequiredID of the sampler to generate from.
Body parameters
promptobjectrequiredThe prompt as raw tokens: an object with a token_ids array.
num_samplesintegerrequiredNumber of independent samples to generate from the same prompt.
sampling_paramsobjectDecoding controls: max_tokens, temperature, top_p and stop.
include_prompt_logprobsbooleanAlso return logprobs over the prompt tokens, not just the generated tokens.
topk_prompt_logprobsintegerReturn the top-k alternative logprobs at each prompt position.
Each sample carries its token_ids, per-token logprobs and a stop_reason telling you
whether decoding hit max_tokens or a stop condition.
curl https://api.nucleus-ai.io/v1/samplers/sess_A1:sample:0/sample \
-H "Authorization: Bearer $NUCLEUS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": { "token_ids": [151644, 872, 198] },
"num_samples": 1,
"sampling_params": { "max_tokens": 3, "temperature": 0.7 }
}'{
"samples": [
{
"token_ids": [12095, 11, 8574],
"logprobs": [-0.31, -0.08, -1.22],
"stop_reason": "max_tokens"
}
]
}Compute prompt logprobs
/v1/samplers/{sampler_id}/logprobsscope: inferReturns per-token logprobs over the prompt without generating anything. Useful for scoring sequences under the served model, for example when computing rewards or importance weights.
Path parameters
sampler_idstringrequiredID of the sampler to score with.
Body parameters
promptobjectrequiredThe tokens to score: an object with a token_ids array.
The response array is position-aligned with token_ids. null marks positions where no
logprob could be computed, typically the first token, which has nothing to condition on.
curl https://api.nucleus-ai.io/v1/samplers/sess_A1:sample:0/logprobs \
-H "Authorization: Bearer $NUCLEUS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt": {"token_ids": [151644, 872, 198, 3838]}}'{
"logprobs": [null, -2.41, -0.88, -1.05]
}Close a sampler
/v1/samplers/{sampler_id}scope: inferReleases the inference session. A sampler stays open, and keeps holding capacity, until you close it, so close every sampler you are done with, especially samplers opened programmatically inside loops.
Path parameters
sampler_idstringrequiredID of the sampler to close.
DELETE is safe to retry: closing an already-closed sampler does no harm.
curl -X DELETE https://api.nucleus-ai.io/v1/samplers/sess_A1:sample:0 \
-H "Authorization: Bearer $NUCLEUS_API_KEY"(empty body)