Serving and exporting

Fine-tuning ends with weights, not with a deployment. Nucleus keeps both halves of the promise: serve the model on Nucleus with a single API call, or export the weights in a standard format and run them on your own infrastructure. This guide covers both paths, and the operational habits that keep a served model healthy.

Where servable weights come from

Servable weights are identified by a nucleus:// path. You get one from either workflow:

• A finished fine-tuning job reports its output in result_model.
• A sampler checkpoint you wrote yourself during a training run with Create a checkpoint.

Either way, the path has the same shape:

nucleus://run_3Bz:train:0/sampler_weights/000240

Everything in this guide starts from that path.

Serving on Nucleus

Create a sampler

A sampler is an inference session: it loads the weights and holds them ready for requests. Create one from a base model name or your nucleus:// path. Sampler routes require the infer scope.

curl https://api.nucleus-ai.io/v1/samplers \
  -H "Authorization: Bearer $NUCLEUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model_path": "nucleus://run_3Bz:train:0/sampler_weights/000240"}'

The 201 Created response confirms which weights sit behind the session:

{
  "sampler_id": "sess_A1:sample:0",
  "base_model": "Qwen/Qwen3-8B",
  "model_path": "nucleus://run_3Bz:train:0/sampler_weights/000240"
}

Token-level sampling

Generate samples is the low-level surface. It takes exact token IDs, applies no chat template, and returns token IDs and logprobs. That makes it the right route inside RL loops and anywhere you need full control over what the model sees. Render conversations to token IDs with Render a conversation and decode the output with Detokenize token IDs.

curl https://api.nucleus-ai.io/v1/samplers/sess_A1:sample:0/sample \
  -H "Authorization: Bearer $NUCLEUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": {"token_ids": [3923, 374, 279, 6864, 315, 9822, 30]},
    "num_samples": 1,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

The compatibility routes

For standard chat serving, skip the token plumbing and use the compatibility routes. They accept a base model name or your nucleus:// path as the model parameter and render messages server-side, so any OpenAI or Anthropic SDK works after a base URL change and nothing else. This is the recommended way to serve a chat model in an existing application.

curl https://api.nucleus-ai.io/v1/chat/completions \
  -H "Authorization: Bearer $NUCLEUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nucleus://run_3Bz:train:0/sampler_weights/000240",
    "messages": [{"role": "user", "content": "What is 17 x 23?"}]
  }'

See OpenAI compatibility and Anthropic compatibility for the full parameter surface, streaming and tool use.

Operational hygiene

Two habits keep serving predictable:

• Close samplers you are done with. Each sampler holds an inference session. When an experiment or deployment ends, release it with Close a sampler; the DELETE returns 204 No Content and frees the session.
• Keep production checkpoints from expiring. If a checkpoint backing a model you serve has an expiry set, clear it by passing ttl_seconds: null to Update checkpoint expiry, or extend it with a longer TTL, so the weights never expire out from under a live sampler.

Exporting weights

To run the model outside Nucleus, export the checkpoint with Export weights: POST /v1/training-runs/{run_id}/checkpoints/{checkpoint_id}/export (scope train). Choose a format for where the weights will run:

Choose a destination: signed-url gives you a time-limited download URL when the export completes, and hf-hub pushes the weights to a Hugging Face repository named in hf_repo.

The export runs asynchronously. The request returns 202 Accepted with an operation ID; poll Poll an operation until it finishes, or register a webhook to be notified. Export allocates work, so send an Idempotency-Key header if your client retries automatically. See Errors and retries.

curl https://api.nucleus-ai.io/v1/training-runs/run_3Bz/checkpoints/ckpt_000240/export \
  -H "Authorization: Bearer $NUCLEUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"format": "merged-safetensors", "destination": "signed-url"}'

# 202 Accepted: {"operation": "op_91c4", "format": "merged-safetensors", "status": "running"}

curl https://api.nucleus-ai.io/v1/operations/op_91c4 \
  -H "Authorization: Bearer $NUCLEUS_API_KEY"

Running exported weights

A merged-safetensors export loads like any other checkpoint directory:

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained('./support-assistant-export')

A lora-adapter export composes with its base model (for example Qwen/Qwen3-8B) in any stack that applies LoRA adapters, and a gguf export runs directly under llama.cpp.

One last route to know about: Get a download URL returns a signed URL for a checkpoint's raw archive. The archive is Nucleus's internal storage format, and the bytes do not stream through the API host. It is useful for backup and transfer between Nucleus environments, but for running the model anywhere else, export is what you want.