TokenReply

Loading…

Reload

max_tokens is too small

The request leaves too little output budget for the model to return the expected answer.

Last checked · Sources: TokenReply code and documentation review; upstream wording may vary

Area
Generation settings
Typical response
Truncated or empty output
Retry?
Yes, after changing the limit

Symptoms

  • The response ends early or reports a length finish reason.
  • Reasoning or media generation consumes budget before visible output appears.

Likely causes

  • max_tokens was copied from a tiny smoke test.
  • The task needs more output or internal reasoning budget than expected.

How to diagnose

  • Inspect finish reason and token usage when available.
  • Retry one controlled request with a larger output allowance.
  • Check whether the endpoint uses max_tokens or another protocol-specific field.

Fixes

  • Raise the output limit within the model and account constraints.
  • Ask for a shorter answer or split the work into stages.
  • Avoid unbounded maximums in public or automated workloads.

How to verify

The response completes with the expected finish reason and content.

Resources