Symptoms
- Short conversations work and long histories fail.
- Removing tools, documents, or earlier messages restores the request.
Likely causes
- The client sends more tokens than the route accepts.
- Hidden system prompts, tool schemas, or retrieved documents consume the remaining context.
How to diagnose
- Count all messages, system text, tool definitions, and retrieved context.
- Reproduce with a shortened history and the same model.
- Do not infer a context window from the model name alone.
Fixes
- Trim or summarize history and retrieved documents.
- Reduce tool schemas and reserve space for the output.
- Choose a route with a documented larger context only when verified.
How to verify
The reduced request succeeds with enough output headroom for the application.