Context Window & Compaction

Long sessions — especially with coding agents like Claude Code — keep growing the conversation until it no longer fits the model’s context window. This page explains the window we publish, how to calibrate your client to it, and how to keep a session going with compaction instead of hitting an error.

Why context windows exist

Every model can only read a limited number of tokens at once — its context window. The whole conversation (your messages, the model’s replies, tool output, and files the agent has read) counts against it. When the running total approaches the window, the next request can be rejected as prompt too long.

Our published value: read it from the API

We publish the real, verified window for each live model — don’t hard-guess 200k. Read it from the model list so your client calibrates to the true number:

# Public catalog (no key) — includes context_window per model
curl https://api.nexotao.com/models
 
# OpenAI-format list (requires key) — used by clients like Claude Code, n8n
curl https://api.nexotao.com/v1/models \
  -H "Authorization: Bearer sk-nexo-..."

Each model object carries a context_window field (token count).

This field can be null. When present, the number is authoritative — in sync with what the gateway actually serves. But context_window is only published for models whose hard limit we have verified. Right now 6 of the 12 live models carry a number; the rest are null because the provider has not published a firm limit. Don’t read null as “unlimited”, and don’t substitute a 200k guess — treat it as unknown, then rely on client-side compaction plus the explicit 400 you get when a prompt is genuinely too long.

The currently published values:

The remaining models (DeepSeek-V4-Flash, gpt-5-mini, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, grok-4.3) return null. This table can change — read it from the API rather than copying it.

Honest note. For the Claude models, our published window is larger than the legacy 200k that many clients still assume by default — 350k tokens across the Opus 4.x line and Sonnet 4.6, and 1M tokens for claude-opus-5. The figure is per model: don’t hard-guess 200k, and don’t carry one model’s window over to another. Because the real window is bigger than the old 200k default, a client that hasn’t been calibrated will compact earlier than it needs to — you simply lose some usable context, nothing breaks.

Client-side compaction

Compaction replaces a long history with a shorter summary so the session can continue. Clients in the Claude Code family do this for you:

• Autocompact — as the conversation nears the context window, the client automatically summarizes older turns and continues. The threshold is set from the window the client believes the model has. Many clients still assume the old 200k default, which is smaller than our real window, so they autocompact earlier than necessary. Calibrate the client to our published context_window and autocompact fires at the true threshold, letting you use the full window.
• /compact — run this command anytime to compact manually. Useful before a big task, or right after the agent has read a lot of files. You can add a hint, e.g. /compact keep the API design decisions and the failing test.
• /clear — start a fresh context when the current task is done and you don’t need the history. The cheapest way to reclaim the full window.

Point the client at Nexotao and the right models

So compaction calibrates correctly, point the client at our base URL and the live model ids:

The sonnet tier is available — claude-sonnet-4-6 is a live model with its own rate, so use it directly in the sonnet slot. What is not available is the haiku tier, so the background slot also points at claude-sonnet-4-6 (the cheapest Claude model we carry). If your client still sends claude-haiku-4-5, the gateway routes it to claude-opus-4-8 and bills it at the Opus rate.

Older versions of this guide mentioned ANTHROPIC_SMALL_FAST_MODEL. Our generator now emits ANTHROPIC_DEFAULT_SONNET_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL — use those two. See Claude Code for the full config and the interactive generator.

Per-client setup

Claude Code (and the Claude Code family)

1. Set the variables above (or use the generator on the Claude Code page to produce ~/.claude/settings.json).
2. Window: Claude Code derives the window from the model. Read the exact number from context_window in /v1/models. For the Claude models it’s larger than the legacy 200k some clients still assume, so a stale assumption only makes the client compact early (you lose usable context, nothing breaks). Where your client lets you set the model’s context window, set it to the published value so autocompact and /compact trigger at the true threshold.
3. Run claude and use /compact / /clear to manage long sessions.

Other OpenAI-compatible clients

For tools that speak the OpenAI format (n8n, scripts, custom apps):

• Base URL: https://api.nexotao.com/v1
• Models: claude-opus-4-8 (Anthropic-style via /v1/messages) or gpt-5-mini (OpenAI-style via /v1/chat/completions).
• These clients usually don’t compact automatically — keep your own history short: trim or summarize old turns before each call, and read context_window from /v1/models to know your budget. If the field is null for the model you use, trim against your own history size and treat a 400 upstream.context_length_exceeded as the signal to summarize.

When a prompt exceeds the window

If a request is too large, our gateway returns an honest upstream error (not a generic 502), so your client can react:

Recovery options, easiest first:

1. Compact — run /compact (Claude Code) to summarize the history and continue the same task.
2. Start a fresh session — /clear, or open a new chat, when the current task is done. Paste only the summary you still need.
3. Split the input — if a single file or paste is the problem, send it in smaller chunks rather than one oversized message.

Usage is billed from your Rupiah balance the same way regardless of how you manage context — see Billing & Pricing. More on error codes is in the Error Codes; common questions are in the FAQ.