Fixed#
Claude's band had leaked out of Claude — 1.54's second chapter, and three
defects with one root. ±10% is the estimator's error measured against
Claude's tokenizer over twenty-one samples, and --exact-tokens counts with
Anthropic's endpoint. Both facts were true, neither was enforced, and three
claims escaped the family they were measured in.
optimize --exact-tokens handed the user's own model id to Anthropic.
countTokensAnthropic({ apiKey, model: result.usage.model }) — correct on a
Claude model, and on gpt-5 either a confusing upstream error or a number
counted with the wrong tokenizer and labelled exact, which is the strongest
word this tool uses about a count. It refuses by name now, naming the model and
its family and pointing at that provider's own tooling. The family check runs
before the key check, because telling somebody on another family to go find an
ANTHROPIC_API_KEY sends them after a credential that could not have helped
them.
The context-overflow advisory said "The call will fail" to every family. The most absolute sentence this product produces, with no dollar figure to soften it — and the margin behind it was Claude's. On a family nobody has measured, an estimate over the window is always uncertain however far over it looks, because the margin that would settle it is the unknown. The fix is not a second, wider band: inventing a number would be the same mistake with worse arithmetic.
And both context advisories sent every reader to --exact-tokens. "Settle
it with --exact-tokens; the counting endpoint is free" — pointing most of the
seven priced families at a counter for a different tokenizer, and, since this
release, at a command that refuses them. The advice is bounded rather than
deleted: the family it works for still gets it.
BAND_CALIBRATED_PROVIDER and bandGoverns() are exported so the fact has a
name. A missing provider reads as not covered, never as covered — the
flattering reading of missing information is the one this project does not take.
Two things this found in itself. The provider-less refusal said the model
"is not a model in the price catalogue at all", which is false in the only
case that reaches it: provider is optional on a priced model and --pricing
lets anybody supply one, so the model is in the catalogue and its family is what
is missing. Found by running the branch rather than reading it. And the new
guard's first draft asked for a prompt 1.01x a 1,000,000-token window but grew
the text by doubling, producing 1,966,080 tokens — landing in the certain
branch and failing a correct implementation over a badly built input.
threshold-honesty.test.js legitimately failed nine models, because it
recognises hedging by vocabulary and the new sentences hedge in words its list
did not know. may be became \bmay\b, plus "not knowable" and "cannot be
said". Widening a guard's accept-list to make a change pass is how a guard
becomes decoration, so the pattern is now handed the flat sentences it exists to
refuse and must reject all three.
Added#
The band harness measures four families, not two — 1.54's first chapter.
Trazum prices seven providers with an estimator calibrated on one, and
ROADMAP.md has called the per-family error "the one number that settles"
whether to take a real tokenizer as a dependency. Nobody had measured it, and
until 1.53 nobody honestly could: the harness can only reach an endpoint this
repository already trusts with a credential.
The 1.53 arc made two more of them facts. OpenAI measures through
https://api.openai.com/v1/chat/completions and Google through
https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent
— both the gateway's own upstreams, both reading a count this repository already
knows how to read. Google deliberately does not use a counting endpoint. One
may well exist; nobody here has ever sent a key to it, and an endpoint recalled
rather than committed is exactly what the previous arc spent itself refusing. So
it measures with :generateContent at one output token and pays for it, rather
than guessing and saving pennies.
Neither has been run against the real service. Nothing in this environment has a key. What ships is the shape; the numbers arrive when somebody runs it.
The measuring script is tied to the gateway's allowlist. It sends a real API
key to every family it measures — the same act the gateway performs, from a file
nobody thinks of as security-sensitive, and in fact the file where DeepSeek's
endpoint sat while the gateway was still refusing DeepSeek as unsupported. Its
origins must now be ones the gateway forwards to, so adding a family requires
the same deliberate edit to security.test.js that adding an upstream does. A
measuring script is not a side door.
And exactly one family may claim the published band, counted from the harness source rather than from the fixtures — which do not exist on a clean checkout, the same absence that once let this suite report "0 failures" for a directory that was not there.
Fixed#
"Nothing else has been measured" told a reader on GPT nothing about their own
figures. The suite emitted one skip naming deepseek, because that was the
only other provider the day the sentence was written — so two families arriving
at 1.53 would have gone unmentioned by a message bounded by what happened to
exist when it was typed. Each measurable family now gets its own named skip
carrying its own command, derived from the harness, and drops out of the list
the moment its fixture appears. The twelfth time this project has bounded a
message by its neighbour rather than by its subject.
A check that could never have fired. The new "the harness reaches nothing the gateway does not front" assertion would never have run against anything it should reject: a brand-new host fails the decision check above it first. The case it actually exists for is subtler — a host already decided about, and decided to be something other than an upstream, which would satisfy every other check in the file. It is handed exactly that now. Third occurrence this session of a filter over today's correct values proving nothing.