Assist Signal
This board reads every workspace's assist telemetry. It requires a staff claim and is not reachable from a customer console.
Assist Signal is the read side of the Assist data loop. Every assist turn writes two documents: the exchange, which holds the question a customer typed and the answer they got, and the signal, which holds what the turn cited, how it stopped, what it cost and how it was rated. The board mines the signals.
The split matters for reading this page. The signal carries no prose and no user id and never expires; the exchange carries both and is deleted after 180 days. So an old failure still shows its counts long after its words are gone, and a row with no question attached is not a bug.
The workflow this board exists for
A top row of Docs gaps becomes a docs issue. That is the whole loop: the assistant tells you which pages are failing, you fix those pages, and the assistant gets better because its corpus did. Nothing else on the board is a task list — the cost panels are for tuning, and the prose panel is evidence you read while writing the fix.
Work the board in this order:
- Questions the docs could not answer — these are missing pages. Highest value, because no amount of editing an existing page creates one.
- Docs gaps — these are pages that exist and are not doing their job.
- What people actually asked — the words behind the failures in 1 and 2, to work out what to write.
Fleet
Totals across the scanned sample: messages, tokens, estimated cost, the thumbs tally, and how turns stopped. Every dollar on this page is what the models cost us, not what the workspaces drew in credits — the two differ wherever a marked-up model served, and this page exists to answer the first question.
Answered free is the share of turns served with no model call at all — either quoted straight from the docs index or replayed from the answer cache — with the count beside it. It is the number that decides whether Assist is affordable to leave switched on, and it is the one to watch after a docs rewrite: writing a page that people were asking about moves this figure up and the estimated cost down, in the same week.
Read it against estimated cost, never on its own. The two can move the same way for opposite reasons: a rise in the free share is good news, but so is a fall in it when the reason is that people have stopped asking questions the docs already answered.
Two of these read differently from the rest. Stop reasons separate a refusal from
a truncation — both look like a short answer in the data, but a rising refusal rate is
a prompt problem and a rising max_tokens rate is a ceiling problem, and they need
different fixes. And the unrated count is usually the largest of the three thumbs
figures; that is normal, not a broken control. Ratings are volunteered.
Free tier today is the one chip about money that is not a sample: it is today's
(UTC) free-tier provider spend against the platform's daily ceiling, with the number of
free requests behind it. Free workspaces carry AI credits with no invoice behind them,
so this is the figure that can switch every Free workspace's generation off at once —
amber once staff have been mailed at 80% of the ceiling, red and marked paused once
the ceiling is reached, at which point every Free workspace is refused generation until
the day rolls and paid workspaces carry on. If the chip is red on an ordinary day, the
per-workspace cost table below says who spent it; if it is red because of a wave, the
ai-generate lock on Staff → Lockdown is the manual stop.
The cache-read rate, and what a bad number looks like
The cache-read rate is the share of prompt tokens served from the prompt cache, out of everything the prompts were billed as: sent at full input rates, read from the cache, and written to it. It is the single most load-bearing margin number on the board, and it is on the board because it could not be settled any other way.
Assist sends a large static system prompt on every turn, ordered so the unchanging part comes first and carries the cache breakpoints. That prefix measures close to the model's minimum cacheable length — near enough that whether it caches at all is an empirical question rather than an arithmetic one, and it moves with the model, because the minimum differs between model tiers.
A prefix below the minimum does not cache, and it does not say so. There is no error and no warning; the requests simply cost full input rates. The bill is the only place it shows, and this rate is the bill's early warning.
What the numbers mean:
- A healthy rate is high — most turns after the first should be reading the prefix from cache, so expect a clear majority of prompt tokens to be cache reads.
- A rate at or near zero, with real traffic, means the prefix is not caching at all. That is the failure. The usual cause is the static prefix having shrunk below the model's minimum, or a per-workspace value having been edited into a block that is supposed to be identical for everyone — one per-org byte inside a cached block gives every workspace its own copy, and a copy that is written and never read costs strictly more than not caching.
- A rate that falls after a prompt change or a model change points at that change. Both are the things that move it.
Cache writes cost more than plain input tokens and cache reads cost far less, so a prefix that is written on every turn and read on none is the worst of the three outcomes — and it looks, from every other panel, exactly like normal traffic.
Where the money goes
Costs on this board are our estimated provider cost at the serving model's list rates. They are telemetry for tuning price against margin. They are not a bill, they are not charged to anyone, and they are not revenue — Assist spend is recorded as cost of goods sold and deliberately never billed per token.
This panel splits spend two ways, and both splits carry dollars and message counts, because the questions are money questions that a count cannot answer:
- By tier settles "is the free tier eating the margin, or are paying workspaces?" A tier can be a minority of turns and a majority of spend, and in exactly that case a turn count says the opposite of the truth.
- By model settles "would a cheaper model on the common path fix this?" — where the entire point is that turns and dollars do not move together.
Both lists are ordered dearest first, so the expensive line is the top line.
Tokens by kind
What each kind of model request costs and weighs, one row per kind, dearest first:
the console assistant (assist), the Besigner's copy modes (element, section,
blog) and each generation job kind. Only turns that reached a model are counted; a
turn answered from the docs or the answer cache bought nothing and has no tokens to
split.
- Per request is the estimated provider cost divided by that kind's requests. For a generation job a request is one step, so a job that plans and then builds counts two.
- In, cache reads, cache writes and out are the four token counts the provider reports, summed.
- p95 out is the answer size 95 of every 100 requests of that kind stayed within.
A kind's output ceiling is sized against it: a ceiling well above it is headroom,
and a ceiling at or under it is where answers are cut off — the same thing a rising
max_tokensstop reason says from the other side. - Cache hit is the share of that kind's prompt tokens read from the cache, out of everything its prompts were billed as. Writes count against it for the reason given in the cache-read rate: a prefix written on every other request is the expensive failure, and a rate that left writes out would read it as healthy.
A signal recorded before kinds were kept is read by its route: the Besigner modes name
themselves, a generation job reads as job, and every other turn as assist.
Docs gaps
Cited pages ranked by thumbs-down first, then by how often a question landed there. Not by volume.
Volume alone would produce a popularity list. The most-cited page in the corpus is whichever page answers the most common question, and that page is usually working — it gets cited constantly precisely because retrieval keeps finding it and it keeps being right. Ranking by volume puts your best page on top and buries the one that is being found and is not answering, which is the only row that tells you to do something.
Volume is still the tiebreak, and it is doing real work there. Between two pages with the same number of thumbs-down, the one more people hit is the more expensive one to leave broken.
Read a top row as: this page is being found, and it is not answering. The docs issue writes itself, with its evidence attached.
Questions the docs could not answer
A turn with no cited docs paths is a question the corpus could not match at all, so the model answered ungrounded — from its own knowledge of the product, with nothing to link.
These cannot appear in the Docs gaps ranking, and the reason is structural rather than a filtering choice: that ranking is keyed on the docs paths a turn cited, and a missing page cites nothing. A gap in the documentation is therefore invisible to any path-keyed ranking, no matter how the sort is written. It has to be counted on a different key or it is not counted at all.
So ungrounded turns are counted by route — the console page the person was looking at when they hit the gap. That is deliberately the most useful key available: there is no docs page to name, but there is a console page, and "eleven people asked something ungroundable from the bookings settings page" is a docs assignment.
This is the sharper of the two gap signals. A thumbs-down means an answer was unsatisfying; an ungrounded question means the documentation had nothing to say.
What people actually asked
The verbatim question behind each failing turn, with the answer it got.
The shortlist is deliberately not every turn. Words are fetched only for turns that failed — rated thumbs-down, or grounded in nothing — because those are the ones where the counts cannot say what went wrong and the words can. Reading the successful ones would be surveillance with a dashboard on it.
Prose is kept for 180 days and then deleted, so an older failure shows its counts with the words gone. If a row you want the words for has none, it is past its retention period, and that is the retention working rather than a fault.
What Assist costs, by workspace
The same estimated cost as Where the money goes, per workspace, dearest first, with each workspace's thumbs-down count beside it.
This is where a single workspace burning an unusual amount shows up before the monthly staff margin alert fires. A workspace high on this list and high on thumbs-down is the worst combination on the board: expensive turns that are not landing.
Reading the sample honestly
The board scans up to 20,000 signal documents. When there are more, a warning appears at the top and every number below it describes that sample, not the fleet.
This is surfaced rather than hidden because a ranking cut short looks exactly like a complete one, and this ranking decides where documentation effort goes. Treat a truncated run as a sample: the ordering is usually still informative, the totals are not fleet totals.
Related
- Feature Flags — Assist is behind a release flag as well as a plan entitlement, so a workspace sees it only when both allow it.
- Lockdown — the
ai-assistfeature kill switch, for a provider incident or a cost runaway. - Platform Health — where the staff margin alert for assist spend is surfaced.