Catalog task WEIRD-005
A model does not choose a word. At every position it produces a distribution over the whole vocabulary and something downstream picks one entry. You only ever see the winner. Here the runners-up are stacked above it, ghosted in proportion to their probability, so a confident position looks empty and an uncertain one looks crowded.
Ghosted words above each token are what the model nearly said, opacity proportional to probability. An amber underline marks a position where the runner-up was within a factor of two of the winner. A red token was chosen with under 50% probability.
| capture | tokens | close calls | least certain token | its probability |
|---|---|---|---|---|
ambiguous | 21 | 4 | ·task | 43.3% |
creative | 28 | 6 | ·a | 20.6% |
factual | 7 | 0 | The | 95.1% |
uncertain | 15 | 0 | ·approximately | 90.7% |
Compare factual and uncertain. One answers a question with a
single correct answer. The other answers how many hairs are on an average human head,
which the model cannot know and which no one can state to the precision it gives. Both were
produced with almost no hesitation. Whatever these probabilities measure, it is agreement among
plausible continuations rather than correspondence with the world, and the interface makes that
visible in a way a chat window never does.
top_logprobs returns the top k of a vocabulary of tens of thousands of tokens,
so its probabilities sum to at most 1 and usually less. Rescaling them to fill the width would
make the bars look finished and would misstate what happened, so the unreported mass gets its
own slice labelled everything else. At a confident position it is nearly invisible. At
an uncertain one it is a real part of the picture.
The entropy shown with each position is computed over the reported alternatives only, so it is a lower bound. Unseen mass can only add uncertainty, never remove it.