Logit bias

Inside the model, raw scores called "logits" are produced for every token in the vocabulary at each step. Those logits get passed through softmax (and through temperature, top-p, top-k) to become a probability distribution from which the next token gets sampled. Logit bias steps in before that pipeline: it adds (or subtracts) a number from the logit of specific tokens of your choice.

Bias values are typically in the range -100 to +100. A bias of -100 effectively zeros out the token's probability — the model will never pick it. A bias of +100 makes the token nearly inevitable. Mid-range values (say, ±5) are gentle nudges.

Practical uses:

• Format enforcement: bias the JSON delimiters { : , } upward, suppress markdown tokens, force structured output without elaborate prompting.
• Topic suppression: ban tokens that name competitors, profanity, or specific terms you do not want surfaced.
• Style steering: nudge toward particular vocabulary (e.g., bias the British spelling "colour" over "color").

The technique is precise but coarse. Tokens are subword units, so banning "Coca" does not necessarily ban "Coca-Cola" (the second word might tokenize as -Cola or Cola, depending on the model's tokenizer). Effective use requires inspecting the tokenizer to figure out which tokens make up the strings you actually care about.

Most chat APIs expose logit_bias as a parameter, mapping token IDs to bias values. Token IDs are model-specific: the bias map you build for GPT-4 will not transfer directly to Claude or Gemini. The OpenAI Tokenizer playground and Anthropic's tokenizer endpoint exist for exactly this reason.

In practice, logit bias is used more for production guardrails and structured output than for everyday chat tuning.