Bitzu

AI-901 ยท AI models and how to configure them

How a generative model produces text

Tokens and tokenization, embeddings, the transformer idea with attention, next-token prediction, training vs inference, context window, and why models can hallucinate.

Key points

  • Text is split into tokens; each token becomes an embedding (a vector of numbers).
  • Attention: transformer layers weigh which earlier tokens matter most.
  • The model predicts the next token, appends it, repeats.
  • Training sets weights; inference only uses them.
  • Context window: caps how many tokens the model sees at once.
  • Hallucination: fluent but false text, as the model picks likely tokens without checking facts.

Practice questions

Single choice

In a language model, what is a token?

  1. A chunk of text such as a word, part of a word or punctuation mark
  2. A secret key used to authenticate calls to the model
  3. A single trained weight inside the neural network
  4. A complete sentence that the model treats as one unit
Show answer

Answer: A. A chunk of text such as a word, part of a word or punctuation mark

A token is the unit of text the model reads and writes: a whole word, part of a word, or punctuation. It is not always a full word and not a character-by-character count.

Select all that apply

Which statements about the context window are correct?

  1. It counts both the input tokens and the tokens the model generates
  2. It grows automatically as the user keeps chatting
  3. It limits how much text the model can take into account in one request
  4. It equals the number of parameters in the model
  5. It guarantees that facts placed inside it are always repeated correctly
Show answer

Answer: A. It counts both the input tokens and the tokens the model generates, C. It limits how much text the model can take into account in one request

The window is the token budget for prompt plus generated answer, so it limits how much the model can consider at once. It is unrelated to parameter count, is not extended by chatting, and does not make content true.

True or false

True or false: while a deployed model answers a prompt (inference), its weights are updated to learn from that prompt.

Show answer

Answer: False

False. Weights are fixed after training. Inference only runs the trained model; anything it seems to remember within a chat comes from text sent in the context window.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AI-901 topics in this module