AIF-C01 ยท Generative AI concepts
Transformers, LLMs, multi-modal and diffusion models
How transformer-based LLMs, foundation models, multi-modal models and diffusion models differ and what each generates.
Key points
- The transformer is the architecture that uses attention to relate all parts of the input to each other.
- A large language model is a transformer-based foundation model trained on huge amounts of text to predict the next token.
- A multi-modal model accepts or produces more than one kind of data, such as text plus images.
- A diffusion model generates images by starting from noise and removing it step by step.
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/key-definitions.html
Practice questions
Which model type typically generates an image by gradually removing noise?
- Diffusion model
- Text-only LLM
- Embedding model
- Classification model
Show answer
Answer: A. Diffusion model
Diffusion models learn to reverse a noising process. An LLM predicts tokens, and an embedding model outputs vectors rather than images.
Which statements about foundation models and LLMs are correct?
- A foundation model is trained on large, diverse data and can serve many tasks
- Most modern LLMs are built on the transformer architecture
- A foundation model can only perform the one task it was trained for
- An LLM is guaranteed to produce factually correct text
- Some foundation models generate images rather than text
Show answer
Answer: A. A foundation model is trained on large, diverse data and can serve many tasks, B. Most modern LLMs are built on the transformer architecture, E. Some foundation models generate images rather than text
Foundation models are broadly trained and adaptable, and LLMs are transformer-based models that work with text. Foundation models are not limited to one narrow task, and LLMs do not guarantee facts.
True or false: a large language model understands facts by looking them up in a database at every response.
Show answer
Answer: False
False. An LLM generates text by predicting likely next tokens from patterns learned in training. Looking up data requires an extra step such as retrieval augmented generation.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
