AIF-C01 ยท Training and fine-tuning foundation models
Instruction tuning, domain adaptation, transfer learning, continued pre-training
Instruction tuning, domain adaptation, transfer learning and continuous pre-training, and choosing one for a goal.
Key points
- Instruction tuning trains on prompt and response examples so the model follows a task format.
- Domain adaptation trains on domain text (jargon, style) to specialize the model.
- Transfer learning reuses a pre-trained model's knowledge as the starting point for a new task.
- Continued pre-training keeps training on large unlabeled text to broaden knowledge.
Source: https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-fine-tuning.html
Practice questions
Which fine-tuning method uses examples of prompts paired with desired responses?
- Domain adaptation
- Instruction tuning
- Continued pre-training
- Quantization
Show answer
Answer: B. Instruction tuning
Instruction tuning teaches the model to respond to instructions using prompt/response pairs. Domain adaptation uses plain domain text.
Which are valid reasons to fine-tune a pre-trained model rather than train from scratch (transfer learning)?
- Knowledge from pre-training is reused as a starting point
- Less task-specific data and compute are needed
- The pre-trained weights are discarded first
- It guarantees the model never hallucinates
- It removes the need for any evaluation
Show answer
Answer: A. Knowledge from pre-training is reused as a starting point, B. Less task-specific data and compute are needed
Transfer learning reuses learned representations, so you need less data and compute. It does not guarantee zero errors, and the pre-trained weights are the starting point, not discarded.
True or false: fine-tuning changes the model's weights, whereas adding examples to a prompt (in-context learning) does not.
Show answer
Answer: True
True. Fine-tuning is further training that updates weights; in-context examples only change the input for one request.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
