Bitzu

AIF-C01 ยท Training and fine-tuning foundation models

Instruction tuning, domain adaptation, transfer learning, continued pre-training

Instruction tuning, domain adaptation, transfer learning and continuous pre-training, and choosing one for a goal.

Key points

  • Instruction tuning trains on prompt and response examples so the model follows a task format.
  • Domain adaptation trains on domain text (jargon, style) to specialize the model.
  • Transfer learning reuses a pre-trained model's knowledge as the starting point for a new task.
  • Continued pre-training keeps training on large unlabeled text to broaden knowledge.

Source: https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-fine-tuning.html

Practice questions

Single choice

Which fine-tuning method uses examples of prompts paired with desired responses?

  1. Domain adaptation
  2. Instruction tuning
  3. Continued pre-training
  4. Quantization
Show answer

Answer: B. Instruction tuning

Instruction tuning teaches the model to respond to instructions using prompt/response pairs. Domain adaptation uses plain domain text.

Select all that apply

Which are valid reasons to fine-tune a pre-trained model rather than train from scratch (transfer learning)?

  1. Knowledge from pre-training is reused as a starting point
  2. Less task-specific data and compute are needed
  3. The pre-trained weights are discarded first
  4. It guarantees the model never hallucinates
  5. It removes the need for any evaluation
Show answer

Answer: A. Knowledge from pre-training is reused as a starting point, B. Less task-specific data and compute are needed

Transfer learning reuses learned representations, so you need less data and compute. It does not guarantee zero errors, and the pre-trained weights are the starting point, not discarded.

True or false

True or false: fine-tuning changes the model's weights, whereas adding examples to a prompt (in-context learning) does not.

Show answer

Answer: True

True. Fine-tuning is further training that updates weights; in-context examples only change the input for one request.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AIF-C01 topics in this module