Bitzu

AIF-C01 ยท Designing foundation model applications

How do FM customization approaches compare on cost and effort?

Comparing in-context learning, RAG, fine-tuning, model distillation and pre-training by cost, effort and when each is justified.

Key points

  • Cheapest first; move up only when quality demands it.
  • In-context learning: examples in the prompt, no training, pay for longer prompts.
  • RAG: retrieval infrastructure, vector store, no training.
  • Fine-tuning: labelled data, training job, custom model to host.
  • Distillation: smaller student trained on a larger teacher's outputs, cheaper to run.
  • Pre-training: enormous data and compute, rarely justified.

Source: https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html

Practice questions

Single choice

Which customization approach needs no training job and only adds examples to the prompt?

  1. Pre-training
  2. Fine-tuning
  3. Model distillation
  4. In-context learning
Show answer

Answer: D. In-context learning

In-context learning places instructions and examples directly in the prompt at inference time. Fine-tuning and pre-training both involve training runs; distillation trains a student model.

Select all that apply

Which statements about the cost of customizing a model are correct?

  1. Fine-tuning is billed for training tokens processed and for storing the custom model
  2. Pre-training is usually the cheapest way to add company knowledge
  3. In-context examples avoid training but increase input tokens on every request
  4. RAG needs a full training run before it can answer
  5. Customization never adds cost on top of inference
Show answer

Answer: A. Fine-tuning is billed for training tokens processed and for storing the custom model, C. In-context examples avoid training but increase input tokens on every request

Training-based customization is charged by tokens processed in training (data tokens times epochs) plus model storage, so it adds cost beyond inference. In-context learning needs no training, but longer prompts mean more input tokens on every call. Pre-training is not the cheapest route, and RAG does not require training.

True or false

Fine-tuning a model is the best way to keep it up to date with documents that change daily.

Show answer

Answer: False

False. Fine-tuning captures a snapshot of the training data, and each refresh means another training run. RAG is better for fast-changing knowledge because you only re-index the changed documents.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AIF-C01 topics in this module