AIF-C01 ยท Designing foundation model applications
How do FM customization approaches compare on cost and effort?
Comparing in-context learning, RAG, fine-tuning, model distillation and pre-training by cost, effort and when each is justified.
Key points
- Cheapest first; move up only when quality demands it.
- In-context learning: examples in the prompt, no training, pay for longer prompts.
- RAG: retrieval infrastructure, vector store, no training.
- Fine-tuning: labelled data, training job, custom model to host.
- Distillation: smaller student trained on a larger teacher's outputs, cheaper to run.
- Pre-training: enormous data and compute, rarely justified.
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html
Practice questions
Which customization approach needs no training job and only adds examples to the prompt?
- Pre-training
- Fine-tuning
- Model distillation
- In-context learning
Show answer
Answer: D. In-context learning
In-context learning places instructions and examples directly in the prompt at inference time. Fine-tuning and pre-training both involve training runs; distillation trains a student model.
Which statements about the cost of customizing a model are correct?
- Fine-tuning is billed for training tokens processed and for storing the custom model
- Pre-training is usually the cheapest way to add company knowledge
- In-context examples avoid training but increase input tokens on every request
- RAG needs a full training run before it can answer
- Customization never adds cost on top of inference
Show answer
Answer: A. Fine-tuning is billed for training tokens processed and for storing the custom model, C. In-context examples avoid training but increase input tokens on every request
Training-based customization is charged by tokens processed in training (data tokens times epochs) plus model storage, so it adds cost beyond inference. In-context learning needs no training, but longer prompts mean more input tokens on every call. Pre-training is not the cheapest route, and RAG does not require training.
Fine-tuning a model is the best way to keep it up to date with documents that change daily.
Show answer
Answer: False
False. Fine-tuning captures a snapshot of the training data, and each refresh means another training run. RAG is better for fast-changing knowledge because you only re-index the changed documents.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
