AIF-C01 ยท Designing foundation model applications
How do you choose a foundation model for an application?
Cost, modality, latency, multi-lingual support, model size and complexity, customization, input and output length and prompt caching.
Key points
- Compare candidates against the requirements, not against a leaderboard: cost (token price), modality (text, image, audio in or out), latency, language coverage, model size and complexity, how far it can be customized, and input and output length limits.
- Larger models are usually more capable but slower and pricier.
- Prompt caching can cut cost and latency when a long prompt prefix repeats.
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
Practice questions
Which model attribute mainly determines whether a model can accept a photo together with a text question?
- Context window length
- Modality
- Temperature setting
- Price per token
Show answer
Answer: B. Modality
Modality describes which kinds of data a model accepts and produces. A multimodal model can take images and text; a text-only model cannot, regardless of size or price.
Which factors directly affect the per-request cost of calling a model on demand?
- Number of input and output tokens
- Colour theme of the console
- Whether the response is rendered as markdown
- The price tier of the chosen model
- The length of the project name
Show answer
Answer: A. Number of input and output tokens, D. The price tier of the chosen model
On-demand pricing is token based, so the number of input tokens and output tokens drives cost, and the model's price tier sets the rate. The model's license colour or the Region name of a console tab do not change the bill.
The largest available model is always the best choice because bigger models are both more accurate and cheaper per request.
Show answer
Answer: False
False. Larger models often score higher on hard tasks but cost more per token and respond more slowly. A smaller model is frequently the better fit for simple, high-volume tasks.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
