AIF-C01 ยท Generative AI concepts
FM lifecycle and token-based pricing
Data selection, model selection, pre-training, fine-tuning, evaluation, deployment and feedback, and how token-based pricing drives inference cost and performance.
Key points
- Lifecycle: select data, select a model, pre-train (usually the provider), fine-tune, evaluate, deploy, collect feedback and loop back.
- On-demand: billed per token processed; longer prompts and answers and bigger models cost more and take longer.
- Batch inference: can be cheaper for non-urgent jobs.
- Provisioned Throughput: billed per model unit per hour, for steady, predictable load.
Source: https://aws.amazon.com/bedrock/pricing/
Practice questions
In on-demand mode, what mainly determines the cost of one Amazon Bedrock model call?
- The number of input and output tokens processed
- The number of users signed in to the console
- The time of day the request is sent
- The size of the application's source code
Show answer
Answer: A. The number of input and output tokens processed
On-demand inference is billed by the number of input and output tokens processed, at a rate that depends on the model. Number of users logged in or the time of day is not the billing unit.
Which factors increase the inference cost or latency of a generative AI application?
- Longer prompts with a lot of retrieved context
- Very long generated answers
- Trimming the prompt to essentials
- Choosing a larger, more capable model
- Setting a lower maximum output length
Show answer
Answer: A. Longer prompts with a lot of retrieved context, B. Very long generated answers, D. Choosing a larger, more capable model
More tokens in and out and larger models cost more and respond more slowly. A smaller prompt does the opposite, and a shorter answer limit reduces output tokens.
True or false: the foundation model lifecycle ends once the model is deployed, because feedback is only used for the next unrelated project.
Show answer
Answer: False
False. The lifecycle is a loop: feedback and monitoring after deployment feed back into data selection, fine-tuning and evaluation.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
