AI-901 ยท AI models and how to configure them
Deploying a model in Foundry
Deploying a model to use it: deployment names and endpoints, standard (pay per token) vs provisioned throughput, global vs regional deployment, quotas and capacity, and model catalog sources.
Key points
- Deploying a catalog model gives a named deployment behind your resource's endpoint; you choose the name.
- Standard: pay-per-token.
- Provisioned: reserved capacity in PTUs, predictable throughput.
- Batch: large asynchronous jobs, lower cost.
- Global: any Azure region. Data Zone: within the US, EU or APAC zone.
- Plain Standard or regional provisioned: within an Azure geography.
- Quota: tokens per minute, caps deployment traffic.
Source: https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/deployment-types
Practice questions
What does the quota assigned to a model deployment limit?
- How many tokens per minute the deployment can process
- How many people in the tenant may sign in
- The maximum length of the deployment name
- Which natural languages the model may answer in
Show answer
Answer: A. How many tokens per minute the deployment can process
Quota is expressed in tokens per minute, so it caps the throughput a deployment can use before requests are rate limited. It does not limit users, name length or languages.
Which statements describe a Global Standard deployment?
- You pay for the tokens you use
- Prompts and responses can be processed in any Azure region
- You must buy provisioned throughput units in advance
- Processing is guaranteed to stay inside one Azure geography
- It is meant only for evaluating fine-tuned models for 24 hours
Show answer
Answer: A. You pay for the tokens you use, B. Prompts and responses can be processed in any Azure region
Global Standard is billed per token and may process inference in any Azure region. It does not require purchasing PTUs and does not pin processing to a single geography.
True or false: a deployment name must be identical to the name of the model it serves.
Show answer
Answer: False
False. You choose the deployment name when you create the deployment, and applications call that name. It can differ from the model name, for example a deployment called 'chat-prod' serving a GPT model.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
