Bitzu

AI-901 ยท AI models and how to configure them

Deploying a model in Foundry

Deploying a model to use it: deployment names and endpoints, standard (pay per token) vs provisioned throughput, global vs regional deployment, quotas and capacity, and model catalog sources.

Key points

  • Deploying a catalog model gives a named deployment behind your resource's endpoint; you choose the name.
  • Standard: pay-per-token.
  • Provisioned: reserved capacity in PTUs, predictable throughput.
  • Batch: large asynchronous jobs, lower cost.
  • Global: any Azure region. Data Zone: within the US, EU or APAC zone.
  • Plain Standard or regional provisioned: within an Azure geography.
  • Quota: tokens per minute, caps deployment traffic.

Source: https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/deployment-types

Practice questions

Single choice

What does the quota assigned to a model deployment limit?

  1. How many tokens per minute the deployment can process
  2. How many people in the tenant may sign in
  3. The maximum length of the deployment name
  4. Which natural languages the model may answer in
Show answer

Answer: A. How many tokens per minute the deployment can process

Quota is expressed in tokens per minute, so it caps the throughput a deployment can use before requests are rate limited. It does not limit users, name length or languages.

Select all that apply

Which statements describe a Global Standard deployment?

  1. You pay for the tokens you use
  2. Prompts and responses can be processed in any Azure region
  3. You must buy provisioned throughput units in advance
  4. Processing is guaranteed to stay inside one Azure geography
  5. It is meant only for evaluating fine-tuned models for 24 hours
Show answer

Answer: A. You pay for the tokens you use, B. Prompts and responses can be processed in any Azure region

Global Standard is billed per token and may process inference in any Azure region. It does not require purchasing PTUs and does not pin processing to a single geography.

True or false

True or false: a deployment name must be identical to the name of the model it serves.

Show answer

Answer: False

False. You choose the deployment name when you create the deployment, and applications call that name. It can differ from the model name, for example a deployment called 'chat-prod' serving a GPT model.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AI-901 topics in this module