Machine Learning Associate ยท Deploying and serving models
Splitting traffic between served models
Serving multiple model versions behind one endpoint and splitting traffic by percentage, e.g. for A/B tests and gradual rollouts.
Key points
- One serving endpoint can host several served entities (e.g. two model versions), each with a unique name.
- traffic_config routes: each pairs a served_model_name with a traffic_percentage; examples add up to 100, e.g. 90/10 champion/challenger.
- Change the split: PUT /api/2.0/serving-endpoints/{name}/config, no new URL.
- Test one model: /serving-endpoints/{endpoint}/served-models/{model-name}/invocations, traffic settings ignored.
Practice questions
Which setting of an endpoint configuration assigns a percentage of requests to each served model?
- scale_to_zero_enabled
- workload_size
- traffic_config
- route_optimized
Show answer
Answer: C. traffic_config
traffic_config.routes pairs each served_model_name with a traffic_percentage. The other names are not configuration fields for splitting.
Which statements about splitting traffic on one serving endpoint are correct?
- Each route refers to a served entity by its name
- The split can be changed by updating the endpoint configuration
- Clients must send a header with the target percentage
- A new split creates a new endpoint URL
- Traffic settings apply per client IP address
Show answer
Answer: A. Each route refers to a served entity by its name, B. The split can be changed by updating the endpoint configuration
Each route points to a named served entity with a percentage, and the split can be changed by updating the endpoint config. Percentages are not set per client, and the URL does not change when the split changes.
True or false: when you call a single served model through its served-models invocations path, the endpoint's traffic percentages still decide whether the request reaches it.
Show answer
Answer: False
False. Querying an individual served model bypasses the split, which is useful to test a challenger directly.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
