Machine Learning Associate ยท Deploying and serving models
Custom model Model Serving endpoints
Deploying a custom model to a Model Serving endpoint, scale-to-zero and workload sizing, and querying it with REST or the SDK using the expected input format.
Key points
- Model Serving endpoint: hosts a registered model at the HTTPS URL /serving-endpoints/<name>/invocations.
- Workload size: Small, Medium, Large, by expected concurrent requests.
- scale_to_zero_enabled: optional; releases idle capacity but adds cold-start latency, not advised for production.
- Request: JSON with a key such as dataframe_split (recommended for pandas-style data).
- Response: output under predictions.
Source: https://docs.databricks.com/aws/en/machine-learning/model-serving/score-custom-model-endpoints
Practice questions
Which JSON key is the recommended way to send a pandas DataFrame to a custom model endpoint?
- dataframe_split
- frame_rows
- pandas_json
- dataset_table
Show answer
Answer: A. dataframe_split
dataframe_split sends the DataFrame in split orientation and keeps column order explicit. dataframe_records works but does not guarantee column order; the other names are not accepted formats.
Which are accepted input formats for a custom model Model Serving endpoint?
- dataframe_records
- table_rows
- inputs
- sql_query
- csv_body
Show answer
Answer: A. dataframe_records, C. inputs
The documented formats are dataframe_split, dataframe_records, instances (tensor, row format) and inputs (tensor, columnar format). The others are invented.
True or false: with scale to zero enabled, the first request after an idle period is served as fast as any other.
Show answer
Answer: False
False. When the endpoint has scaled down, capacity must be provisioned again, which adds cold-start latency and is not guaranteed to be available immediately.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
