Bitzu

Machine Learning Associate ยท Deploying and serving models

Custom model Model Serving endpoints

Deploying a custom model to a Model Serving endpoint, scale-to-zero and workload sizing, and querying it with REST or the SDK using the expected input format.

Key points

  • Model Serving endpoint: hosts a registered model at the HTTPS URL /serving-endpoints/<name>/invocations.
  • Workload size: Small, Medium, Large, by expected concurrent requests.
  • scale_to_zero_enabled: optional; releases idle capacity but adds cold-start latency, not advised for production.
  • Request: JSON with a key such as dataframe_split (recommended for pandas-style data).
  • Response: output under predictions.

Source: https://docs.databricks.com/aws/en/machine-learning/model-serving/score-custom-model-endpoints

Practice questions

Single choice

Which JSON key is the recommended way to send a pandas DataFrame to a custom model endpoint?

  1. dataframe_split
  2. frame_rows
  3. pandas_json
  4. dataset_table
Show answer

Answer: A. dataframe_split

dataframe_split sends the DataFrame in split orientation and keeps column order explicit. dataframe_records works but does not guarantee column order; the other names are not accepted formats.

Select all that apply

Which are accepted input formats for a custom model Model Serving endpoint?

  1. dataframe_records
  2. table_rows
  3. inputs
  4. sql_query
  5. csv_body
Show answer

Answer: A. dataframe_records, C. inputs

The documented formats are dataframe_split, dataframe_records, instances (tensor, row format) and inputs (tensor, columnar format). The others are invented.

True or false

True or false: with scale to zero enabled, the first request after an idle period is served as fast as any other.

Show answer

Answer: False

False. When the endpoint has scaled down, capacity must be provisioned again, which adds cold-start latency and is not guaranteed to be available immediately.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module