Bitzu

Machine Learning Associate ยท Deploying and serving models

Batch inference with pandas vs a Spark UDF

Loading a registered model (pyfunc, by alias or version) and scoring data with pandas, versus Spark UDF for large tables.

Key points

  • Load with mlflow.pyfunc.load_model and a models:/ URI: fixed version (models:/catalog.schema.model/3) or alias (models:/catalog.schema.model@Champion).
  • Alias URIs pick up a promoted version without code changes.
  • model.predict(pdf): scores pandas on a single node, fine when data fits in driver memory.
  • Large tables: mlflow.pyfunc.spark_udf(spark, model_uri) scores in parallel across the cluster.

Source: https://docs.databricks.com/aws/en/machine-learning/manage-model-lifecycle/

Practice questions

Single choice

Which call loads a registered model as a generic Python function for scoring?

  1. mlflow.pyfunc.load_model(model_uri)
  2. mlflow.models.open_model(model_uri)
  3. mlflow.pandas.read_model(model_uri)
  4. mlflow.registry.fetch(model_uri)
Show answer

Answer: A. mlflow.pyfunc.load_model(model_uri)

mlflow.pyfunc.load_model returns a PyFuncModel whose predict accepts a pandas DataFrame. The other names do not exist as model-loading calls.

Select all that apply

Which URIs are valid ways to load a Unity Catalog registered model with pyfunc?

  1. models:/main.ml.churn/4
  2. models:/main.ml.churn@Champion
  3. models:/churn/Production
  4. models:/main.ml.churn:latest-alias
  5. main.ml.churn.pkl
Show answer

Answer: A. models:/main.ml.churn/4, B. models:/main.ml.churn@Champion

Unity Catalog models are addressed by three-level name plus either /version or @alias. Workspace stages and a bare name are not the Unity Catalog form.

True or false

True or false: a batch job that loads models:/main.ml.churn@Champion keeps scoring with the same model version even after the Champion alias is moved to a newer version.

Show answer

Answer: False

False. The alias is resolved when the model is loaded, so the next run of the job uses whatever version the alias points to then. Only a fixed-version URI stays pinned.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module