Bitzu

Machine Learning Associate ยท Hyperparameter tuning and validation

Parallelizing tuning with SparkTrials

Distributing the tuning of single-node models: Hyperopt SparkTrials with parallelism, and the difference from distributed training of one model.

Key points

  • Hyperopt SparkTrials runs each trial as a Spark task on a worker, so many single-node models (e.g. scikit-learn) are tuned at once with no other code change.
  • Pass trials=SparkTrials(parallelism=n) to fmin.
  • parallelism caps concurrent trials (default: number of executors, maximum 128).
  • Distributed training spreads one model (MLlib, Horovod) across the cluster: use the default Trials class.

Source: https://docs.databricks.com/aws/en/machine-learning/automl-hyperparam-tuning/hyperopt-concepts

Practice questions

Single choice

Which Hyperopt class distributes trials of single-node models across Spark workers?

  1. SparkTrials
  2. Trials
  3. STATUS_OK
  4. ParamGridBuilder
Show answer

Answer: A. SparkTrials

SparkTrials is the Spark-aware Trials implementation. Plain Trials runs on the driver; STATUS_OK is a result flag; ParamGridBuilder belongs to Spark ML grids.

Select all that apply

Which statements about SparkTrials are accurate?

  1. parallelism sets the maximum number of trials running at the same time
  2. It is intended for single-machine libraries such as scikit-learn
  3. Very high parallelism gives TPE less past information per new trial, which can hurt result quality
  4. It is the recommended class for tuning MLlib models
  5. Setting parallelism above the executor count always speeds the run up
Show answer

Answer: A. parallelism sets the maximum number of trials running at the same time, B. It is intended for single-machine libraries such as scikit-learn, C. Very high parallelism gives TPE less past information per new trial, which can hurt result quality

parallelism caps concurrency, SparkTrials targets single-node libraries, and more concurrency means each suggestion sees fewer finished results. MLlib models are already distributed, so they use plain Trials; extra parallelism beyond the cluster's task slots is reduced.

True or false

Setting a higher parallelism in SparkTrials always produces a better tuned model.

Show answer

Answer: False

False. Higher parallelism finishes a fixed number of trials sooner, but adaptive algorithms like TPE then choose settings with fewer completed results, so quality can drop.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module