Bitzu

Machine Learning Associate ยท Exploring and preparing data

One-hot encoding

Encoding categorical features with one-hot encoding, and where it is or is not appropriate (high cardinality, tree-based models, ordinal data, dummy variable trap).

Key points

  • Each category becomes a 0/1 column, so a nominal feature gets no fake order.
  • Good for linear models, logistic regression, neural networks.
  • Poor for very high cardinality; unnecessary for most tree models.
  • Ordinal data (S < M < L): ordered integers.
  • Dummy variable trap: k dummies plus intercept are collinear; drop one (Spark: last by default; scikit-learn: drop="first").
  • Spark: StringIndexer, then OneHotEncoder.

Source: https://spark.apache.org/docs/latest/ml-features.html

Practice questions

Single choice

Why is plain integer label encoding (red=1, green=2, blue=3) risky for a nominal feature in logistic regression?

  1. The model reads the integers as having an order and distance that the colors do not have
  2. It deletes rows containing the color blue
  3. Integers cannot be stored in a Spark column
  4. It always doubles the number of features
Show answer

Answer: A. The model reads the integers as having an order and distance that the colors do not have

The model treats the integers as magnitudes, so it would assume blue is 'more' than red. One-hot columns avoid that false order. The integers do not change the number of rows or make the column categorical.

Select all that apply

Which statements about one-hot encoding are correct?

  1. With an intercept, k full dummy columns are perfectly collinear (dummy variable trap)
  2. In Spark, string columns are typically indexed with StringIndexer before OneHotEncoder
  3. A genuinely ordinal feature such as S < M < L can often be kept as ordered integers
  4. One-hot encoding adds a fixed single column regardless of the number of categories
  5. Decision trees cannot be trained at all without one-hot encoding
Show answer

Answer: A. With an intercept, k full dummy columns are perfectly collinear (dummy variable trap), B. In Spark, string columns are typically indexed with StringIndexer before OneHotEncoder, C. A genuinely ordinal feature such as S < M < L can often be kept as ordered integers

The dummy variable trap comes from k dummies summing to the intercept column; Spark's OneHotEncoder takes category indices (from StringIndexer for strings); and a truly ordered feature often keeps its order as integers. One-hot adds one column per category, and trees do not strictly need it.

True or false

True or false: one-hot encoding is the best choice for an ordinal feature such as low / medium / high, because it avoids assuming any order.

Show answer

Answer: False

False. The feature does have a meaningful order, and one-hot throws it away. Ordered integers (ordinal encoding) keep that information and use only one column.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module