Variance threshold

Variance threshold removes low-variance features.

VarianceThreshold

The simplest filter: dropping columns whose variance is below a threshold. Constant or near-constant columns give no information to any model:

from sklearn.feature_selection import VarianceThreshold
sec = VarianceThreshold(threshold=0.01)
X2 = sec.fit_transform(X)
print(sec.get_support())   # which columns were kept

Variance depends on scale; that's why the threshold is applied before scaling (on scaled data all variances are 1). For 0/1 columns the variance is p(1-p).

get_support

All scikit-learn selectors give a boolean mask with get_support() and the indices of the kept columns with get_support(indices=True). They are used inside pipelines, and names are tracked with get_feature_names_out:

print(sec.get_support(indices=True))
print(sec.variances_.round(3))

The variance threshold doesn't look at the target; that's why it can't separate noise columns, only drop constants.

Exercise Drop constant columns

Apply VarianceThreshold(threshold=0) to the digit data. Print the number of remaining columns and the indices of the dropped columns (with ~sec.get_support()) on separate lines.

Show example solution

import numpy as np
from sklearn.datasets import load_digits
from sklearn.feature_selection import VarianceThreshold

X, y = load_digits(return_X_y=True)

sec = VarianceThreshold(threshold=0)
X2 = sec.fit_transform(X)
print(X2.shape[1])
print(np.where(~sec.get_support())[0])

Test yourself

Roughly what is the variance of a 0/1 column whose values are 99% zeros?
- 0
- 0.0099
- 0.99
- 0.5
Show answer

0.0099 — p(1-p) = 0.01 · 0.99 ≈ 0.0099.

Get the indices of the kept columns.

idx = sec.get_support(____=True)

Show answer

idx = sec.get_support(indices=True)

Match the concepts.

VarianceThreshold

Drops constants
get_support()

Boolean mask
variances_

Column variances
p(1-p)

0/1 column variance

Next lesson: SelectKBest

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies