Machine Learning
Estimator API
All scikit-learn models share fit() and predict() methods.
The Estimator API
In scikit-learn everything is an estimator: an object that learns something from data. There are three roles:
- Predictor:
fit + predict (models). - Transformer:
fit + transform (scalers, encoders, PCA). - Evaluator:
score.
sc = StandardScaler() # transformer
sc.fit(X_train)
Z = sc.transform(X_train)
m = LogisticRegression() # predictor
m.fit(Z, y_train)
m.predict(sc.transform(X_test))
Hyperparameters are given in the constructor; learned values are stored afterwards in attributes ending with _.
clone and get_params
An estimator's hyperparameters are read with get_params() and changed with set_params(). clone produces a new, untrained copy with the same settings; cross-validation does this in every fold:
from sklearn.base import clone
m = LogisticRegression(C=0.5).fit(X_train, y_train)
yeni = clone(m)
print(yeni.get_params()["C"]) # 0.5
print(hasattr(yeni, "coef_")) # False: untrained
Thanks to this design, classes you write yourself work with all the tools once they derive from BaseEstimator.
Exercise Copy with clone
Train a LogisticRegression(C=0.1, max_iter=1000). Copy it with clone; print the copy's C value and whether it has a coef_ attribute on one line. Then change the copy with set_params(C=10), train it and print the test accuracy with round(..., 3).
Show example solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
from sklearn.base import clone
from sklearn.linear_model import LogisticRegression
m = LogisticRegression(C=0.1, max_iter=1000).fit(X_train, y_train)
yeni = clone(m)
print(yeni.get_params()["C"], hasattr(yeni, "coef_"))
yeni.set_params(C=10).fit(X_train, y_train)
print(round(yeni.score(X_test, y_test), 3))
Test yourself
-
Which estimator role does PCA have?
- Evaluator
- None of them
- Transformer
- Predictor
Show answer
Transformer — PCA learns the components with fit and transforms the data with transform; it doesn't predict.
-
Create an untrained copy with the same settings.
yeni = ____(m)
Show answer
yeni = clone(m)
-
Match the roles.
- StandardScaler
- Transformer
- LogisticRegression
- Predictor
- clone
- Untrained copy
- set_params
- Changing settings
Next lesson: The fit method
