Advantages
- Machine Learning course
- Intermediate
- Lesson 10/112
- 5 min
Pipeline prevents data leakage and keeps code clean.
What a pipeline gives you
A pipeline gives three concrete gains:
- Correctness: leakage is prevented structurally.
- Convenience: one
fit, onepredict; in grid search the settings of all steps are searched together. - Portability: the whole chain is saved as one object and moved to production.
import joblib
joblib.dump(pipe, "model.joblib")
yuklu = joblib.load("model.joblib")
yuklu.predict(yeni_veri) # raw data is enoughRaw data arrives in production; if preprocessing isn't saved with the model, the same transformation as in training must be rewritten, which creates a risk of errors.
A grid inside a pipeline
In a grid search, both preprocessing and model settings are searched together; each combination is retrained cleanly in every fold:
izgara = {
"olcek": [StandardScaler(), MinMaxScaler()],
"model__C": [0.1, 1, 10],
}
gs = GridSearchCV(pipe, izgara, cv=5).fit(X_train, y_train)
print(gs.best_params_)Both the scaler choice and the C value are decided by validation.
Practice Save and load
Save the trained pipeline to the file "hat.joblib" with joblib.dump and load it back with joblib.load. Print on separate lines that the loaded model's test predictions match the original's, using (... == ...).all(), and the loaded model's test accuracy with round(..., 3).
Starter code
import joblib
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipe = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
# buraya yazSample solution
import joblib
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipe = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
joblib.dump(pipe, "hat.joblib")
yuklu = joblib.load("hat.joblib")
print((yuklu.predict(X_test) == pipe.predict(X_test)).all())
print(round(yuklu.score(X_test, y_test), 3))Expected output
True 0.958
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does forgetting to save the scaler separately when moving a model to production lead to?
- The model runs faster
- Nothing
- The model retrains itself
- The model makes wrong predictions on unscaled data Correct answer
Explanation: The model was trained on scaled data; raw data is in a different space.
2 Save the pipeline to a file.
joblib.____(pipe, "model.joblib")joblib.dump(pipe, "model.joblib")Explanation: joblib.dump writes the object to disk; joblib.load reads it back.
3 Match the gains.
| Term | Meaning |
|---|---|
Leak prevention |
Correctness |
A single fit |
Convenience |
joblib.dump |
Portability |
"model__C" |
Joint search |