What is a Pipeline?
- Machine Learning course
- Intermediate
- Lesson 7/112
- 5 min
Pipeline combines preprocessing and model steps in a single flow.
Why is a pipeline needed?
Real projects have many steps before the model: missing-value filling, scaling, encoding, feature selection. Managing them by hand carries two risks: forgetting a step on the test data, or accidentally fitting on the test data.
from sklearn.pipeline import Pipeline
pipe = Pipeline([
("doldur", SimpleImputer(strategy="median")),
("olcek", StandardScaler()),
("model", LogisticRegression()),
])
pipe.fit(X_train, y_train) # each step learns in order
pipe.predict(X_test) # the same transformations are applied in orderA pipeline behaves like a single estimator; it can be given directly to cross-validation and grid search.
The effect of leakage
For simple steps like scaling the effect of leakage is small; but for steps that look at the target, like feature selection, it is huge. It can produce a high score even on random noise:
X = rng.normal(size=(100, 1000)); y = rng.integers(0, 2, 100) # completely random
X_sec = SelectKBest(k=10).fit_transform(X, y) # WRONG: selection on all data
print(cross_val_score(LogisticRegression(), X_sec, y, cv=5).mean()) # ~0.7+When the same selection is done inside a pipeline, the score drops to a realistic value close to chance.
Practice Measure the leakage
On random data, first try the wrong way: apply SelectKBest(k=10) to all the data and validate logistic regression with 5 folds. Then do the same selection inside make_pipeline(SelectKBest(k=10), LogisticRegression()). Print the two means with round(..., 2) on separate lines.
Starter code
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(0)
X = rng.normal(size=(100, 1000))
y = rng.integers(0, 2, 100)
# buraya yazSample solution
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(0)
X = rng.normal(size=(100, 1000))
y = rng.integers(0, 2, 100)
X_sec = SelectKBest(k=10).fit_transform(X, y)
print(round(cross_val_score(LogisticRegression(), X_sec, y, cv=5).mean(), 2))
pipe = make_pipeline(SelectKBest(k=10), LogisticRegression())
print(round(cross_val_score(pipe, X, y, cv=5).mean(), 2))Expected output
0.78 0.62
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does scaling all the data before cross-validation, without a pipeline, lead to?
- An error
- Nothing
- Leakage from the validation folds into training Correct answer
- A more accurate score
Explanation: In every fold, the validation data's statistics are mixed into the scale.
2 Build a pipeline from named steps.
pipe = ____([("olcek", StandardScaler()), ("model", SVC())])pipe = Pipeline([("olcek", StandardScaler()), ("model", SVC())])Explanation: Pipeline takes a list of (name, object) pairs.
3 Match the risks.
| Term | Meaning |
|---|---|
Fit on the test set |
Leakage |
Forgetting a step |
Inconsistent prediction |
Selection that looks at the target |
Large leakage |
Pipeline |
Prevents both |