CodeBlanks Logo Icon CodeBlanks
Login Register

Feature extraction

Feature extraction derives new features from existing ones.

Feature extraction

Producing numbers the model can use from raw data (dates, text, images) is called feature extraction. Dates are the classic example:

from datetime import date
d = date(2024, 3, 15)
print(d.year, d.month, d.weekday())   # 2024 3 4  (0 = Monday)
hafta_sonu = d.weekday() >= 5

The model doesn't understand the text "2024-03-15"; but pieces like "month 3", "Friday", "not a weekend" carry a lot of information for sales prediction.

Features from text

The simplest method for text is counting words. CountVectorizer opens a column for each word:

from sklearn.feature_extraction.text import CountVectorizer
metin = ["ucuz ürün ucuz", "kaliteli ürün"]
cv = CountVectorizer()
X = cv.fit_transform(metin).toarray()
print(cv.get_feature_names_out())   # ['kaliteli' 'ucuz' 'ürün']
print(X)                            # [[0 2 1] [1 0 1]]

Simple counts like length, number of exclamation marks and capital-letter ratio are strong features for spam detection too.

Practice Count words

Turn the yorumlar list into a count matrix with CountVectorizer. Print the words with get_feature_names_out() and the matrix with toarray() on separate lines.

Starter code

model.py
from sklearn.feature_extraction.text import CountVectorizer

yorumlar = ["harika film", "kötü film kötü", "harika oyuncu"]

# buraya yaz

Sample solution

model.py
from sklearn.feature_extraction.text import CountVectorizer

yorumlar = ["harika film", "kötü film kötü", "harika oyuncu"]

cv = CountVectorizer()
X = cv.fit_transform(yorumlar)
print(cv.get_feature_names_out())
print(X.toarray())

Expected output

['film' 'harika' 'kötü' 'oyuncu']
[[1 1 0 0]
 [1 0 2 0]
 [0 1 0 1]]

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 If <code>date(2024, 3, 16).weekday()</code> is 5, is that day a weekend?

  • No, Monday
  • It can't be determined
  • Yes, Saturday Correct answer
  • No, Friday

Explanation: weekday 0 = Monday ... 5 = Saturday, 6 = Sunday.

2 Get the names of the word columns.

Question 
adlar = cv.____()
Answer 
adlar = cv.get_feature_names_out()

Explanation: get_feature_names_out gives the words in column order.

3 Match the raw data with the features.

Term Meaning
Date Month, weekday
Text Word counts
Location Distance to centre
Image Pixel values

All lessons in Beginner

1 What is ML?

  1. What is machine learning?
  2. Supervised and unsupervised
  3. Data and model
  4. Training and testing
  5. Feature and label notation
  6. Summary - ML fundamentals

2 NumPy basics

  1. What is NumPy?
  2. Creating arrays
  3. Array properties
  4. Array indexing
  5. Reshaping arrays
  6. Summary - NumPy basics

3 Pandas basics

  1. What is Pandas?
  2. Creating a DataFrame
  3. Column operations
  4. Selecting and filtering data
  5. Viewing first rows
  6. Summary - Pandas basics

4 Data preprocessing

  1. Data cleaning
  2. Missing data
  3. Data types
  4. Label encoding
  5. Data splitting
  6. Summary - Data preprocessing

5 Statistics basics

  1. Mean
  2. Median
  3. Standard deviation
  4. Variance
  5. Correlation
  6. Summary - Statistics

6 Linear algebra

  1. Vectors
  2. Matrices
  3. Matrix multiplication
  4. Transpose
  5. Identity matrix
  6. Summary - Linear algebra

7 Project: Data analysis

  1. Loading data
  2. Cleaning data
  3. Computing statistics
  4. Printing results

8 Regression

  1. What is regression?
  2. Linear regression
  3. Model training
  4. Prediction
  5. Error measurement
  6. Summary - Regression

9 KNN

  1. What is KNN?
  2. The K parameter
  3. Distance measurement
  4. Classification
  5. Model training
  6. Summary - KNN

10 Decision trees

  1. What is a decision tree?
  2. Splitting criterion
  3. Depth
  4. Classification
  5. Overfitting
  6. Summary - Decision trees

11 Classification

  1. What is classification?
  2. Binary classification
  3. Multi-class
  4. Accuracy
  5. Confusion matrix
  6. Summary - Classification

12 Model evaluation

  1. Train-test split
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Model evaluation

13 Project: Classification

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

14 Scaling

  1. What is scaling?
  2. StandardScaler
  3. MinMaxScaler
  4. When to use?
  5. Pipeline
  6. Summary - Scaling

15 Feature engineering

  1. What is a feature?
  2. Feature selection
  3. Feature extraction
  4. Categorical features
  5. Numerical features
  6. Summary - Feature engineering

16 Model selection

  1. What is model selection?
  2. Cross-validation
  3. Grid search
  4. Score comparison
  5. Hyperparameter
  6. Summary - Model selection

17 Project: Regression

  1. Preparing data
  2. Training model
  3. Prediction
  4. Error measurement

18 Metrics

  1. What is a metric?
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Metrics

19 Data visualization

  1. What is visualization?
  2. Scatter plot
  3. Histogram
  4. Line plot
  5. Box plot
  6. Summary - Visualization

20 Beginner final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4