CodeBlanks Logo Icon CodeBlanks
Login Register

Selecting and filtering data

Boolean filtering is used to select rows by condition in a DataFrame.

Filtering with a condition

The boolean mask from NumPy works in pandas too:

genc = df[df["yas"] < 30]
zengin_genc = df[(df["yas"] < 30) & (df["maas"] > 4000)]
print(len(genc))

Parentheses are required when combining conditions with & and |; Python's and/or keywords don't work here.

loc and iloc

loc selects by label, iloc by position; both use the [row, column] form:

df.loc[df["yas"] > 30, "ad"]   # ad column of the matching rows
df.iloc[0]                     # first row
df.iloc[:2, 1]                 # second column of the first two rows

Doing the filter and the column selection in one step with loc is readable and also avoids the "SettingWithCopy" warning.

Practice Filter and select

Select the names of people older than 25 with a salary above 4500 using loc, and print them as a list with tolist().

Starter code

model.py
import pandas as pd

df = pd.DataFrame({
    "ad": ["Ali", "Ece", "Can", "Su"],
    "yas": [34, 28, 45, 22],
    "maas": [5000, 4200, 7100, 3000],
})

# buraya yaz

Sample solution

model.py
import pandas as pd

df = pd.DataFrame({
    "ad": ["Ali", "Ece", "Can", "Su"],
    "yas": [34, 28, 45, 22],
    "maas": [5000, 4200, 7100, 3000],
})

secilen = df.loc[(df["yas"] > 25) & (df["maas"] > 4500), "ad"]
print(secilen.tolist())

Expected output

['Ali', 'Can']

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Why does <code>df[df["yas"] &gt; 30 and df["maas"] &gt; 0]</code> error?

  • 30 must be text
  • It doesn't error
  • and needs a single truth value of a Series; &amp; and parentheses are needed Correct answer
  • The column doesn't exist

Explanation: For element-wise "and" you write (condition1) &amp; (condition2).

2 Select the first row by position.

Question 
ilk = df.____[0]
Answer 
ilk = df.iloc[0]

Explanation: iloc works with integer positions.

3 Match the selectors.

Term Meaning
loc Selection by label
iloc Selection by position
& Element-wise and
| Element-wise or

All lessons in Beginner

1 What is ML?

  1. What is machine learning?
  2. Supervised and unsupervised
  3. Data and model
  4. Training and testing
  5. Feature and label notation
  6. Summary - ML fundamentals

2 NumPy basics

  1. What is NumPy?
  2. Creating arrays
  3. Array properties
  4. Array indexing
  5. Reshaping arrays
  6. Summary - NumPy basics

3 Pandas basics

  1. What is Pandas?
  2. Creating a DataFrame
  3. Column operations
  4. Selecting and filtering data
  5. Viewing first rows
  6. Summary - Pandas basics

4 Data preprocessing

  1. Data cleaning
  2. Missing data
  3. Data types
  4. Label encoding
  5. Data splitting
  6. Summary - Data preprocessing

5 Statistics basics

  1. Mean
  2. Median
  3. Standard deviation
  4. Variance
  5. Correlation
  6. Summary - Statistics

6 Linear algebra

  1. Vectors
  2. Matrices
  3. Matrix multiplication
  4. Transpose
  5. Identity matrix
  6. Summary - Linear algebra

7 Project: Data analysis

  1. Loading data
  2. Cleaning data
  3. Computing statistics
  4. Printing results

8 Regression

  1. What is regression?
  2. Linear regression
  3. Model training
  4. Prediction
  5. Error measurement
  6. Summary - Regression

9 KNN

  1. What is KNN?
  2. The K parameter
  3. Distance measurement
  4. Classification
  5. Model training
  6. Summary - KNN

10 Decision trees

  1. What is a decision tree?
  2. Splitting criterion
  3. Depth
  4. Classification
  5. Overfitting
  6. Summary - Decision trees

11 Classification

  1. What is classification?
  2. Binary classification
  3. Multi-class
  4. Accuracy
  5. Confusion matrix
  6. Summary - Classification

12 Model evaluation

  1. Train-test split
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Model evaluation

13 Project: Classification

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

14 Scaling

  1. What is scaling?
  2. StandardScaler
  3. MinMaxScaler
  4. When to use?
  5. Pipeline
  6. Summary - Scaling

15 Feature engineering

  1. What is a feature?
  2. Feature selection
  3. Feature extraction
  4. Categorical features
  5. Numerical features
  6. Summary - Feature engineering

16 Model selection

  1. What is model selection?
  2. Cross-validation
  3. Grid search
  4. Score comparison
  5. Hyperparameter
  6. Summary - Model selection

17 Project: Regression

  1. Preparing data
  2. Training model
  3. Prediction
  4. Error measurement

18 Metrics

  1. What is a metric?
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Metrics

19 Data visualization

  1. What is visualization?
  2. Scatter plot
  3. Histogram
  4. Line plot
  5. Box plot
  6. Summary - Visualization

20 Beginner final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4