CodeBlanks Logo Icon CodeBlanks
Login Register

Standard deviation

Standard deviation shows how much data deviates from the mean.

Standard deviation

The standard deviation measures how far the values move from the mean on average. If it is small the data is tight; if large, spread out.

a = np.array([10, 10, 10, 10])
b = np.array([0, 5, 15, 20])
print(a.std(), b.std())   # 0.0 7.905...

Both arrays have the same mean (10) but very different spreads; the mean alone isn't enough.

ddof: sample and population

By default NumPy computes the population standard deviation (divides by n). When estimating the population from a sample, ddof=1 divides by n-1; that is the pandas default:

x = np.array([2, 4, 4, 4, 5, 5, 7, 9])
print(x.std())          # 2.0
print(x.std(ddof=1))    # 2.138...

With large datasets the difference becomes negligible; but when NumPy and pandas results differ, this is usually why.

Practice Measure the spread

Print the population standard deviation (the default) of array x. Then print how many standard deviations each value is from the mean ((x - mean) / std).

Starter code

model.py
import numpy as np

x = np.array([2, 4, 4, 4, 5, 5, 7, 9])

# buraya yaz

Sample solution

model.py
import numpy as np

x = np.array([2, 4, 4, 4, 5, 5, 7, 9])

s = x.std()
print(s)
print((x - x.mean()) / s)

Expected output

2.0
[-1.5 -0.5 -0.5 -0.5  0.   0.   1.   2. ]

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What is the standard deviation of an array whose values are all the same?

  • 1
  • Equal to the mean
  • Undefined
  • 0 Correct answer

Explanation: No value deviates from the mean, so the spread is zero.

2 Compute the sample standard deviation.

Question 
s = x.std(____)
Answer 
s = x.std(ddof=1)

Explanation: ddof=1 makes the denominator n-1.

3 Match the values.

Term Meaning
small std() Data is tight
large std() Data is spread out
ddof=0 Dividing by n
ddof=1 Dividing by n-1

All lessons in Beginner

1 What is ML?

  1. What is machine learning?
  2. Supervised and unsupervised
  3. Data and model
  4. Training and testing
  5. Feature and label notation
  6. Summary - ML fundamentals

2 NumPy basics

  1. What is NumPy?
  2. Creating arrays
  3. Array properties
  4. Array indexing
  5. Reshaping arrays
  6. Summary - NumPy basics

3 Pandas basics

  1. What is Pandas?
  2. Creating a DataFrame
  3. Column operations
  4. Selecting and filtering data
  5. Viewing first rows
  6. Summary - Pandas basics

4 Data preprocessing

  1. Data cleaning
  2. Missing data
  3. Data types
  4. Label encoding
  5. Data splitting
  6. Summary - Data preprocessing

5 Statistics basics

  1. Mean
  2. Median
  3. Standard deviation
  4. Variance
  5. Correlation
  6. Summary - Statistics

6 Linear algebra

  1. Vectors
  2. Matrices
  3. Matrix multiplication
  4. Transpose
  5. Identity matrix
  6. Summary - Linear algebra

7 Project: Data analysis

  1. Loading data
  2. Cleaning data
  3. Computing statistics
  4. Printing results

8 Regression

  1. What is regression?
  2. Linear regression
  3. Model training
  4. Prediction
  5. Error measurement
  6. Summary - Regression

9 KNN

  1. What is KNN?
  2. The K parameter
  3. Distance measurement
  4. Classification
  5. Model training
  6. Summary - KNN

10 Decision trees

  1. What is a decision tree?
  2. Splitting criterion
  3. Depth
  4. Classification
  5. Overfitting
  6. Summary - Decision trees

11 Classification

  1. What is classification?
  2. Binary classification
  3. Multi-class
  4. Accuracy
  5. Confusion matrix
  6. Summary - Classification

12 Model evaluation

  1. Train-test split
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Model evaluation

13 Project: Classification

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

14 Scaling

  1. What is scaling?
  2. StandardScaler
  3. MinMaxScaler
  4. When to use?
  5. Pipeline
  6. Summary - Scaling

15 Feature engineering

  1. What is a feature?
  2. Feature selection
  3. Feature extraction
  4. Categorical features
  5. Numerical features
  6. Summary - Feature engineering

16 Model selection

  1. What is model selection?
  2. Cross-validation
  3. Grid search
  4. Score comparison
  5. Hyperparameter
  6. Summary - Model selection

17 Project: Regression

  1. Preparing data
  2. Training model
  3. Prediction
  4. Error measurement

18 Metrics

  1. What is a metric?
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Metrics

19 Data visualization

  1. What is visualization?
  2. Scatter plot
  3. Histogram
  4. Line plot
  5. Box plot
  6. Summary - Visualization

20 Beginner final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4