Summary - Linear algebra

In this group you learned vectors, matrices, multiplication, and transpose.

Summary: linear algebra

The language of ML is linear algebra:

Vector: an example or the weights; dot product = linear prediction.
Matrix: the dataset or layer weights; row operations through broadcasting.
Matrix multiplication (@): (m, n) @ (n, p) = (m, p), batch prediction.
Transpose, identity, inverse: the normal equation and Ridge.

tahmin = X @ w + b
w = np.linalg.solve(X.T @ X + 0.1 * np.eye(X.shape[1]), X.T @ y)   # Ridge

When you get a shape error, check the inner dimensions of the matrix multiplication.

Exercise Ridge closed form

Find the Ridge weights with alpha = 1.0 using np.linalg.solve(X.T @ X + alpha * np.eye(2), X.T @ y), round them to 3 digits (np.round) and print them.

Show example solution

import numpy as np

X = np.array([[1, 0], [0, 1], [1, 1]])
y = np.array([1, 2, 3])
alpha = 1.0

w = np.linalg.solve(X.T @ X + alpha * np.eye(2), X.T @ y)
print(np.round(w, 3))

Test yourself

For <code>X</code> (100, 4) and <code>W</code> (4, 8), what does <code>X @ W</code> represent?
- 100 outputs for 8 examples
- The mean of 4 features
- An error
- 8 outputs for each of 100 examples
Show answer

8 outputs for each of 100 examples — The result has shape (100, 8): a layer's output in 8 neurons for 100 examples.

Add alpha to the diagonal.

A = X.T @ X + alpha * np.____(X.shape[1])

Show answer

A = X.T @ X + alpha * np.eye(X.shape[1])

Match the concepts with their ML counterparts.

Vector

A single example
Matrix

Dataset
Dot product

Linear prediction
Matrix multiplication

Batch prediction

Next lesson: Loading data

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies