What is a neural network?
- Machine Learning course
- Advanced
- Lesson 1/78
- 5 min
A neural network is a layered model inspired by biological neural networks.
What is a neural network?
An artificial neural network is a model made of simple computing units (neurons) connected in layers. Each layer changes its input with a linear transformation and passes it through a non-linear function; stacked layers let the model learn very complex relationships.
# input (4 features) -> hidden layer (8 neurons) -> output (3 classes)
h = relu(X @ W1 + b1) # (n, 8)
cikti = h @ W2 + b2 # (n, 3)Logistic regression is a single-layer neural network; deep learning is the many-layered version of this structure.
Why now?
The idea of neural networks has existed since the 1950s. Recent success has three causes: large labelled datasets, the parallel matrix multiplication power of GPUs and better training techniques (ReLU, good initialisation, Adam, dropout).
- Images: convolutional networks (CNN).
- Sequences and text: RNN, LSTM and especially the transformer.
- Tabular data: gradient boosting is often still better.
Practice Composition of linear layers
Print with np.allclose that applying two linear layers (W1, W2) in a row without activation gives the same result as the single matrix W1 @ W2. Then print that the result is no longer the same once a ReLU (np.maximum(0, ·)) is put between them.
Starter code
import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(5, 3))
W1 = rng.normal(size=(3, 4))
W2 = rng.normal(size=(4, 2))
# buraya yazSample solution
import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(5, 3))
W1 = rng.normal(size=(3, 4))
W2 = rng.normal(size=(4, 2))
print(np.allclose((X @ W1) @ W2, X @ (W1 @ W2)))
print(np.allclose(np.maximum(0, X @ W1) @ W2, X @ (W1 @ W2)))Expected output
True False
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What would happen without the non-linear function between hidden layers?
- Nothing would change
- The whole network would reduce to a single linear transformation Correct answer
- The network would be more powerful
- Training would be impossible
Explanation: The composition of two linear transformations is still linear: W2(W1x) = (W2W1)x.
2 Compute the hidden layer.
h = np.maximum(0, X ____ W1 + b1)h = np.maximum(0, X @ W1 + b1)Explanation: @ is matrix multiplication; np.maximum(0, ·) is ReLU.
3 Match the concepts.
| Term | Meaning |
|---|---|
neuron |
Simple computing unit |
layer |
Group of neurons |
W |
Learned weights |
ReLU |
Non-linear function |