Layers

A neural network has input, hidden, and output layers.

Layers

A layer is a group of neurons receiving the same input. All the neurons' weights are collected in a matrix; the layer's output is then computed with a single matrix multiplication:

# 3 inputs, a 4-neuron layer
W = rng.normal(size=(3, 4))   # each column is a neuron
b = np.zeros(4)
h = X @ W + b                  # (n, 3) @ (3, 4) -> (n, 4)

The input layer takes the data, hidden layers produce intermediate representations, the output layer gives the prediction.

Depth and width

A network's width is the number of neurons per layer, its depth the number of layers. Deep networks learn hierarchical features: in images the first layers learn edges, later ones shapes, the top ones objects.

katmanlar = [784, 128, 64, 10]   # input, two hidden, output
param = sum(a * b + b for a, b in zip(katmanlar[:-1], katmanlar[1:]))
print(param)   # 109386

As the parameter count grows the model gets more flexible but needs more data and regularisation.

Exercise Parameter count

For a network with layer sizes [784, 256, 128, 10], print each layer's parameter count (weights + biases) as a list, then the total.

Show example solution

katmanlar = [784, 256, 128, 10]

param = [a * b + b for a, b in zip(katmanlar[:-1], katmanlar[1:])]
print(param)
print(sum(param))

Test yourself

How many parameters does a layer with 10 inputs and 32 neurons have?
- 320
- 42
- 32
- 352
Show answer

352 — Weights 10 × 32 = 320, biases 32: 352 in total.
Compute the layer's output.
```
h = X @ W ____ b
```
Show answer
```
h = X @ W + b
```
Match the layers.

input layer

Takes the data
hidden layer

Intermediate representation
output layer

Prediction
column of W

One neuron

Next lesson: Activation

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies