Generator
- Machine Learning course
- Expert
- Lesson 30/44
- 5 min
The generator produces realistic data from random noise.
The generator
The generator is a network mapping a vector z taken from a simple distribution (usually standard normal) to the data space. In image GANs z ~ N(0, I) is 100-dimensional and the output is a 64×64×3 image:
G = nn.Sequential(
nn.Linear(100, 256), nn.ReLU(),
nn.Linear(256, 784), nn.Tanh()) # 28x28 image, [-1, 1]In its simplest one-dimensional form the generator can be G(z) = a·z + b: if z ~ N(0, 1), then G(z) ~ N(b, a²). If the target is N(4, 1.5²), the ideal generator is a = 1.5, b = 4.
Mode collapse
A well-known problem of GANs is mode collapse: the generator finds a single kind of sample that fools D and always produces it. When the real data has two peaks (modes), the generator may cover only one:
gercek = np.concatenate([rng.normal(-3, 0.5, 500), rng.normal(3, 0.5, 500)])
cokmus = rng.normal(3, 0.5, 1000) # only the right peak
print((gercek < 0).mean(), (cokmus < 0).mean()) # 0.5 and 0.0Remedies: the Wasserstein loss (WGAN), minibatch discrimination, multiple generators.
Practice A linear generator
Generate 100 000 samples of z ~ N(0, 1). Write the G(z, a, b) function. For the target N(4, 1.5²), print the sample mean and standard deviation of the correct (a=1.5, b=4) and the wrong (a=1, b=0) generators with round(..., 1) on one line each. Then print the shares of negative samples in the two-peaked real data and in the collapsed generator with round(..., 2) on one line.
Starter code
import numpy as np
rng = np.random.default_rng(0)
z = rng.normal(size=100_000)
gercek = np.concatenate([rng.normal(-3, 0.5, 500), rng.normal(3, 0.5, 500)])
cokmus = rng.normal(3, 0.5, 1000)
# buraya yazSample solution
import numpy as np
rng = np.random.default_rng(0)
z = rng.normal(size=100_000)
gercek = np.concatenate([rng.normal(-3, 0.5, 500), rng.normal(3, 0.5, 500)])
cokmus = rng.normal(3, 0.5, 1000)
def G(z, a, b):
return a * z + b
for a, b in [(1.5, 4), (1, 0)]:
s = G(z, a, b)
print(round(s.mean(), 1) + 0.0, round(s.std(), 1))
print(round((gercek < 0).mean(), 2), round((cokmus < 0).mean(), 2))Expected output
4.0 1.5 0.0 1.0 0.5 0.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 If G(z) = 2z − 1 and z ~ N(0, 1), what are the mean and standard deviation of G(z)?
- Mean 2, std −1
- Mean 0, std 1
- Mean −1, std 4
- Mean −1, std 2 Correct answer
Explanation: A linear transformation shifts the mean and multiplies the std by |a|.
2 Squash the image output into [-1, 1].
nn.Linear(256, 784), nn.____())nn.Linear(256, 784), nn.Tanh())Explanation: Tanh outputs [-1, 1]; Sigmoid gives [0, 1].
3 Match.
| Term | Meaning |
|---|---|
z |
Noise input |
a·z + b |
N(b, a²) |
mode collapse |
A single kind of sample |
WGAN |
A loss against collapse |