Machine Learning
Generator
The generator produces realistic data from random noise.
The generator
The generator is a network mapping a vector z taken from a simple distribution (usually standard normal) to the data space. In image GANs z ~ N(0, I) is 100-dimensional and the output is a 64×64×3 image:
G = nn.Sequential(
nn.Linear(100, 256), nn.ReLU(),
nn.Linear(256, 784), nn.Tanh()) # 28x28 image, [-1, 1]
In its simplest one-dimensional form the generator can be G(z) = a·z + b: if z ~ N(0, 1), then G(z) ~ N(b, a²). If the target is N(4, 1.5²), the ideal generator is a = 1.5, b = 4.
Mode collapse
A well-known problem of GANs is mode collapse: the generator finds a single kind of sample that fools D and always produces it. When the real data has two peaks (modes), the generator may cover only one:
gercek = np.concatenate([rng.normal(-3, 0.5, 500), rng.normal(3, 0.5, 500)])
cokmus = rng.normal(3, 0.5, 1000) # only the right peak
print((gercek < 0).mean(), (cokmus < 0).mean()) # 0.5 and 0.0
Remedies: the Wasserstein loss (WGAN), minibatch discrimination, multiple generators.
Exercise A linear generator
Generate 100 000 samples of z ~ N(0, 1). Write the G(z, a, b) function. For the target N(4, 1.5²), print the sample mean and standard deviation of the correct (a=1.5, b=4) and the wrong (a=1, b=0) generators with round(..., 1) on one line each. Then print the shares of negative samples in the two-peaked real data and in the collapsed generator with round(..., 2) on one line.
Show example solution
import numpy as np
rng = np.random.default_rng(0)
z = rng.normal(size=100_000)
gercek = np.concatenate([rng.normal(-3, 0.5, 500), rng.normal(3, 0.5, 500)])
cokmus = rng.normal(3, 0.5, 1000)
def G(z, a, b):
return a * z + b
for a, b in [(1.5, 4), (1, 0)]:
s = G(z, a, b)
print(round(s.mean(), 1) + 0.0, round(s.std(), 1))
print(round((gercek < 0).mean(), 2), round((cokmus < 0).mean(), 2))
Test yourself
-
If G(z) = 2z − 1 and z ~ N(0, 1), what are the mean and standard deviation of G(z)?
- Mean 2, std −1
- Mean 0, std 1
- Mean −1, std 4
- Mean −1, std 2
Show answer
Mean −1, std 2 — A linear transformation shifts the mean and multiplies the std by |a|.
-
Squash the image output into [-1, 1].
nn.Linear(256, 784), nn.____())
Show answer
nn.Linear(256, 784), nn.Tanh())
-
Match.
- z
- Noise input
- a·z + b
- N(b, a²)
- mode collapse
- A single kind of sample
- WGAN
- A loss against collapse
Next lesson: Discriminator
