Summary - GAN and VAE
- Machine Learning course
- Expert
- Lesson 34/44
- 6 min
In this group you learned GAN and VAE generative models.
Summary: GAN and VAE
In this group:
- GAN: a game between the generator and the discriminator; at equilibrium D says 0.5 to every sample.
- The non-saturating loss
bce(D(G(z)), 1)is used for the generator; mode collapse is a known problem. - Generator quality can be measured with a new classifier separating real/fake (C2ST).
- VAE: the encoder gives μ and log σ²; with reparameterization z = μ + σ·ε.
- VAE loss = reconstruction + β·KL; a linear autoencoder is equivalent to PCA.
# GAN: sharp samples, unstable training
# VAE: stable training, smooth latent space, blurry samples
# Diffusion models: combine the strengths of both (Stable Diffusion)Practice Compute the VAE loss
With the given x, encoder outputs mu, logvar and eps, compute z with reparameterization. The decoder is linear (z @ W). Print the reconstruction error (sum of squared errors), the KL and the total loss for β = 1 and β = 4 with round(..., 3) on one line.
Starter code
import numpy as np
x = np.array([1.0, 2.0, 0.5])
mu = np.array([0.8, -0.2])
logvar = np.array([-1.0, -0.5])
eps = np.array([0.1, -0.3])
W = np.array([[1.0, 2.0, 0.5], [0.5, -1.0, 0.0]])
# buraya yazSample solution
import numpy as np
x = np.array([1.0, 2.0, 0.5])
mu = np.array([0.8, -0.2])
logvar = np.array([-1.0, -0.5])
eps = np.array([0.1, -0.3])
W = np.array([[1.0, 2.0, 0.5], [0.5, -1.0, 0.0]])
z = mu + np.exp(0.5 * logvar) * eps
x_hat = z @ W
yk = float(((x_hat - x) ** 2).sum())
kl = float(-0.5 * np.sum(1 + logvar - mu ** 2 - np.exp(logvar)))
print(round(yk, 3), round(kl, 3), round(yk + kl, 3), round(yk + 4 * kl, 3))Expected output
0.156 0.577 0.733 2.465
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which model has stable training and a smooth latent space but blurry samples?
- GAN
- GPT
- BERT
- VAE Correct answer
Explanation: The VAE's reconstruction loss pushes toward an averaged output.
2 Draw the latent vector from the standard normal for a new sample.
Question
z = rng.____(size=(16, 2))
Answer
z = rng.normal(size=(16, 2))Explanation: The prior is N(0, I).
3 Match.
| Term | Meaning |
|---|---|
GAN |
Adversarial training |
VAE |
Probabilistic autoencoder |
KL |
Closeness to the prior |
mode collapse |
GAN problem |