Summary - GAN and VAE

In this group you learned GAN and VAE generative models.

Summary: GAN and VAE

In this group:

GAN: a game between the generator and the discriminator; at equilibrium D says 0.5 to every sample.
The non-saturating loss bce(D(G(z)), 1) is used for the generator; mode collapse is a known problem.
Generator quality can be measured with a new classifier separating real/fake (C2ST).
VAE: the encoder gives μ and log σ²; with reparameterization z = μ + σ·ε.
VAE loss = reconstruction + β·KL; a linear autoencoder is equivalent to PCA.

# GAN: sharp samples, unstable training
# VAE: stable training, smooth latent space, blurry samples
# Diffusion models: combine the strengths of both (Stable Diffusion)

Exercise Compute the VAE loss

With the given x, encoder outputs mu, logvar and eps, compute z with reparameterization. The decoder is linear (z @ W). Print the reconstruction error (sum of squared errors), the KL and the total loss for β = 1 and β = 4 with round(..., 3) on one line.

Show example solution

import numpy as np

x = np.array([1.0, 2.0, 0.5])
mu = np.array([0.8, -0.2])
logvar = np.array([-1.0, -0.5])
eps = np.array([0.1, -0.3])
W = np.array([[1.0, 2.0, 0.5], [0.5, -1.0, 0.0]])

z = mu + np.exp(0.5 * logvar) * eps
x_hat = z @ W
yk = float(((x_hat - x) ** 2).sum())
kl = float(-0.5 * np.sum(1 + logvar - mu ** 2 - np.exp(logvar)))
print(round(yk, 3), round(kl, 3), round(yk + kl, 3), round(yk + 4 * kl, 3))

Test yourself

Which model has stable training and a smooth latent space but blurry samples?
- GAN
- GPT
- BERT
- VAE
Show answer

VAE — The VAE's reconstruction loss pushes toward an averaged output.
Draw the latent vector from the standard normal for a new sample.
```
z = rng.____(size=(16, 2))
```
Show answer
```
z = rng.normal(size=(16, 2))
```
Match.

GAN

Adversarial training
VAE

Probabilistic autoencoder
KL

Closeness to the prior
mode collapse

GAN problem

Next lesson: What is RL?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies