Machine Learning
Discriminator
The discriminator distinguishes real from fake data.
The discriminator
The discriminator is an ordinary binary classifier: real samples get label 1, the generator's samples get 0. For images it is usually a convolutional network and ends with a sigmoid:
D = nn.Sequential(
nn.Linear(784, 256), nn.LeakyReLU(0.2),
nn.Linear(256, 1), nn.Sigmoid())
LeakyReLU is preferred over ReLU in GANs because it passes gradient in the negative region too; this way the generator always receives feedback.
Using the discriminator as a measure
An intuitive way to measure a generator's quality is to train a new classifier on real and fake samples (the C2ST test). The more the classifier struggles, the better the generator:
X = np.r_[gercek, sahte].reshape(-1, 1)
y = np.r_[np.ones(n), np.zeros(n)]
skor = cross_val_score(LogisticRegression(), X, y, cv=5).mean()
Note: a linear classifier can only see a difference in means; a difference in variance needs a more flexible model or a feature (x²).
Exercise Testing generators with C2ST
Write the c2st(gercek, sahte) function: concatenate the samples, use [x, x**2] as features, and return the 5-fold cross-validation accuracy of a LogisticRegression. Print the scores for the given three generators (bad, medium, good) with round(..., 2) on one line.
Show example solution
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
rng = np.random.default_rng(0)
n = 2000
gercek = rng.normal(4, 1.5, n)
ureticiler = [rng.normal(0, 1, n), rng.normal(4, 0.5, n), rng.normal(4, 1.5, n)]
def c2st(gercek, sahte):
x = np.r_[gercek, sahte]
X = np.c_[x, x ** 2]
y = np.r_[np.ones(len(gercek)), np.zeros(len(sahte))]
return cross_val_score(LogisticRegression(max_iter=1000), X, y, cv=5).mean()
print(*[round(c2st(gercek, s), 2) for s in ureticiler])
Test yourself
-
If the discriminator's accuracy drops to 0.5, what does it mean?
- The generator collapsed
- The data is mislabelled
- Real and fake are indistinguishable; the generator succeeds
- The discriminator is very good
Show answer
Real and fake are indistinguishable; the generator succeeds — 0.5 accuracy is chance level in binary classification.
-
Pass gradient in the negative region too in the discriminator.
nn.Linear(784, 256), nn.____(0.2)
Show answer
nn.Linear(784, 256), nn.LeakyReLU(0.2)
-
Match.
- real label
- 1
- fake label
- 0
- accuracy ≈ 0.5
- Good generator
- accuracy ≈ 1
- Bad generator
Next lesson: What is VAE?
