Generative Adversarial Networks (GANs)

Generative Adversarial Networks
(GANs)
Hoàng Minh Thanh (21C11029)
Nguyễn Trần Khánh Nguyên (21C11017)
Nguyễn Thanh Tùng (18120104)
Lê Minh Đức (18120164)
Team 07
(*) The Big Book of Dashboards Visualizing Your Data Using Real-World Business Scenarios by Steve Wexler Jeffrey Shaffer Andy Cotgreave (z-lib.org)
1
1. Introduction of GANs
2. DCGANs
Content
3. Cycle GANs
4. Experiment & Demo
2
Overview of GAN
● GANs introduced by Goodfellow et al. (2014)
● Generate synthetic images (fake)
● 2 Networks: Generator and Discriminator
3
What are GANs ?
4
What are GANs ?
5
6 Steps for Training GANs
● Step 1: Generate a noise vector
● Step 2: Pass this noise through Generator to generate an
actual image
● Step 3: Mix authentic images and synthetic images
● Step 4: Using this mixed set for training Discriminator
● Step 5: Once again generate random noise and label each
noise vector as a “real image”
● Step 6: Train the GAN using the noise vectors and “real
image” labels
6
GANs Training
● Two losses to minimize: One loss for the Discriminator and

a second loss for the Generator.
● Goal is to not seek a minimum loss value but instead find
some equilibrium between the two
7
Types of GANs
● Foundation level
○ Deep Convolutional GAN (DCGAN)
● Extensions
○ Conditional GAN (cGAN)
○ Information Maximizing GAN (InfoGAN)
○ Stacked GAN (StackGAN)
○ Auxiliary Classifier GAN (AC-GAN)
○ Context Encoders
○ Pix2Pix
● Advanced level
○ Cycle-Consistent GAN (CycleGAN)
○ Big GAN (BigGAN)
○ Style-Based GAN (StyleGAN)
○ Wasserstein GAN (WGAN)
○ Progressive Growing GAN (Progressive GAN)
8
Applications of GANs
9
Generate Examples for Image Dataset

10
Generate Photographs of Human Faces

(Progressive GAN) 11
Generate Realistic Photographs (BigGAN)
12
Photo Inpainting (Context Encoders)

13
● Generate Examples for Image Datasets
● Photograph Editing
● Video Prediction
● Super Resolution
● Generate Photographs of Human Faces
● Generate Cartoon Characters
● Image-to-Image Translation
● Text-to-Image Translation
● Photos to Emojis
14
DCGAN
15
What is the goal of GAN?
The goal: The main focus for GAN (Generative Adversarial
Networks) is to generate data from scratch, mostly images
but other domains including music have been done.
16
Main idea of DCGAN
• Generator: generates fake images to fool the discriminator.

• Discriminator: tries to distinguish between real and fake
images.
- Train them against each other.
- Finally, the discriminator cannot distinguish between real and fake.
- It means the fake images look like real
Picasso painting Fake painting 17

Main idea of DCGAN
• Generator: a forger who wants to create a
fake Picasso painting.
• Discriminator: an art dealer.
• Train them against each other.
• Forger mixes his fake with authentic paintings and ask art dealer to assess
them.
• Art dealer gives feedback about what makes a Picasso look like a Picasso.
• Forger improves his competence at imitating the style of Picasso.
• Art dealer improves his competence at distinguishing real and fake Picasso.
• Finally, the art dealer cannot distinguish between real and
fake.
18
GAN: Model Overview
19
Generator: From Vector to Image
20
Generator: From Vector to Image
21
Discriminator is a classification
22
Discriminator: From Image to Binary
23
How to training
Alternating minimization
Repeat the 2 steps:

1. Update the discriminator network;
2. Update the generator network.
24
Update the Discriminator
1. Generate a batch of fake images by the generator;
2. Randomly sample a batch of real images;
3. Inputs : X = [real_images, fake_images];
4. Targets: y = [True, ⋯ , True, False, ⋯ , False];
5. Update the discriminator network using X and y
25
Update the Generator
Connect the generator and discriminator

(freeze discriminator’s parameters).
26
Connect the generator and discriminator (freeze
discriminator’s parameters).
Minimize Loss = Dist (True, ) w.r.t.generator.
(Encourage be True.)
27
• Use the same vector.
• Generate a fake image using the updated generator.
• The discriminator thinks the new fake image more “real”
than before.
28
Loss function
Probability of real image: D(x)
Probability of fake image: 1 - D(x)
p_data(x) : distribution of real image
p_z(z) : distribution of fake image
max D : Maximum loss function by discriminator

min G : Minimum loss function by generator
29
Discriminator Shouldn’t Be Too Good
• Generator: a forger who wants to create a fake Picasso painting.
• Discriminator: an art dealer providing feedbacks.
What if the art dealer is 100% correct at judging

Picasso painting?
• Whatever forged painting sent to the art dealer is recognized as

fake.
• The forger cannot learn anything from the feedback.
• No positive case to follow.
• The forger need some success.
• So he will know what kind of fake painting can fool the dealer.
30
Discriminator Shouldn’t Be Too Bad
… Vanishing gradient… !
• Suppose the discriminator is perfect.
• Whatever the generators forged is recognized fake by the
discriminator.
→The gradient is near zero.
31
Discriminator Shouldn’t Be Too Good
• Generator: a forger who wants to create a fake Picasso painting.
• Discriminator: an art dealer providing feedbacks.
What if the art dealer cannot distinguish between real

and fake paintings?
• The art dealer’s judgement is almost random guess.

• The forger cannot learn anything from the feedback.
→ When the forger’s skill is good, getting amateurish art dealer’s
feedback is a not helpful.
32
Solutions to training DCGAN
1. Carefully tune the learning rates.
• If the discriminator improves too fast,
→ classification accuracy can be 100%,
→ vanishing gradient,
→ the generator is dead.
• If the discriminator improves too slowly,
→ the discriminator cannot provide useful feedback,
→ the generator has to wait for the discriminator,
→ slow convergence.
33
Solutions to training DCGAN
2. Add noise to the real and fake images; decay
the noise over time.
• When training the discriminator, perturb the inputs (both
real and fake images).
3. Add noise to the labels.

• When training the discriminator, perturb the labels (both
real and fake images).
● Real = 1 -> Real ~ Uniform(0.7, 1.2).
● Fake = 0 -> Fake ~ Uniform(0.0, 0.3).
34
Experiment
35
Experiment
● Datasets:
MNIST
Celeb-A
36
Experiment
● DCGAN Architecture for MNIST
Generator
Discriminator
37
Experiment
● DCGAN Architecture for Celeb-A
Generator
Discriminator
38
Demo
39
Cycle GAN
40
Introduction Cycle GAN
41
CycleGAN Architecture
Domain X: Photos of the zebra

Domain Y: Photos of the horse
Cycle GAN have two GAN models:

● GAN A: Translate photos of Domain X (zebra) to photos
of Domain Y (horse)
● GAN B: Translate photos of Domain Y (horse) to photos
of Domain X (zebra)
42
CycleGAN Architecture
43
Cycle Consistency Loss
44
Cycle Consitency Loss
forward cycle-consistency loss backward cycle-consistency loss

x → G(x) → F(G(x)) ≈ x y → F(y) → G(F(x)) ≈ y
45
Applications of CycleGAN
Collection style transfer
46
Object transfiguration
47
Photo generation from paintings

48
Limitations of CycleGAN
49
Resource
DCGAN Report (Overleaf) :
https://www.overleaf.com/7297748965xsdcptbdddbk
DCGAN Colab:
https://colab.research.google.com/drive/1pRVH4srQyT_ri-I
a4c3_uauTfcIMSQN8?usp=sharing#scrollTo=eJHN40X0Mr
bV
CycleGAN Colab:
https://colab.research.google.com/drive/1pRVH4srQyT_ri-I
a4c3_uauTfcIMSQN8?usp=sharing#scrollTo=eJHN40X0Mr
bV
50
Q&A
51

Generative Adversarial Networks (GANs)

Uploaded by

Copyright:

Available Formats

Generative Adversarial Networks (GANs)

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Generative Adversarial Networks (GANs)

Uploaded by

Copyright:

Available Formats

Generative Adversarial Networks

4. Experiment & Demo

● GANs introduced by Goodfellow et al. (2014)

● Generate synthetic images (fake)

● 2 Networks: Generator and Discriminator

● Two losses to minimize: One loss for the Discriminator and

Generate Examples for Image Dataset

Generate Photographs of Human Faces

Generate Realistic Photographs (BigGAN)

Photo Inpainting (Context Encoders)

• Generator: generates fake images to fool the discriminator.

Picasso painting Fake painting 17

Repeat the 2 steps:

1. Generate a batch of fake images by the generator;

2. Randomly sample a batch of real images;

3. Inputs : X = [real_images, fake_images];

4. Targets: y = [True, ⋯ , True, False, ⋯ , False];

5. Update the discriminator network using X and y

Connect the generator and discriminator

max D : Maximum loss function by discriminator

What if the art dealer is 100% correct at judging

• Whatever forged painting sent to the art dealer is recognized as

What if the art dealer cannot distinguish between real

• The art dealer’s judgement is almost random guess.

3. Add noise to the labels.

Domain X: Photos of the zebra

Cycle GAN have two GAN models:

forward cycle-consistency loss backward cycle-consistency loss

Collection style transfer

Photo generation from paintings

You might also like