GANs in Slanted Land - Solution
GANs in Slanted Land - Solution
GANs in Slanted Land - Solution
Activity Overview
Generative adversarial networks (GANs) are a clever way of training a generative model by framing the
problem as a supervised learning problem. The GAN model architecture involves two sub-models:
When training begins, the generator produces obviously fake data, and the discriminator quickly learns to
tell that it's fake. As training progresses, the generator gets closer to producing output that can fool the
discriminator. Finally, if the generator training goes well, the discriminator gets worse at telling the difference
between real and fake. It starts to classify fake data as real, and its accuracy decreases.
For this activity, we have adapted the code in this (https://github.com/luisguiserrano/gans) GitHub
repository, where you can also find a link to a YouTube video that provides useful insight on the code.
In this notebook, we build a very simple pair of GANs where the data consists of 2x2 black and white
images, where the images we want to generate (the faces) are backwards diagonals (\). Our goal is to define
a GANs that is able to distinguish a real image from a fake one.
This assignment is designed to help you apply the machine learning algorithms you have learned using
packages in Python . Python concepts, instruction, and starter code are embedded within this Jupyter
Notebook to help guide you as you progress through the assignment. Remember to run the code of each
code cell prior to submitting the assignment. Upon completing the assignment, we encourage you to
compare your work against the solution file to perform a self-assessment.
Index:
Week 7: Generative Adversarial Networks in Slanted Land
Depending on the set of tasks you wish to accomplish, you can import as many different libraries as you
wish.
In the code cell below, we have imported the libraries that we will be using in this activity.
In [1]: # Imports
You can think of Python functions as a self-contained bundle of code used to carry out specific tasks.
Functions can be defined for you within a library. For very specific tasks, you may need to define a function
on your own. These are called user-defined functions and they have the following general structure:
def function_name(parameters):
"""
optional description of the function
"""
statement(s)
return
Where:
The keyword def that marks the start of the function header.
A function name that uniquely identifies the function.
Parameters through which we pass values to a function. Depending on the function, they can be
optional.
A colon (:) that marks the end of the function header.
Optional documentation string (docstring) which describes what the function does.
One or more valid Python statements that make up the function body. Statements must have the
same indentation level. They define the task(s) that your function is supposed to execute.
An optional return statement which returns a value from the function.
In the code cell, we have defined the function view_samples used to generate and display subplots with
desired characteristics. This function takes three arguments, samples , m , and n , where:
This function unpacks the values we wish to plot into the desired number of subplots and creates each one
of them without axis and using the black-and-white gradient colormap.
In [2]: ##### Drawing function
Back to top
As we have seen in the YouTube video, in this Slanted Land world, the characters look slightly elongated to a
45 degree angle and are represented by 2X2 black-and-white pixel screens.
In the code below, we have used the function array to create four characters. Notice that, because each
face is identified by 4 pixel, each array contains four values between 0 and 1.
In the code cell below, practice modifying the definition of the second face from the left using the values
0.8, 0.1, 0.2, 0.9 .
Next, we would like to visualize the faces we have just created by using the function view_samples that
we have defined above.
In the code cell below, we have decided to visualize the values generated above in one row and four
columns.
It is possible to tell real faces apart from fake ones by noticing that top-left and the bottom-right corners
have large values because the pixels are dark, whereas the other two corners have small values and
therefore their pixels are light. Notice that the values are not always exactly 0 and 1 because we can have
noisy images.
In [4]: faces_view = view_samples(faces, 1, 4)
Question:
1) In the case where we wanted to visualize the faces on two rows and two columns, how would you need to
change the values of m and n in view_samples ? Feel free to practice your answer by modifying the
code cell above.
2) Suppose we generated 12 faces and we wanted to display them on four rows and three columns. What
would the values of m and n be in this case?
1) m =2 and n =2
2) m =4 and n =3
Back to top
In the code cell below, we have created 20 images, each one with four pixels. The function
generate_random_image generates a random number for each one of the four pixels.
In the code cell below, practice modifying the code so that it generates 25 random images.
In a similar way as we did in Part 2, we use the function view_samples to visualize the fake faces we
have generated above.
In [6]: noise_view = view_samples(noise, 4,5)
As you may notice, each one of the images below is a lot more noisy than the ones we have defined in Part
2, and some of them don't have higher values on the diagonal as real faces.
Question:
Assuming you generate 25 fake images, and you wanted to display them on a 5X5 grid, how would you
change the parameters in the code cell above? Feel free to test your answer by modifying the code cell
above.
view_samples(noise, 5,5)
Distinguishing faces from noise
As a rule of thumb for this exercise, all we need to do is sum the value in the top-left and bottom-right
corners and subtract the sum of the values in the top-right and bottom-left corners.
In a real face, this score will be high, whereas in a noisy image will be low.
Back to top
As we can see from the image above, the sigmoid function takes values close to 1 when x is high and
close to 0 when x is low.
For example, an image with score 1 (x=1) takes a value for the sigmoid function equal to 0.73. We can
interpret this as a 73% probability that the discriminator will consider this image real.
In the code cell below, we have defined a function sigmoid , which takes as input a value x and
evaluates it according to the mathematical definition of the function to compute the probability.
As an exercise, in the code cell below, we call the function sigmoid with the value 1.
In [8]: sigmoid(1)
Out[8]: 0.7310585786300049
Question:
If an image has a score of 1.5, what is the probability that the discriminator will recognize that image as real?
Feel free to practice your answer by using the code cell above.
0.81
Back to top
The discriminator in a GAN is simply a classifier. It tries to distinguish real data from fake data created by the
generator.
Real data instances, such as real pictures of people. The discriminator uses these instances as positive
examples during training
Fake data instances created by the generator. The discriminator uses these instances as negative
examples during training
1) The discriminator classifies both real data and fake data from the generator
2) The discriminator loss function penalizes the discriminator for misclassifying a real instance as fake or
a fake instance as real
3) The discriminator updates its weights through backpropagation from the discriminator loss function
through the discriminator network
The code provided in the code cell below provides the complete definition for the discriminator. Note that
the code starts with the keyword class . Python classes provide a means of bundling data and
functionality together, which belong to the object defined by the class (in this case, Discriminator ).
We notice that because the discriminator takes data from real and fake images, we have functions to
compute the error and update the weights for both cases.
NOTE: The code below can be quite advanced if you have no prior coding knowledge. For this reason,
details are left out.
In [9]: class Discriminator():
def __init__(self):
#self.weights = np.array([0.0 for i in range(4)])
#self.bias = 0.0
self.weights = np.array([np.random.normal() for i in range(4)]
)
self.bias = np.random.normal()
In our GAN, the generator is not directly connected to the loss that we're trying to affect. The generator loss
function penalizes the generator for producing a sample that the discriminator network classifies as fake.
The code provided in the code cell below provides the complete definition for the generator. Again, note that
the generator is defined as a class .
We notice that because the generator is fed through backpropagation by the discriminator, we only have one
function to compute the error and update the weights.
NOTE: The code below can be quite advanced if you have no prior coding knowledge. For this reason,
details are left out.
In [10]: class Generator():
#generate random number
def __init__(self):
self.weights = np.array([np.random.normal() for i in range(4)]
)
self.biases = np.array([np.random.normal() for i in range(4)])
We saw that the generator and the discriminator have different training processes. So how do we train the
GAN as a whole?
It is important to keep the generator constant during the discriminator training phase. As the discriminator
training tries to figure out how to distinguish real data from fake data, it has to learn how to recognize the
generator's flaws.
Similarly, we keep the discriminator constant during the generator training phase. Otherwise, the generator
would be trying to hit a moving target and might never converge.
It's this back and forth that allows GANs to deal with generative problems.
In the code cell below, we have given you the code that simulates this process. Let's try to understand it.
First of all, for reproducibility of the results, we need to set a random seed by using the function seed()
from the module random in NumPy .
Next, we set the learning rate for our GANs. The learning rate is a hyperparameter that controls how much
to change the model in response to the estimated error each time the model weights are updated.
We also set the number of times we want to repeat the training process ( epochs ).
Next, we define the discriminator D and the generator G for this exercise.
Finally, we train the set following the steps outlined above. Notice the presence of a for loop. In Python ,
for loops are used to repeatedly execute instructions for the desired number of iterations (in this case
epochs ).
In [11]: # Set random seed
np.random.seed(42)
# Hyperparameters
learning_rate = 0.01
epochs = 1000
z = random.rand()
noise = G.forward(z)
Question:
Choosing a proper value for the learning rate can be challenging. What do you expect to happen to the
training process if you choose a value that is too large or too small?
CLICK ON THIS CELL TO TYPE YOUR ANSWER
Choosing the learning rate can be challenging as a value too small may result in a long stagnant training
process, whereas a value too large may result, for example, in an unstable training process.
The first part of the code plots the values stored in the variable errors_generator using the library
matplotlib . The second part of the code plots the error values for the discriminator.
Question
Practice modifying the code above by setting the learning_rate equal to 0.05 and train your model
again.
Although this change may seem small compared to its original value, what changes do you observe in the
plots of the errors for the generator and the discriminator?
The errors in the generator seem to take longer to stabilize. The errors in the discriminator seem to keep
going up and they decrease toward to end.
Question
Practice modifying the code above by setting the learning_rate equal to 0.07 and train your model
again.
What changes do you observe in the plots of the discriminator errors compared to the values we had when
the learning rate was equal to 0.05?
Question
Practice modifying the code above by increasing both the learning_rate to 0.07 and epochs to
10000 . Train your model again and plot the results. What do you observe?
Both the generator and the discriminator errors stabilize after so many epochs, meaning that although we
have increased the learning rate we have trained the network long enough to stabilize the errors.