Scribd Logo
Upload

Welcome to Scribd!

  • 98 views

    DL - Assignment 12 Solution

    Uploaded by

    swathisreejith6
    1) The document contains a 10 question multiple choice quiz about deep learning topics including variational autoencoders, generative adversarial networks, and neural network activations functions. 2) Each question includes 4 multiple choice answers and a detailed explanation of the correct answer. 3) The questions cover topics such as the computational graph of a variational autoencoder, the reparameterization trick, convergence of a GAN, losses used to optimize generators and discriminators, effects of not using KL divergence in VAEs, and properties of latent spaces.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    DL - Assignment 12 Solution

    Uploaded by

    swathisreejith6
    98 views7 pages
    1) The document contains a 10 question multiple choice quiz about deep learning topics including variational autoencoders, generative adversarial networks, and neural network activations functions. 2) Each question includes 4 multiple choice answers and a detailed explanation of the correct answer. 3) The questions cover topics such as the computational graph of a variational autoencoder, the reparameterization trick, convergence of a GAN, losses used to optimize generators and discriminators, effects of not using KL divergence in VAEs, and properties of latent spaces.

    Original Description:

    Original Title

    DL_Assignment 12 Solution

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    1) The document contains a 10 question multiple choice quiz about deep learning topics including variational autoencoders, generative adversarial networks, and neural network activations functions. 2) Each question includes 4 multiple choice answers and a detailed explanation of the correct answer. 3) The questions cover topics such as the computational graph of a variational autoencoder, the reparameterization trick, convergence of a GAN, losses used to optimize generators and discriminators, effects of not using KL divergence in VAEs, and properties of latent spaces.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    98 views7 pages

    DL - Assignment 12 Solution

    Uploaded by

    swathisreejith6
    1) The document contains a 10 question multiple choice quiz about deep learning topics including variational autoencoders, generative adversarial networks, and neural network activations functions. 2) Each question includes 4 multiple choice answers and a detailed explanation of the correct answer. 3) The questions cover topics such as the computational graph of a variational autoencoder, the reparameterization trick, convergence of a GAN, losses used to optimize generators and discriminators, effects of not using KL divergence in VAEs, and properties of latent spaces.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 7

    NPTEL Online Certification Courses

    Indian Institute of Technology Kharagpur

    Deep Learning
    Assignment- Week 12
    TYPE OF QUESTION: MCQ/MSQ
    Number of questions: 10 Total mark: 10 X 1 = 10
    ______________________________________________________________________________

    QUESTION 1:
    Which one of the following graphical models fully represents a Variational Auto-encoder (VAE)
    realization?

    Correct Answer: a

    Detailed Explanation:
    For practical realization of VAE, we have an encoder 𝑄(∙) which receives an input signal, 𝑥 and
    generates a latent code, 𝑧. This part of the network can be denoted by 𝑄(𝑧|𝑥) and directed from
    𝑥 to 𝑧. Next, we have a decoder section which takes the encoded z vector to reconstruct the input
    signal, 𝑥. This part of the network is represented by 𝑃(𝑥|𝑧) and should be directed from 𝑧 to 𝑥.
    ______________________________________________________________________________

    QUESTION 2:
    Which one of the following computational graphs correctly depict the re-parameterization trick
    deployed for practical Variational Auto-encoder (VAE) implementation? Circular nodes
    represent random nodes in the models and the quadrilateral nodes represent deterministic
    nodes.
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    a b c d

    Correct Answer: a

    Detailed Solution:
    With the re-parameterization trick, the only random component in the network is the node of ∊
    which is sampled from 𝑁(0, 𝐼). The other nodes of μ and σ are deterministic. Since ∊ is sampled
    from outside the computational graph, the overall z vector also becomes deterministic
    component for a given set of μ, σ and ∊. Also, if z is not deterministic, we cannot back propagate
    gradients through it. Also, in the computation graph, the forward arrows will emerge from μ,σ,ϵ
    towards z for computing the z vector.
    ______________________________________________________________________________

    QUESTION 3:
    When the GAN game has converged to its Nash equilibrium (when the Discriminator randomly
    makes an error in distinguishing fake samples from real samples), what is the probability (of
    belongingness to real class) given by the Discriminator to a fake generated sample?

    a. 1
    b. 0.5
    c. 0
    d. 0.25

    Correct Answer: b

    Detailed Solution:
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    Nash equilibrium is reached when the generated distribution, 𝑝𝑔 (𝑥) equals the original data
    distribution, 𝑝𝑑𝑎𝑡𝑎 (𝑥), which leads to 𝐷(𝑥) = 0.5 for all 𝑥.
    ______________________________________________________________________________

    QUESTION 4:
    Why is re-parameterization trick used in VAE?

    a. Without re-parameterization, the mean vector of latent code of VAE encoder


    with tend towards zero
    b. Sampling from a VAE encoder latent space is non-differentiable and thus we
    cannot back propagate gradient during optimization using gradient descent
    c. We need to re-parameterize Normal distribution over latent space to Bernoulli
    distribution
    d. None of the above

    Correct Answer: b

    Detailed Solution:
    We cannot sample in a differentiable manner from within a computational graph present in a
    neural network. Re-parameterization enables the sampling function to be present outside the
    main computational graph which enables us to do regular gradient descent optimization.
    ______________________________________________________________________________
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    QUESTION 5:
    Which of the following losses can be used to optimize for generator’s objective (while training a
    Generative Adversarial network) by MINIMIZING with gradient descent optimizer? Consider
    cross-entropy loss,

    CE(a, b) = - [ a*log(b) + (1-a)*log(1-b)]

    and D(G(z)) = probability of belonging to real class as output by the Discriminator for a given
    generated sample G(z).

    a. CE(1, D(G(z)))
    b. CE(1, -D(G(z)))
    c. CE(1, 1 - D(G(z)))
    d. CE(1, 1 / D(G(z)))

    Correct Answer: a

    Detailed Solution:
    Except for option (a) none of the other objective function are minimized at D(G(z)) = 1 which is
    the goal of the Generator, i.e. to force the Discriminator to output probability=1 for a generated
    sample. Loss function in option (a) is the only choice which keeps on decreasing as D(G(z))
    increases. Also, it is required that D(G(z)) ∈ [0,1].
    ____________________________________________________________________________

    QUESTION 6:
    What will happen if we do not enforce KL divergence loss in VAE latent code space?

    a. The latent code distribution will be mimic zero mean and unit variance Normal
    distribution
    b. Network will learn to form distinctive clusters with high standard deviation for
    each cluster
    c. Network will learn to form distinctive clusters with low standard deviation for
    each cluster
    d. None of the above

    Correct Answer: c
    Detailed Solution:

    With zero KL loss, the encoder part of VAE will try to form separated clusters (by
    increasing the distance of the mean vectors) and simultaneously reduce the standard
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    deviation of each cluster to reduce confusion for the decoder part of the network. This will
    allow to efficiently reduce the reconstruction loss only present as the loss component in the
    network. So, without the KL loss, the network will reduce to a simple autoencoder network.

    ____________________________________________________________________________

    QUESTION 7:
    Which of the following is an INVALID activation function inside a neural network?

    a. 𝑓(𝑥) = 𝑚𝑎𝑥(0, 2𝑥)


    b. 𝑓(𝑥) = 𝑚𝑖𝑛(0, 2𝑥)
    c. 𝑓(𝑥) = 𝑡𝑎𝑛ℎ(𝑥)
    d. None of the above

    Correct Answer: d

    Detailed Solution:

    Options (a), (b) and (c) are the valid activation functions.

    ______________________________________________________________________________

    QUESTION 8:
    Which one of the following statements is True about Variational Autoencoder (VAE)?

    a. VAE can only be applied on monochrome images


    b. VAE reconstructions tend to be blurry
    c. VAE reconstructions always have high frequency preserving details
    d. VAE latent space is designed to be NOT smooth

    Correct Answer: b

    Detailed Solution:

    All other options are False. One of the primary reasons for the blurry reconstructions by VAE is
    the explicit reconstruction loss component present in the loss function.

    ____________________________________________________________________________
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    QUESTION 9:
    Figure shows latent vector addition of two concepts of “man without a hat” and “hat”. What is
    expected from the resultant vector?

    a. Hat without man


    b. Man with hat
    c. Woman with hat
    d. Woman without hat

    Correct Answer: b
    Detailed Solution:

    It is expected that VAE latent space follows semantic vector arithmetic. Thus the resultant
    vector is a vector addition of the two semantic concepts which will result in the final vector
    to represent a MAN WITH HAT.

    ______________________________________________________________________________

    QUESTION 10:
    Which one of the following statements is preferable situation for GAN training?

    a. Discriminator loss should monotonically increase


    b. Generator loss should monotonically increase
    c. Discriminator loss should monotonically go down while the Discriminator loss
    should monotonically go up
    d. Neither of the Discriminator or Generator monotonically loss goes up or down
    monotonically

    Correct Answer: d

    Detailed Solution:
    NPTEL Online Certification Courses
    Indian Institute of Technology Kharagpur

    Since GAN game is played under a zero-sum non-cooperative game; if one of the wins, the
    opponent losses and the Nash equilibrium is not reached.

    ____________________________________________________________________________

    ______________________________________________________________________________

    ************END*******

    You might also like