Unit 1

Deep Learning
By
Prof Nirali Arora

Rizvi College of Engineering
Syllabus and books to be referred
T Detailed syllabus and teaching plan of Deep learning subject is as follows
RCOE CMSys | View Teaching Plan
Deep learning books to be referred
Deep learning using python and tensor flow by Venkata Reddy Konasani
Deep learning by Iresh Dhotre
Deep learning by Lovelyn Rose

Biological Neuron
Functionality of Biological Neuron

Biological neuron working
Brain is a neural network. One really really complex neural network. It has 1011 neurons.
And each of these neurons is connected to approximately 104 other neurons. These
interconnected neurons use electrical pulses to "communicate" with each other
Above is a highly simplified diagram of a neuron. In the center, is the cell body. This houses
the actual organelles of the nerve cell. To the left of the diagram are incoming connections.
The diagram shows only a few of them, but in physical neurons, the count is usually in
thousands. And to the right of the diagram is axon, the output of the neuron.
The output of the neuron is an electrical voltage. And this voltage is decided by the cell
body, based on the inputs it receives. This output goes to several other nerve cells, where it
acts as an input. Thus, this single neuron can "contribute" to controlling several other
neurons.
Now let's talk about the inputs into the nerve cell. The nerve cell
has several dendrites. These dendrites gather information from the
axon of other nerve cells. The way in which an electrical pulse is
transferred from an axon to a dendrite is interesting.
The point of contact of the axon and the dendrite is called a
synapse. It is at the synapse that the electrical pulse actually jumps
from the axon to the dendrite. This happens by means of a complex
chemical reaction.
.
T
By "summing up" the inputs. This happens because all the inputs are
electrical voltages, and they all merge at a single point. So, if the total
input (after passing and getting amplified/reduced through the synapse)
is greater than some particular value, the cell fires an event. Again, this
happens through a complex chemical reaction.
Once the cell fires, it must restore its chemical equilibrium (and prepare
for future signals). So the neuron remains inactive for a small amount of
time (called the refractory period). Thus several neurons may remain
inactive at any given time while several others are active
ANN: Artificial Neural Network
ANN is an efficient computing system whose central theme is

borrowed from the analogy of biological neural networks. ANNs
are also named as “artificial neural systems,” or “parallel
distributed processing systems,” or “connectionist systems.” ANN
acquires a large collection of units that are interconnected in
some pattern to allow communication between the units. These
units, also referred to as nodes or neurons, are simple processors
which operate in parallel.
Every neuron is connected with other neuron through a
connection link. Each connection link is associated with a weight
that has information about the input signal. This is the most
useful information for neurons to solve a particular problem
because the weight usually excites or inhibits the signal that is
being communicated. Each neuron has an internal state, which is
called an activation signal. Output signals, which are produced
after combining the input signals and activation rule, may be sent
to other units.
History of ANN
● 1943 − It has been assumed that the concept of neural network started
with the work of physiologist, Warren McCulloch, and mathematician,
Walter Pitts, when in 1943 they modeled a simple neural network using
electrical circuits in order to describe how neurons in the brain might
work.
● 1949 − Donald Hebb’s book, The Organization of Behavior, put forth the
fact that repeated activation of one neuron by another increases its
strength each time they are used.
● 1956 − An associative memory network was introduced by Taylor.
● 1958 − A learning method for McCulloch and Pitts neuron model named
● 1961 − Rosenblatt made an unsuccessful attempt but proposed the
“backpropagation” scheme for multilayer networks.
● 1964 − Taylor constructed a winner-take-all circuit with inhibitions
among output units.
● 1969 − Multilayer perceptron
● MLP
● MLP was invented by Minsky and Papert.
● 1971 − Kohonen developed Associative memories.
● 1976 − Stephen Grossberg and Gail Carpenter developed Adaptive
● 1982 − The major development was Hopfield’s Energy approach.
● 1985 − Boltzmann machine was developed by Ackley, Hinton, and Sejnowski.
● 1986 − Rumelhart, Hinton, and Williams introduced Generalised Delta Rule.
● 1988 − Kosko developed Binary Associative Memory
● BAM
● BAM and also gave the concept of Fuzzy Logic in ANN.
Mc Culloch and Pitts model
Explain McCuloch-Pitts Network?
The first computational model of a neuron was proposed by Warren MuCulloch

(neuroscientist) and Walter Pitts (logician) in 1943.
It may be divided into 2 parts. The first part, g takes an input (ahem dendrite ahem), performs an
aggregation and based on the aggregated value the second part, f makes a decision.
Lets suppose that I want to predict my own decision, whether to watch a random football game or not on
TV. The inputs are all boolean i.e., {0,1} and my output variable is also boolean {0: Will watch it, 1: Won’t
watch it}.
● So, x_1 could be isPremierLeagueOn (I like Premier League more)
● x_2 could be isItAFriendlyGame (I tend to care less about the friendlies)
● x_3 could be isNotHome (Can’t watch it when I’m running errands. Can
I?)
● x_4 could be isManUnitedPlaying (I am a big Man United fan. GGMU!)
and so on.
These inputs can either be excitatory or inhibitory. Inhibitory inputs are those
that have maximum effect on the decision making irrespective of other inputs i.e.,
Mathematically this is happening
We can see that g(x) is just doing a sum of the inputs — a simple aggregation. And theta here is called thresholding parameter. For example, if I always watch the game
when the sum turns out to be 2 or more, the theta is 2 here. This is called the Thresholding Logic.
Question :
Q1. Model a simple add function using Mcculloch -Pitts unit?

Difference between Biological and Artificial
Neurons
Biological Neurons
Major components: Axions, Dendrites, Synapse
Artificial Neurons
Major Components: Nodes, Inputs, Outputs, Weights, Bias
Biological Neurons working:Information from other neurons, in the form of
electrical impulses, enters the dendrites at connection points called synapses.
The information flows from the dendrites to the cell where it is processed. The
output signal, a train of impulses, is then sent down the axon to the synapse of
other neurons.
ANN working :The arrangements and connections of the neurons made up the
network and have three layers. The first layer is called the input layer and is the
only layer exposed to external signals. The input layer transmits signals to the
Different types of Neural Network Architectures
Perceptron
Types of Perceptron models
1. Single-layer Perceptron Model

2. Multi-layer Perceptron model
Single Layer Perceptron Model:
This is one of the easiest Artificial neural networks (ANN) types. A single-layered perceptron model consists feed-forward network and also includes a threshold
transfer function inside the model. The main objective of the single-layer perceptron model is to analyze the linearly separable objects with binary outcomes.
In a single layer perceptron model, its algorithms do not contain recorded data, so it begins with inconstantly allocated input for weight parameters. Further, it
sums up all inputs (weight). After adding all inputs, if the total sum of all inputs is more than a pre-determined value, the model gets activated and shows the
output value as +1.
If the outcome is same as pre-determined or threshold value, then the performance of this model is stated as satisfied, and weight demand does not change.
Multilayer Perceptron model
Multilayer Perceptron model

Like a single-layer perceptron model, a multi-layer perceptron model also has the same model
structure but has a greater number of hidden layers.
The multi-layer perceptron model is also known as the Backpropagation algorithm, which executes
in two stages as follows:
● Forward Stage: Activation functions start from the input layer in the forward stage and
terminate on the output layer.
● Backward Stage: In the backward stage, weight and bias values are modified as per the
model's requirement. In this stage, the error between actual output and demanded originated
backward on the output layer and ended on the input layer.
Define Deep Learning
Deep learning is a branch of machine learning which is completely based

on artificial neural networks, as neural network is going to mimic the
human brain so deep learning is also a kind of mimic of human brain. In
deep learning, we don’t need to explicitly program everything. The
concept of deep learning is not new. It has been around for a couple of
years now. It’s on hype nowadays because earlier we did not have that
much processing power and a lot of data. As in the last 20 years, the
processing power increases exponentially, deep learning and machine
learning came in the picture.
Why is Deep learning called as Deep ? (Viva)
Think???
Difference between Deep Learning and
machine learning?
Answer to above question..
Difference between Machine and deep learning - Google Docs

Applications of Deep learning
Deep learning applications - Google Docs

Unit 1

Uploaded by

Copyright:

Available Formats

Unit 1

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unit 1

Uploaded by

Copyright:

Available Formats

Deep Learning

Prof Nirali Arora

T Detailed syllabus and teaching plan of Deep learning subject is as follows

RCOE CMSys | View Teaching Plan

Deep learning books to be referred

Deep learning by Iresh Dhotre

Deep learning by Lovelyn Rose

Functionality of Biological Neuron

ANN is an efﬁcient computing system whose central theme is

Explain McCuloch-Pitts Network?

The first computational model of a neuron was proposed by Warren MuCulloch

● x_2 could be isItAFriendlyGame (I tend to care less about the friendlies)

● x_4 could be isManUnitedPlaying (I am a big Man United fan. GGMU!)

Q1. Model a simple add function using Mcculloch -Pitts unit?

1. Single-layer Perceptron Model

Single Layer Perceptron Model:

Multilayer Perceptron model

Deep learning is a branch of machine learning which is completely based

Answer to above question..

Difference between Machine and deep learning - Google Docs

Deep learning applications - Google Docs

You might also like