Introduction To Artificial Neural Network

12/18/20
M. Tech 1 st Sem CSE, 2020
ARTIFICIAL NEURAL NETWORK
PGCSE 105A
Introduction to Artificial
Neural Network
The Nervous System:

• The human nervous system can be broken down
into three stages that may be represented in block
diagram form as:
• The receptors collect information from the

environment – e.g. photons on the retina.
• The effectors generate interactions with the
environment – e.g. activate muscles.
• The flow of information/activation is represented by
arrows – feedforward and feedback.
• In this course we will be primarily concerned with
the neural network in the middle.
1
12/18/20
The generic biological neuron
Some Basic Components of Biological Neurons
• The majority of neurons encode their activations or outputs

as a series of brief electrical pulses (i.e. spikes or actions).
• The neuron’s cell body (soma) processes the incoming

activations and converts them into output activations.
• Dendrites are fibers which emanate from the cell body and
provide the receptive zones that receive activation from other
neurons.
• Axons are fibers acting as transmission lines that send

activation to other neurons.
• The junctions that allow signal transmission between the

axons and dendrites are called synapses.
2
12/18/20
Characteristics of Biological Neuron
• Low speed processors (ms) with limited computing

power.
– Massive parallelism.
• They can learn and generalize from training data –

so there is no need for enormous feats of
programming.
• They are particularly fault tolerant –“graceful

degradation”.
• They are very noise tolerant.
A simplified model of human brain
Mathematical representation or Computational

models of neurons
3
12/18/20
Mathematical representation or Computational

models of neurons
Non-linearity
• But we do not want simple summation, but answers.

• The neuron calculates a weighted sum of inputs and compares
it to a threshold. If the sum is higher than the threshold, the
output is set to 1, otherwise to 0 / -1.
The structure of Artificial Neural

Networks
So, What are Artificial Neural Networks ?
• Neural Networks (NNs) are networks of neurons, as found in

real (i.e. biological) brains.
• Artificial Neurons are crude approximations of the neurons

found in brains.
ü They may be physical devices,
ü Or Purely mathematical constructs.
• Artificial Neural Networks (ANNs) are networks of Artificial

Neurons, and hence constitute crude approximations to parts
of real brains.
ü They may be physical devices, or simulated on
conventional computers.
4
12/18/20
What are Artificial Neural Networks ?
• From a practical point of view, an ANN is just a parallel

computational system consisting of many simple processing
elements connected together in a specific way in order to
perform a particular task.
• One should never lose sight of how crude the

approximations are, and how over-simplified our ANNs are
compared to real brains.
Characteristics of ANN
Why are ANN worth studying vis-a-vis personal
computers?
• ICs are extremely powerful computational devices (10 -3s) but
Neurons may not as powerful as ICs (10-9 s).
• But Massive parallelism makes them very efficient.
• They can learn and generalize from training data – so there is

no need for enormous feats of programming.
• They are particularly fault tolerant – this is equivalent to the

“graceful degradation” found in biological systems.
• They are very noise tolerant – so they can cope with situations
where normal symbolic systems would have difficulty.
History Of ANN
• 1943 McCulloch and Pitts proposed the McCulloch-Pitts

neuron model.
• 1949 Hebb published his book The Organization of

Behavior, in which the Hebbian learning rule was
proposed.
• 1958 Rosenblatt introduced the simple single layer

networks now called Perceptrons.
• 1969 Minsky and Papert’s book Perceptrons

demonstrated the limitation of single layer perceptrons,
and almost the whole field went into hibernation or
death.
5
12/18/20
• 1982 Hopfield published a series of papers on Hopfield

networks.
• 1982 Kohonen developed the Self-Organising Maps that now

bear his name.
• 1986 The Back-Propagation learning algorithm for Multi-Layer

Perceptrons was rediscovered and the whole field took off
again.
• 1990s The sub-field of Radial Basis Function Networks was

developed.
• 2000s The power of Ensembles of Neural Networks and

Support Vector Machines becomes apparent.
Application of ANN: Pattern Analysis

• Pattern analysis- The automatic detection of patterns in data
from the same source.
– Make predictions of new data coming from the same source.
– Data may take many forms: images, text, records of commercial
transactions, genome sequences, family tree.
• Pattern is a composite of traits or features characteristic of

an individual.
• Pattern analysis is done using any of the following methods:

1. template matching,
2. statistical methods,
3. syntactic methods and
4. neural networks.
5. Etc…
6
12/18/20
“Hot” Applications
• Recommendation systems
– Amazon, Netflix
• Targeted advertising
Application of ANN: Classification
• Given a set of items that have several classes, and given

the past instances (training instances) with their associated
class, Classification is the process of predicting the class of
a new item.
• Example: A bank wants to classify its Home Loan

Customers into groups according to their response to bank
advertisements. The bank might use the classifications
“Responds Rarely, Responds Sometimes, Responds
Frequently”.

• Face recognition example
7
12/18/20

• Fingerprint recognition example
Application of ANN: Regression

• Regression- Predicting the value of random variable y from
measurement x.
• Finding a relationship (a function f(x)) between the input and

output training data generated by an unknown but fixed
distribution
Application of ANN: Regression
• “Regression deals with the prediction of a value, rather than

a class.”
– Example: Find out if there is a relationship between smoking patients

and cancer related illness.
• A simple approach:
– Given values: X 1, X 2... X n
– Objective predict variable Y
– One way is to predict coefficients a0, a1, a2
– Y = a0 + a1X 1 + a2X 2 + … anX n
• Linear Regression
8
12/18/20
Application of ANN: Clustering

• Partitioning a given dataset with known or un-known
distribution into homogeneous subgroups.
• Example: Insurance company could use clustering to group

clients by their age, location and types of insurance
purchased.
• Object categorization/classification from remote sensed
image example
Books
• Neural network: A comprehensive review by Simon Haykin
• Neural Computing - An Introduction by R Beale, T Jackson.
• Neural Network Design by Martin T. Hagan, Howard B. Demuth, Mark H.
Beale.
• Neural Networks for Pattern Recognition by Christopher M. Bishop.
• Neural Networks-Algorithm, Applications and Programming Techniques,
James A Freeman, David M. Skapura
Easy book:
• Introduction to Neural Networks Using Matlab 6.0 by S. N. Sivanandam, S.
N Deepa, Tata McGraw-Hill.
For Lab- PGCSE-192A :

• Introducing to MATLAB Programming toolbox and simulink by Jaydeep
Charavorty, Universities Press
• Introduction to Neural Networks Using Matlab 6.0 by S. N. Sivanandam, S.
N Deepa, Tata McGraw-Hill.

Introduction To Artificial Neural Network

Uploaded by

Copyright:

Available Formats

Introduction To Artificial Neural Network

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Artificial Neural Network

Uploaded by

Copyright:

Available Formats

12/18/20

M. Tech 1 st Sem CSE, 2020

ARTIFICIAL NEURAL NETWORK

The Nervous System:

• The receptors collect information from the

The generic biological neuron

Some Basic Components of Biological Neurons

• The majority of neurons encode their activations or outputs

• The neuron’s cell body (soma) processes the incoming

• Axons are fibers acting as transmission lines that send

• The junctions that allow signal transmission between the

Characteristics of Biological Neuron

• Low speed processors (ms) with limited computing

• They can learn and generalize from training data –

• They are particularly fault tolerant –“graceful

• They are very noise tolerant.

A simplified model of human brain

Mathematical representation or Computational

Mathematical representation or Computational

• But we do not want simple summation, but answers.

The structure of Artificial Neural

So, What are Artificial Neural Networks ?

• Neural Networks (NNs) are networks of neurons, as found in

• Artificial Neurons are crude approximations of the neurons

• Artificial Neural Networks (ANNs) are networks of Artificial

What are Artificial Neural Networks ?

• From a practical point of view, an ANN is just a parallel

• One should never lose sight of how crude the

• But Massive parallelism makes them very efficient.

• They can learn and generalize from training data – so there is

• They are particularly fault tolerant – this is equivalent to the

• 1943 McCulloch and Pitts proposed the McCulloch-Pitts

• 1949 Hebb published his book The Organization of

• 1958 Rosenblatt introduced the simple single layer

• 1969 Minsky and Papert’s book Perceptrons

• 1982 Hopfield published a series of papers on Hopfield

• 1982 Kohonen developed the Self-Organising Maps that now

• 1986 The Back-Propagation learning algorithm for Multi-Layer

• 1990s The sub-field of Radial Basis Function Networks was

• 2000s The power of Ensembles of Neural Networks and

Application of ANN: Pattern Analysis

• Pattern is a composite of traits or features characteristic of

• Pattern analysis is done using any of the following methods:

Application of ANN: Classification

• Given a set of items that have several classes, and given

• Example: A bank wants to classify its Home Loan

Application of ANN: Classification

Application of ANN: Classification

Application of ANN: Regression

• Finding a relationship (a function f(x)) between the input and

Application of ANN: Regression

• “Regression deals with the prediction of a value, rather than

– Example: Find out if there is a relationship between smoking patients

Application of ANN: Clustering

• Example: Insurance company could use clustering to group

For Lab- PGCSE-192A :

You might also like