Unit-5

Neural Pattern
Recognition
Syllabus
• Introduction to Neural Networks: Neurons and Neural Nets, Neural
Network Structures for PR Applications, Physical Neural Networks,
The Artificial Neural Network Model.
• Introduction to Neural Pattern Associators and Matrix Approaches:
Neural Network Based Pattern Associators, Matrix Approaches (Linear
Associative Mappings) and Examples .
Introduction to Neural Networks
• Neural network is the fusion of artificial intelligence and brain-
inspired design that reshapes modern computing.
• With intricate layers of interconnected artificial neurons, these
networks emulate the workings of the human brain, enabling
remarkable feats in machine learning.
• .There are different types of neural networks, from feedforward to
recurrent and convolutional, each tailored for specific tasks.
• This unit covers its real-world applications across industries like
image recognition, natural language processing, and more.
What are Neural Networks?
• Neural networks mimic the basic functioning of the human brain and
are inspired by how the human brain interprets information.
• They solve various real-time tasks because of its ability to perform
computations quickly and its fast responses.
• Artificial Neural Network has a huge number of interconnected processing
elements, known as Nodes.
• These nodes are connected with other nodes using a connection link.
• The connection link contains weights, these weights contain the information
about the input signal. Each iteration and input in turn leads to updation of
these weights.
• After inputting all the data instances from the training data set, the final
weights of the Neural Network along with its architecture is known as the
Trained Neural Network. This process is called Training of Neural Networks.
• These trained neural networks solve specific problems as defined in the
problem statement.
• Types of tasks that can be solved using an artificial neural network include
Classification problems, Pattern Matching, Data Clustering, etc.
What is Neuron?
• An artificial neuron is a connection point in an artificial neural network.
• Artificial neural networks, like the human body's biological neural
network, have a layered architecture and each network node
(connection point) has the capability to process input and forward
output to other nodes in the network.
• In both artificial and biological architectures, the nodes are called
neurons and the connections are characterized by synaptic weights,
which represent the significance of the connection.
• As new data is received and processed, the synaptic weights change and
this is how learning occurs.
What is Neural Nets?
• A neural network is a method in artificial intelligence that teaches
computers to process data in a way that is inspired by the human
brain.
• It is a type of machine learning process, called deep learning, that
uses interconnected nodes or neurons in a layered structure that
resembles the human brain.
Importance of Neural Networks

• We use artificial neural networks because they learn very efficiently

and adaptively.
• They have the capability to learn “how” to solve a specific problem
from the training data it receives.
• After learning, it can be used to solve that specific problem very
quickly and efficiently with high accuracy.
• Some real-life applications of neural networks include Air Traffic
Control, Optical Character Recognition as used by some scanning apps
like Google Lens, Voice Recognition, etc.
 What are Neural Networks Used For?
• Identifying objects, faces, and understanding spoken language in applications like self-driving cars
and voice assistants.
• Analyzing and understanding human language, enabling sentiment analysis, chatbots, language
translation, and text generation.
• Diagnosing diseases from medical images, predicting patient outcomes, and drug discovery.
• Predicting stock prices, credit risk assessment, fraud detection, and algorithmic trading.
• Personalizing content and recommendations in e-commerce, streaming platforms, and social media.
• Powering robotics and autonomous vehicles by processing sensor data and making real-time
decisions.
• Enhancing game AI, generating realistic graphics, and creating immersive virtual environments.
• Monitoring and optimizing manufacturing processes, predictive maintenance, and quality control.
• Analyzing complex datasets, simulating scientific phenomena, and aiding in research across
disciplines.
• Generating music, art, and other creative content.
Types of Neural Network in Machine
Learning
(i) ANN
• ANN is also known as an artificial neural network.
• It is a feed-forward neural network because the inputs are sent in the
forward direction.
• It can also contain hidden layers which can make the model even denser.
• They have a fixed length as specified by the programmer.
• It is used for Textual Data or Tabular Data.
• A widely used real-life application is Facial Recognition. It is
comparatively less powerful than CNN and RNN.
(ii) CNN
• Convolutional Neural Networks is mainly used for Image Data.
• It is used for Computer Vision.
• Some of the real-life applications are object detection in autonomous
vehicles.
• It contains a combination of convolutional layers and neurons. It is
more powerful than both ANN and RNN.
(iii) RNN
• It is also known as Recurrent Neural Networks.
• It is used to process and interpret time series data.
• In this type of model, the output from a processing node is fed back
into nodes in the same or previous layers.
• The most known types of RNN are LSTM (Long Short Term Memory)
Networks
What is LSTM and why is used?

• LSTMs are predominantly used to learn, process, and classify

sequential data because these networks can learn long-term
dependencies between time steps of data.
• Common LSTM applications include sentiment analysis, language
modeling, speech recognition, and video analysis.
• Now that we know the basics about Neural Networks, We know that
Neural Networks’ learning capability is what makes it interesting.
There are 3 types of learnings in Neural networks, namely
• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning
Reinforcement Learning

• Reinforcement Learning (RL) is the science of decision making.

• It is about learning the optimal behavior in an environment to obtain
maximum reward.
• In RL, the data is accumulated from machine learning systems that
use a trial-and-error method.
• Data is not part of the input that we would find in supervised or
unsupervised machine learning.
• Reinforcement learning uses algorithms that learn from outcomes
and decide which action to take next.
• After each action, the algorithm receives feedback that helps it
determine whether the choice it made was correct, neutral or
incorrect.
• It is a good technique to use for automated systems that have to
make a lot of small decisions without human guidance.
• Reinforcement learning is an autonomous, self-teaching system that
essentially learns by trial and error.
• It performs actions with the aim of maximizing rewards, or in other
words, it is learning by doing in order to achieve the best outcomes.
Example:
The problem is as follows: We have an agent and a reward, with many
hurdles in between. The agent is supposed to find the best possible path
to reach the reward. The following problem explains the problem more
easily.
• The image shows the robot, diamond, and fire.
• The goal of the robot is to get the reward that is the diamond and
avoid the hurdles that are fired.
• The robot learns by trying all the possible paths and then choosing
the path which gives him the reward with the least hurdles.
• Each right step will give the robot a reward and each wrong step will
subtract the reward of the robot.
• The total reward will be calculated when it reaches the final reward
that is the diamond.
Main points in Reinforcement learning –

• Input: The input should be an initial state from which the model will
start
• Output: There are many possible outputs as there are a variety of
solutions to a particular problem
• Training: The training is based upon the input, The model will return a
state and the user will decide to reward or punish the model based on
its output.
• The model keeps continues to learn.
• The best solution is decided based on the maximum reward.
What is Artificial Neural Network?

• The term "Artificial Neural Network" is derived from Biological neural

networks that develop the structure of a human brain.
• Similar to the human brain that has neurons interconnected to one
another, artificial neural networks also have neurons that are
interconnected to one another in various layers of the networks.
These neurons are known as nodes.
What does a neuron look like?
• Dendrites from Biological Neural Network
represent inputs in Artificial Neural Networks,
• cell nucleus represents Nodes,
• synapse represents Weights,
• and Axon represents Output.

The given figure illustrates the typical diagram of Biological Neural Network.
• The typical Artificial Neural Network looks something like the given
figure.
 :

Relationship between Biological neural

network and artificial neural network

Biological Neural Network Artificial Neural Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output
The architecture of an artificial neural network:
• Input Layer:As the name suggests, it accepts inputs in several different formats
provided by the programmer.
• Hidden Layer:The hidden layer presents in-between input and output layers. It
performs all the calculations to find hidden features and patterns.
• Output Layer: The input goes through a series of transformations using the hidden
layer, which finally results in output that is conveyed using this layer. The artificial
neural network takes input and computes the weighted sum of the inputs and
includes a bias. This computation is represented in the form of a transfer function.

• It determines weighted total is passed as an input to an activation function to

produce the output. Activation functions choose whether a node should fire or
not. Only those who are fired make it to the output layer. There are distinctive
activation functions available that can be applied upon the sort of task we are
performing.
Advantages of Artificial Neural Network
(ANN)
• Parallel processing capability:
• Artificial neural networks have a numerical value that can perform more than one task simultaneously.
• Storing data on the entire network:
• Data that is used in traditional programming is stored on the whole network, not on a database. The disappearance
of a couple of pieces of data in one place doesn't prevent the network from working.
• Capability to work with incomplete knowledge:
• After ANN training, the information may produce output even with inadequate data. The loss of performance here
relies upon the significance of missing data.
• Having a memory distribution:
• For ANN is to be able to adapt, it is important to determine the examples and to encourage the network according
to the desired output by demonstrating these examples to the network. The succession of the network is directly
proportional to the chosen instances, and if the event can't appear to the network in all its aspects, it can produce
false output.
• Having fault tolerance:
• Extortion of one or more cells of ANN does not prohibit it from generating output, and this feature makes the
network fault-tolerance.
Disadvantages of Artificial Neural Network:

• Assurance of proper network structure:

• There is no particular guideline for determining the structure of artificial neural networks. The appropriate
network structure is accomplished through experience, trial, and error.
• Unrecognized behavior of the network:
• It is the most significant issue of ANN. When ANN produces a testing solution, it does not provide insight
concerning why and how. It decreases trust in the network.
• Hardware dependence:
• Artificial neural networks need processors with parallel processing power, as per their structure. Therefore,
the realization of the equipment is dependent.
• Difficulty of showing the issue to the network:
• ANNs can work with numerical data. Problems must be converted into numerical values before being
introduced to ANN. The presentation mechanism to be resolved here will directly impact the performance of
the network. It relies on the user's abilities.
• The duration of the network is unknown:
• The network is reduced to a specific value of the error, and this value does not give us optimum results.
 How do artificial neural networks work?

• Artificial Neural Network can be best represented as a weighted

directed graph, where the artificial neurons form the nodes.
• The association between the neurons outputs and neuron inputs can
be viewed as the directed edges with weights.
• The Artificial Neural Network receives the input signal from the
external source in the form of a pattern and image in the form of a
vector.
• These inputs are then mathematically assigned by the notations x(n)
for every n number of inputs.
• Afterward, each of the input is multiplied by its corresponding weights ( these
weights are the details utilized by the artificial neural networks to solve a
specific problem ). In general terms, these weights normally represent the
strength of the interconnection between neurons inside the artificial neural
network. All the weighted inputs are summarized inside the computing unit.
• If the weighted sum is equal to zero, then bias is added to make the output
non-zero or something else to scale up to the system's response. Bias has the
same input, and weight equals to 1. Here the total of weighted inputs can be
in the range of 0 to positive infinity. Here, to keep the response in the limits of
the desired value, a certain maximum value is benchmarked, and the total of
weighted inputs is passed through the activation function.
• The activation function refers to the set of transfer functions used to achieve
the desired output. There is a different kind of the activation function, but
primarily either linear or non-linear sets of functions. Some of the commonly
used sets of activation functions are the Binary, linear, and Tan hyperbolic
sigmoidal activation functions.
Let us take a look at each of them in details:
• Binary:In binary activation function, the output is either a one or a 0.
Here, to accomplish this, there is a threshold value set up. If the net
weighted input of neurons is more than 1, then the final output of the
activation function is returned as one or else the output is returned as 0.
• Sigmoidal Hyperbolic:The Sigmoidal Hyperbola function is generally
seen as an "S" shaped curve. Here the tan hyperbolic function is used to
approximate output from the actual net input. The function is defined
as:
• F(x) = (1/1 + exp(-????x))
• Where ???? is considered the Steepness parameter.
 Types of Artificial Neural Network:
• Feedback ANN: In this type of ANN, the output returns into the network to
accomplish the best-evolved results internally. As per the University of
Massachusetts, Lowell Centre for Atmospheric Research. The feedback
networks feed information back into itself and are well suited to solve
optimization issues. The Internal system error corrections utilize feedback
ANNs.
• Feed-Forward ANN: A feed-forward network is a basic neural network
comprising of an input layer, an output layer, and at least one layer of a neuron.
Through assessment of its output by reviewing its input, the intensity of the
network can be noticed based on group behavior of the associated neurons,
and the output is decided. The primary advantage of this network is that it
figures out how to evaluate and recognize input patterns.
Pattern Association or Associative
Networks
• To an extent, learning is forming associations.
• Human memory associates
– similar items,
– contrary/opposite items,
– items close in proximity,
– items close in succession (e.g., in a song)c etc.
Types of Associative Networks
• Heteroassociative Networks
• Autoassociative Networks
 Pattern Association
• Pattern association is the process of memorizing input-output patterns in
a heteroassociative network architecture, or input patterns only in an
autoassociative network, in order to recall the patterns when a new input
pattern is presented.
• It is not required that the new input pattern be exactly the same as one
that is memorized.
• It can be different, but similar to it.
• Three exemplar patterns are shown in figure .
• After memorizing them in a system, a new pattern is presented, a
corrupted variant of the pattern 3.
• An associative memory system should associate this pattern with one
of the memorized patterns.
• This is a task for an autoassociative network architecture.
• An associative network is a single-layer net in which the weights are
determined in such a way that the net can store a set of pattern
associations.
• Each association is an input-output vector pair s:t
• Auto-associative Network: If vector t is the same as s, the net is auto-
associative.
• If the t’s are different from the s’s, the net is hetero-associative.
• Whether auto- or hetero-associative, the net can associate not only
the exact pattern pairs used in training, but is also able to obtain
associations if the input is similar to one on which it has been
trained.
Examples of Hetero-association

• If the t’s are different from the s’s, the net is hetero-associative.
• Example 1: Mapping from 4-inputs to 2-outputs. Whenever the net
is shown a 4-bit input pattern, it produces a 2-bit output pattern

• Input Output
1000 10
1100 10
0001 01
0011 01
• Example 2: Input similar to a training input
• The net has been trained on the mapping:
Input Output
1000 10

1100 10
0001 01
0011 01

• Now, if the input is x = (0 1 0 0), which is different from the second

training input in one location, the net still produces (1 0) as the output.
•Example 3: Character Recognition

We can train the net to learn the mapping from the larger patterns to the smaller patterns, pairwise

Now, if the input is noisy in some places, the net still produces the
same map. Here up to 1/3rd of the input bits are erroneous
 Example of Auto-association

• Example 1: We can train a 4 input, 4 output network to store just one vector
• s = (1 1 1 -1)
• It will remember this vector s, i.e., if we give s as input it will produce s as output.
• In addition, if there are some errors in the input, it will be able to map it to s as
well.
• For example, it does the following maps:
• (-1 1 1 -1 ) à (1 1 1 -1)
• (1 -1 1 -1 ) à (1 1 1 -1)
• (1 1 -1 -1 ) à (1 1 1 -1)
 Auto Associative Memory

• This is a single layer neural network in which the input training vector and the
output target vectors are the same. The weights are determined so that the
network stores a set of patterns.
Training Algorithm
For training, this network is using the Hebb or Delta learning rule.

Step 1 − Initialize all the weights to zero as wij = 0

i=1 to n,j=1 to n
Step 2 − Perform steps 3-4 for each input vector.
Step 3 − Activate each input unit as follows −
xi=si(i=1 to n)
Step 4 − Activate each output unit as follows −
yj=sj(j=1 to n)
Step 5 − Adjust the weights as follows −
wij(new)= wij(old) + xiyj
Hetero Associative memory

• Similar to Auto Associative Memory network, this is also a single layer

neural network. However, in this network the input training vector
and the output target vectors are not the same. The weights are
determined so that the network stores a set of patterns. Hetero
associative network is static in nature, hence, there would be no non-
linear and delay operations.
Training Algorithm
For training, this network is using the Hebb or Delta learning
rule.
Step 1 − Initialize all the weights to zero as wij = 0 i=1 to n,j=1to m
Step 2 − Perform steps 3-4 for each input vector.
Step 3 − Activate each input unit as follows −
xi=si(i=1 to n)
Step 4 − Activate each output unit as follows −
yj=sj(j=1 to m)
Step 5 − Adjust the weights as follows −
wij(new) = wij(old)+xiyj
References
• http://www.cs.uccs.edu/~
jkalita/work/cs587/2014/05PatternAssoc.pdf
• https://www.youtube.com/watch?v=hHjMXL4LY10
Thank You!!!