Biological Neurons and Neural Networks, Artificial Neurons
Biological Neurons and Neural Networks, Artificial Neurons
Biological Neurons and Neural Networks, Artificial Neurons
L2-1
The Nervous System
The human nervous system can be broken down into three stages that may be
represented in block diagram form as:
Neural
Stimulus Receptors Network/ Effectors Response
Brain
The receptors collect information from the environment – e.g. photons on the retina.
The effectors generate interactions with the environment – e.g. activate muscles.
Naturally, in this module we will be primarily concerned with the neural network in the
middle.
L2-2
Levels of Brain Organization
The brain contains both large scale and small scale anatomical structures and different
functions take place at higher and lower levels.
The ANNs we study in this module are crude approximations to levels 5 and 6.
L2-3
Brains versus Computers : Some numbers
1. There are approximately 10 billion neurons in the human cortex, compared with
10 of thousands of processors in the most powerful parallel computers.
4. The human brain is extremely energy efficient, using approximately 10-16 joules
per operation per second, whereas the best computers today use around 10-6 joules
per operation per second.
5. Brains have been evolving for tens of millions of years, computers have been
evolving for tens of decades.
L2-4
Structure of a Human Brain
L2-5
Slice Through a Real Brain
http://medlib.med.utah.edu/WebPath/HISTHTML/NEURANAT/NEURANCA.html
L2-6
Basic Components of Biological Neurons
2. The neuron’s cell body (soma) processes the incoming activations and converts
them into output activations.
3. The neuron’s nucleus contains the genetic material in the form of DNA. This
exists in most types of cells, not just neurons.
4. Dendrites are fibres which emanate from the cell body and provide the receptive
zones that receive activation from other neurons.
5. Axons are fibres acting as transmission lines that send activation to other neurons.
6. The junctions that allow signal transmission between the axons and dendrites are
called synapses. The process of transmission is by diffusion of chemicals called
neurotransmitters across the synaptic cleft.
L2-7
Schematic Diagram of a Biological Neuron
L2-8
The McCulloch-Pitts Neuron
This vastly simplified model of real neurons is also known as a Threshold Logic Unit :
in1
in2
∑ out
inn
L2-9
Some Useful Notation
We often need to talk about ordered sets of related numbers – we call them vectors, e.g.
Two vectors of the same length may be added to give another vector, e.g.
Two vectors of the same length may be multiplied to give a scalar, e.g.
n
p = x.y = x1y1 + x2 y2 + …+ xnyn = ∑ xi yi
i =1
To any ambiguity/confusion, we will mostly use the component notation (i.e. explicit
indices and summation signs) throughout this module.
L2-10
The Power of the Notation : Matrices
We can use the same vector component notation to represent complex things with many
more dimensions/indices. For two indices we have matrices, e.g. an m × n matrix
L2-11
Some Useful Functions
1.0
1 if x≥ 0 0.8
sgn( x )=
sgn(x)
0.6
0 if x < 0 0.4
0.2
0.0
-4 -2 0 2 4
x
1.0
1 0.8
Sigmoid( x ) = Sigmoid(x)
1 + e− x
0.6
0.4
0.2
0.0
-8 -4 0 4 8
x
L2-12
The McCulloch-Pitts Neuron Equation
Using the above notation, we can now write down a simple equation for the output out
of a McCulloch-Pitts neuron as a function of its n inputs ini :
n
out = sgn(∑ ini − θ )
i =1
L2-13
Overview and Reading
Reading
L2-14