Neural Networks: Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22

Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 18

Neural Networks

Teacher: Assistant:
Elena Kees Jong
Marchiori S2.22
R4.47 [email protected]
[email protected]

Neural Netw NN 1 1
Course Outline

Basics of neural network theory and practice for


supervised and unsupervised learning.

Most popular Neural Network models:


architectures
learning algorithms
applications

Neural Netw NN 1 2
Course Outline

Rules: - 4 s.p
- Final mark is based on two assignments, which will
be available at the end of the course.
- one assignment is on theory (to do alone).
- one assignment is on practice (to do in couples).
- Programming in Matlab 5.3.
- Registration: send email to [email protected]

Neural Netw NN 1 3
Course Organization

There is no text book.

Course schedule, slides and exercises will be


available at

http://www.cs.vu.nl/~elena/nn.html

Neural Netw NN 1 4
Neural Networks
A NN is a machine learning approach inspired by the
way in which the brain performs a particular learning
task:
Knowledge about the learning task is given in the form of
examples.

Inter neuron connection strengths (weights) are used to


store the acquired information (the training examples).

During the learning process the weights are modified in


order to model the particular learning task correctly on the
training examples.

Neural Netw NN 1 5
Learning
Supervised Learning
Recognizing hand-written digits, pattern recognition,
regression.
Labeled examples (input , desired output)
Neural Network models: perceptron, feed-forward, radial
basis function, support vector machine.
Unsupervised Learning
Find similar groups of documents in the web, content
addressable memory, clustering.
Unlabeled examples (different realizations of the input
alone)
Neural Network models: self organizing maps, Hopfield
networks.

Neural Netw NN 1 6
Network architectures

Three different classes of network architectures

single-layer feed-forward neurons are organized


multi-layer feed-forward in acyclic layers
recurrent

The architecture of a neural network is linked with the


learning algorithm used to train

Neural Netw NN 1 7
Single Layer Feed-forward

Input layer Output layer


of of
source nodes neurons

Neural Netw NN 1 8
Multi layer feed-forward

3-4-2 Network

Input Output
layer layer

Hidden Layer

Neural Netw NN 1 9
Recurrent network
Recurrent Network with hidden neuron(s): unit delay operator z-1
implies dynamic system

z-1

input
z-1 hidden
output

z-1

Neural Netw NN 1 10
Neural Network Architectures

Neural Netw NN 1 11
The Neuron
The neuron is the basic information processing unit of
a NN. It consists of:
1 A set of synapses or connecting links, each link
characterized by a weight:
W1, W2, , Wm
2 An adder function (linear combiner) which
computes the weighted sum of m
the inputs: u wjxj
j 1

3 Activation function (squashing function) for


limiting the amplitude of the
output of the neuron. y (u b)
Neural Netw NN 1 12
The Neuron
Bias
b
x1 w1
Activation
Local function
Field


Output
Input x2 w2 v () y
signal

Summing
function

xm wm
Synaptic
weights

Neural Netw NN 1 13
Bias of a Neuron
Bias b has the effect of applying an affine
transformation to u
v=u+b
v is the induced field of the neuron
v

u
m
u wjxj
j 1

Neural Netw NN 1 14
Bias as extra input
Bias is an external parameter of the neuron. Can be
modeled by adding an extra input. m
v wj xj
w0 j 0
x0 = +1
w0 b
x1 w1 Activation
Local function
Field


Input Output
signal x2 w2
v () y
Summing
function

xm Synaptic
wm
weights
Neural Netw NN 1 15
Dimensions of a Neural
Network
Various types of neurons

Various network architectures

Various learning algorithms

Various applications

Neural Netw NN 1 16
Face Recognition

90% accurate learning head pose, and recognizing 1-of-20 faces


Neural Netw NN 1 17
Handwritten digit recognition

Neural Netw NN 1 18

You might also like