Main Contents

FPGA
•
Implementation of
Neural Networks: A Brief Introduction
• Practical Example: Character Recognition using MATLAB
• VHDL: An Instant Approach

Neural Networks •
•
A Neuron Implementation
Activation Function Implementation
• Overview of the Implemented Network
Presented by Nariman Dorafshan

Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Computational Intelligence
Neural Networks • A branch of AI, which implements the nature inspired algorithms is called
computational intelligence.
• Introduction to Neural Networks • Its goal is to find ways for System Modeling, Pattern Recognition, Series
o Computational Intelligence Prediction, Optimization, etc.
o Artificial Neuron • Understanding the performance of brain inspired the creation of artificial neural
o Network Structures networks (ANNs) as an information processing system to generate human
• MLP & Back-propagation recognition models based on biological nervous systems.
o Regulation

• Practical Example: Character Recognition

o Problem Definition
o Training the Network using MATLAB

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

1
 Biological Neuron Artificial Neural Networks (ANNs)
• A single biological neuron is composed of three major parts including the cell • An artificial neuron network (ANN) is an information-processing system that has
body (called soma), the axon, and the dendrite. certain performance characteristics in common with biological neural networks.
• The cell body of a neuron is connected to the dendrite of a neighbouring neuron.
• Based on examples, together with some feedback from a “teacher,” we learn easily
• Signal communications between neurons are continuously generated which is
delivered from one neuron to the others by firing an electrical signal generated to recognize the letter A or distinguish a cat from a bird.
through a chemical reaction.
• More experience allows us to refine our responses and improve our performance.
• The other neurons receive the signal through the interfaces with the neighbouring
neurons, referred to as a synapse. • Although eventually, we may be able to describe rules by which we can make such
• This system is capable of learning, recalling, and
decisions, these do not necessarily reflect the actual process we use.
generating output corresponding to external signals.
• Even without a teacher, we can group similar patterns together.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Artificial Neuron Artificial Neuron in a layer

• ANNs has been developed as generalization of mathematical models of human
cognition or neural biology, based on the assumptions that:
o Information processing occurs at many simple elements called neurons, units,
cells, or nodes.
o Signals are passed between neurons over connection links.
o Each connection link has an associated weight, which, in a typical neural net,
multiplies the signal transmitted.
o Each neuron applies an activation function
(usually nonlinear) to its net input (sum of
weighted input signals) to determine its
output signal.
It is also called “Transfer Function”.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

2
 Artificial Neuron Characteristics & Applications
• Each neuron is connected to other neurons by means of directed communication • A neural network is characterized by:
links, each with an associated weight, o Its pattern connections between the neurons (called architecture)
• The weights represent information being used by the net to solve a problem. o Its method of determining the weights on the connections (called training, or
learning algorithms)
o Its activation function
• Those above also depends on training data spaces.

• Neural nets can be applied to a wide variety of problems, such as storing and
recalling data or patterns, classifying patterns, performing general mappings
from input patterns to output patterns, grouping similar patterns (clustering), or
finding solutions to constrained optimization problems.
• Most common applications lies in “Fitting a Function”, “Recognizing Patterns”,
“Clustering Data”, and “Time Series Prediction”.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Learning: Supervision Activation Functions

• Supervised training: • The output of the neuron is related to the network input via a linear or nonlinear
 Training is accomplished by presenting a sequence of training vectors, or transformation called the activation function or transfer function.
patterns, each with an associated target output vector. • Based on different applications and network structure, we can use different type
 The weights are then adjusted according to a learning algorithm. of activation functions in neuron layers.

• Unsupervised training:
 A sequence of input vectors is provided, but no target vectors are specified. 1 n  0
a  f ( n)  
 The net modifies the weights so that the most similar input vectors are 0 n  0

assigned to the same output (or cluster) unit.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

3
 Activation Functions Network Structures: Connections
• Multiple neurons are interconnected to form a network to facilitate distributed
computing.
• This configuration can be described with a directed graph.
1
a  f (n)  n a  f ( n) 
1  e n • Based on the applications, we use appropriate structure with regards to training
data spaces, learning algorithms, etc.
• As is the case with most neural networks, the aim is to train the net to achieve a
balance between the ability to respond correctly to the input patterns that are
used for training (memorization) and the ability to give reasonable responses to
1  en
a  f ( n)  e  n
2
a  f ( n) 
1  e n input that is similar, but not identical, to that used in training (generalization).

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Network Topologies Network Structures

Acyclic Topology Cyclic Topology
• Consists of no feedback loops. • Contains at least one feedback loop.
• Also called feedforward topology. • Also known as a recurrent network.
• Often used to approximate a • Leads to a nonlinear dynamic system
nonlinear mapping between its model that contains internal memory.
inputs and outputs. • Used in competitive structure.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

4
 MLP & Back-propagation MLP & Back-propagation
• For problems that involve mapping a given set of inputs to a specified set of target
outputs (which is supervised learning), using a multi-layer perceptron is very
common.
• The goal is to achieve a balance between:
o respond correctly to the input patterns that are used for training (memorization)
o give reasonable responses to input that is similar, but not identical, to that used in
training (generalization)

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

MLP & Back-propagation Mean Square Error (MSE)

• Error Back-propagation • The Delta rule, or Least Mean Square (LMS) rule is used as error parameter in
training.
• The learning rule minimizes the mean squared error (MSE) between the activation
and the target values of each layer.
• This allows the net to continue learning on all training patterns even after the
correct output value is generated for some patterns.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

5
 Universal Approximators Universal Approximators
• One use of neural network is to approximate a continuous mapping f. • Kolmogorov Mapping Neural Network Existence Theorem:
 A powerful tool for modelling the issues in which the relations between A feedforward neural network with two layers of neurons (input units, a sigmoid
different variables present no clearly defined forms. layer and a linear output layer) can represent any function with a finite number
of discontinuities.
 Very important in modelling the laboratory samples.

• The question is, how well a multilayer net can perform this task?
o How many layer?
o How many units?
o What about precision?
o Which structure? This network can be used as a general function approximator.
It can approximate any function with a finite number of discontinuities arbitrarily well,
given sufficient neurons in the hidden layer.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

NN Implementation Practical Example

• Neural Networks are highly parallel. • One of the most common applications of neural networks is “Pattern Recognition”.
• A basic problem in all forms of parallel computing is how best to map applications
onto hardware. • Likewise other supervised training processes, associated target vectors are used.
• FPGAs, based on their concurrent nature, are a promising candidate for • For each input vector, only one of the output values would be high, representing
construction of design, implementation, and use of large scale parallel
architectures for neural network applications. the associated pattern, while the others would be low.
• In case of MLP and back-propagation, the goal is to completely parallelize the • An Excel file has been prepared that contains all the input and output vectors in
entire computations.
bipolar format.

• Now let’s define the patterns and move on to Matlab workspace.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

6
 NN Location in process Input Vectors
• In pre-processing unit, input forms has been converted into binary strings.
0011000
0001000
0001000
0010100
0010100
0111110
0100010
0100010
• Pre-processing units such as image processing units, databases, , filters, and etc., 0100010
provide usable data for NN unit. It may be even a manual process. 1110111
They extract input vectors from raw data.
• Post-processing units may be used for inference & decision making, optimization,
Input String:
data compression, etc.
0011000000100000010000010100001010001111100100010010001001000101110111

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Training Vectors Training Vectors

• Three input forms are applied in binary string format to train an MLP network.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

7
 Other Input Vectors Neural Networks: Any Questions?

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

VHDL Basics: Code Structure

VHDL: An Instant Approach • A standalone piece of VHDL code is composed of at least three fundamental
sections:
• VHDL Basics o LIBRARY: a collection of commonly used pieces of code.
o Code Structure
o Arrays
o ENTITY: a list with specifications of all input and output pins (PORTS) of the circuit.
o Operations o ARCHITECTURE: a description of how the circuit should behave (function).
• Concurrent versus Sequential Code
• Data Values
• Miscellaneous Examples
o DFF plus NAND gate
o Sine Wave Generation
o A L U

• Memory Design: ROM

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

8
 VHDL Basics: Library VHDL Basics: Entity
• A collection of commonly used pieces of code. Placing such pieces inside a library
• A list with specifications of all input and output pins (PORTS) of the circuit.
allows them to be reused or shared by other designs.

• Signal Modes:

• Entity: something which exists apart from other things, having its own
independent existence.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

VHDL Basics: Architecture VHDL Basics: Arrays

• A description of how the circuit should behave (function). • Arrays are collections of objects of the same type.
• They can be one-dimensional (1D), two-dimensional (2D), or one-dimension-by-
one-dimensional (1D×1D).

• Has two parts:

o Declarative part (optional): signals and constants are declared.
o Code part (from BEGIN down)
(a) a single value (scalar), (b) a vector (1D),
(c) an array of vectors (1D×1D), and (d) an array of scalars (2D)

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

9
 VHDL Basics: Arrays VHDL Basics: Port Array
• To specify a new array type: • There are no pre-defined data types of more than one dimension.
• We might need to specify the ports as arrays of vectors, in the specification of the
• 1D×1D array example: The leftmost bit is MSB
input or output pins (PORTS).
• The solution is to declare user-defined data types in a PACKAGE. It will be visible
to the whole design (the TYPE declarations are not allowed in an ENTITY).
• 2D array example: The topmost row is row 0

• Array initialization:

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

VHDL Basics: Operations VHDL Basics: Operations

• VHDL provides several kinds of pre-defined operators: • Assignment Operators:
o Assignment operators o <= Used to assign a value to a SIGNAL.
o Logical operators o := Used to assign a value to a VARIABLE, CONSTANT, or GENERIC. Used also for
establishing initial values.
o Arithmetic operators
o => Used to assign values to individual vector elements or with OTHERS.
o Relational operators
o Shift operators
o Concatenation operators
• Assignments:

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

10
 VHDL Basics: Operations VHDL Design Styles (Architecture)
VHDL Design
Styles

(BEHAVIORAL)
NON-
DATAFLOW STRUCTURAL SYTHESIZABLE
SYNTHESIZABLE
“concurrent” components and “sequential” statements
statements interconnects • State machines
• Gates • Registers
• The constructs that are not synthesizable (or have little synthesis support) are • Simple Comb. Logic • Complex Comb. Logic
marked with the “ ” symbol.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Concurrent vs. Sequential Logic Concurrent vs. Sequential Code

Combinational Logic Sequential Logic • VHDL is inherently concurrent (parallel); so to implement any clocked circuits we
have to “force” VHDL to be sequential.
• The output of the circuit depends • The output depend on previous • Only statements placed inside a PROCESS, FUNCTION, or PROCEDURE are
solely on the current inputs. inputs. sequential.
• Requires no memory, can be • Storage elements are required, which • Though within these blocks the execution is sequential, the block, as a whole, is
implemented using conventional are connected to the combinational concurrent with any other (external) statements.
logic. logic block through a feedback loop. • Concurrent processes with sequential execution within a process offers maximum
flexibility
• Sequential code also referred to as “Behavioral code”.
• Concurrent code is also called “dataflow code”.
o Dataflow is a software architecture based on the idea that changing the value of a
variable should automatically force the recalculation of the variables which depends
on its value.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

11
 VHDL Behavior Models Concurrent Mechanism
Concurrent Sequential • Basic granularity of concurrency is the process
o Processes are executed concurrently
o Concurrent signal assignment statements are one-line processes.
• Mechanism for achieving concurrency:
o Processes communicate with each other via signals.
o Signal assignments require delay before new value is assumed.
o Simulation time advances only when all active processes complete.
o Effect is concurrent processing.
VHDL provides concurrent statements using • So, the order in which processes are actually executed by simulator does
`structural’ statements and serial operations using not affect behaviour.
`behavioral statements’.
FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Concurrent Code: When Sequential Code: Wait

• Example: synchronous reset of an 8-bit register:

WAIT UNTIL WAIT ON

• Intended for simulation only.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

12
 Data Values Data Values: Generics
• VHDL provides objects for dealing with values: • A generic parameter is a static parameter that can be easily modified and adapted
o For established (static) values: CONSTANT and GENERIC to different applications. The purpose is more flexibility and reusability.
o For non-static data values: SIGNAL and VARIABLE • A GENERIC statement must be declared in the ENTITY.
• The specified parameter will be truly global (visible to the whole design).
• CONSTANT
o Serves to establish default values.
• Example:
o Can be global (seen by the whole code)
o Can be declared in a PACKAGE, ENTITY, or ARCHITECTURE.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Data Values: Generics Data Values: Signals

• Generics allow a design entity to be described so that, for each use of that • SIGNAL serves to pass values in and out the circuit, and between its internal units.
component, its structure and behaviour can be changed by generic values. • A signal represents circuit interconnections (wires).
• In general they are used to construct parameterized hardware components. • All PORTS of an ENTITY are signals by default.
• "GENERIC" is a great asset when you use your design at many places with slight
changes in the register sizes, input sizes, etc.
• But if the design is very unique you don’t need to have generic parameters.
• Its declaration can be made in the same places as the declaration of a CONSTANT.
• The assignment operator for a SIGNAL is “<=”.
• The initial value in the syntax above is not synthesizable, being only considered in
simulations.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

13
 Data Values: Signals Data Values: Variables
• When used inside a section of sequential code (e.g. PROCESS) • Represents only local information.
 Its update is not immediate. o Can only be used inside a sequential code.
 Its new value should not be expected to be ready before the corresponding o Its declaration can only be done in the declarative part of a sequential code.
PROCESS, FUNCTION or PROCEDURE. o Its values can not be passed out directly. It must be assigned to a SIGNAL first.
• Its update is immediate.
o The new value can be promptly used in the next line of code.

• The assignment operator for a variable is “:=”.

• The initial values are synthesizable.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Data Values: Signals vs. Variables Examples: DFF plus NAND gate

• Line 10 is executed concurrently

with block 11-15.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

14
 Examples: Sine Wave Generation Examples: ALU

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Memory Design: ROM Memory Design: ROM

• Since it is read-only memory, no clock signal or write-enable pin is necessary.
• The circuit is a pile of pre-stored words, being the one selected by the address
input (addr) presented at the output (data).

• A LUT is simply a ROM.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

15
 Memory Design: ROM VHDL: Any Questions?

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

A Layer Structure: Schematic

A Neuron Implementation
• A Layer Structure
• Small Scale Approach
• Large Scale Approach

Activation function implementation will be discussed in the next section.

The main difference between these two solutions is that the first code is not as
generic, and therefore adequate for specific, small designs. The second solution,
being generic, is reusable and easily adaptable to any NN size.
NN implementation with only one input for the weights

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

16
 A Layer Structure: Description Small Scale Approach

• This approach is appropriate for general purpose NNs (that is, with
programmable weights). • Has the advantage of being simple, easily understandable, and self contained in
• It employs only one input to load all weights (thus saving on chip pins). the main code.
• The weights are shifted in sequentially until each register is loaded with its • Its limitation is that the inputs (x) and outputs (y) are specified one by one.
respective weight.
• The weights are then multiplied by the inputs and accumulated to produce the
desired outputs.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Small Scale Approach: Entity Small Scale Approach: Architecture

specified one by one

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

17
 Small Scale Approach: Architecture Large Scale Approach
• We use a more generic approach which is appropriate for large scale networks.
• The idea is to use arrays for representing inputs, outputs, and weights.
• Since type declaration is not permitted in the entity, we use a package for it.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Large Scale Approach: Entity Large Scale Approach: Architecture

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

18
 Large Scale Approach: Architecture Neuron Implementation:Any Questions?

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Implementation Problem
Activation Function Implementation • The sigmoid and hyperbolic tangent functions are usually used as the activations
in ANNs.
• Straightforward implementation of these functions in hardware is not practical
due to their exponential nature.
• Implementation Problem
• Several different approaches exist for this hardware approximation, including
• Look-Up Tables (LUT) lookup tables (LUT), piecewise linear approximation, piecewise non-linear
approximation, and hybrid methods.
• Linear Approximation Methods
• Range Addressable LUTs (RALUT)
• Hybrid Methods

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

19
 Lookup Tables (LUT) Piecewise Linear Approximation
• Generally, LUT implementations are the fastest, while they consume a large area of • Piecewise linear approximations are slow, but consume less area; they are also the
silicon. most common way of implementing the activation functions.
• They are based on ROM design. Get an input range or input address.

LUT RALUT

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Comparison of Implementation Methods Complexity comparison

• Generally, LUT implementations are the fastest, while they consume a large area of
silicon.
• Piecewise linear approximations are slow, but consume less area; they are also the
most common way of implementing the activation functions.
• Piecewise non-linear approximations achieve excellent approximation (with low
maximum error) but are the slowest type of approximation, as they usually make
use of multipliers in their architecture.
• Hybrid methods typically achieve good performance regarding area and delay as
they employ a combination of approaches, such as the use of piecewise
approximation with lookup tables.

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

20
 Activation Function: Any Questions?
The Entire Design

FPGA Implementation of Neural Networks Semnan University – Spring 2012 FPGA Implementation of Neural Networks Semnan University – Spring 2012

Any Questions?

FPGA Implementation of Neural Networks Semnan University – Spring 2012

21