Distributed system:

• Forms parallelism.

• Classification architecture :
A.Flynn Taxonomy:
1. SISD.
2. SIMD.
3. MISD.
4. MIMD.

B. According connection memory - processor :

1. Share memory:
-According access” UMA, NUMA, COMA”.
-According connection: buses, crossing switch, multistage network.

2. Message system:
-according network topology.

3. Hybrid system (combination access).

4. Distributed share memory.

1
Form Parallelism
1. Processors:
-sequence: execute always only one instruction.
-”look ahead”: looking also to next instruction:
A technique for speeding up the process of fetching and decoding instructions in a
computer program, A central processing unit wherein instruction fetch and execution is
performed by a mechanism featuring an instruction look ahead mechanism whereby
fetching and processing of the next software instruction is commenced as a last step of
the currently executing software instruction, and the currently executing software
instruction is terminated by the first portion of the next software instruction.

2. Overlapping phase read and decode with executing instruction.

3. Function parallelism:
1) Multi function units:
Multifunctional unit (MFU) of the processor. This reconfigurable functional unit can
hold several instructions or sequences of instructions that can be provided by a
special compiler, and loaded by the processor when needed.

2
 2) Chining-”pipeline”:
Pipelining imparts an implicit execution parallelism in the different cycles
of processing an instruction. Suppose execution of an instruction consists of the
following stages:
1. Fetch – Get the instruction from memory.
2. Decode – Determine what the instruction is.
3. Execute – Perform the instruction decode.
4. Write – Store the results to memory.

4. Chining (Next parts function units).

1) Implicit processing vector operand.
2) Explicit vector operand.

5. Register –register.
- Single Instruction Multiple Data “SIMD”:
ex: processor array “EPIC” ( Explicitly Parallel Instruction Computing ),
Very Long Instruction Word ”VLIW”:

Very long instruction word (VLIW) describes a computer processing architecture

in which a language compiler or pre-processor breaks program instruction down into
basic operations that can be performed by the processor in parallel (that is, at the same
time). These operations are put into a very long instruction word which the processor can
then take apart without further analysis, handing each operation to an appropriate
functional unit.

3
 “EPIC” ( Explicitly Parallel Instruction Computing ):
Its architecture should enable the CISC processors to take a big enough step to
overtake the RISC processors. By using a technique called VLIW, still experimental at
the time, and creating the EPIC model, explicitly parallel. That is, multiple instructions can
easily be executed at once, assuming that the hardware supports it.

6. Memory – memory.
- Multiple Instructions Multiple Data “MIMD”:
Ex: Multiprocessor, Multicomputer.

4
Flynn Taxonomy
 Michael J. Flynn (1966).
 How much instruction is in given moment running and over
how much data elements?

Instruction Stream

Single Multiple

Single SISD MISD

Data

Stream
Multiple SIMD MIMD

FLYNN’S TAXONOMY OF COMPUTER ARCHITECTURE

The most popular taxonomy of computer architecture was defined

by Flynn in 1966.

Flynn’s classification scheme is based on the notion of a stream of

information. Two types of information flow into a processor:
instructions and data. The instruction stream is defined as the sequence
of instructions performed by the processing unit. The data stream is
defined as the data traffic exchanged between the memory and the
processing unit. According to Flynn’s classification, either of the
5
instruction or data streams can be single or multiple. Computer
architecture can be classified into the following four distinct categories:

. single-instruction single-data streams (SISD);

. single-instruction multiple-data streams (SIMD);
. multiple-instruction single-data streams (MISD); and
. multiple-instruction multiple-data streams (MIMD).

Conventional single-processor von Neumann computers are classified as

SISD systems. Parallel computers are either SIMD or MIMD. When
there is only one control unit and all processors execute the same
instruction in a synchronized fashion, the parallel machine is classified
as SIMD. In a MIMD machine, each processor has its own control unit
and can execute different instructions on different data. In the MISD
category, the same stream of data flows through a linear array of
processors executing different instruction streams. In practice, there is
no viable MISD machine; however, some authors have considered
pipelined machines (and perhaps systolic-array computers) as examples
for MISD.
An extension of Flynn’s taxonomy was introduced by D. J. Kuck in
1978. In his classification, Kuck extended the instruction stream further
to single (scalar and array) and multiple (scalar and array) streams. The
data stream in Kuck’s classification is called the execution stream and is
also extended to include single (scalar and array) and multiple (scalar
and array) streams. The combination of these streams results in a total of
16 categories of architectures.

6
1) SISD ARCHITECTURE:

1. One control unit, one flow control.

2. Always read/write only one value data from memory.
3. Classical von Neumann architecture.

Today architecture of SISD:

7
2) SIMD ARCHITECTURE:

1. Single control unit (single instruction cache) control multi equal

(synchronize) executed units.
2. Equal operation will execute with various data.
3. Vector processors: operate over vector execute in single instruction
cycle.
4. (Increment over component scalar multiplies).
5. Work with special HW “VLIW, EPIC”.
6. Characteristic :
1) Count execution units (EU).
2) Count scalar instruction.
3) Semaphore.
4) Count bridge scheme.
5) Count vector instruction.

Note: VLIW (Very Long Instruction Word),

EPIC ( Explicitly Parallel Instruction Computing ).
Scalar (in computing), an atomic quantity that can hold only one value at a time.

The SIMD model of parallel computing consists of two parts: a front-end computer
of the usual von Neumann style and a processor array as shown in Figure, The
processor array is a set of identical synchronized processing elements capable of
simultaneously performing the same operation on different data. Each processor in
the array has a small amount of local memory where the distributed data resides
while it is being processed in parallel. The processor array is connected to the
memory bus of the front end so that the front end can randomly access the local
processor memories as if it were another memory. Thus, the front end can issue
special commands that cause parts of the memory to be operated on simultaneously
or cause data to move around in the memory. A program can be developed and
executed on the front end using a traditional serial programming language. The
application program is executed by the front end in the usual serial way, but issues
commands to the processor array to carry out SIMD operations in parallel.
The similarity between serial and data parallel programming is one of the strong
points of data parallelism. Synchronization is made irrelevant by the lock–step
8
synchronization of the processors. Processors either do nothing or exactly the same
operations at the same time. In SIMD architecture, parallelism is exploited by
applying simultaneous operations across large sets of data. This paradigm is most
useful for solving problems that have lots of data that need to be updated on a
wholesale basis. It is especially powerful in many regular numerical calculations.

There are two main configurations that have been used in SIMD machines. In the
first scheme, each processor has its own local memory. Processors can
communicate with each other through the interconnection network. If the
interconnection network does not provide direct connection between a given pair
of processors, then this pair can exchange data via an intermediate processor.

9
3) MISD ARCHITECTURE:

1. Multiple Various operation over equal data :

-Data pass at once to all execution units (different parts
single data flow).
-Data pass to first execution unit and his output is input to
next …”Micro- pipeline”.

2. Ex: Carnegie-Mellon: Multi mini processor,

”C.mmp” (16p,SIMD,MISD,MIMD)

3. “Data flow” architecture, systolic array.

4. Only for integrity, Practically unused.

4) MIMD ARCHITECTURE:

1. Multi flow “control and data” _ each processor can execute

different program on itself data.
2. Can work asynchronous (on different from MISD).
3. Processors synchronous for access to share (resources)
data.
4. Computer with multi processor (SM-MIMD).
5. Multi computer connected in network (DM-MIMD).

10
Multiple-instruction multiple-data streams (MIMD) parallel architectures are made
Of multiple processors and multiple memory modules connected together via some
interconnection network. They fall into two broad categories: shared memory or
Message passing. the general architecture of these two categories.
Processors exchange information through their central shared memory in
shared memory systems, and exchange information through their interconnection
network in message passing systems.
A shared memory system typically accomplishes interprocessor coordination
through a global memory shared by all processors. These are typically server
systems that communicate through a bus and cache memory controller. The bus/

11
cache architecture alleviates the need for expensive multi ported memories and
interface circuitry as well as the need to adopt a message-passing paradigm when
developing application software. Because access to shared memory is balanced,
these systems are also called SMP (symmetric multiprocessor) systems. Each
processor has equal opportunity to read/write to memory, including equal access
speed.
A message passing system (also referred to as distributed memory) typically
combines the local memory and processor at each node of the interconnection
network.
There is no global memory, so it is necessary to move data from one local memory
to another by means of message passing. This is typically done by a Send/Receive
pair of commands, which must be written into the application software by a
programmer.

How can seem MIMD/SIMD?

 Multiprocessors:
-Narrow band- communicates through share memory.
-Single (share) addresses space.
- Assigned data getting by searching program on
Share memory locations.
-Semaphore, mutex, barrier.
 Multicomputer:
-Free band-communication message system.
-Each process have self memory and address space.
-Located assigned data by effected step (instruction) in program.

12