CS4231 Parallel and Distributed Algorithms

CS4231
Parallel and Distributed Algorithms
AY 2006/2007 Semester 2
Lecture 3 (26/01/2006)
Instructor: Haifeng YU
Review of Last Lecture
Why do we need synchronization primitives
Busy waiting waste CPU
But synchronization primitives need OS support
Semaphore:
Using semaphore to solve dining philosophers problem
Avoiding deadlocks
Monitor:
Easier to use than semaphores / more popular
Two kinds of monitors
Using monitor to solve producer-consumer and reader-writer
problem
CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 2

Today’s Roadmap
 Chapter 4 “Consistency Conditions”, Chapter 5.1, 5.2
“Wait-Free Synchronization”
 What is consistency? Why do we care about it?
 Sequential consistency
 Linearizability
 Consistency models for registers

Abstract Data Type
 Abstract data type: A piece of data with allowed
operations on the data
 Integer X, read(), write()
 Integer X, increment()
 Queue, enqueue(), dequeue()
 Stack, push(), pop()
 We consider shared abstract data type

 Can be accessed by multiple processes
 Shared object as a shorthand

What Is Consistency?
 Consistency – a term with thousand definitions
 Definition in this course:
 Consistency specifies what behavior is allowed when a shared
object is accesses by multiple processes
 When we say something is “consistent”, we mean it satisfies the
specification (according to some given spec)
 Specification:
 No right or wrong – anything can be a specification
 But to be useful, must:
 Be sufficiently strong – otherwise the shared object cannot be used
in a program
 Can be implemented (efficiently) – otherwise remains a theory
 Often a trade-off

Mutual Exclusion Problem: x = x+1
process 0 process 1
 Data type 1:
read x into a register
 Integer x (value read: 0)
 read(), write() we implicitly assumed
increment the the value read must
register (1) be the value written –
 Data type 2: one possible def of
write value in consistency
 Integer x register back to x (1)
 increment() read x into a register
(value read: 1)
increment the
 We were using register (2)
type 1 to write value in
implement type 2 register back to x (2)

Mutual Exclusion Problem
 Data type 1: process 0 process 1
 Integer x
 read(), write() (value read: 0)
increment the
 Data type 2: register (1)
 Integer x
 increment()
(value read: 0)
increment the
 By saying the register (1)
execution is not “good”,
we implicitly assumed write value in
some consistency def register back to x (1)
for Data type 2 write value in
register back to x (1)

Why do we care about consistency?
 Mutual exclusion is actually a way to ensure
consistency (based on some consistency definition)
 Thus we actually already dealt with consistency…
 Our goal in the lecture

 To formalize such consistency definition
 Explore some other useful consistency definitions

Formalizing a Parallel System
 Operation: A single invocation of a single method of a
single shared object by some process
 e being an operation
 proc(e): The invoking process
 obj(e): The object wall clock time /
 inv(e): Invocation event (start time)
physical time /
 resp(e): Reply event (finish time)
real time
 Two invocation events are the same if invoker, invokee,
parameters are the same – inv(p, read, X)
 Two response events are the same if invoker, invokee,
response are the same – resp(p, read, X, 1)

(Execution) History
 (The definitions in the book is based on operations
are slightly harder to understand. We will use a
different kind of definitions. But the two are
equivalent.)
 A history H is a sequence of invocations and
responses ordered by wall clock time
 For any invocation in H, the corresponding response is
required to be in H
 One-to-one mapping between all histories and all possible
execution of a parallel system
inv(p, read, X) inv(q, write, X, 1) resp(p, read, X, 0) resp(q, write, X, OK)

Sequential History and Concurrent History
 A history H is sequential if
 Any invocation is always immediately followed by its
response
 I.e. No interleaving (but interleaving allowed between
operations)
 Otherwise called concurrent
Sequential:
inv(p, read, X) resp(p, read, X, 0) inv(q, write, X, 1) resp(q, write, X, OK)
concurrent:
inv(p, read, X) inv(q, write, X, 1) resp(p, read, X, 0) resp(q, write, X, OK)

Sequential Legal History and Subhistory
 A sequential history H is legal if
 All responses satisfies the sequential semantics of the object
 Sequential means: Any invocation is always immediately
followed by its response
 It is possible for a sequential history not to be legal
 Process p’s process subhistory of H (H | p):

 The subsequence of all events of p
 Thus a process subhistory is always sequential
 Object o’s object subhistory of H (H | o):

 The subsequence of all events of o

Equivalency and Process Order
 Two histories are equivalent if they have the exactly
same set of events
 Same events imply all responses are the same
 Ordering of the events may be different
 User only cares about responses – ordering not observable
 Process order is a partial order among all events

 For any two events of the same process, process order is
the same as execution order
 No other additional orders

Sequential Consistency
 First defined by Lamport: “…the results … is the
same as if the operations of all the processors were
executed in some sequential order, and the
operations of each individual processor appear in this
sequence in the order specified by the program”
 Use sequential order as a comparison point
 Focus on results only – that is what users care about (what if
we only allow sequential histories?)
 Require that program order is preserved
 All our formalism earlier served to formalize the

above

Sequential Consistency
 A history H is sequentially consistent if it is equivalent
to some legal sequential history S that preserves
process order
 Use sequential order as a comparison point
 Focus on results only – that is what users care about (what if
we only allow sequential histories?)
 Require that program order is preserved

Examples
write(x,1) ok() write(x,1) ok()

P P
read(x) ok(0)  read(x) ok(0)
Q Q

P P
Q Q

Examples
write(x,1) ok() read(x) ok(0)

P
read(x) ok(0)  ?
Q
write(x,1) ok()
P
read(x) ok(2)  ?
Q

Motivation for Linearizability
 Sequential consistency arguably the most widely used
consistency definition
 E.g. Almost all commercial databases
 Many multi-processors
 But sometimes not strong enough

P P
Q Q

More Formalism
 A history H uniquely induces the following “<“ partial order
among operations
 o1 “<“ o2 iff response of o1 appears in H before invocation of o2
 What is the partial order induced by a sequential history?

P P
o1 read(x) ok(0)  read(x) ok(0)
Q Q
o2

Linearizability
 Intuitive definition: The execution is equivalent to
some execution such that each operation happens
instantaneously at some point (called linearization
point) between the invocation and response

P P
Q Q
The above history is sequentially consistent but not linearizable

Linearizability
 Intuitive definition: The execution is equivalent to some
execution such that each operation happens instantaneously at
some point between the invocation and response
 You can directly formalize the above intuition, OR
 Formal definition from the textbook: A history H is linearizable if

1. It is equivalent to some legal sequential history S, and
2. The operation partial order induced by H is a subset of the
operation partial order induced by S
 Are the two definitions the same?

Linearizability
 A linearizable history must be sequentially consistent
 Linearizability is arguable the strongest form of consistency
people have ever defined
write(x,1) o1 ok()
P
read(x) ok(1)
Q
o2 read(x) ok(0)
R
o3
Another interesting example

Local Property
 Linearizability is a local property in the sense that H
is linearizable if and only if for any object x, H | x is
linearizable
 Sequential consistency is not a local property

Sequential Consistency is Not Local Property
x, y are initially empty queues
enque(x,1) ok() enque(y,1) ok() deque(x) ok(2)

P
enque(y,2) ok() enque(x,2) ok() deque(y) ok(1)
Q
H | x is sequentially consistent:
enque(x,1) ok() deque(x) ok(2)

P
enque(x,2) ok()
Q
Similarly, H | y is sequentially consistent as well

Sequential Consistency is Not Local Property
x, y have initial value of 0
enque(x,1) ok() enque(y,1) ok() deque(x) ok(2)

P
enque(y,2) ok() enque(x,2) ok() deque(y) ok(1)

Q
But H is not sequentially consistent

Proving That Linearizability is a Local Property
 Intuitive definition: The execution is equivalent to

some execution such that each operation happens
instantaneously at some point between the invocation
and response
 The proof is trivial if we use the above definition (after

formalizing it)

 Definition from the textbook: A history H is linearizable if
1. It is equivalent to some legal sequential history S, and
2. The operation partial order induced by H is a subset of the
operation partial order (in fact, total order) induced by S
 Want to show H is linearizable if for any x, H | x is linearizable --

We construct a directed graph among all operations as
following: A directed edge is created from o1 to o2 if
 o1 and o2 is on the same obj and o1 is before o2 when linearizing
H | x (we say o1o2 due to obj), OR
 o1 < o2 as induced by H (i.e., response of o1 appears before
invocation of o2 in H) (we say o1o2 due to H)

operations on o1
operations on o2
operations on o3
 Lemma: The resulting directed graph does not have a cycle (we
prove this later)
 Any topological sorting of the above graph gives us a legal
serial history S, and the edges in the graph is a subset of the
total order induced by S – Theorem proved

operations on obj1
operations on obj2
operations on obj3
 Lemma: The resulting directed graph does not have a cycle

 Prove by contradiction

operations on obj1
operations on obj2
operations on obj3
 The red cycle are composed of

 Edges due to obj3, followed by
 Edges due to H, followed by
 Edges due to obj2, followed by

operations on obj1
operations on obj2
operations on obj3
 Impossible to have
 Edges due to some object x, followed by
 Edges due to some object y

operations on obj1
operations on obj2
operations on obj3
 To generalize, any cycle must be composed of:

 Edges due to some object x, followed by
 Edges due to some object y, followed by
 ….

edges due to x H | x is equivalent to a serial

history S – S is a total order
edges due to H the partial order induced by

H is transitive
 To generalize, any cycle must be composed of:

 Edges due to some object x  A single edge due to x
 Edges due to H  A single edge due to H
 Edges due to some object y  A single edge due to y
 Edges due to H  A single edge due to H
 ….

A
edge due to H
edge due to x
D
B
edge due to y edge due to H
C
 We must have a cycle in the form of

 A single edge due to x, followed by
 A single edge due to H, followed by
 A single edge due to y, followed by
 A single edge due to H , followed by
 ….

A
edge due to H
edge due to x
D
B
edge due to y edge due to H
C
in H:
D A
From the edge DA:
B
From the edge AB:
C
From the edge BC:
It is impossible to have CD due to y

Consistency Definitions for Registers
 Register is a kind of abstract data type: A single
value that can be read and written
 An (implementation of a) register is called atomic if

the implementation always ensures linearizability of
the history
 An (implementation of a) register is called ?? if the

implementation always ensures sequential
consistency of the history

 An (implementation of a) register is called regular if the
implementation always ensures that a read returns one of the
values written by
 Most recent writes, OR
 Overlapping writes (if any) An atomic register must
be regular
 Definition of most recent writes
write read
the write whose

response time is the
latest
Regular  Sequential Consistency ?
Sequential Consistency  Regular ?
assuming the initial value of the register is 0
write(1)
P Sequentially consistent
read(0) but not regular
Q
write(1)
P Regular but not
read(1) read(0) sequentially consistent
Q

 An (implementation of a) register is called safe if the implementation always ensures that if a read does not
have overlapping writes, then it returns one of the values written by
 Most recent writes
 If there are overlapping writes, a read can return anything
 Safe  Sequential Consistency ?

A regular register must
 Sequential Consistency  Safe ?
be safe

Summary
 What is consistency? Why do we care about it?
 Sequential consistency
 Linearizability
 Linearizability is a local property
 Consistency models for registers

Homework Assignment
 Page 62:
 Problem 4.1
 If you believe a history is linearizable/serializable, give the
equivalent sequential history
 If you believe a history is not linearizable/serializable, prove that no
equivalent sequential history exists
 Slide 26: Prove that linearizability is a local property using the
definition on slide 26 (formalize the definition first)
 Think about Slide 21: Prove that the two definitions of
linearizability are equivalent
 Homework due a week from today
 Read Chapter 7

CS4231 Parallel and Distributed Algorithms

Uploaded by

Copyright:

Available Formats

CS4231 Parallel and Distributed Algorithms

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CS4231 Parallel and Distributed Algorithms

Uploaded by

Copyright:

Available Formats

CS4231

Parallel and Distributed Algorithms

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 2

 What is consistency? Why do we care about it?

 Consistency models for registers

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 3

 We consider shared abstract data type

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 4

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 5

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 6

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 7

 Our goal in the lecture

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 8

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 9

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 10

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 11

 Process p’s process subhistory of H (H | p):

 Object o’s object subhistory of H (H | o):

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 12

 Process order is a partial order among all events

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 13

 All our formalism earlier served to formalize the

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 14

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 15

write(x,1) ok() write(x,1) ok()

write(x,1) ok() write(x,1) ok()

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 16

write(x,1) ok() read(x) ok(0)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 17

write(x,1) ok() write(x,1) ok()

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 18

write(x,1) ok() write(x,1) ok()

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 19

write(x,1) ok() write(x,1) ok()

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 20

 You can directly formalize the above intuition, OR

 Formal definition from the textbook: A history H is linearizable if

 Are the two definitions the same?

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 21

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 22

 Sequential consistency is not a local property

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 23

enque(x,1) ok() enque(y,1) ok() deque(x) ok(2)

enque(x,1) ok() deque(x) ok(2)

Similarly, H | y is sequentially consistent as well

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 24

enque(x,1) ok() enque(y,1) ok() deque(x) ok(2)

enque(y,2) ok() enque(x,2) ok() deque(y) ok(1)

But H is not sequentially consistent

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 25

 Intuitive definition: The execution is equivalent to

 The proof is trivial if we use the above definition (after

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 26

 Want to show H is linearizable if for any x, H | x is linearizable --

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 27

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 28

 Lemma: The resulting directed graph does not have a cycle

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2 29

 The red cycle are composed of