See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220695832

Concurrent Programming in Java: Design Principles and Patterns

Book · January 1997

Source: DBLP

CITATIONS READS
423 15,717

1 author:

Doug Lea
State University of New York at Oswego
105 PUBLICATIONS   2,859 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Doug Lea on 06 September 2014.

The user has requested enhancement of the downloaded file.

 Concurrent
Programming in Java
J a v a
i n
P r o g r a m m i n g

Doug Lea
C o n c u r r e n t

State University of New York at Oswego

dl@cs.oswego.edu

http://gee.cs.oswego.edu

1
 Topics
Concurrency
J a v a

Models, design forces, Java

i n

Designing objects for concurrency

P r o g r a m m i n g

Immutability, locking, state dependence, containment, splitting

Introducing concurrency into applications

Autonomous loops, oneway messages, interactive messages,

cancellation
C o n c u r r e n t

Concurrent application architectures

Flow, parallelism, layering

Libraries

Using, building, and documenting reusable concurrent classes

2
 About These Slides ...
Some slides are based on joint presentations with David Holmes,
Macquarie University, Sydney Australia.
J a v a

More extensive coverage of most topics can be found in the book

i n

Concurrent Programming in Java, Addison-Wesley

P r o g r a m m i n g

and the online supplement

http://gee.cs.oswego.edu/dl/cpj

The printed slides contain much more material than can be covered
in a tutorial. They include extra backgound, examples, and
C o n c u r r e n t

extensions. They are not always in presentation order.

Java code examples often omit qualifiers, imports, etc for space
reasons. Full versions of most examples are available from the
CPJ online supplement.

None of this material should be construed as official Sun

information.

Java is a trademark of Sun Microsystems, Inc.

3
 Concurrency
Why?
J a v a

Availability
Minimize response lag, maximize throughput
i n

Modelling
P r o g r a m m i n g

Simulating autonomous objects, animation

Parallelism
Exploiting multiprocessors, overlapping I/O
Protection
C o n c u r r e n t

Isolating activities in threads

Why Not?
Complexity
Dealing with safety, liveness, composition
Overhead
Higher resource usage
4
 Common Applications
I/O-bound tasks
J a v a

• Concurrently access web pages, databases, sockets ...

GUIs
i n
P r o g r a m m i n g

• Concurrently handle events, screen updates

Hosting foreign code

• Concurrently run applets, JavaBeans, ...

Server Daemons
C o n c u r r e n t

• Concurrently service multiple client requests

Simulations

• Concurrently simulate multiple real objects

Common examples

• Web browsers, web services, database servers,

programming development tools, decision support tools
5
 Concurrent Programming
Concurrency is a conceptual property of software.
J a v a

Concurrent programs might or might not:

i n

Operate across multiple CPUs Share access to resources

symmetric multiprocessor objects, memory, displays,
P r o g r a m m i n g

(SMPs), clusters, special- file descriptors, sockets,

purpose architectures, ... ...
Parallel programming mainly Distributed programming
deals with mapping mainly deals with
software to multiple CPUs concurrent programs that
to improve performance. do NOT share system
C o n c u r r e n t

resources.

Concurrent programming mainly deals with concepts and

techniques that apply even if not parallel or distributed.

• Threads and related constructs run on any Java platform

• This tutorial doesn’t dwell much on issues specific to

parallelism and distribution.
6
 Concurrent Object-Oriented
Programming
Concurrency has always been a part of OOP (since Simula67)
J a v a

• Not a factor in wide-scale embrace of OOP (late 1980s)

i n

• Recent re-emergence, partly due to Java

P r o g r a m m i n g

Concurrent OO programming differs from ...

Sequential OO programming
• Adds focus on safety and liveness
• But uses and extends common design patterns
C o n c u r r e n t

Single-threaded Event-based programming (as in GUIs)

• Adds potential for multiple events occuring at same time
• But uses and extends common messaging strategies
Multithreaded systems programming
• Adds encapsulation, modularity
• But uses and extends efficient implementations
7
 Object Models
Models describe how to think about objects (formally or informally)
J a v a

Common features
• Classes, state, references, methods, identity, constraints
i n

• Encapsulation
P r o g r a m m i n g

— Separation between the insides and outsides of objects

Four basic computational operations
• Accept a message
• Update local state
C o n c u r r e n t

• Send a message
• Create a new object
Models differ in rules for these operations. Two main categories:
• Active vs Passive
• Concurrent models include features of both
• Lead to uniquely concurrent OO design patterns
8
 Active Object Models
J a v a

state, acquaintances

message anAction {
i n

trigger
update state
oneway
P r o g r a m m i n g

send a message
make an object
}

Every object has a single thread of control (like a process) so can

do only one thing at a time.
Most actions are reactive responses to messages from objects
C o n c u r r e n t

• But actions may also be autonomous

• But need not act on message immediately upon receiving it
All messages are oneway. Other protocols can be layered on.
Many extensions and choices of detailed semantics
• Asynchronous vs synchronous messaging, queuing, pre-
emption and internal concurrency, multicast channels, ...
9
 Passive Object Models
In sequential programs, only the single Program object is active
J a v a

• Passive objects serve as the program’s data

i n

State: program counter, object addresses

main
P r o g r a m m i n g

trigger I/O
interpret() {
Program ...
}

a passive object
C o n c u r r e n t

State: instance vars

Methods: byte codes other passive objects

In single-threaded Java, Program is the JVM (interpretor)

• Sequentially simulates the objects comprising the program

• All internal communication based on procedure calls

10
 Concurrent Object Models
Mixtures of active and passive objects
Normally many fewer threads than passive objects
J a v a

Dumber Active Objects Smarter Passive Objects

i n

• Can perform only one • May simultaneously

P r o g r a m m i n g

activity participate in multiple

threads
— in Java, ‘run()’
• Protect themselves
from engaging in
• Share most resources conflicting activities
with other threads
• Communicate with
C o n c u r r e n t

objects participating
• Require scheduling in in other threads
order to coexist
• Initiate and control
new threads

11
 Hardware Mappings
remote messages remote messages
memory cells
J a v a

state of
an object
CPU CPU
i n
P r o g r a m m i n g

Cache Cache
state of an object

Shared memory multiprocessing

• All objects visible in same (virtual) machine
C o n c u r r e n t

• Can use procedural message passing

• Usually many more threads than CPUs
Remote message passing
• Only access objects via Remote references or copying
• Must marshal (serialize) messages
Mixed models including database mediation (‘‘three tier’’)
12
 Vertical Objects
Most OO systems and applications operate at multiple levels
J a v a

Objects at each level manipulate, manage, and coordinate

lower-level ground objects as resources.
Once considered an arcane systems design principle.
i n

But now applies to most applications

P r o g r a m m i n g

Concurrency
• Thread-objects interpret passive objects
Networking and Distribution
• Server-objects pass around resources
C o n c u r r e n t

Persistence and Databases

• Database-objects manage states of ground objects
Component Frameworks
• Design tools build applications from JavaBeans, etc
Layered Applications
• Design patterns based on reflection, interpretation, ...
13
 Design Forces
J a v a

Three main aspects of concurrent OO design

i n

Policies & Protocol Object structures Coding techniques

P r o g r a m m i n g

System-wide Design patterns, Idioms,

design rules microarchitecture neat tricks,
workarounds

Four main kinds of forces that must be addressed at each level

C o n c u r r e n t

Safety — Integrity requirements

Liveness — Progress requirements

Efficiency — Performance requirements

Reusability — Compositional requirements

14
 Systems = Objects + Activities
J a v a
i n
P r o g r a m m i n g

Objects
• ADTs, aggregate components, JavaBeans, monitors,
business objects, remote RMI objects, subsystems, ...
• May be grouped according to structure, role, ...
C o n c u r r e n t

• Usable across multiple activities — focus on SAFETY

Activities
• Messages, call chains, threads, sessions, scenarios,
scripts, workflows, use cases, transactions, data flows,
mobile computations, ...
• May be grouped according to origin, function, ...
• Span multiple objects — focus on LIVENESS
15
 Safe Objects
Perform method actions only when in consistent states
J a v a

method1
legal
temp
i n

method2 transient
P r o g r a m m i n g

states
method3 ??
legal temp

method4
C o n c u r r e n t

Usually impossible to predict consequences of actions attempted

when objects are in temporarily inconsistent states
• Read/write and write/write conflicts
• Invariant failures
• Random-looking externally visible behavior
Must balance with liveness goals
• Clients want simultanous access to services
16
 State Inconsistency Examples

A figure is drawn while it is in the midst of being moved

J a v a

• Could draw at new X-value, old Y-value

i n
P r o g r a m m i n g

• Draws at location that figure never was at

Withdraw from bank account while it is the midst of a transfer

• Could overdraw account

C o n c u r r e n t

• Could lose money

A storage location is read in the midst of being written

• Could result in reading some old bytes and some new

bytes

• Normally, a nonsense value

17
 Live Activities
Every activity should progress toward completion
J a v a

• Every called method should eventually execute

i n
P r o g r a m m i n g

Related to efficiency
• Every called method should execute as soon as possible
C o n c u r r e n t

An activity might not complete if

• An object does not accept a message
• A method blocks waiting for an event, message or
condition that should be, but isn’t produced by another
activity
• Insufficient or unfairly scheduled resources
• Failures and errors of various kinds
18
 Design Dualities
Two extreme approaches:
J a v a

Safety-first Liveness-first
i n

Ensure that each class is Design live ground-level

safe, then try to improve code, then try to layer on
P r o g r a m m i n g

liveness as optimization safety features such as

measure. locking and guarding.
• Characteristic of top- • Characteristic of
down OO Design multithreaded
systems
programming
C o n c u r r e n t

• Can result in slow, • Can result in buggy

deadlock-prone code code full of races

Effective, practical, middle-out approaches combine these.

For example, iteratively improving initial designs to be safe

and live across different contexts
19
 Guaranteeing Safety

“Nothing bad ever happens”

J a v a

Concurrent safety is an extended sense of type safety

i n
P r o g r a m m i n g

• Adds a temporal dimension

• Not completely enforceable by compilers

Low-level view High-level view

• Bits are never • Objects are
C o n c u r r e n t

misinterpreted accessible only when

in consistent states
• Protect against
storage conflicts on • Objects must maintain
memory cells state and
representation
— read/write and invariants
— write/write • Presents subclass
conflicts obligations

20
 Guaranteeing Liveness
“Something eventually happens”
J a v a

Availability
• Avoiding unnecessary blocking
i n

Progress
P r o g r a m m i n g

• Avoiding resource contention among activities

• Avoiding deadlocks and lockouts
• Avoiding unfair scheduling
• Designing for fault tolerance, convergence, stability
C o n c u r r e n t

Citizenship
• Minimizing computational demands of sets of activities
Protection
• Avoiding contention with other programs
• Preventing denial of service attacks
• Preventing stoppage by external agents
21
 Concurrency and Efficiency
Concurrency can be expensive
J a v a

• Performance profiles may vary across platforms

Resources
i n

• Threads, Locks, Monitors

P r o g r a m m i n g

Computation
• Construction, finalization overhead for resources
• Synchronization, context switching, scheduling overhead
Communication
C o n c u r r e n t

• Interaction overhead for threads mapped to different CPUs

• Caching and locality effects
Algorithmic efficiency
• Cannot use some fast but unsafe sequential algorithms
Paying for tunability and extensibility
• Reduces opportunities to optimize for special cases
22
 Concurrency and Reusability
Added Complexity
J a v a

• More stringent correctness criteria than sequential code

i n

— Usually not automatically statically checkable

P r o g r a m m i n g

• Nondeterminism impedes debuggability, understandability

Added Context Dependence (coupling)

• Components only safe/live when used in intended contexts

C o n c u r r e n t

— Need for documentation

• Can be difficult to extend via subclassing

— “Inheritance anomalies”

• Can be difficult to compose

— Clashes among concurrency control techniques

23
 Reuse and Design Policies
Think locally. Act globally.
J a v a

Example design policy domains

State-dependence Service availability Flow constraints

i n

What to do if Constraints on Establishing

P r o g r a m m i n g

a request concurrent message

logically access to directionality
cannot be methods and layering
performed rules

Combat complexity
C o n c u r r e n t

• High-level design rules and architectural constraints avoid

inconsistent case-by-case decisions
• Policy choices are rarely ‘‘optimal’’, but often religiously
believed in anyway.
Maintain openness
• Accommodate any component that obeys a given policy
• Fail but don’t break if they do not obey policy
24
 Three Approaches to Reusability
Patterns Reusing design knowledge
J a v a

• Record best practices, refine them to essences

• Analyze for safety, liveness, efficiency, extensibility, etc

i n
P r o g r a m m i n g

• Provide recipes for construction

Frameworks Reusing policies and protocols

• Create interfaces and classes that establish policy choices

for a suite of applications

• Provide utilities and support classes

C o n c u r r e n t

• Mainly use by creating application-dependent (sub)classes

Libraries Reusing code

• Create interfaces that apply in many contexts

• Provide high-quality implementations

• Allow others to create alternative implementations

25
 Java Overview
Core Java is a relatively small, boring object-oriented language
J a v a

Main differences from Smalltalk:

i n

• Static typing
P r o g r a m m i n g

• Support for primitive data types (int, float, etc)

• C-based syntax

Main differences from C++:

• Run-time safety via Virtual Machine

C o n c u r r e n t

— No insecure low-level operations

— Garbage collection

• Entirely class-based: No globals

• Relative simplicity: No multiple inheritance, etc

• Object-based implementations of Array, String, Class, etc

• Large predefined class library: AWT, Applets, net, etc

26
 Java Features
Java solves some software development problems
J a v a

Packaging: Objects, classes, components, packages

Portability: Bytecodes, unicode, transports

i n
P r o g r a m m i n g

Extensibility: Subclassing, interfaces, class loaders

Safety: Virtual machine, GC, verifiers

Libraries: java.* packages

Ubiquity: Run almost anywhere

C o n c u r r e n t

But new challenges stem from new aspects of programming:

Concurrency: Threads, locks, ...

Distribution: RMI, CORBA, ...

Persistence: Serialization, JDBC, ...

Security: Security managers, Domains, ...

27
 Basic Java Constructs
Classes Descriptions of object features
J a v a

Instance variables Fields representing object state

Methods Encapsulated procedures
i n

Statics Per-class variables and methods

P r o g r a m m i n g

Constructors Operations performed upon object creation

Interfaces Sets of methods implemented by any class
Subclasses Single inheritance from class Object
Inner classes Classes within other classes and methods
C o n c u r r e n t

Packages Namespaces for organizing sets of classes

Visibility control private, public, protected, per-package
Qualifiers Semantic control: final, abstract, etc
Statements Nearly the same as in C/C++
Exceptions Throw/catch control upon failure
Primitive types byte, short, int, long, float, char, boolean
28
 Particle Applet
import java.awt.*;
import java.applet.*;
J a v a

public class ParticleApplet extends Applet {

public void init() {
add(new ParticleCanvas(10));
i n

}
}
P r o g r a m m i n g

class ParticleCanvas extends Canvas {

Particle[] particles;
ParticleCanvas(int nparticles) {
setSize(new Dimension(100, 100));
particles = new Particle[nparticles];
for (int i = 0; i < particles.length; ++i) {
C o n c u r r e n t

particles[i] = new Particle(this);

new Thread(particles[i]).start();
}
}

public void paint(Graphics g) {

for (int i = 0; i < particles.length; ++i)
particles[i].draw(g);
}
} // (needs lots of embellishing to look nice)
29
 Particle Class
public class Particle implements Runnable {
private int x = 0, y = 0;
J a v a

private Canvas canvas;

public Particle(Canvas host) { canvas = host; }
i n

synchronized void moveRandomly() {

x += (int) (((Math.random() - 0.5) * 5);
P r o g r a m m i n g

y += (int) (((Math.random() - 0.5) * 5);

}

public void draw(Graphics g) {

int lx, ly;
synchronized (this) { lx = x; ly = y; }
g.drawRect(lx, ly, 10, 10);
C o n c u r r e n t

}
public void run() {
for(;;) {
moveRandomly();
canvas.repaint();
try { Thread.sleep((int)(Math.random()*10);}
catch (InterruptedException e) { return; }
}
}
}
30
 Java Concurrency Support
Thread class represents state of an independent activity
J a v a

• Methods to start, sleep, etc

• Very weak guarantees about control and scheduling
i n

• Each Thread is a member of a ThreadGroup that is used

P r o g r a m m i n g

for access control and bookkeeping

• Code executed in threads defined in classes implementing:
interface Runnable { public void run(); }
synchronized methods and blocks control atomicity via locks
• Java automates local read/write atomicity of storage and
C o n c u r r e n t

access of values of type byte, char, short, int, float,

and Object references, but not double and long
• synchronized statement also ensures cache flush/reload
• volatile keyword controls per-variable flush/reload
Monitor methods in class Object control suspension and
resumption:
• wait(), wait(ms), notify(), notifyAll()
31
 Class Thread
Constructors
J a v a

Thread(Runnable r) constructs so run() calls r.run()

— Other versions allow names, ThreadGroup placement
i n

Principal methods
P r o g r a m m i n g

start() activates run() then returns to caller

isAlive() returns true if started but not stopped
join() waits for termination (optional timeout)
interrupt() breaks out of wait, sleep, or join
isInterrupted() returns interruption state
C o n c u r r e n t

getPriority() returns current scheduling priority

setPriority(int priorityFromONEtoTEN) sets it
Static methods that can only be applied to current thread
currentThread() reveals current thread
sleep(ms) suspends for (at least) ms milliseconds
interrupted() returns and clears interruption status
32
 Designing Objects for
Concurrency
J a v a

Patterns for safely representing and managing state

i n

Immutability
P r o g r a m m i n g

• Avoiding interference by avoiding change

Locking

• Guaranteeing exclusive access

State dependence
C o n c u r r e n t

• What to do when you can’t do anything

Containment

• Hiding internal objects

Splitting

• Separating independent aspects of objects and locks

33
 Immutability
Synopsis
J a v a

• Avoid interference by avoiding change

i n

• Immutable objects never change state

P r o g r a m m i n g

• Actions on immutable objects are always safe and live

Applications

• Objects representing values

C o n c u r r e n t

— Closed Abstract Data Types

— Objects maintaining state representations for others

— Whenever object identity does not matter

• Objects providing stateless services

• Pure functional programming style

34
 Stateless Service Objects
class StatelessAdder {
int addOne(int i) { return i + 1; }
J a v a

int addTwo(int i) { return i + 2; }

}
i n

There are no special concurrency concerns:

P r o g r a m m i n g

• There is no per-instance state

➔ No storage conflicts

• No representational invariants

➔ No invariant failures
C o n c u r r e n t

• Any number of instances of addOne and/or addTwo can

safely execute at the same time. There is no need to
preclude this.

➔ No liveness problems

• The methods do not interact with any other objects.

➔ No concurrent protocol design

35
 Freezing State upon Construction
class ImmutableAdder {
private final int offset_; // blank final
J a v a

ImmutableAdder(int x) { offset_ = x; }
i n

int add(int i) { return i + offset_; }

P r o g r a m m i n g

Still no safety or liveness concerns

Java (blank) finals enforce most senses of immutablity

• Don’t cover cases where objects eventually latch into

values that they never change from
C o n c u r r e n t

Immutability is often used for closed Abstract Data Types in Java

• java.lang.String

• java.lang.Integer

• java.awt.Color

• But not java.awt.Point or other AWT graphical

representation classes (A design error?)
36
 Applications of Immutability
Immutable references to mutable objects delegate
relay server
J a v a

class Relay {
private final Server delegate;
Relay(Server s) { delegate = s; }
i n

void serve() { delegate.serve(); }

}
P r o g r a m m i n g

Partial immutability next next next

element element element

Methods dealing with immutable aspects of state do not

require locking
C o n c u r r e n t

class FixedList { // cells with fixed successors

private final FixedList next; // immutable
FixedList(FixedList nxt) { next = nxt; }
FixedList successor() { return next; }

private Object elem = null; // mutable

synchronized Object get() { return elem; }
synchronized void set(Object x) { elem = x; }
}

37
 Locking
client
J a v a

internal state
lock
action { ... }
i n

host
client
P r o g r a m m i n g

Locking is a simple message accept mechanism

• Acquire object lock on entry to method, release on return
Precludes storage conflicts and invariant failures
C o n c u r r e n t

• Can be used to guarantee atomicity of methods

Introduces potential liveness failures
• Deadlock, lockouts
Applications
• Fully synchronized (atomic) objects
• Most other reusable objects with mutable state
38
 Synchronized Method Example
J a v a

class Location {

private double x_, y_;

i n
P r o g r a m m i n g

Location(double x, double y) { x_ = x; y_ = y; }

synchronized double x() { return x_; }

double y() {
synchronized (this) {
return y_;
C o n c u r r e n t

}
}

synchronized void moveBy(double dx, double dy) {

x_ += dx;
y_ += dy;
}
}
39
 Java Locks
Every Java Object possesses one lock
J a v a

• Manipulated only via synchronized keyword

• Class objects contain a lock used to protect statics

i n

• Scalars like int are not Objects so can only be locked via
P r o g r a m m i n g

their enclosing objects

Synchronized can be either method or block qualifier

synchronized void f() { body; } is equivalent to:

void f() { synchronized(this) { body; } }

C o n c u r r e n t

Java locks are reentrant

• A thread hitting synchronized passes if the lock is free or

it already possesses the lock, else waits

• Released after passing as many }’s as {’s for the lock

— cannot forget to release lock

Synchronized also has the side-effect of clearing locally cached

values and forcing reloads from main storage
40
 Storage Conflicts
class Even {
int n = 0;
J a v a

public int next(){ // POST?: next is always even

++n;
++n;
i n

return n;
P r o g r a m m i n g

}
}

Postcondition may fail due to storage conflicts. For example, one

possible execution trace when n starts off at 0 is:
Thread 1 Thread 2
read 0
write 1
C o n c u r r e n t

read 1
write 2
read 2 read 2
write 3
return 3
write 3
return 3

Declaring next method as synchronized precludes conflicting

traces, as long as all other methods accessing n are also
synchronized
41
 Locks and Caching
Locking generates messages between threads and memory
J a v a

Lock acquisition forces reads from memory to thread cache

Lock release forces writes of cached updates to memory

i n
P r o g r a m m i n g

memory cells
CPU CPU

Cache Cache

unlock lock
state of object
C o n c u r r e n t

Without locking, there are NO promises about if and when caches

will be flushed or reloaded

➔ Can lead to unsafe execution

➔ Can lead to nonsensical execution

42
 Memory Anomalies
Should acquire lock before use of any field of any object, and
release after update
J a v a

If not, the following are possible:

i n

• Seeing stale values that do not reflect recent updates

P r o g r a m m i n g

• Seeing inconsistent states due to out-of-order writes

during flushes from thread caches

• Seeing incompletely initialized new objects

Can declare volatile fields to force per-variable load/flush.

C o n c u r r e n t

• Has very limited utility.

• volatile never usefully applies to reference variables

— The referenced object is not necessarily loaded/

flushed, just the reference itself.

— Instead, should use synchronization-based

constructions
43
 Fully Synchronized Objects
Objects of classes in which all methods are synchronized
• Always safe, but not always live or efficient
J a v a

Only process one request at a time client host

aMessage
i n

• All methods are locally sequential Ready

P r o g r a m m i n g

Accept new messages only when ready

➔ No other thread holds lock ... actions...
➔ Not engaged in another activity
return
• But methods may make self-calls to
other methods during same activity
without blocking (due to reentrancy)
C o n c u r r e n t

Constraints
• All methods must be synchronized: Java unsynchronized
methods execute even when lock held.
• No public variables or other encapsulation violations
• Methods must not suspend or infinitely loop
• Re-establish consistent state after exceptions

44
 Deadlock
class Cell {
private long value_;
J a v a

synchronized long getValue() { return value_;}

synchronized void setValue(long v) {value_ = v;}
i n
P r o g r a m m i n g

synchronized void swapValue(Cell other) {

long t = getValue();
long v = other.getValue();
setValue(v);
other.setValue(t);
}
}
C o n c u r r e n t

SwapValue is a transactional method. Can deadlock in trace:

thread1 thread2
enter cell1.swapValue
t = getValue()
enter cell2.swapValue
t = getValue()
v = other.getValue()
v = other.getValue()

45
 Lock Precedence
Can prevent deadlock in transactional methods via resource-
J a v a

ordering based on Java hash codes (among other solutions)

class Cell {
i n

long value;
P r o g r a m m i n g

void swapValue(Cell other) {

if (other == this) return; // alias check

Cell fst = this; // order via hash codes

Cell snd = other;
if (fst.hashCode() > snd.hashCode()) {
fst = other; snd = this;
C o n c u r r e n t

}
synchronized(fst) {
synchronized (snd) {
long t = fst.value;
fst.value = snd.value;
snd.value = t;
}
}
}
}
46
 Holding Locks
class Server {
double state;
J a v a

Helper helper;
public synchronized void svc() {
state = illegalValue;
i n

helper.operation();
P r o g r a m m i n g

state = legalValue;
}
}
Potential problems with holding locks during downstream calls
Safety: What if helper.operation throws exceptions?
Liveness: What if helper.operation causes deadlock?
C o n c u r r e n t

Availability: Cannot accept new svc requests during helper op

Rule of Thumb (with many variants and exceptions):
Always lock when updating state
Never lock when sending message
Redesign methods to avoid holding locks during downstream calls,
while still preserving safety and consistency
47
 Synchronization of Accessor Methods
class Queue {
private int sz_ = 0; // number of elements
J a v a

public synchronized void put(Object x) {

// ... increment sz_ ...
i n

}
public synchronized Object take() {
P r o g r a m m i n g

// ... decrement sz_ ...

}
public int size() { return sz_; } // synch?
}
Should size() method be synchronized?
Pro:
C o n c u r r e n t

• Prevents clients from obtaining stale cached values

• Ensures that transient values are never returned
— For example, if put temporarily set sz_ = -1 as flag
Con:
• What could a client ever do with this value anyway?
Sync always needed for accessors of mutable reference variables
48
 Locking and Singletons
Every Java Class object has a lock. Both static and instance
J a v a

methods of Singleton classes should use it.

public class Singleton { // lazy initialization

i n

private int a;
private Singleton(){ a = 1;}
P r o g r a m m i n g

private static Class lock = Singleton.class;

private static Singleton ref = null;

public static Singleton instance(){

synchronized(lock) {
if (ref == null) ref = new Singleton();
C o n c u r r e n t

return ref;
}
}
public int getA() {
synchronized(lock) { return a; }
}
public void setA(int v){
synchronized(lock) { a = v; }
}
}
49
 State Dependence
Two aspects of action control:
J a v a

• A message from a client

• The internal state of the host
i n

Design Steps:
P r o g r a m m i n g

• Choose policies for dealing with actions that can succeed

only if object is in particular logical state
• Design interfaces and protocols to reflect policy
• Ensure objects able to assess state to implement policy
C o n c u r r e n t

There is not a
state, acquaintances
separate accept
mechanism in
message anAction { ... } Java. So must
accept
implement policies
policy control in action methods
themselves.

50
 Examples of State-Dependent Actions

Operations on collections, streams, databases

J a v a

• Remove an element from an empty queue

i n

Operations on objects maintaining constrained values

P r o g r a m m i n g

• Withdraw money from an empty bank account

Operations requiring resources

• Print a file
C o n c u r r e n t

Operations requiring particular message orderings

• Read an unopened file

Operations on external controllers

• Shift to reverse gear in a moving car

51
 Policies for State Dependent Actions
Some policy choices for dealing with pre- and post- conditions
J a v a

Blind action Proceed anyway; no guarantee of outcome

i n

Inaction Ignore request if not in right state

P r o g r a m m i n g

Balking Fail (throw exception) if not in right state

Guarding Suspend until in right state

Trying Proceed, check if succeeded; if not, roll back

C o n c u r r e n t

Retrying Keep trying until success

Timing out Wait or retry for a while; then fail

Planning First initiate activity that will achieve right state

52
 Interfaces and Policies
Boring running example
J a v a

interface BoundedCounter {
static final long MIN = 0;
static final long MAX = 10;
i n
P r o g r a m m i n g

long value(); // INV: MIN <= value() <= MAX

// INIT: value() == MIN

void inc(); // PRE: value() < MAX

void dec(); // PRE: value() > MIN
}

Interfaces alone cannot convey policy

C o n c u r r e n t

• But can suggest policy

— For example, should inc throw exception? What kind?

— Different methods can support different policies

• But can use manual annotations

— Declarative constraints form basis for implementation

53
 Balking
Check state upon method entry client host
aMessage
• Must not change state in course
J a v a

of checking it !inRightState

• Relevant state must be explicitly throw

i n

represented, so can be checked

P r o g r a m m i n g

upon entry inRightState

Exit immediately if not in right state

• Throw exception or return special ... actions...
error value
return
• Client is responsible for handling
failure
C o n c u r r e n t

The simplest policy for fully synchronized objects

• Usable in both sequential and concurrent contexts
— Often used in Collection classes (Vector, etc)
• In concurrent contexts, the host must always take
responsibility for entire check-act/check-fail sequence
— Clients cannot preclude state changes between check
and act, so host must control
54
 Balking Counter Example
class Failure extends Exception { }
J a v a

class BalkingCounter {
protected long count_ = MIN;
synchronized long value() { return count_;}
i n

synchronized void inc() throws Failure {

P r o g r a m m i n g

if (count_ >= MAX) throw new Failure();

++count_;
}

synchronized void dec() throws Failure {

if (count_ <= MIN) throw new Failure();
--count_;
C o n c u r r e n t

}
}
// ...
void suspiciousUsage(BalkingCounter c) {
if (c.value() > BalkingCounter.MIN)
try { c.dec(); } catch (Failure ignore) {}
}
void betterUsage(BalkingCounter c) {
try { c.dec(); } catch (Failure ex) {cope();}
}
55
 Collection Class Example
class Vec { // scaled down version of Vector
protected Object[] data_; protected int size_=0;
J a v a

public Vec(int cap) { data_=new Object[cap]; }

i n

public int size() { return size_; }

P r o g r a m m i n g

public synchronized Object at(int i)

throws NoSuchElementException {
if (i < 0 || i >= size_ )
throw new NoSuchElementException();
return data_[i];
}
public synchronized void append(Object x) {
C o n c u r r e n t

if (size_ >= data_.length) resize();

data_[size_++] = x;
}
public synchronized void removeLast()
throws NoSuchElementException {
if (size_ == 0)
throw new NoSuchElementException();
data_[--size_] = null;
}
}
56
 Policies for Collection Traversal
How to apply operation to collection elements without interference
J a v a

Balking iterators
• Throw exception on access if collection was changed.
Implement via version numbers updated on each change
i n

— Used in JDK1.2 collections

P r o g r a m m i n g

• But can be hard to recover from exceptions

Snapshot iterators
• Make immutable copy of base collection elements. Or
conversely, copy-on-write during each update.
• But can be expensive
C o n c u r r e n t

Indexed traversal
• Clients externally synchronize when necessary
• But coupled to particular locking policies
Synchronized aggregate methods
• Support apply-to-all methods in collection class
• But deadlock-prone
57
 Synchronized Traversal Examples

interface Procedure { void apply(Object obj); }

J a v a

class XVec extends Vec {

synchronized void applyToAll(Procedure p) {
i n

for (int i=0;i<size_;++i) p.apply(data_[i]);

}
P r o g r a m m i n g

class App {
void printAllV1(XVec v) { // aggregate synch
v.applyToAll(new Procedure() {
public void apply(Object x) {
System.out.println(x);
C o n c u r r e n t

}});
}

void printAllV2(XVec v) { // client-side synch

synchronized (v) {
for (int i = 0; i < v.size(); ++i)
System.out.println(v.at(i));
}
}
}
58
 Guarding
Generalization of locking for state-dependent actions
• Locked: Wait until ready (not engaged in other methods)
J a v a

• Guarded: Wait until an arbitrary state predicate holds

i n

Check state upon entry

P r o g r a m m i n g

• If not in right state, wait

• Some other action in some client host

other thread may eventually aMessage
cause a state change that inRightState
enables resumption
C o n c u r r e n t

... actions...
Introduces liveness concerns
return
• Relies on actions of other
threads to make progress
• Useless in sequential
programs

59
 Guarding via Busy Waits
class UnsafeSpinningBoundedCounter { // don’t use
protected volatile long count_ = MIN;
J a v a

long value() { return count_; }

void inc() {
i n

while (count_ >= MAX); // spin

++count_;
P r o g r a m m i n g

}
void dec() {
while (count_ <= MIN); // spin
--count_;
}
}
Unsafe — no protection from read/write conflicts
C o n c u r r e n t

Wasteful — consumes CPU time

But busy waiting can sometimes be useful; generally when
• The conditions latch
— once set true, they never become false
• You are sure that threads are running on multiple CPUs
— Java doesn’t provide a way to determine or control this
60
 Guarding via Suspension
class GuardedBoundedCounter {
protected long count_ = MIN;
J a v a

synchronized long value() { return count_; }

i n

synchronized void inc()

throws InterruptedException {
P r o g r a m m i n g

while (count_ >= MAX) wait();

++count_;
notifyAll();
}

synchronized void dec()

throws InterruptedException {
C o n c u r r e n t

while (count_ <= MIN) wait();

--count_;
notifyAll();
}
}
Each wait relies on a balancing notification
• Generates programmer obligations
Must recheck condition upon resumption
61
 Java Monitor Methods
Every Java Object has a wait set
J a v a

• Accessed only via monitor methods, that can only be

invoked under synchronization of target
wait()
i n

• Suspends thread
P r o g r a m m i n g

• Thread is placed in wait set for target object

• Synch lock for target is released
notify()
• If one exists, any thread T is chosen from target’s wait set
C o n c u r r e n t

• T must re-acquire synch lock for target

• T resumes at wait point
notifyAll() is same as notify() except all threads chosen
wait(ms)is same as wait() except thread is automatically notified
after ms milliseconds if not already notified
Thread.interrupt causes a wait (also sleep, join) to abort.
Same as notify except thread resumed at the associated catch
62
 Monitors and Wait Sets
class X {
J a v a

synchronized void w() {

before(); wait(); after();
}
i n

synchronized void n() { notifyAll(); }

}
P r o g r a m m i n g

One possible trace for three threads accessing instance x:

T1 T2 T3
enter x.w()
before(); enter x.w()
wait();
release lock before();
C o n c u r r e n t

wait(); enter x.n()

release lock notifyAll();
after();
after();
acquire lock
acquire lock

x
T1 T2
waitset

63
 Interactions with Interruption
Effect of Thread.interrupt():
J a v a

• If thread not waiting, set the isInterrupted() bit

i n

• If thread is waiting, force to exit wait and throw

P r o g r a m m i n g

InterruptedException upon resumption

notify, notifyAll, timeout, interrupt

Acquiring exitAcquire
Running Waiting
Lock
enterAcquire wait
C o n c u r r e n t

Interrupt Interrupt Thread.interrupted, wait

exitAcquire
Acquiring Running
Lock + +
Interrupted interrupted
interrupt enterAcquire interrupt

64
 Fairness in Java
Fairness is a system-wide progress property:
J a v a

Each blocked activity will eventually continue when its

enabling condition holds. (Many variants of definition)
➔Threads waiting for lock eventually enter when lock free
i n
P r o g r a m m i n g

➔Guarded wait loops eventually unblock when condition true

Usually implemented via First-in-First-Out scheduling policies
• FIFO lock and wait queues
• Sometimes, along with preemptive time-slicing
Java does not guarantee fairness
C o n c u r r e n t

• Potential starvation
— A thread never gets a chance to continue because other
threads are continually placed before it in queue
• FIFO usually not strictly implementable on SMPs
• But JVM implementations usually approximate fairness
• Manual techniques available to improve fairness properties
65
 Timeouts
Intermediate points between balking and guarding
J a v a

• Can vary timeout parameter from zero to infinity

Useful for heuristic detection of failures

i n
P r o g r a m m i n g

• Deadlocks, crashes, I/O problems, network disconnects

But cannot be used for high-precision timing or deadlines

• Time can elapse between wait and thread resumption

Java implementation constraints

C o n c u r r e n t

• wait(ms)does not automatically tell you if it returns

because of notification vs timeout

• Must check for both. Order and style of checking can

matter, depending on

— If always OK to proceed when condition holds

— If timeouts signify errors

66
 Timeout Example
class TimeOutBoundedCounter {
protected long TIMEOUT = 5000;
J a v a

// ...
synchronized void inc() throws Failure {
i n

if (count_ >= MAX) {

long start = System.currentTimeMillis();
P r o g r a m m i n g

long waitTime = TIMEOUT;

for (;;) {
if (waitTime <= 0) throw new Failure();
try { wait(waitTime); }
catch (InterruptedException e) {
throw new Failure();
}
C o n c u r r e n t

if (count_ < MAX) break;

long now = System.currentTimeMillis();
waitTime = TIMEOUT - (now - start);
}
}
++count_;
notifyAll();
}
synchronized void dec() throws Failure;//similar
}
67
 Buffer Supporting Multiple Policies
class BoundedBuffer {
Object[] data_;
J a v a

int putPtr_ = 0, takePtr_ = 0, size_ = 0;

BoundedBuffer(int capacity) {
data_ = new Object[capacity];
i n

}
P r o g r a m m i n g

protected void doPut(Object x){ // mechanics

data_[putPtr_] = x;
putPtr_ = (putPtr_ + 1) % data_.length;
++size_;
notifyAll();
}
C o n c u r r e n t

protected Object doTake() { // mechanics

Object x = data_[takePtr_];
data_[takePtr_] = null;
takePtr_ = (takePtr_ + 1) % data_.length;
--size_;
notifyAll();
return x;
}
boolean isFull(){ return size_ == data_.length;}
boolean isEmpty(){ return size_ == 0; }
68
 Buffer (continued)
J a v a

synchronized void put(Object x)

throws InterruptedException {
while (isFull()) wait();
i n

doPut(x);
}
P r o g r a m m i n g

synchronized Object take() {

throws InterruptedException {
while (isEmpty()) wait();
return doTake();
}
C o n c u r r e n t

synchronized boolean offer(Object x) {

if (isFull()) return false;
doPut(x);
return true;
}

synchronized Object poll() {

if (isEmpty()) return null;
return doTake();
}
69
 Buffer (continued)
J a v a

synchronized boolean offer(Object x, long ms) {

if (isFull()) {
if (ms <= 0) return false;
i n

long start = System.currentTimeMillis();

long waitTime = ms;
P r o g r a m m i n g

for (;;) {
try { wait(waitTime); }
catch (InterruptedException e) {
return false;
}
if (!isFull()) break;
long now = System.currentTimeMillis();
C o n c u r r e n t

waitTime = ms - (now - start);

if (waitTime <= 0) return false;
}
}
return doTake();
}

synchronized Object poll(long ms); // similar

}
70
 Containment
client
J a v a

part1 subpart1
lock
i n

outer part2
client
P r o g r a m m i n g

Structurally guarantee exclusive access to internal objects

• Control their visibility

• Provide concurrency control for their methods

C o n c u r r e n t

Applications

• Wrapping unsafe sequential code

• Eliminating need for locking ground objects and variables

• Applying special synchronization policies

• Applying different policies to the same mechanisms

71
 Containment Example
class Pixel {
private final java.awt.Point pt_;
J a v a

Pixel(int x, int y) { pt_ = new Point(x, y); }

i n

synchronized Point location() {

P r o g r a m m i n g

return new Point(pt_.x, pt_.y);

}

synchronized void moveBy(int dx, int dy){

pt_.x += dx; pt_.y += dy;
}
}
C o n c u r r e n t

Pixel provides synchronized access to Point methods

• The reference to Point object is immutable, but its fields

are in turn mutable (and public!) so is unsafe without
protection

Must make copies of inner objects when revealing state

• This is the most common way to use java.awt.Point,

java.awt.Rectangle, etc
72
 Implementing Containment
Strict containment creates islands of isolated objects
J a v a

• Applies recursively
i n

• Allows inner code to run faster

P r o g r a m m i n g

Inner code must be communication-closed

• No unprotected calls in to or out from island

Outer objects must never leak identities of inner objects

C o n c u r r e n t

• Can be difficult to enforce and check

Outermost objects must synchronize access

• Otherwise, possible thread-caching problems

Seen in concurrent versions of many delegation-based patterns

• Adapters, decorators, proxies

73
 Hierarchical Locking
Applies when logically contained parts are not hidden from clients
J a v a

client
part1 subpart1
owner
i n
P r o g r a m m i n g

client part2

Avoids deadlocks that could occur if parts fully synchronized

m() part1 holds self lock

part1 part2 needs part2 lock
C o n c u r r e n t

m() part2 holds self lock

part1 part2
needs part1 lock

Can eliminate this potential deadlock if all locking in all

methods in all Parts relies on the common owner’s lock.
Extreme case: one Giant Lock for entire subsystem
Can use either internal or external conventions
74
 Internal Hierarchical Locking
Visible components protect themselves using their owners’ locks:
J a v a

class Part {
protected Container owner_; // never null
i n

public Container owner() { return owner_; }

P r o g r a m m i n g

void bareAction() { /* ... unsafe ... */ }

public void m() {

synchronized(owner()) { bareAction(); }
}
}
C o n c u r r e n t

Or implement using inner classes — Owner is outer class:

class Container {
class Part {
public void m() {
synchronized(Container.this){ bareAction();}
} } }

Can extend to frameworks based on shared Lock objects,

transaction locks, etc rather than synchronized blocks
75
 External Hierarchical Locking
Rely on callers to provide the locking
J a v a

class Client {

void f(Part p) {
i n

synchronized (p.owner()) { p.bareAction(); }

P r o g r a m m i n g

}
}

Used in AWT

• java.awt.Component.getTreeLock()

Can sometimes avoid more locking overhead, at price of fragility

C o n c u r r e n t

• Can manually minimize use of synchronized

• Requires that all callers obey conventions

• Effectiveness is context dependent

— Breaks encapsulation

— Doesn’t work with fancier schemes that do not directly

rely on synchronized blocks or methods for locking
76
 Containment and Monitor Methods
class Part {
protected boolean cond_ = false;
J a v a

synchronized void await() {

while (!cond_)
i n

try { wait(); }
P r o g r a m m i n g

catch(InterruptedException ex) {}
}

synchronized void signal(boolean c) {

cond_ = c; notifyAll();
}
}
C o n c u r r e n t

class Whole {
final Part part_ = new Part();

synchronized void rely() { part_.await(); }

synchronized void set(boolean c){

part_.signal(c); }
}

What happens when Whole.rely() called?

77
 Nested Monitors
J a v a

whole
part
wait set: T ...
i n

holds lock to
P r o g r a m m i n g

If thread T calls whole.rely

• It waits within part

C o n c u r r e n t

• The lock to whole is retained while T is suspended

• No other thread will ever unblock it via whole.set

➔ Nested Monitor Lockout

Policy clash between guarding by Part and containment by Whole

Never wait on a hidden contained object in Java while holding lock

78
 Avoiding Nested Monitors
Adapt internal hierarchical locking pattern
J a v a

Can use inner classes, where Part waits in Whole’s monitor

i n

class Whole { // ...

P r o g r a m m i n g

class Part { // ...

public void await() {
synchronized (Whole.this) {
while (...) Whole.this.wait() // ...
} } }

Create special Condition objects

C o n c u r r e n t

• Condition methods are never invoked while holding locks

• Some concurrent languages build in special support for

Condition objects

— But generally only deal with one-level nesting

• Can build Condition class library in Java

79
 Splitting Objects and Locks
Synopsis
J a v a

• Isolate independent aspects of state and/or behavior of a

host object into helper objects
i n

• The host object delegates to helpers

P r o g r a m m i n g

• The host may change which helpers it uses dynamically

Applications
• Atomic state updates
— Conservative and optimistic techniques
C o n c u r r e n t

• Avoiding deadlocks
— Offloading locks used for status indicators, etc
• Improving concurrency
— Reducing lock contention for host object
• Reducing granularity
— Enabling fine-grained concurrency control
80
 Isolating Dependent Representations
Does Location provide strong enough semantic guarantees?
J a v a

class Location { // repeated

private double x_, y_;
synchronized double x() { return x_; }
i n

synchronized double y() { return y_; }

synchronized void moveBy(double dx, double dy) {
P r o g r a m m i n g

x_ += dx; y_ += dy;
}
}
No protection from interleaving problems such as:
Thread 1: x=loc.x(); ...............; y=loc.y();
C o n c u r r e n t

Thread 2: .........; loc.moveBy(1,6);.........;

Thread 1 can have incorrect view (old x, new y)
Avoid by splitting out dependent representations in separate class
Location xy XY
x()
XY xy() y()
moveBy(dx, dy)

81
 Conservative Representation Updates
class XY { // immutable
private final double x_, y_;
J a v a

XY(double x, double y) { x_ = x; y_ = y; }
double x() { return x_; }
double y() { return y_; }
i n

}
P r o g r a m m i n g

class LocationV2 {
private XY xy_;

LocationV2(double x, double y) {
xy_ = new XY(x, y);
}
synchronized XY xy() { return xy_; }
C o n c u r r e n t

synchronized void moveBy(double dx,double dy) {

xy_ = new XY(xy_.x() + dx, xy_.y() + dy);
}
}

Locking moveBy() ensures that the two accesses of xy_ do not

get different points

Locking xy() avoids thread-cache problems by clients

82
 Optimistic Representation Updates

class LocationV3 {
J a v a

private XY xy_;

private synchronized boolean commit(XY oldp,

i n

XY newp){
boolean success = (xy_ == oldp);
P r o g r a m m i n g

if (success) xy_ = newp;

return success;
}

LocationV3(double x,double y){xy_=new XY(x,y);}

synchronized XY xy() { return xy_; }

C o n c u r r e n t

void moveBy(double dx,double dy) {

while (!Thread.interrupted()){
XY oldp = xy();
XY newp = new XY(oldp.x()+dx, oldp.y()+dy);
if (commit(oldp, newp)) break;
Thread.yield();
}
}
}
83
 Optimistic Update Techniques
Every public state update method has four parts:
J a v a

➔ Record current version

Easiest to use reference to immutable representation
i n

— Or can assign version numbers, transaction IDs, or time

stamps to mutable representations
P r o g r a m m i n g

➔ Build new version, without any irreversible side effects

All actions before commit must be reversable
— Ensures that failures are clean (no side effects)
— No I/O or thread construction unless safely cancellable
C o n c u r r e n t

— All internally called methods must also be reversable

➔ Commit to new version if no other thread changed version
Isolation of state updates to single atomic commit method
can avoid potential deadlocks
➔ Otherwise fail or retry
Retries can livelock unless proven wait-free in given
context
84
 Optimistic State-Dependent Policies
As with optimistic updates, isolate state client host
into versions, and isolate state aMessage
J a v a

changes to commit method

... actions...
i n

In each method:
P r o g r a m m i n g

• Record current version return

inRightState

• Build new version

!inRightState
• Commit to version if success and
no one changed version
... undo...
• Otherwise fail or retry
throw
C o n c u r r e n t

Retry policy is a tamed busy wait. Can be

more efficient than guarded waits if
• Conflicts are rare
• Guard conditions usually hold
• Running on multiple CPUs

85
 Optimistic Counter
class OptimisticBoundedCounter {
private Long count_ = new Long(MIN);
J a v a

long value() { return count().longValue(); }

synchronized Long count() { return count_;}
private synchronized boolean commit(Long oldc,
i n

Long newc){
boolean success = (count_ == oldc);
P r o g r a m m i n g

if (success) count_ = newc;

return success;
}

public void inc() throws InterruptedException{

for (;;) { // retry-based
if (Thread.interrupted())
C o n c u r r e n t

throw new InterruptedException();

Long c = count();
long v = c.longValue();
if (v < MAX && commit(c, new Long(v+1)))
break;
Thread.yield();
}
}
public void dec() // symmetrical
}
86
 Splitting Locks and Behavior
Associate a helper object with an independent subset of state and
functionality.
J a v a

Delegate actions to helper via pass-through method

class Shape {
i n

// Assumes size & dimension are independent

P r o g r a m m i n g

int height_ = 0;
int width_ = 0;

synchronized void grow() { ++height_; ++width_;}

Location l = new Location(0,0); // fully synched

C o n c u r r e n t

void shift() { l.moveBy(1, 1); } // Use l’s synch

}
grow and shift can execute simultaneously
When there is no existing object to delegate independent actions:
• Use an arbitrary Object as a lock, and protect associated
methods using synchronized block on that lock
— Useful for concurrent data structures
87
 Concurrent Queue
class TwoLockQueue {
final static class Node {
J a v a

Object value; Node next = null;

Node(Object x) { value = x; }
}
i n

private Node head_ = new Node(null); // dummy hdr

private Node last_ = head_;
P r o g r a m m i n g

private Object lastLock_ = new Object();

void put(Object x) {
synchronized (lastLock_) {
last_ = last_.next = new Node(x);
}
}
C o n c u r r e n t

synchronized Object poll() { // null if empty

Object x = null;
Node first = head_.next; // only contention pt
if (first != null) {
x = first.value; first.value = null;
head_ = first; // old first becomes header
}
return x;
}
}
88
 Concurrent Queue (continued)

queue
J a v a

last
head
i n

hdr first ... (null)

next
P r o g r a m m i n g

value
(null)

puts and polls can run concurrently

• The data structure is crafted to avoid contending access
C o n c u r r e n t

— Rely on Java atomicity guarantees at only potential

contention point
• But multiple puts and multiple polls disallowed
Weakens semantics
• poll may return null if another thread is in midst of put
• Balking policy for poll is nearly forced here
— But can layer on blocking version
89
 Introducing Concurrency into
Applications
J a v a

Three sets of patterns

i n

Each associated with a reason to introduce concurrency

P r o g r a m m i n g

Autonomous Loops

Establishing independent cyclic behavior

Oneway messages

Sending messages without waiting for reply or termination

C o n c u r r e n t

• Improves availability of sender object

Interactive messages

Requests that later result in reply or callback messages

• Allows client to proceed concurrently for a while

Most design ideas and semantics stem from active object models.
90
 Autonomous Loops
Simple non-reactive active objects contain a run loop of form:
J a v a

public void run() {

while (!Thread.interrupted())
i n

doSomething();
}
P r o g r a m m i n g

Normally established with a constructor containing:

new Thread(this).start();
Perhaps also setting priority and daemon status
Normally also support other methods called from other threads
C o n c u r r e n t

Requires standard safety measures

Common Applications
• Animations
• Simulations
• Message buffer Consumers
• Polling daemons that periodically sense state of world
91
 Autonomous Particle Class
public class Particle implements Runnable {
private int x = 0, y = 0;
J a v a

private Canvas canvas;

public Particle(Canvas host) { canvas = host; }
i n

synchronized void moveRandomly() {

x += (int) (((Math.random() - 0.5) * 5);
P r o g r a m m i n g

y += (int) (((Math.random() - 0.5) * 5);

}

public void draw(Graphics g) {

int lx, ly;
synchronized (this) { lx = x; ly = y; }
g.drawRect(lx, ly, 10, 10);
C o n c u r r e n t

}
public void run() {
for(;;) {
moveRandomly();
canvas.repaint();
try { Thread.sleep((int)(Math.random()*10);}
catch (InterruptedException e) { return; }
}
}
}
92
 Particle Applet
import java.awt.*;
import java.applet.*;
J a v a

public class ParticleApplet extends Applet {

public void init() {
add(new ParticleCanvas(10));
i n

}
}
P r o g r a m m i n g

class ParticleCanvas extends Canvas {

Particle[] particles;
ParticleCanvas(int nparticles) {
setSize(new Dimension(100, 100));
particles = new Particle[nparticles];
for (int i = 0; i < particles.length; ++i) {
C o n c u r r e n t

particles[i] = new Particle(this);

new Thread(particles[i]).start();
}
}

public void paint(Graphics g) {

for (int i = 0; i < particles.length; ++i)
particles[i].draw(g);
}
} // (needs lots of embellishing to look nice)
93
 Oneway Messages
J a v a

state, acquaintances
Client
accept react {
i n

update state oneway

Host send message Handler
P r o g r a m m i n g

Conceptually oneway messages are sent with

• No need for replies

C o n c u r r e n t

• No concern about failure (exceptions)

• No dependence on termination of called method

• No dependence on order that messages are received

But may sometimes want to cancel messages or resulting activities

94
 Oneway Message Styles

Events Mouse clicks, etc

J a v a

Notifications Status change alerts, etc

i n

Postings Mail messages, stock quotes, etc

P r o g r a m m i n g

Activations Applet creation, etc

Commands Print requests, repaint requests, etc
Relays Chain of responsibility designs, etc

Some semantics choices

C o n c u r r e n t

Asynchronous: Entire message send is independent

— By far, most common style in reactive applications
Synchronous: Caller must wait until message is accepted
— Basis for rendezvous protocols
Multicast: Message is sent to group of recipients
— The group might not even have any members
95
 Messages in Java
Direct method invocations
J a v a

• Rely on standard call/return mechanics

Command strings
i n

• Recipient parses then dispatches to underlying method

P r o g r a m m i n g

• Widely used in client/server systems including HTTP

EventObjects and service codes
• Recipient dispatches
• Widely used in GUIs, including AWT
Request objects, asking to perform encoded operation
C o n c u r r e n t

• Used in distributed object systems — RMI and CORBA

Class objects (normally via .class files)
• Recipient creates instance of class
• Used in Java Applet framework
Runnable commands
• Basis for thread instantiation, mobile code systems
96
 Design Goals for Oneway Messages
Object-based forces
J a v a

Safety

• Local state changes should be atomic (normally, locked)

i n
P r o g r a m m i n g

— Typical need for locking leads to main differences vs

single-threaded Event systems

• Safe guarding and failure policies, when applicable

Availability

• Minimize delay until host can accept another message

C o n c u r r e n t

Activity-based forces

Flow

• The activity should progress with minimal contention

Performance

• Minimize overhead and resource usage

97
 Design Patterns for Oneway Messages
Thread-per-Message
J a v a

client host handler

i n

start new thread

P r o g r a m m i n g

Thread-per-Activity via Pass-throughs

client host handler

same thread
C o n c u r r e n t

Thread-per-Object via Worker Threads (variants: Pools, Listeners)

client host handler

put take

channel worker thread

98
 Reactive Methods
Code scaffolding for illustrating patterns:
J a v a

class Host {
// ...
i n

private long localState_; // Or any state vars

private Handler handler_; // Message target
P r o g r a m m i n g

public void react(...) {

updateState(...);
sendMessage(...);
}

private synchronized void updateState(...) {

C o n c u r r e n t

// Assign to localState_;
}

private void sendMessage(...) {

// Issue handler.process(...)
}
}

react() may be called directly from client, or indirectly after

decoding command, event, etc
99
 Thread-per-Message
class Host { //...
public void react(...) {
J a v a

updateState(...);
sendMessage(...);
}
i n

synchronized void sendMessage(...) {

P r o g r a m m i n g

Runnable command = new Runnable() { // wrap

final Handler dest = handler_;
public void run() {
dest.process(...);
}
};
C o n c u r r e n t

new Thread(command).start(); // run

}
}
Runnable is the standard Java interface describing argumentless,
resultless command methods (aka closures, thunks)
Synchronization of sendMessage desirable if handler_ or
process() arguments not fixed/final
Variants: Thread-per-connection (sockets)
100
 Thread-per-Message Protocol
client host handler command
J a v a

react
i n

... updateState...
P r o g r a m m i n g

start/run

process
C o n c u r r e n t

101
 Multicast TPM
J a v a

client host handlers

react
i n
P r o g r a m m i n g

return
C o n c u r r e n t

Multicasts can either

• Generate one thread per message, or

• Use a single thread for all messages

Depends on whether OK to wait each one out before sending

next one
102
 TPM Socket-based Server
class Server implements Runnable {
public void run() {
J a v a

try {
ServerSocket socket = new ServerSocket(PORT);
for (;;) {
i n

final Socket connection = socket.accept();

new Thread(new Runnable() {
P r o g r a m m i n g

public void run() {

new Handler().process(connection);
}}).start();
}
}
catch(Exception e) { /* cleanup; exit */ }
}
C o n c u r r e n t

class Handler {
void process(Socket s) {
InputStream i = s.getInputStream();
OutputStream o = s.getOutputStream();
// decode and service request, handle errors
s.close();
}
}
103
 Thread Attributes and Scheduling
Each Thread has an integer priority
J a v a

• From Thread.MIN_PRIORITY to Thread.MAX_PRIORITY

(currently 1 to 10)
• Initial priority is same as that of the creating thread
i n

• Can be changed at any time via setPriority

P r o g r a m m i n g

• ThreadGroup.setMaxPriority establishes a ceiling for

all threads in the group
JVM schedulers give preference to threads with higher priority
• But preference is left vague, implementation-dependent
• No guarantees about fairness for equal-priority threads
C o n c u r r e n t

— Time-slicing is permitted but not required

• No guarantees whether highest-priority or longest-waiting
threads acquire locks or receive notifications before others
Priorities can only be used heuristically
• Build custom Queues to control order of sequential tasks
• Build custom Conditions to control locking and notification
104
 Adding Thread Attributes
Thread
specific
J a v a

attributes
i n
P r o g r a m m i n g

Thread
specific
attributes

Thread objects can hold non-public Thread-Specific contextual

attributes for all methods/objects running in that thread
C o n c u r r e n t

• Normally preferable to static variables

Useful for variables that apply per-activity, not per-object
• Timeout values, transaction IDs, Principals, current
directories, default parameters
Useful as tool to eliminate need for locking
• Used internally in JVMs to optimize memory allocation,
locks, etc via per-thread caches
105
 Implementing Thread-Specific Storage
class GameThread extends Thread { // ...
private long movementDelay_ = 3;
J a v a

static GameThread currentGameThread() {

return (GameThread)(Thread.currentThread());
}
i n

static long getDelay() {

return currentGameThread().movementDelay_;
P r o g r a m m i n g

}
static long setDelay(long t) {
currentGameThread().movementDelay_ = t;
}
}
class Ball { // ...
void move() { // ...
C o n c u r r e n t

Thread.sleep(GameThread.getDelay()));
}
}
class Main { ... new GameThread(new Game()) ... }
Define contextual attributes in special Thread subclasses
• Can be accessed without locking if all accesses are always
via Thread.currentThread()
• Enforce via static methods in Thread subclass
106
 Using ThreadLocal
java.lang.ThreadLocal available in JDK1.2
J a v a

• An alternative to defining special Thread subclasses

Uses internal hash table to associate data with threads

i n
P r o g r a m m i n g

• Avoids need to make special Thread subclasses when

adding per-thread data

— Trade off flexibility vs strong typing and performance

class Ball {
static ThreadLocal delay = new ThreadLocal();
void move() { // ...
C o n c u r r e n t

if (delay.get()==null) delay.set(new Long(3));

long d = ((Long)(delay.get())).longValue();
Thread.sleep(d);
}
}

Can extend to implement inherited Thread contexts

Where new threads by default use attributes of the parent

thread that constructed them
107
 Other Scoping Options

Choices for maintaining context information

J a v a
i n

per Class
P r o g r a m m i n g

per Application
per Principal per Object
per Role
per Session
per Thread
per Group
per Method
per Site
per Block
C o n c u r r e n t

per Domain
per Aggregate

per System
per Version

108
 Choosing among Scoping Options
Reusability heuristics
J a v a

• Responsibility-driven design

• Factor commonalities, isolate variation

i n

• Simplify Programmability
P r o g r a m m i n g

— Avoid long parameter lists

— Avoid awkward programming constructions

— Avoid opportunities for errors due to policy conflicts

— Automate propagation of bindings

C o n c u r r e n t

Conflict analysis

Example: Changing per-object bindings via tuning interfaces

can lead to conflicts when objects support multiple roles

• Settings made by one client impact others

• Common error with Proxy objects

• Replace with per-method, per-role, per-thread

109
 Thread-per-Activity via Pass-Throughs
class Host { //...
J a v a

void reactV1(...) { // no synch

updateState(); // isolate in synched method
sendMessage(...);
i n

}
void sendMessage(...) { // no synch
P r o g r a m m i n g

handler_.process(...); // direct call

}
}
A kind of forwarding — conceptually removing host from call chain
Callers of react must wait client host handler
for handler.process react
C o n c u r r e n t

to terminate, or
generate their own
threads ... updateState...
Host can respond to process
another react call from
another thread
immediately after
updating state

110
 Using Pass-Throughs
Common approach to writing AWT Event handlers, JavaBeans
methods, and other event-based components.
J a v a

But somewhat fragile:

• There is no “opposite” to synchronized
i n

— Avoid self calls to react from synchronized methods

P r o g r a m m i n g

• Need care in accessing representations at call-point

— If handler_ variable or process arguments not fixed,
copy values to locals while under synchronization
• Callers must be sure to create thread around call if they
cannot afford to wait or would lock up
C o n c u r r e n t

Variants
Bounded Thread-per-Message
• Keep track of how many threads have been created. If too
many, fall back to pass-through.
Mediated
• Register handlers in a common mediator structured as a
pass-through.
111
 Multicast Pass-Throughs
class Host { //...
CopyOnWriteSet handlers_;
J a v a

synchronized void addHandler(Handler h) {

handlers_.add(h); // copy
i n

}
P r o g r a m m i n g

void sendMessage(...) {
Iterator e = handlers_.iterator();
while (e.hasNext())
((Handler)(e.next())).process(...);
}
}
C o n c u r r e n t

Normally use copy-on-write to implement target collections

• Additions are much less common than traversals

AWT uses java.awt.AWTEventMulticaster class

• Employs variant of FixedList class design

• But coupled to AWT Listener framework, so cannot be

used in other contexts
112
 Thread-Per-Object via Worker Threads
Establish a producer-consumer chain
J a v a

Producer
Reactive method just places message in a channel
i n

Channel might be a buffer, queue, stream, etc

P r o g r a m m i n g

Message might be a Runnable command, event, etc

Consumer
Host contains an autonomous loop thread of form:
while (!Thread.interrupted()) {
m = channel.take();
process(m);
C o n c u r r e n t

}
Common variants
Pools
Use more than one worker thread
Listeners
Separate producer and consumer in different objects
113
 Worker Thread Example
interface Channel { // buffer, queue, stream, etc
void put(Object x);
J a v a

Object take();
}
i n

class Host { //...

Channel channel_ = ...;
P r o g r a m m i n g

void sendMessage(...) {
channel_.put(new Runnable() { // enqueue
public void run(){
handler_.process(...);
}});
}
C o n c u r r e n t

Host() { // Set up worker thread in constructor

// ...
new Thread(new Runnable() {
public void run() {
while (!Thread.interrupted())
((Runnable)(channel_.take())).run();
}
}).start();
}
}
114
 Worker Thread Protocol
client host handler command channel
J a v a

react
i n

... updateState...
P r o g r a m m i n g

put

take
run !empty
C o n c u r r e n t

run

process

115
 Channel Options
Unbounded queues
J a v a

• Can exhaust resources if clients faster than handlers

Bounded buffers
i n

• Can cause clients to block when full

P r o g r a m m i n g

Synchronous channels
• Force client to wait for handler to complete previous task
Leaky bounded buffers
• For example, drop oldest if full
C o n c u r r e n t

Priority queues
• Run more important tasks first
Streams or sockets
• Enable persistence, remote execution
Non-blocking channels
• Must take evasive action if put or take fail or time out
116
 Example: The AWT Event Queue Thread
AWT uses one thread and a single java.awt.EventQueue
J a v a

• Single thread makes visual updates appear more coherent

• Browsers may add per-Applet threads and queues
i n

AWT queue
P r o g r a m m i n g

anEvent pass-through applet

anEvent dequeue button actionPerformed(e) {
... dispatch ...
click mouseEvent
AWT Thread }

Events implement java.util.EventObject

C o n c u r r e n t

• Include both ‘‘Low-level’’ and ‘‘Semantic’’ events

Event dequeuing performed by AWT thread
repaint() places drawing request event in queue.
• The request may beoptimized away if one already there
• update/paint is called when request dequeued
— Drawing is done by AWT thread, not your threads
117
 AWT Example
J a v a

class MyApplet extends Applet

implements ActionListener {
i n

Button button = new Button(“Push me”);

boolean onOff = false;
P r o g r a m m i n g

public void init() {

button.addActionListener(this); // attach
add(button); // add to layout
}

public void ActionPerformed(ActionEvent evt) {

C o n c u r r e n t

if (evt.getSource() == button) // dispatch

toggle(); // update state
repaint(); // issue event(not necessary here)
}

synchronized void toggle() {

onOff = !onOff;
}
}
118
 Using AWT in Concurrent Programs
Most conservative policy is to perform all GUI-related state updates
in event handling methods
J a v a

• Define and generate new EventObjects if necessary

i n

• Consider splitting GUI-related state into separate classes

P r o g r a m m i n g

• Do not rely on thread-safety of GUI components

Define drawing and event handling methods in reactive form

• Do not hold locks when sending messages

C o n c u r r e n t

• Do not block or delay caller thread (the AWT thread)

• Generate threads to arrange GUI-unrelated processing

— Explicitly set their ThreadGroups

• Generate events to arrange GUI-related asynch processing

— Swing includes some utility classes to make this easier

119
 Thread Pools

client
J a v a

put handler
i n

handler
P r o g r a m m i n g

channel take

handler

handler
C o n c u r r e n t

Use a collection of worker threads, not just one

• Can limit maximum number and priorities of threads
Often faster than thread-per-message
• But slower than single thread working off a multislot buffer
unless handler actions permit parallelism
• Often works well for I/O-bound actions
120
 Listeners
J a v a

client host listener handler

put take
i n

channel worker thread

P r o g r a m m i n g

House worker thread in a different object

• Even in a different process, connected via socket
But full support for remote listeners requires frameworks for
C o n c u r r e n t

• Naming remote acquaintances (via registries, jndi etc)

• Failure, reliability, fault tolerance
• Security, protocol conformance, ...
Can make more transparent via Proxies
• Channels/Listeners that duplicate interface of Handler, but
wrap each message as queued command for later
execution
121
 Remote Worker Threads

class Host { // ...

J a v a

ObjectOutputStream c; // connected to a Socket

void sendMessage(...) {
c.writeObject(new SerializableRunnable() {
i n

public void run(){

new Handler().process(...);
P r o g r a m m i n g

}
});
}
}

class Listener { // instantiate on remote machine

ObjectInputStream c; // connected to a Socket
C o n c u r r e n t

Listener() {
c = new ...
Thread me = new Thread(new Runnable() {
public void run() {
for (;;) {
((Runnable)(c.readObject())).run();
}}});
me.start();
}
}
122
 Synchronous Channels
Synchronous oneway messages same as asynchronous, except:
J a v a

• Caller must wait at least until message is accepted

Simplest option is to use synchronized methods
i n

• Caller must wait out all downstream processing

P r o g r a m m i n g

Increase concurrency via synchronous channel to worker thread

• Every put must wait for take
• Every take must wait for put
Basis for synchronous message passing frameworks (CSP etc)
• Enables more precise, deterministic, analyzable, but
C o n c u r r e n t

expensive flow control measures.

• Relied on in part because CSP-inspired systems did not
allow dynamic construction of new threads, so required
more careful management of existing ones.
Variants
• Barrier: Threads wait but do not exchange information
• Rendezvous: Bidirectional message exchange at wait
123
 Synchronous Channel Example

class SynchronousChannel {
J a v a

Object item_ = null;

boolean putting_ = false;//disable multiple puts
i n

synchronized void put(Object e) {

if (e == null) return;
P r o g r a m m i n g

while (putting_) try { wait(); } catch ...

putting_ = true;
item_ = e;
notifyAll();
while (item_ != null) try { wait(); } catch ...
putting_ = false;
notifyAll();
C o n c u r r e n t

synchronized Object take() {

while (item_ == null) try { wait(); } catch ...
Object e = item_;
item_ = null;
notifyAll();
return e;
}
}
124
 Some Pattern Trade-Offs
J a v a

Thread-per- Pass-Through Worker Threads

Message
i n

+ Simple + Low overhead + Tunable

P r o g r a m m i n g

semantics: semantics and

When in doubt, - Fragile structure
make a new - No within-activity
thread + Can bound
concurrency resource
- Can be hard to usage
limit resource
usage - Higher overhead
C o n c u r r e n t

- Thread start-up - Can waste

overhead threads
- May block caller
(if buffer full
etc)

125
 Interactive Messages
Synopsis
J a v a

• Client activates Server with a oneway message

oneway
i n

client server
P r o g r a m m i n g

• Server later invokes a callback method on client

client callback server

Callback can be either oneway or procedural

C o n c u r r e n t

Callback can instead be sent to a helper object of client

Degenerate case: inform only of task completion

Applications

• Observer designs

• Completion indications from file and network I/O

• Threads performing computations that yield results

126
 Observer Designs
The oneway calls are change
notifications subject observer
J a v a

changeValue(v)
The callbacks are state queries
Examples
i n

changeNotification
• Screen updates
P r o g r a m m i n g

val==v

• Constraint frameworks
return
• Publish/subscribe
• Hand-built variants of currentValue
wait and notifyAll
Notifications must use oneway
C o n c u r r e n t

design pattern return(val)

display
Otherwise:
changeNotification
thread1 val==v cache == val

can deadlock against:

currentValue
thread2

127
 Observer Example

class Subject {
J a v a

protected double val_ = 0.0; // modeled state

public synchronized double getValue(){
return val_;}
i n

protected synchronized void setValue(double d){

val_ = d;}
P r o g r a m m i n g

protected CopyOnWriteSet obs_ = new COWImpl();

public void attach(Observer o) { obs_.add(o); }

public void changeValue(double newstate) {

setValue(newstate);
Iterator it = obs_.iterator();
C o n c u r r e n t

while (it.hasNext()){
final Observer o = (Observer)(it.next());
new Thread(new Runnable() {
public void run() {
o.changeNotification(this);
}
}).start();
}
}
} // More common to use pass-through calls instead of threads
128
 Observer Example (Continued)
J a v a

class Observer {
protected double cachedState_;//last known state
protected Subject subj_; // only one here
i n

Observer(Subject s) {
P r o g r a m m i n g

subj_ = s; cachedState_ = s.getValue();

display();
}

synchronized void changeNotification(Subject s){

if (s != subj_) return; // only one subject
C o n c u r r e n t

double oldState = cachedState_;

cachedState_ = subj_.getValue(); // probe

if (oldState != cachedState_) display();

}

synchronized void display() { // default version

System.out.println(cachedState_);
}
}
129
 Completion Callbacks
The asynch messages are service
activations client server
J a v a

app start/run
The callbacks are continuation
calls that transmit results
i n

return
• May contain a message ... try action...
P r o g r a m m i n g

ID or completion token to
tell client which task has success
completed completed

Typically two kinds of callbacks return

Success – analog of return

Failure – analog of throw
failure
C o n c u r r e n t

failed

Client readiness to accept

return
callbacks may be state-
dependent
• For example, if client can
only process callbacks in
a certain order

130
 Completion Callback Example
Callback interface
J a v a

interface FileReaderClient {
void readCompleted(String filename);
void readFailed(String filename,IOException ex);
i n

}
P r o g r a m m i n g

Sample Client

class FileReaderApp implements FileReaderClient {

private byte[] data_;

void readCompleted(String filenm) {

// ... use data ...
C o n c u r r e n t

}
void readFailed(String fn, IOException e){
// ... deal with failure ...
}

void app() {
new Thread(new FileReader(“file”,
data_,this)).start();
}
}
131
 Completion Callbacks (continued)
Sample Server
J a v a

class FileReader implements Runnable {

final String nm_;
final byte[] d_;
i n

final FileReaderClient client_; // allow null

P r o g r a m m i n g

public FileReader(String name, byte[] data,

FileReaderClient c) {
nm_ = name; d_ = data; client_ = c;
}

void run() {
try {
C o n c u r r e n t

// ... read...
if (client_ != null)
client_.readCompleted(nm_);
}
catch (IOException ex) {
if (client_ != null)
client_.readFailed(nm_, ex);
}
}
}
132
 Threads and I/O
Java I/O calls generally block
J a v a

• Thread.interrupt causes them to unblock

— (This is broken in many Java implementations)
i n

• Time-outs are available for some Socket operations

P r o g r a m m i n g

— Socket.setSoTimeOut
• Can manually set up classes to arrange time-out interrupts
for other kinds of I/O
Common variants of I/O completion callbacks
• Issue callback whenever there is enough data to process,
rather than all at once
C o n c u r r e n t

• Send a Runnable completion action instead of callback

• Use thread pools for either I/O or completion actions
Alternatives
• Place the I/O and the subsequent actions all in same
method, run in same thread.
• Read into a buffer serviced by a worker thread
133
 Rerouting Exceptions
Callbacks can be used instead of exceptions in any asynchronous
messaging context, not just those directly constructing threads
J a v a

Variants seen in Adaptors that call methods throwing exceptions

that clients do not know how to handle:
i n

interface Server { void svc() throws SException; }

P r o g r a m m i n g

interface EHandler { void handle(Exception e); }

class SvcAdapter {
Server server = new ServerImpl();
EHandler handler;
void attachHandler(EHandler h) { handler = h; }
public void svc() { // no throw clause
try { server.svc(); }
C o n c u r r e n t

catch (SException e) {
if (handler != null) handler.handle(e); }
}
}
Pluggable Handlers can do anything that a normal catch clause can
• Including cancelling all remaining processing in any thread
• But are less structured and sometimes more error-prone
134
 Joining Threads
Thread.join() may be used instead of callbacks when
J a v a

• Server does not need to call back client with results

• But client cannot continue until service completion

i n
P r o g r a m m i n g

Usually the easiest way to express termination dependence

• No need to define callback interface or send client ref as

argument

• No need for server to explicitly notify or call client

• Internally implemented in java by

C o n c u r r e n t

— t.join() calls t.wait()

— terminating threads call notifyAll()

Can use to simulate futures and deferred calls found in other

concurrent OO languages

• But no syntactic support for futures

135
 Join Example

public class PictureDisplay {

J a v a

private final PictureRenderer myRenderer_;

// ...
i n

public void show(final byte[] rawPic) {

class Waiter implements Runnable {
P r o g r a m m i n g

Picture result = null;

public void run() {
result = myRenderer_.render(rawPic); }
};
Waiter waiter = new Waiter();
Thread t = new Thread(waiter);
t.start();
C o n c u r r e n t

displayBorders(); // do other things

displayCaption(); // while rendering

try { t.join(); }
catch(InterruptedException e) { return; }

displayPicture(waiter.result);
}
}
136
 Join Protocol

picturedisplay thread renderer

J a v a

waiter
show start
i n

run
isAlive
P r o g r a m m i n g

render

... other actions...

return(im)

return
C o n c u r r e n t

!isAlive

join
!isAlive

return

displayPicture(result)
...

137
 Futures
Encapsulate waits for results of operations performed in threads
J a v a

• Futures are ‘‘data’’ types that wait until results ready

— Normally requires use of interfaces for types
i n

Clients wait only upon trying to use results

P r o g r a m m i n g

interface Pic { byte[] getImage(); }

interface Renderer { Pic render(byte[] raw); }

class AsynchRenderer implements Renderer {

static class FuturePic implements Pic { //inner
byte[] img_ = null;
synchronized void setImage(byte[] img) {
C o n c u r r e n t

img_ = img;
notifyAll();
}
public synchronized byte[] getImage() {
while (img_ == null)
try { wait(); }
catch (InterruptedException e) { ... }
return img_;
}
} // continued
138
 Futures (continued)
// class AsynchRender, continued
J a v a

public Pic render(final byte[] raw) {

final FuturePic p = new FuturePic();
new Thread(new Runnable() {
i n

public void run() {

P r o g r a m m i n g

p.setImage(doRender(raw));
}
}).start();
return p;
}

private Pic doRender(byte[] r); // ...

}
C o n c u r r e n t

class App { // sample usage

void app(byte[] r) {
Pic p = new AsynchRenderer().render(r);
doSomethingElse();
display(p.getImage()); // wait if not yet ready
}
}

Could alternatively write join-based version.

139
 Cancellation
Threads normally terminate after completing their run methods
J a v a

May need to cancel asynchronous activities before completion

i n

• Applet.stop() called
P r o g r a m m i n g

• User hits a CANCEL button

• Threads performing computations that are not needed

• I/O or network-driven activites that encounter failures

Options
C o n c u r r e n t

Asynchronous cancellation: Thread.stop

Polling and exceptions: Thread.interrupt

Terminating program: System.exit

Minimizing contention: setPriority(MIN_PRIORITY)

Revoking permissions: SecurityManager methods

Unlinking resources known to cause failure exceptions

140
 Asynchronous Cancellation
Thread.stop stops thread by throwing ThreadDeath exception
J a v a

Deprecated in JDK1.2 because it can corrupt object state:

class C {
i n

private int v; // invariant: v >= 0

P r o g r a m m i n g

synchronized void f() {

v = -1; // temporarily set to illegal value
compute(); // call some other method
v = 1; // set to legal value
}
synchronized void g() { // depend on invariant
while (v != 0) { --v; something(); } }
}
C o n c u r r e n t

What happens if stop occurs during compute()?

In principle, could catch(ThreadDeath)

• But this would only work well if done after just about every
line of code in just about every Java class. Impractical.

• Most other thread systems (including POSIX) either do not

support or severely restrict asynchronous cancellation
141
 Interruption
Safety can be maintained by each object checking cancellation
status only when in an appropriate state to do so, relying on:
J a v a

thread.isInterrupted
i n

• Returns current interruption status.

P r o g r a m m i n g

(static) Thread.interrupted
• Clears status for current thread, returning previous status.
thread.interrupt
• Sets interrupted status, and also causes applicable
methods to throw InterruptedException
C o n c u r r e n t

• Threads that are blocked waiting for synchronized

method or block entry are NOT awakened by interrupt
InterruptedException
• Thrown by Thread.sleep, Thread.join, Object.wait
if blocked during interruption, also clearing status
• Blocking IO methods in the java.io package respond to
interrupt by throwing InterruptedIOException
142
 Implementing a Cancellation Policy
Best-supported policy is:
J a v a

Thread.isInterrupted() means cancelled

Any method sensing interruption should

i n
P r o g r a m m i n g

• Assume current task is cancelled.

• Exit as quickly and cleanly as possible.

• Ensure that callers are aware of cancellation. Options:

Thread.currentThread().interrupt()
C o n c u r r e n t

throw new InterruptedException()

Alternatives

• Local recovery and continuation

• Centralized error recovery objects

• Always ignoring/resetting status

143
 Detecting Cancellation
Cancellation can be checked as a precondition for any method
J a v a

if (Thread.currentThread().isInterrupted())
cancellationCode();
• Also in loop headers of looping methods, etc
i n

Can be caught, thrown, or rethrown as an exception

P r o g r a m m i n g

try { somethingThrowingInterruptedException(); }
catch (InterruptedException ex) {
cancellationCode();
}
• Or as a subclass of a general failure exception, as in
InterruptedIOException
C o n c u r r e n t

Placement, style, and poll frequency require engineering tradeoffs

• How important is it to stop now?
• How hard is it to stop now?
• Will another object detect and deal with at a better time?
• Is it too late to stop an irreversable action?
• Does it really matter if the thread is stopped?
144
 Responses to Cancellation
Early return
J a v a

• Clean up and exit without producing or signalling errors —

May require rollback or recovery
i n

• Callers can poll status if necessary to find out why action

was not carried out.
P r o g r a m m i n g

• Reset (if necessary) interruption status before return:

Thread.currentThread().interrupt()

Continuation (ignoring cancellation status)

• When it is too dangerous to stop

C o n c u r r e n t

• When partial actions cannot be backed out

• When it doesn’t matter (but consider lowering priority)

Throwing InterruptedException

• When callers must be alerted on method return

Throwing a general failure Exception

• When interruption is one of many reasons method can fail

145
 Multiphase Cancellation
Foreign code running in thread might not respond to cancellation.
J a v a

Dealing with this forms part of any security framework. Example:

static boolean terminate(Thread t) {

i n

if (!t.isAlive()) return true; // already dead

P r o g r a m m i n g

// phase 1 -- graceful cancellation

t.interrupt();
try { t.join(maxWaitToDie); }
catch(InterruptedException e){} // ignore
if (!t.isAlive()) return true; // success

// phase 2 -- trap all security checks

C o n c u r r e n t

theSecurityMgr.denyAllChecksFor(t); // made-up
try { t.join(maxWaitToDie); }
catch(InterruptedException ex) {}
if (!t.isAlive()) return true;

// phase 3 -- minimize damage

t.setPriority(Thread.MIN_PRIORITY);
// or even unsafe last-resort t.stop()
return false;
}
146
 Shutting Down Applets
Applets can create threads instantiate
— usually in Applet.start
J a v a

and terminate them

init invoke on load
i n

— usually in Applet.stop
P r o g r a m m i n g

These threads should be cancellable

start main action
• Otherwise, it is impossible to move
predict lifecycle off/on
page
• No guarantees about when
stop leave page
browsers will destroy, or
whether threads automatically
C o n c u r r e n t

revisit/reload
killed when unloading after
finalized
destroy finalization
Guidelines
• Explicitly cancel threads (normally in Applet.stop)
• Ensure that activities check cancellation often enough
• Consider last-resort Thread.stop in Applet.destroy

147
 Concurrent Application
Architectures
J a v a

Establishing application- (or subsystem-) wide Policies

• Communication directionality, synchronization
i n

• Avoid inconsistent case-by-case decisions

P r o g r a m m i n g

Samplings from three styles

Flow systems
Wiring together processing stages
— Illustrated with Push Flow designs
C o n c u r r e n t

Parallel execution
Partitioning independent tasks
— Illustrated with Group-based designs
Layered services
Synchronization and control of ground objects
— Illustrated with Before/After designs
148
 Push Flow Systems
Systems in which (nearly) all activities are performed by objects
J a v a

issuing oneway messages along paths from sources to sinks

• Each message transfers information and/or objects
i n

Examples
P r o g r a m m i n g

tempSensor comparator heater

Control systems
Assembly systems
supplier
Workflow systems invoices returns

Event processing approval

contractor
Chain of command invoices payment
C o n c u r r e n t

Pipeline algorithms
Requires common directionality and locality constraints
• Precludes many safety and liveness problems
• Success relies on adherence to design rules
— potentially formally checkable
The simplest and sometimes best open systems protocol

149
 Stages in Flow Systems
Every stage is a producer and/or consumer
Stages implement common interface
J a v a

with method of form put

producer consumer
i n

void put(Object item)

P r o g r a m m i n g

May have multiple successors

Outgoing elements may be
— multicasted or splitter put

— routed
May have multiple predecessors
C o n c u r r e n t

Incoming elements may be

— combined or put merger

— collected
Normally require explicit linkages
— only one stage per connection
Each stage can define put using any appropriate oneway message
implementation pattern — may differ across stages

150
 Exclusive Ownership of Resources
Elements in most flow systems act like physical resources in that
J a v a

• If you have one, then you can do something (with it) that
you couldn’t do otherwise.
i n

• If you have one, then no one else has it.

P r o g r a m m i n g

• If you give it to someone else, then you no longer have it.

• If you destroy it, then no one will ever have it.

Examples

• Invoices
C o n c u r r e n t

• Network packets

• File and socket handles

• Tokens

• Mail messages

• Money
151
 Accessing Resources

How should stages manage resource objects?

J a v a

class Stage {
i n

Resource res;
void put(Resource r) { /* ... */ }
P r o g r a m m i n g

Both reference-passing ‘‘shared memory’’ and copy-based

‘‘message passing’’ policies can encounter problems:

Shared memory Message passing

C o n c u r r e n t

put(r) put(r)
stage1 stage2 stage1 stage2

access access access

access

resource resource copy

Synchronize access to resource Deal with identity differences

152
 Ownership Transfer
Transfer policy
J a v a

At most one stage refers to any resource at any time

Require each owner to forget about each resource after revealing it
to any other owner as message argument or return value
i n
P r o g r a m m i n g

• Implement by nulling out instance variables refering to

resources after hand-off
— Or avoiding such variables
• Resource Pools can be used to hold unused resources
— Or just let them be garbage collected
C o n c u r r e n t

Before message After message

put(r) put(r)
stage1 stage2 stage1 stage2

access access

(null) (null)
resource resource

153
 Assembly Line Example
J a v a
i n
P r o g r a m m i n g

Boxes are flow elements

• Have adjustable dimension and color
• Can clone and draw themselves
C o n c u r r e n t

Sources produce continuous stream of BasicBoxes

Boxes are pushed through stages
• Stages paint, transform, combine into composite boxes
A viewer applet serves as the sink
See CPJ p233-248 for most code omitted here
• Some code here differs in minor ways for sake of
illustration
154
 Interfaces

start
J a v a

PushSource

interface PushSource { void start(); }

i n
P r o g r a m m i n g

putA
PushStage

interface PushStage { void putA(Box p); }

C o n c u r r e n t

putA
DualInput
PushStage

putB

interface DualInputPushStage extends PushStage {

public void putB(Box p);
}
155
 Adapters
putA DualInput putB
Adapter
J a v a

class DualInputAdapter implements PushStage {

i n

protected final DualInputPushStage stage_;

P r o g r a m m i n g

DualInputAdapter(DualInputPushStage stage) {
stage_ = stage;
}

void putA(Box p) { stage_.putB(p); }

}

Allows all other stages to issue putA

C o n c u r r e n t

• Use adapter when necessary to convert to putB

• Simplifies composition

Alternatively, could have used a single put(command) interface

• Would require each stage to decode type/sense of

command
156
 Connections
J a v a

SingleOutput
PushStage next1
i n

class SingleOutputPushStage {
protected PushStage next1_= null;
P r o g r a m m i n g

void attach1(PushStage s) { next1_ = s; }

}
next1

DualOutput
PushStage

next2
C o n c u r r e n t

class DualOutputPushStage
extends SingleOutputPushStage {
protected PushStage next2_ = null;
void attach2(PushStage s) { next2_ = s; }
}
Alternatively, could have used a collection (Vector etc) of nexts
We assume/require all attaches to be performed before any puts
157
 Linear Stages
J a v a

putA putA
Painter next1
i n
P r o g r a m m i n g

class Painter extends SingleOutputPushStage

implements PushStage {
protected final Color color_;

public Painter(Color c) {
super();
color_ = c;
}
C o n c u r r e n t

public void putA(Box p) {

p.color(color_);
next1_.putA(p);
}
}

Painter is immutable after initialization

158
 Dual Input Stages
J a v a

putA
putA
Collector next1
i n

putB
P r o g r a m m i n g

public class Collector

extends SingleOutputPushStage
implements DualInputPushStage {

public synchronized void putA(Box p) {

next1_.putA(p);
C o n c u r r e n t

public synchronized void putB(Box p) {

next1_.putA(p);
}

Synchronization used here to illustrate flow control, not safety

159
 Joiners
class Joiner extends SingleOutputPushStage
implements DualInputPushStage {
J a v a

protected Box a_ = null; // incoming from putA

protected Box b_ = null; // incoming from putB
protected abstract Box join(Box p, Box q);
i n

protected synchronized Box joinFromA(Box p) {

while (a_ != null) // wait until last consumed
P r o g r a m m i n g

try { wait(); }
catch (InterruptedException e){return null;}
a_ = p;
return tryJoin();
}
protected synchronized Box tryJoin() {
if (a_ == null || b_ == null) return null;
C o n c u r r e n t

Box joined = join(a_, b_); // make combined box

a_ = b_ = null; // forget old boxes
notifyAll(); // allow new puts
return joined;
}
void putA(Box p) {
Box j = joinFromA(p);
if (j != null) next1_.putA(j);
}
} // (mechanics for putB are symmetrical)
160
 Dual Output Stages
putA next1
putA
Cloner
J a v a

putA next2
i n

class Cloner extends DualOutputPushStage

P r o g r a m m i n g

implements PushStage {

protected synchronized Box dup(Box p) {

return p.duplicate();
}

public void putA(final Box p) {

Box p2 = dup(p); // synched update (not nec.)
C o n c u r r e n t

Runnable r = new Runnable() {

public void run() { next1_.putA(p); }
};
new Thread(r).start(); // use new thread for A
next2_.putA(p2); // current thread for B
}
}

Using second thread for second output maintains liveness

161
 Configuration
J a v a

BBSource Painter Alternator DIAdaptor

i n
P r o g r a m m i n g

HorizJoin Collector

BBSource Painter DIAdaptor

C o n c u r r e n t

All setup code is of form

Stage aStage = new Stage();

aStage.attach(anotherStage);

Would be nicer with a visual scripting tool

162
 Parallel Execution
Classic parallel programming deals with
J a v a

Tightly coupled, fine-grained multiprocessors

i n

Large scientific and engineering problems

P r o g r a m m i n g

Speed-ups from parallelism are possible in less exotic settings

SMPs, Overlapped I/O

Key to speed-up is independence of tasks

C o n c u r r e n t

Minimize thread communication and synchronization

Minimize sharing of resource objects

Rely on groups of thread-based objects

Worker thread designs

Scatter/gather designs
163
 Interacting with Groups
Group Proxies encapsulate a group of workers and protocol
J a v a

client proxy members

serve
scatter
i n
P r o g r a m m i n g

return
gather

class GroupProxy implements Service {

public Result serve(Data data) {
split the data into parts;
C o n c u r r e n t

for each part p

start up a thread to process p;

for each thread t {

collect results from t;//via callback or join
if (have enough results) // one, all, or some
return aggegrate result;
}
}
164
 Group Service Example
public class GroupPictureRenderer {
J a v a

public Picture[] render(final byte[][] data)

throws InterruptedException {
i n

int n = data.length;
Thread threads[] = new Thread[n];
P r o g r a m m i n g

final Picture results[] = new Picture[n];

for (int k = 0; k < n; k++) {

final int i = k; // inner vars must be final
threads[i] = new Thread(new Runnable() {
public void run() {
PictureRenderer r = new PictureRenderer();
C o n c u r r e n t

results[i] = r.render(data[i]);
}
};
threads[i].start();
}
// block until all are finished
for (int k = 0; k < n; k++) threads[k].join();
return results;
}
}
165
 Iteration using Cyclic Barriers
CyclicBarrier is synchronization tool for iterative group algorithms
J a v a

• Initialize count with number of members

• synch() waits for zero, then resets to initial count
i n

class PictureProcessor { // ...

public void processPicture(final byte[][] data){
P r o g r a m m i n g

final CyclicBarrier barrier =

new CyclicBarrier(NWORKERS);
for (int ii = 0; ii < NWORKERS; ++ii) {
final int i = ii;
Runnable worker = new Runnable() {
public void run() {
C o n c u r r e n t

while (!done()) {
transform(data[i]);
try { barrier.barrier(); }
catch(InterruptedException e){return;}
combine(data[i],data[(i+1)%NWORKERS]);
} } };
new Thread(worker).start();
}
}
}
166
 Implementing Cyclic Barriers
class CyclicBarrier {
J a v a

private int count_;

private int initial_;
private int resets_ = 0;
i n
P r o g r a m m i n g

CyclicBarrier(int c) { count_ = initial_ = c; }

synchronized boolean barrier() throws Inte...{

if (--count_ > 0) { // not yet tripped
int r = resets_; // wait until next reset
do { wait(); } while (resets_ == r);
return false;
}
C o n c u r r e n t

else {
count_ = initial_;
++resets_;
notifyAll();
return true; // return true if caller tripped
}
}
}

167
 Layered Services
client
J a v a

part1 subpart1
Control
i n

client part2
P r o g r a m m i n g

Providing concurrency control for methods of internal objects

• Applying special synchronization policies
• Applying different policies to the same mechanisms
C o n c u r r e n t

Requires visibility control (containment)

• Inner code must be communication-closed
• No unprotected calls in to or out from island
• Outer objects must never leak identities of inner objects
• Can be difficult to enforce and check
Usually based on before/after methods
168
 Three-Layered Application Designs
J a v a

Interaction with external world

generating threads
i n
P r o g r a m m i n g

Concurrency Control
locking, waiting, failing

Basic mechanisms
C o n c u r r e n t

Common across many concurrent applications

Generally easy to design and implement

Maintain directionality of control and locking

169
 Before/After Control
Control access to contained object/action via a method of the form
J a v a

void controlled() {
pre();
try { action(); }
i n

finally { post(); }
}
P r o g r a m m i n g

Used by built-in Java synchronized(obj) { action(); }

Pre: ‘{‘ obtains lock ... Post: ‘}’ releases lock
Control code must be separable from ground action code
• Control code deals only with execution state
C o n c u r r e n t

• Ground code deals only with intrinsic state

Basis for many delegation-based designs
delegate
ControlledService GroundService
service() {
pre(); action() { ... }
delegate.action();
post();
}

170
 Template Method Before/After Designs
Subclassing is one way to implement
before/after containment designs AbstractService
J a v a

public service() {
• Superclass instance variables pre();
action();
and methods are “contained” post();
i n

in subclass instances }
protected pre();
P r o g r a m m i n g

protected action();
protected post();

Template methods
• Isolate ground code and
control code in overridable ConcreteService
protected methods override
pre() or action() or post();
• Public methods call control
C o n c u r r e n t

and ground code in an

established fashion
• Can provide default versions
in abstract classes
• Can override the control code
and/or the ground code in
subclasses

171
 Readers & Writers Policies
J a v a

threads

read data
i n
P r o g r a m m i n g

write

Apply when
• Methods of ground class can be separated into readers
(accessors) vs writers (mutators)
C o n c u r r e n t

— For example, controlling access to data repository

• Any number of reader threads can run simultanously, but
writers require exclusive access
Many policy variants possible
• Mainly surrounding precedence of waiting threads
— Readers first? Writers first? FIFO?
172
 Readers & Writers via Template Methods

public abstract class RW {

J a v a

protected int activeReaders_ = 0; // exec state

protected int activeWriters_ = 0;
protected int waitingReaders_ = 0;
i n

protected int waitingWriters_ = 0;

P r o g r a m m i n g

public void read() {

beforeRead();
try { doRead(); } finally { afterRead(); }
}
public void write(){
beforeWrite();
try { doWrite(); } finally { afterWrite(); }
C o n c u r r e n t

protected boolean allowReader() {

return waitingWriters_ == 0 &&
activeWriters_ == 0;
}
protected boolean allowWriter() {
return activeReaders_ == 0 &&
activeWriters_ == 0;
}
173
 Readers & Writers (continued)
J a v a

protected synchronized void beforeRead() {

++waitingReaders_;
i n

while (!allowReader())
P r o g r a m m i n g

try { wait(); }
catch (InterruptedException ex) { ... }
--waitingReaders_;
++activeReaders_;
}
C o n c u r r e n t

protected abstract void doRead();

protected synchronized void afterRead() {

--activeReaders_;
notifyAll();
}
174
 Readers & Writers (continued)
J a v a

protected synchronized void beforeWrite() {

++waitingWriters_;
i n

while (!allowWriter())
try { wait(); }
P r o g r a m m i n g

catch (InterruptedException ex) { ... }

--waitingWriters_;
++activeWriters_;
}
C o n c u r r e n t

protected abstract void doWrite();

protected synchronized void afterWrite() {

--activeWriters_;
notifyAll();
}
}
175
 Using Concurrency Libraries
Library classes can help separate responsibilities for
J a v a

Choosing a policy; for example

i n

— Exclusive versus shared access

P r o g r a m m i n g

— Waiting versus failing

— Use of privileged resources

Applying a policy in the course of a service or transaction

— These decisions can occur many times within a method

C o n c u r r e n t

Standard libraries can encapsulate intricate synchronization code

But can add programming obligations

• Correctness relies on all objects obeying usage policy

• Cannot automatically enforce

Examples

• Synchronization, Channels, Transactions

176
 Interfaces
Sync encompasses many concurrency control policies
J a v a

Sync
void acquire()
void release()
boolean attempt(long timeOut)
i n
P r o g r a m m i n g

cond
Service ConcreteSync
service(...) {
cond.acquire(); implementations
try { action(); }
finally { cond.release(); }
}
C o n c u r r e n t

public interface Sync {

// Serve as a gate, fail only if interrupted
void acquire() throws InterruptedException;
// Possibly allow other threads to pass the gate
void release();
// Try to pass for at most timeout msecs,
// return false if fail
boolean attempt(long timeOut);
}
177
 Synchronization Libraries
Semaphores
J a v a

• Maintain count of the number of threads allowed to pass

Latches
i n

• Boolean conditions that are set once, ever

P r o g r a m m i n g

Barriers

• Counters that cause all threads to wait until all have

finished

Reentrant Locks
C o n c u r r e n t

• Java-style locks allowing multiple acquisition by same

thread, but that may be acquired and released as needed

Mutexes

• Non-reentrant locks

Read/Write Locks

• Pairs of conditions in which the readLock may be shared,

but the writeLock is exclusive
178
 Semaphores
Conceptually serve as permit holders
J a v a

• Construct with an initial number of permits (usually 0)

• require waits for a permit to be available, then takes one

i n

• release adds a permit

P r o g r a m m i n g

But in normal implementations, no actual permits change hands.

• The semaphore just maintains the current count.

• Enables very efficient implementation

Applications
C o n c u r r e n t

• Isolating wait sets in buffers, resource controllers

• Designs that would otherwise encounter missed signals

— Where one thread signals before the other has even

started waiting

— Semaphores ‘remember’ how many times they were

signalled
179
 Counter Using Semaphores
class BoundedCounterUsingSemaphores {
J a v a

long count_ = MIN;

Sync decPermits_= new Semaphore(0);

i n

Sync incPermits_= new Semaphore(MAX-MIN);

P r o g r a m m i n g

synchronized long value() { return count_; }

void inc() throws InterruptedException {

incPermits_.acquire();
synchronized(this) { ++count_; }
decPermits_.release();
}
C o n c u r r e n t

void dec() throws InterruptedException {

decPermits_.acquire();
synchronized(this) { --count_; }
incPermits_.release();
}
}

This uses native synch for update protection, but only inside permit
blocks. This avoids nested monitor lockouts
180
 Semaphore Synchronous Channel

class SynchronousChannelVS {
J a v a

Object item = null;

Semaphore putPermit = new Semaphore(1);

i n

Semaphore takePermit = new Semaphore(0);

P r o g r a m m i n g

Semaphore ack = new Semaphore(0);

void put(Object x) throws InterruptedException {

putPermit.acquire();
item = x;
takePermit.release();
ack.acquire();
C o n c u r r e n t

Object take() throws InterruptedException {

takePermit.acquire();
Object x = item;
putPermit.release();
ack.release();
return x;
}
}
181
 Using Latches
Conditions starting out false, but once set true, remain true forever
J a v a

• Initialization flags
• End-of-stream conditions
i n

• Thread termination
P r o g r a m m i n g

• Event occurrences
class Worker implements Runnable {
Latch go;
Worker(Latch l) { go = l; }
public void run() {
go.acquire();
doWork();
C o n c u r r e n t

}
}
class Driver { // ...
void main() {
Latch go = new Latch();
for (int i = 0; i < N; ++i) // make threads
new Thread(new Worker(go)).start();
doSomethingElse(); // don’t let run yet
go.release(); // let all threads proceed
} }
182
 Using Barrier Conditions
Count-based latches
J a v a

• Initialize with a fixed count

• Each release monotonically decrements count

i n

• All acquires pass when count reaches zero

P r o g r a m m i n g

class Worker implements Runnable {

Barrier done;
Worker(Barrier d) { done = d; }
public void run() {
doWork();
done.release();
C o n c u r r e n t

}
}
class Driver { // ...
void main() {
Barrier done = new Barrier(N);
for (int i = 0; i < N; ++i)
new Thread(new Worker(done)).start();
doSomethingElse();
done.acquire(); // wait for all to finish
} }
183
 Using Lock Classes
class HandSynched {
private double state_ = 0.0;
J a v a

private Sync lock_;

HandSynched(Sync l) { lock_ = l; }
i n

void changeState(double d) {
P r o g r a m m i n g

try {
lock_.acquire();
try { state_ = d; }
finally { lock_.release(); }
} catch(InterruptedException ex) { }
}
C o n c u r r e n t

double getState() {
double d = 0.0;
try {
lock_.acquire();
try { d = state_; }
finally { lock_.release(); }
} catch(InterruptedException ex){}
return d;
}
}
184
 Wrapper Classes
Standardize common client usages of custom locks using wrappers
J a v a

• Wrapper class supports perform method that takes care

of all before/after control surrounding a Runnable
command sent as a parameter
i n
P r o g r a m m i n g

• Can also standardize failure control by accepting

Runnable action to be performed on acquire failure

Alternative perform methods can accept blocks that return results

and/or throw exceptions

• But need to create new interface type for each kind of block
C o n c u r r e n t

Similar to macros in other languages

• But implement more safely via inner classes

— Wrappers are composable

Adds noticeable overhead for simple usages

• Most useful for controlling “heavy” actions

185
 Before/After Wrapper Example
J a v a

class WithLock {
Sync cond;
public WithLock(Sync c) { cond = c; }
i n

public void perform(Runnable command)

P r o g r a m m i n g

throws InterruptedException {
cond.acquire();
try { command.run(); }
finally { cond.release(); }
}

public void perform(Runnable command,

C o n c u r r e n t

Runnable onInterrupt) {
try { perform(command); }
catch (InterruptedException ex) {
if (onInterrupt != null)
onInterrupt.run();
else // default
Thread.currentThread().interrupt();
}
}
}
186
 Using Wrappers
class HandSynchedV2 { // ...
private double state_ = 0.0;
J a v a

private WithLock withlock_;

HandSynchedV2(Sync l) {
i n

withlock_ = new WithLock(l);

}
P r o g r a m m i n g

void changeState(double d) {
withlock_.perform(
new Runnable() {
public void run() { state_ = d; } },
null); // use default interrupt action
}
C o n c u r r e n t

double getState() {
// (need to define interface & perform version)
try {
return withLock_.perform(new DoubleAction(){
public void run() { return state_; } });
}
catch(InterruptedException ex){return 0.0;}
}
}
187
 Using Conditional Locks

Sync.attempt can be used in conditional locking idioms

J a v a

Back-offs
i n

• Escape out if a lock not available

P r o g r a m m i n g

• Can either retry or fail

Reorderings

• Retry lock sequence in different order if first attempt fails

C o n c u r r e n t

Heuristic deadlock detection

• Back off on time-out

Precise deadlock detection

• Implement Sync via lock manager that can detect cycles

188
 Back-off Example

class Cell {
J a v a

long value;
Sync lock = new SyncImpl();
void swapValue(Cell other) {
i n

for (;;) {
try {
P r o g r a m m i n g

lock.acquire();
try {
if (other.lock.attempt(100)) {
try {
long t = value; value = other.value;
other.value = t;
return;
C o n c u r r e n t

}
finally { other.lock.release(); }
}
}
finally { lock.release(); }
}
catch (InterruptedException ex) { return; }
}
}
}
189
 Lock Reordering Example
class Cell {
long value;
J a v a

Sync lock = new SyncImpl();

private static boolean trySwap(Cell a, Cell b) {
a.lock.acquire();
i n

try {
if (!b.lock.attempt(0)) return false;
P r o g r a m m i n g

try {
long t = a.value;
a.value = b.value;
b.value = t;
return true;
}
finally { other.lock.release(); }
C o n c u r r e n t

}
finally { lock.release(); }
return false;
}

void swapValue(Cell other) {

while (!trySwap(this, other) &&
!tryswap(other, this)) Thread.yield();
}
}
190
 Using Read/Write Locks
public interface ReadWriteLock {
Sync readLock();
J a v a

Sync writeLock();
}
i n

Sample usage using wrapper

P r o g r a m m i n g

class WithRWLock {
ReadWriteLock rw;

public WithRWLock(ReadWriteLock l) { rw = l; }

public void performRead(Runnable readCommand)

throws InterruptedException {
C o n c u r r e n t

rw.readLock().acquire();
try { readCommand.run(); }
finally { rw.readlock().release(); }
}

public void performWrite(...) // similar

}

191
 Transaction Locks
Associate keys with locks
J a v a

• Each key corresponds to a different transaction.

— Thread.currentThread() serves as key in reentrant

i n

Java synchronization locks

P r o g r a m m i n g

• Supply keys as arguments to gating methods

• Security frameworks can use similar interfaces, adding

mechanisms and protocols so keys serve as capabilities

Sample interface

interface TransactionLock {
C o n c u r r e n t

void begin(Object key); // bind key with lock

void end(Object key); // get rid of key

void acquire(Object key)

throws InterruptedException;

void release(Object key);

}
192
 Transactional Classes

Implement a common transaction control interface, for example:

J a v a

interface Transactor {
i n

// enter a new transaction

P r o g r a m m i n g

void join(Object key) throws Failure;

// return true if transaction can be committed

boolean canCommit(Object key);

// update state to reflect current transaction

void commit(Object key) throws Failure;
C o n c u r r e n t

// roll back state

void abort(Object key);
}

Transactors must ensure that all objects they communicate with

are also Transactors

• Control arguments must be propagated to all participants

193
 Per-Method Transaction Control
Add transaction control argument to each method.
J a v a

For example:

interface TransBankAccount extends Transactor {

i n
P r o g r a m m i n g

long balance(Object key)

throws InterruptedException;

void deposit(Object key, long amount)

throws InsufficientFunds,
InterruptedException;

void withdraw(Object key, long amount)

C o n c u r r e n t

throws InsufficientFunds,
InterruptedException;
}

The same interfaces can apply to optimistic transactions

• Use interference detection rather than locking.

• They are generally interoperable

194
 Per -ThreadGroup Transaction Control
Assumes each transaction established in own ThreadGroup
J a v a

class Context { // ...

Object get(Object name);
void bind(Object name, Object val);
i n

}
P r o g r a m m i n g

class XTG extends ThreadGroup { // ...

Context getContext();
}

class Account extends Transactor {

private long balance_;
private TransactionLock tlock_;
C o n c u r r e n t

// ...
void deposit(long amount) throws ... {
tlock_.acquire(((XTG)
(Thread.currentThread().getThreadGroup()))
.getContext().get("TransactionID"));
synchronized (this) {
if (amount >= 0) balance_ += amount; else ...
}
}
}
195
 Integrating Control Policies
Dealing with multiple contextual domains, including
J a v a

• Security: Principal identities, keys, groups, etc

• Synchronization: Locks, conditions, transactions, ...
i n

• Scheduling: Priorities, timing, checkpointing, etc

P r o g r a m m i n g

• Environment: Location, computational resources

Dealing with multiple outcomes
• Block, fail, proceed, save state, commit state, notify, ...
Encapsulating associated policy control information
C o n c u r r e n t

• For example access control lists, lock dependencies

Introducing new policies in sub-actions
• New threads, conditions, rights-transfers, subtransactions
• Avoiding policy conflicts: policy compatibility matrices, ...
Avoiding excessive programming obligations for developers
Tool-based code generation, layered virtual machines
196
 Using Integrated Control

Methods invoke helpers to make control decisions as needed

J a v a

class Account { // ...

i n
P r o g r a m m i n g

void deposit(long amount, ...) {

authenticator.authenticate(clientID);
accessController.checkAccess(clientID, acl);
logger.logDeposit(clientID, transID, amount);
replicate.shadowDeposit(...);
db.checkpoint(this);
lock.acquire();
C o n c u r r e n t

balance += amount;

lock.release();
db.commit(balance, ...);
UIObservers.notifyOfChange(this);
}
}

Not much fun to program.

197
 Implementing Library Classes
Classes based on Java monitor methods can be slow
J a v a

• Involve context switch, locking, and scheduling overhead

i n

• Relative performance varies across platforms

P r o g r a m m i n g

Some performance enhancements

State tracking

• Only notify when state changes known to unblock waits

Isolating wait sets

C o n c u r r e n t

• Only wake up threads waiting for a particular state change

Single notifications

• Only wake up a single thread rather than all waiting threads

Avoiding locks

• Don’t lock if can be sure won’t wait

Can lead to significantly faster, but more complex and fragile code
198
 Tracking State in Guarded Methods
Partition action control state into categories with
same enabling properties
J a v a

top
dec inc
State Condition inc dec from
MAX
to
MAX
i n

top value == MAX no yes

middle MIN < value < MAX yes yes middle
P r o g r a m m i n g

bottom value == MIN yes no dec

to
inc
from
MIN MIN

bottom
Only provide notifications when making a state
transition that can ever unblock another thread
• Here, on exit from top or bottom
C o n c u r r e n t

— When count goes up from MIN or down from MAX

• Still need notifyAll unless add instrumentation
State tracking leads to faster but more fragile code
• Usually many fewer notification calls
• Harder to change guard conditions
• Harder to add subclasses with different conditions

199
 Counter with State Tracking
J a v a

class FasterGuardedBoundedCounter {
protected long count_ = MIN;
i n

synchronized long value() { return count_; }

P r o g r a m m i n g

synchronized void inc()

throws InterruptedException {
while (count_ == MAX) wait();
if (count_++ == MIN) notifyAll();
}
C o n c u r r e n t

synchronized void dec()

throws InterruptedException {
while (count_ == MIN) wait();
if (count_-- == MAX) notifyAll();
}
}
200
 Buffer with State Tracking
class BoundedBufferVST {
Object[] data_;
J a v a

int putPtr_ = 0, takePtr_ = 0, size_ = 0;

protected void doPut(Object x){

i n

data_[putPtr_] = x;
putPtr_ = (putPtr_ + 1) % data_.length;
P r o g r a m m i n g

if (size_++ == 0) notifyAll();
}

protected Object doTake() {

Object x = data_[takePtr_];
data_[takePtr_] = null;
takePtr_ = (takePtr_ + 1) % data_.length;
C o n c u r r e n t

if (size_-- == data_.length) notifyAll();

return x;
}

synchronized void put(Object x) throws Inte... {

while (isFull()) wait();
doPut(x);
}
// ...
}
201
 Inheritance Anomaly Example
class XBuffer extends BoundedBufferVST {
synchronized void putPair(Object x, Object y)
J a v a

throws InterruptedException {
put(x);
put(y);
i n

}
P r o g r a m m i n g

}
PutPair does not guarantee that the pair is inserted contiguously
To ensure contiguity, try adding guard:
while (size_ > data_.length - 2) wait();
But doTake only performs notifyAll when the buffer transitions
C o n c u r r e n t

from full to not full

• The wait may block indefinitely even when space available
• So must rewrite doTake to change notification condition
Would have been better to factor out the notification conditions in a
separate overridable method
• Most inheritance anomalies can be avoided by fine-grained
(often tedious) factoring of methods and classes
202
 Isolating Waits and Notifications
Mixed condition problems
J a v a

• Threads that wait in different methods in the same object

may be blocked for different reasons — for example, not
Empty vs not Full for buffer
i n

• notifyAll wakes up all threads, even those waiting for

P r o g r a m m i n g

conditions that could not possibly hold

Can isolate waits and notifications for different conditions in
different objects — an application of splitting
Thundering herd problems
• notifyAll may wake up many threads
C o n c u r r e n t

• Often, at most one of them will be able to continue

Can solve by using notify instead of notifyAll only when
➔ All threads wait on same condition
➔ At most one thread could continue anyway
• That is, when it doesn’t matter which one is woken, and it
doesn’t matter that others aren’t woken
203
 Implementing Reentrant Locks
final class ReentrantLock implements Sync {
private Thread owner_ = null;
J a v a

private int holds_ = 0;

synchronized void acquire() throws Interru... {

i n

Thread caller = Thread.currentThread();

if (caller == owner_) ++holds_;
P r o g r a m m i n g

else {
try { while (owner_ != null) wait(); }
catch (InterruptedException e) {
notify(); throw e;
}
owner_ = caller; holds_ = 1;
}
C o n c u r r e n t

}
synchronized void release() {
Thread caller = Thread.currentThread();
if (caller != owner_ || holds_ <= 0)
throw new Error("Illegal Lock usage");
if (--holds_ == 0) {
owner_ = null;
notify();
}
}}
204
 Implementing Semaphores
final class Semaphore implements Sync {
int permits_; int waits_ = 0;
J a v a

Semaphore(int p) { permits_ = p; }
i n

synchronized void acquire() throws Interrup.. {

if (permits_ <= waits_) {
P r o g r a m m i n g

++waits_;
try {
do { wait(); } while (permits_ == 0);
}
catch(InterruptedException ex) {
--waits_; notify(); throw ex;
}
C o n c u r r e n t

--waits_;
}
--permits_;
}

synchronized void release() {

++permits_;
notify();
}
}
205
 Implementing Latches
Exploit set-once property to avoid locking using double-check:
J a v a

Check status without even locking

• If set, exit — no possibility of conflict or stale read
i n

• Otherwise, enter standard locked wait

P r o g r a m m i n g

But can have surprising effects if callers expect locking for sake of
memory consistency.
final class Latch implements Sync {
private boolean latched_ = false;

void acquire() throws InterruptedException {

if (!latched_)
C o n c u r r e n t

synchronized(this) {
while (!latched_) wait();
}
}

synchronized void release() {

latched_ = true;
notifyAll();
}
}
206
 Implementing Barrier Conditions
Double-check can be used for any monotonic variable that is tested
only for a threshold value
J a v a

CountDown Barriers monotonically decrement counts

i n

• Tests against zero cannot encounter conflict or staleness

P r o g r a m m i n g

(This technique does not apply to Cyclic Barriers)

class CountDown implements Sync {

private int count_;
CountDown(int initialc) { count_ = initialc; }
C o n c u r r e n t

void acquire() throws InterruptedException {

if (count_ > 0)
synchronized(this) {
while (count_ > 0) wait();
}
}
synchronized void release() {
if (--count_ == 0) notifyAll();
}
}
207
 Implementing Read/Write Locks
class SemReadWriteLock implements ReadWriteLock {
J a v a

// Provide fair access to active slot

Sync active_ = new Semaphore(1);
i n

// Control slot sharing by readers

class ReaderGate implements Sync {
P r o g r a m m i n g

int readers_ = 0;

synchronized void acquire()

throws InterruptedException {
// readers pile up on lock until first passes
if (readers_++ == 0) active_.acquire();
}
C o n c u r r e n t

synchronized void release() {

if (--readers_ == 0) active_.release();
}
}
Sync rGate_ = new ReaderGate();

public Sync writeLock() { return active_; }

public Sync readLock() { return rGate_; }
}
208
 Documenting Concurrency
Make code understandable
J a v a

• To developers who use components

• To developers who maintain and extend components
i n

• To developers who review and test components

P r o g r a m m i n g

Avoid need for extensive documentation by adopting:

• Standard policies, protocols, and interfaces
• Standard design patterns, libraries, and frameworks
• Standard coding idioms and conventions
C o n c u r r e n t

Document decisions
• Use javadoc to link to more detailed descriptions
• Use naming and signature conventions as shorthand clues
• Explain deviations from standards, usage limitations, etc
• Describe necessary data invariants etc
Use checklists to ensure minimal sanity
209
 Sample Documentation Techniques
Patlet references
J a v a

/** ... Uses

* <a href=”tpm.html”>Thread-per-Message</a> **/
void handleRequest(...);
i n

Default naming and signature conventions

P r o g r a m m i n g

Sample rule: Unless specified otherwise, methods that can

block have signature
... throws InterruptedException
Intentional limitations, and how to work around them
/** ... NOT Threadsafe, but can be used with
C o n c u r r e n t

* @see XAdapter to make lockable version. **/

Decisions impacting potential subclassers
/** ... Always maintains a legal value,
* so accessor method is unsynchronized **/
protected int bufferSize;
Certification
/** Passed safety review checklist 11Nov97 **/
210
 Semiformal Annotations
PRE — Precondition (normally unchecked)
J a v a

/** PRE: Caller holds synch lock ...

WHEN — Guard condition (always checked)
i n

/** WHEN not empty return oldest ...

P r o g r a m m i n g

POST — Postcondition (normally unchecked)

/** POST: Resource r is released...
OUT — Guaranteed message send (relays, callbacks, etc)
/** OUT: c.process(buff) called after read...
C o n c u r r e n t

RELY — Required property of other objects/methods

/** RELY: Must be awakened by x.signal()...
INV — Object constraint true at start/end of every activity
/** INV: x,y are valid screen coordinates...
INIT — Object constraint that must hold upon construction
/** INIT: bufferCapacity greater than zero...
211
 Safety Problem Checklist
Storage conflicts
J a v a

• Failure to ensure exclusive access; race conditions

Atomicity errors
i n

• Breaking locks in the midst of logically atomic operations

P r o g r a m m i n g

Representation inconsistencies
• Allowing dependent representations to vary independently
Invariant failures
• Failing to re-establish invariants within atomic methods
for example failing to clean up after exceptions
C o n c u r r e n t

Semantic conflicts
• Executing actions when they are logically prohibited
Slipped Conditions
• A condition stops holding in the midst of an action
requiring it to hold
Memory ordering and visibility
• Using stale cached values
212
 Liveness Problems
Lockout
J a v a

• A called method never becomes available

Deadlock
i n

• Two or more activities endlessly wait for each other

P r o g r a m m i n g

Livelock
• A retried action never succeeds
Missed signals
• A thread starts waiting after it has already been signalled
C o n c u r r e n t

Starvation
• A thread is continually crowded out from passing gate
Failure
• A thread that others are waiting for stops
Resource exhaustion
• Exceeding memory, bandwidth, CPU limitations
213
 Efficiency Problems
Too much locking
J a v a

• Cost of using synchronized

• Cost of blocking waiting for locks

i n

• Cost of thread cache flushes and reloads

P r o g r a m m i n g

Too many threads

• Cost of starting up new threads

• Cost of context switching and scheduling

• Cost of inter-CPU communication, cache misses

C o n c u r r e n t

Too much coordination

• Cost of guarded waits and notification messages

• Cost of layered concurrency control

Too many objects

• Cost of using objects to represent state, messages, etc

214
View publication stats

Reusability Problems
Context dependence
J a v a

• Components that are not safe/live outside original context

Policy breakdown
i n

• Components that vary from system-wide policies

P r o g r a m m i n g

Inflexibility

• Hardwiring control, premature optimization

Policy clashes

• Components with incompatible concurrency control

C o n c u r r e n t

strategies

Inheritance anomalies

• Classes that are difficult or impossible to subclass

Programmer-hostile components

Components imposing awkward, implicit , and/or error-prone

programming obligations
215