tranghongson@gmail.

com

Concurrent, Parallel, and Distributed

Programming in Java

Contents
1 Preparation 2
1.1 Eclipse IDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Concurrent Programming 2
2.1 Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Liveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2.1 Deadlock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2.2 Livelock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.3 Starvation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Dining Philosophers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Parallel Programming 11
3.1 Task Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Functional Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.1 Futures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.2 Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Loop Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Distributed Programming 17
4.1 Client-Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1.1 Unicast Communication (one-to-one) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1.1.1 OneServer - OneClient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1.1.2 OneServer - MultiClients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1.2 Broadcast Communication (one-to-all) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.1.2.1 OneIP - MultiPorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.1.2.2 MultiIPs - OnePort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1.3 Remote Method Invocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2 Message Passing Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2.1 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2.2 Point-to-Point Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.3 Collective Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.4 Reciprocal ArraySum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5 Exercises 27
5.1 Exercise Deadlock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Exercise Futures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3 Exercise Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.4 Exercise OneServer - MultiClients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.5 Exercise Remote Method Invocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.6 Exercise Message Passing Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

1
 [email protected]

1 Preparation
1.1 Eclipse IDE
- Download and install the Eclipse IDE (for Java Developers) from: https://www.eclipse.org/downloads/packages/release

1.2 Getting Started

- Create ’HelloWorld’ project: menu File / New / Java Project:

• Project name: HelloWorld

• JRE: Use default JRE ’...’ and workspace compiler preferences

• Module: uncheck ’Create module-info.java’ file

- Create ’Main’ class (entry-point): right-click ’src’ item / New / Class:

• Name: Main

• Which method stubs would you like to create: check ’public static void main(String[] args)’

1 public class Main {

2 public static void main ( String [] args ) {
3 System . out . println ( " Hello World " ) ;
4 }
5 }

- Run project: right-click ’src/Main.java’ file / Run As / Java Application

2 Concurrent Programming
2.1 Threads
- Create ’ThreadsJoin’ project
- Create ’src/MyUtil.java’ file (https://gist.github.com/tsonkk/678479ab009216375b14e2981edd2142)
1 public class MyUtil {
2 static void doSomething ( long millis ) {
3 try {
4 Thread . sleep ( millis ) ;
5 } catch ( I n t erruptedException ex ) {
6 ex . printStackTrace () ;
7 }
8 }
9 }

- Create ’src/Runner.java’ file (https://gist.github.com/tsonkk/231848114e66ea69a10521c3b5ceb0d1)

1 public class Runner implements Runnable {
2 @Override
3 public void run () {
4 System . out . println ( Thread . currentThread () . getName () + " is running ... " ) ;
5 MyUtil . doSomething (3000) ;
6 System . out . println ( Thread . currentThread () . getName () + " is done ! " ) ;
7 }
8 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/45bac019fc300d332f1f38552bd4bd5c)

2
 [email protected]

1 public class Main {

2 public static void main ( String [] args ) throws InterruptedException {
3 // thread - t1
4 Thread t1 = new Thread ( new Runner () , " t1 " ) ;
5 t1 . start () ;
6 // thread - t0 ( main - thread )
7 System . out . println ( Thread . currentThread () . getName () + " is running ... " ) ;
8 MyUtil . doSomething (1000) ;
9 System . out . println ( Thread . currentThread () . getName () + " is done ! " ) ;
10 }
11 }

- Update ’src/Main.java’ file (https://gist.github.com/tsonkk/4ceec81d8feb7835d2ac3df84aae7518)

1 public class Main {
2 public static void main ( String [] args ) throws InterruptedException {
3 ...
4 // thread - t0 ( main - thread )
5 System . out . println ( Thread . currentThread () . getName () + " is running ... " ) ;
6 MyUtil . doSomething (1000) ;
7 t1 . join () ; // wait for t1 to complete
8 System . out . println ( Thread . currentThread () . getName () + " is done ! " ) ;
9 }
10 }

2.2 Liveness
2.2.1 Deadlock
- Create ’ThreadsDeadlock’ project
- Create ’src/MyUtil.java’ file (https://gist.github.com/tsonkk/0cd021311bdd8f4ee2bb2ccf425fc483)
1 public class MyUtil {
2 static void doSomething ( long millis ) {
3 try {
4 Thread . sleep ( millis ) ;
5 } catch ( I n t erruptedException ex ) {
6 ex . printStackTrace () ;
7 }
8 }
9 }

- Create ’src/Runner.java’ file (https://gist.github.com/tsonkk/2240a435edf7ec9b79f4324c949e4251)

1 public class Runner implements Runnable {
2 private Object obj1 ;
3 private Object obj2 ;

3
 [email protected]

4
5 public Runner ( Object obj1 , Object obj2 ) {
6 this . obj1 = obj1 ;
7 this . obj2 = obj2 ;
8 }
9 @Override
10 public void run () {
11 run1 () ;
12 }
13 // deadlock
14 private void run1 () {
15 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj1 ) ;
16 synchronized ( obj1 ) {
17 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj1 ) ;
18 MyUtil . doSomething (3000) ;
19
20 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj2 ) ;
21 synchronized ( obj2 ) {
22 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj2 ) ;
23 MyUtil . doSomething (5000) ;
24 }
25 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj2 ) ;
26 }
27 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj1 ) ;
28 }
29 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/e291bc49670996d7e760ca85507a010e)

1 public class Main {
2 public static void main ( String [] args ) {
3 Object obj1 = new Object () ;
4 Object obj2 = new Object () ;
5 Thread t1 = new Thread ( new Runner ( obj1 , obj2 ) , " t1 " ) ;
6 Thread t2 = new Thread ( new Runner ( obj2 , obj1 ) , " t2 " ) ;
7 t1 . start () ;
8 t2 . start () ;
9 }
10 }

- Update ’src/Runner.java’ file (https://gist.github.com/tsonkk/c99aa3825649f98af3e65d2a403502a1)

1 public class Runner implements Runnable {
2 ...
3 @Override
4 public void run () {
5 // run1 () ;
6 run2 () ;
7 }
8 ...
9 // avoid nested locks
10 private void run2 () {
11 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj1 ) ;
12 synchronized ( obj1 ) {
13 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj1 ) ;
14 MyUtil . doSomething (3000) ;
15 }

4
 [email protected]

16 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj1 ) ;
17
18 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj2 ) ;
19 synchronized ( obj2 ) {
20 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj2 ) ;
21 MyUtil . doSomething (5000) ;
22 }
23 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj2 ) ;
24 }
25 }

- Update ’src/Runner.java’ file (https://gist.github.com/tsonkk/5a7d85b9ffe5c0edeb9e7f9cc3b7cadc)

1 import java . util . concurrent . locks . Lock ;
2 import java . util . concurrent . locks . ReentrantLock ;
3
4 public class Runner implements Runnable {
5 ...
6 @Override
7 public void run () {
8 // run1 () ;
9 // run2 () ;
10 run3 () ;
11 }
12 ...
13 // using unstructured locks
14 private void run3 () {
15 Lock lock = new ReentrantLock () ;
16
17 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj1 ) ;
18 lock . lock () ;
19 try {
20 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj1 ) ;
21 MyUtil . doSomething (3000) ;
22 } finally {
23 lock . unlock () ;
24 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj1 ) ;
25 }
26
27 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj2 ) ;
28 lock . lock () ;
29 try {
30 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj2 ) ;
31 MyUtil . doSomething (5000) ;
32 } finally {
33 lock . unlock () ;
34 System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj2 ) ;
35 }
36 }
37 }

5
 [email protected]

2.2.2 Livelock
- Create ’ThreadsLivelock’ project
- Create ’src/MyUtil.java’ file (https://gist.github.com/tsonkk/9219542d8ff75612110f95acc2fbe761)
1 public class MyUtil {
2 static void sleep ( long millis ) {
3 try {
4 Thread . sleep ( millis ) ;
5 } catch ( I n t erruptedException ex ) {
6 ex . printStackTrace () ;
7 }
8 }
9 }

- Create ’src/MyObject.java’ file (https://gist.github.com/tsonkk/6d1b1324eb64500e6256d90137647a48)

1 public class MyObject {
2 private int value ;
3 public MyObject ( int value ) {
4 this . value = value ;
5 }
6 public void increment () {
7 value ++;
8 }
9 public void decrement () {
10 value - -;
11 }
12 public int getValue () {
13 return value ;
14 }
15 }

- Create ’src/Runner.java’ file (https://gist.github.com/tsonkk/b2047355de6e74a4332e84cc5cece082)

1 public class Runner implements Runnable {
2 private MyObject x ;
3 private boolean isIncrement ;
4
5 public Runner ( MyObject x , boolean isIncrement ) {
6 this . x = x ;
7 this . isIncrement = isIncrement ;
8 }
9 @Override
10 public void run () {
11 if ( isIncrement ) {
12 run1 () ;
13 } else {
14 run2 () ;
15 }
16 }
17 private void run1 () {
18 int r = 0;

6
 [email protected]

19 do {
20 synchronized ( x ) {
21 x . increment () ;
22 r = x . getValue () ;
23 System . out . println ( Thread . currentThread () . getName () + " : x = " + r ) ;
24 }
25 MyUtil . sleep (1000) ;
26 } while ( r < 2) ;
27 }
28 private void run2 () {
29 int r = 0;
30 do {
31 synchronized ( x ) {
32 x . decrement () ;
33 r = x . getValue () ;
34 System . out . println ( Thread . currentThread () . getName () + " : x = " + r ) ;
35 }
36 MyUtil . sleep (1000) ;
37 } while ( r > -2) ;
38 }
39 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/fac062f3835dd8404f9c51a735600ab8)

1 public class Main {
2 public static void main ( String [] args ) {
3 MyObject x = new MyObject (0) ;
4 Thread t1 = new Thread ( new Runner (x , true ) , " t1 " ) ; // increment
5 Thread t2 = new Thread ( new Runner (x , false ) , " t2 " ) ; // decrement
6 t1 . start () ;
7 t2 . start () ;
8 }
9 }

2.2.3 Starvation
- Create ’ThreadsStarvation’ project
- Create ’src/MyUtil.java’ file (https://gist.github.com/tsonkk/db071a5da426210a705f8e43e4ee203a)
1 public class MyUtil {
2 static void doSomething ( long millis ) {
3 try {
4 Thread . sleep ( millis ) ;
5 } catch ( I n t erruptedException ex ) {
6 ex . printStackTrace () ;
7 }
8 }
9 }

7
 [email protected]

- Create ’src/Runner.java’ file (https://gist.github.com/tsonkk/acf1e1b96d4b0e1ea856db5ee7035dff )

1 public class Runner implements Runnable {
2 private Object obj ;
3
4 public Runner ( Object obj ) {
5 this . obj = obj ;
6 }
7 @Override
8 public void run () {
9 System . out . println ( Thread . currentThread () . getName () + " acquiring lock on " + obj ) ;
10 synchronized ( obj ) {
11 System . out . println ( Thread . currentThread () . getName () + " acquired lock on " + obj ) ;
12 MyUtil . doSomething (3000) ;
13 while ( true ) { } // infinite loop
14 }
15 // System . out . println ( Thread . currentThread () . getName () + " released lock on " + obj ) ;
16 }
17 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/d8733e5b1ea607958092e92416419a6d)

1 public class Main {
2 public static void main ( String [] args ) {
3 Object obj = new Object () ;
4 Thread t1 = new Thread ( new Runner ( obj ) , " t1 " ) ;
5 Thread t2 = new Thread ( new Runner ( obj ) , " t2 " ) ;
6 /* Note that this situation is not a deadlock situation , in deadlock two threads are
waiting for each other forever . But in this case , only the second thread is waiting for
completion of the first thread execution so that it can get the lock of the object .
The first thread is not waiting for any resources or any thread to complete execution ,
it is doing its own job ( i . e . executing infinite while loop ) . */
7 t1 . start () ;
8 t2 . start () ;
9 }
10 }

2.3 Isolation
- Create ’ThreadsIsolation’ project
- Create ’src/Account.java’ file (https://gist.github.com/tsonkk/dafc2d237972261dbe4c8a91a808a42b)
1 public class Account {
2 String name ;
3 int balance ;
4
5 public Account ( int balance ) {
6 this . balance = balance ;
7 }
8 }

- Create ’src/Runner.java’ file (https://gist.github.com/tsonkk/64cbc2bc124713cbb37dbb46c5024680)

1 public class Runner implements Runnable {
2 private Account sender ;
3 private Account receiver ;
4 private int amount ;
5
6 public Runner ( Account sender , Account receiver , int amount ) {
7 this . sender = sender ;

8
 [email protected]

8 this . receiver = receiver ;

9 this . amount = amount ;
10 }
11 @Override
12 public void run () {
13 System . out . println ( Thread . currentThread () . getName () + " is transferring ... " ) ;
14 sender . balance -= amount ;
15 receiver . balance += amount ;
16 System . out . println ( Thread . currentThread () . getName () + " is done ! " ) ;
17 }
18 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/b6aab4cf37d59da16f3d5b26eeb625e6)

1 public class Main {
2 public static void main ( String [] args ) {
3 Account myown = new Account (500) ;
4 Account family = new Account (1000) ;
5 Account daughter = new Account (0) ;
6 for ( int r = 1; r <= 10; r ++) {
7 Thread t1 = new Thread ( new Runner ( myown , family , 100) , " t1 " ) ;
8 Thread t2 = new Thread ( new Runner ( family , daughter , 100) , " t2 " ) ;
9 t1 . start () ;
10 t2 . start () ;
11 if ( family . balance != 1000) {
12 System . out . printf ( " [ R % d ] Family 's balance : % d \ n " , r , family . balance ) ;
13 break ;
14 } else {
15 System . out . printf ( " [ R % d ] Family 's balance : % d \ n " , r , family . balance ) ;
16 }
17 }
18 }
19 }

- Download the ’pcdp-core-0.0.4-SNAPSHOT.jar’ file from: https://drive.google.com/file/d/1703G0YNr1kPXlY7iW5GkkfEkEm3

view?usp=sharing, and copy it into ’libs’ folder

- Right-click project / Build Path / Configure Build Path...

• Libraries tab: Classpath / Add JARs...

• Browse to the ’libs/pcdp-core-0.0.4-SNAPSHOT.jar’ file

- Update ’src/Runner.java’ file (https://gist.github.com/tsonkk/a5d89cd3c164fb893b3265c6e10acbce)
1 import static edu . rice . pcdp . PCDP . isolated ;
2
3 public class Runner implements Runnable {
4 ...

9
 [email protected]

5 @Override
6 public void run () {
7 ...
8 /* sender . balance -= amount ;
9 receiver . balance += amount ; */
10 isolated (() -> {
11 sender . balance -= amount ;
12 receiver . balance += amount ;
13 }) ;
14 ...
15 }
16 }

2.4 Dining Philosophers

- Create ’DiningPhilosophers’ project
- Create ’src/Philosopher.java’ file (https://gist.github.com/tsonkk/5aef5f535b011e71ab16d2898481790a)
1 public class Philosopher implements Runnable {
2 // the forks on either side of this Philosopher
3 private Object leftFork ;
4 private Object rightFork ;
5
6 public Philosopher ( Object leftFork , Object rightFork ) {
7 this . leftFork = leftFork ;
8 this . rightFork = rightFork ;
9 }
10 @Override
11 public void run () {
12 for ( int t = 1; t <= 100; t ++) {
13 // thinking
14 doAction ( " Thinking " + t ) ;
15 synchronized ( leftFork ) {
16 doAction ( " Picked up left fork " ) ;
17 synchronized ( rightFork ) {
18 doAction ( " Picked up right fork = > Eating " ) ;
19 // eating
20 doAction ( " Put down right fork " ) ;
21 }
22 // back to thinking
23 doAction ( " Put down left fork = > Back to thinking " ) ;
24 }
25 }
26 }
27 private void doAction ( String action ) {
28 System . out . println ( Thread . currentThread () . getName () + " : " + action ) ;
29 try {
30 Thread . sleep (( int ) ( Math . random () * 10) ) ;

10
 [email protected]

31 } catch ( I n t erruptedException ex ) {
32 ex . printStackTrace () ;
33 }
34 }
35 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/5b3f7b23d838d221c237efcda93118f7)

1 public class Main {
2 public static void main ( String [] args ) {
3 final int n = 5;
4 // create forks
5 Object [] forks = new Object [ n ];
6 for ( int i = 0; i < forks . length ; i ++) {
7 forks [ i ] = new Object () ;
8 }
9 // create philosophers
10 Philosopher [] philosophers = new Philosopher [ n ];
11 // maybe deadlock
12 for ( int i = 0; i < philosophers . length ; i ++) {
13 Object leftFork = forks [ i ];
14 Object rightFork = forks [( i + 1) % forks . length ];
15 philosophers [ i ] = new Philosopher ( leftFork , rightFork ) ;
16 Thread t = new Thread ( philosophers [ i ] , " Philosopher " + ( i + 1) ) ;
17 t . start () ;
18 }
19 }
20 }

3 Parallel Programming
3.1 Task Parallelism
- Create ’ForkJoinReciprocalArraySum’ project
- Create ’src/SumArray.java’ file (https://gist.github.com/tsonkk/a54877bdeb18df7a54d14efe7203370c)
1 import java . util . concurrent . RecursiveAction ;
2
3 public class SumArray extends RecursiveAction {
4 static int S E Q UENTIAL_THRESHOLD = 1000;
5 int lo , hi ;
6 double [] arr ;
7 double ans ;
8
9 public SumArray ( double [] a , int l , int h ) {
10 lo = l ;
11 hi = h ;
12 arr = a ;
13 }
14 @Override
15 protected void compute () {
16 if ( hi - lo <= SEQUENTIAL_THRESHOLD ) {

11
 [email protected]

17 for ( int i = lo ; i < hi ; i ++) {

18 ans += 1 / arr [ i ];
19 }
20 } else {
21 SumArray left = new SumArray ( arr , lo , ( hi + lo ) / 2) ;
22 SumArray right = new SumArray ( arr , ( hi + lo ) / 2 , hi ) ;
23 left . fork () ; // invoke left . compute ()
24 right . compute () ;
25 left . join () ; // wait for left to complete
26 this . ans = left . ans + right . ans ;
27 }
28 }
29 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/44bfa7bdb9203cae54dcada4032794e3)

1 import java . util . concurrent . ForkJoinPool ;
2
3 public class Main {
4 private static double seqArraySum ( double [] X ) {
5 long startTime = System . nanoTime () ;
6 double sum = 0;
7 for ( int i = 0; i < X . length ; i ++) {
8 sum += 1 / X [ i ];
9 }
10 long timeNanos = System . nanoTime () - startTime ;
11 printResults ( " seqArraySum " , timeNanos , sum ) ;
12 return sum ;
13 }
14 private static double parArraySum ( double [] X ) {
15 long startTime = System . nanoTime () ;
16 SumArray t = new SumArray (X , 0 , X . length ) ;
17 int numTasks = Runtime . getRuntime () . availableProcessors () ;
18 new ForkJoinPool ( numTasks ) . invoke ( t ) ; // invoke t . compute ()
19 double sum = t . ans ;
20 long timeNanos = System . nanoTime () - startTime ;
21 printResults ( " parArraySum " , timeNanos , sum ) ;
22 return sum ;
23 }
24 private static void printResults ( String name , long timeInNanos , double sum ) {
25 System . out . printf ( " % s completed in %8.3 f milliseconds , with sum = %8.5 f \ n " , name ,
timeInNanos / 1 e6 , sum ) ;
26 }
27 public static void main ( String [] args ) {
28 // initialization
29 int n = 100 _000_000 ;
30 double [] X = new double [ n ];
31 for ( int i = 0; i < n ; i ++) {
32 X [ i ] = ( i + 1) ;
33 }
34 // run test
35 for ( int numRun = 1; numRun <= 5; numRun ++) {
36 System . out . printf ( " Run % d \ n " , numRun ) ;
37 seqArraySum ( X ) ;
38 parArraySum ( X ) ;
39 }
40 }
41 }

12
 [email protected]

- Note that the results of the first/second run should be ignored because a JVM warm-up phase is needed, see
more details at:

• https://github.com/biancama/Scala Parallel Programming/wiki/Benchmarking-Parallel-Programs

• https://renaissance.dev/resources/docs/renaissance-suite.pdf

3.2 Functional Parallelism

3.2.1 Futures
- Create ’ForkJoinReciprocalArraySumFuture’ project
- Create ’src/SumArray.java’ file (https://gist.github.com/tsonkk/b381f27cf24c9bef0ab4a71199d33a11)
1 import java . util . concurrent . RecursiveTask ;
2
3 public class SumArray extends RecursiveTask < Double > {
4 static int S E Q UENTIAL_THRESHOLD = 1000;
5 int lo , hi ;
6 double [] arr ;
7
8 public SumArray ( double [] a , int l , int h ) {
9 lo = l ;
10 hi = h ;
11 arr = a ;
12 }
13 @Override
14 protected Double compute () {
15 if ( hi - lo <= SEQUENTIAL_THRESHOLD ) {
16 double sum = 0;
17 for ( int i = lo ; i < hi ; i ++) {
18 sum += 1 / arr [ i ];
19 }
20 return sum ;
21 } else {
22 SumArray left = new SumArray ( arr , lo , ( hi + lo ) / 2) ;
23 SumArray right = new SumArray ( arr , ( hi + lo ) / 2 , hi ) ;
24 left . fork () ; // invoke left . compute ()
25 double rightSum = right . compute () ;
26 double leftSum = left . join () ; // wait for left to complete
27 return leftSum + rightSum ;
28 }
29 }
30 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/863001b1fb22ddd38371dc11703821b1)

1 import java . util . concurrent . ForkJoinPool ;
2

13
 [email protected]

3 public class Main {

4 private static double seqArraySum ( double [] X ) {
5 long startTime = System . nanoTime () ;
6 double sum = 0;
7 for ( int i = 0; i < X . length ; i ++) {
8 sum += 1 / X [ i ];
9 }
10 long timeNanos = System . nanoTime () - startTime ;
11 printResults ( " seqArraySum " , timeNanos , sum ) ;
12 return sum ;
13 }
14 private static double parArraySumFuture ( double [] X ) {
15 long startTime = System . nanoTime () ;
16 SumArray t = new SumArray (X , 0 , X . length ) ;
17 int numTasks = Runtime . getRuntime () . availableProcessors () ;
18 double sum = new ForkJoinPool ( numTasks ) . invoke ( t ) ; // invoke t . compute ()
19 long timeNanos = System . nanoTime () - startTime ;
20 printResults ( " parArraySum " , timeNanos , sum ) ;
21 return sum ;
22 }
23 private static void printResults ( String name , long timeInNanos , double sum ) {
24 System . out . printf ( " % s completed in %8.3 f milliseconds , with sum = %8.5 f \ n " , name ,
timeInNanos / 1 e6 , sum ) ;
25 }
26 public static void main ( String [] args ) {
27 // initialization
28 int n = 100 _000_000 ;
29 double [] X = new double [ n ];
30 for ( int i = 0; i < n ; i ++) {
31 X [ i ] = ( i + 1) ;
32 }
33 // run test
34 for ( int numRun = 1; numRun <= 5; numRun ++) {
35 System . out . printf ( " Run % d \ n " , numRun ) ;
36 seqArraySum ( X ) ;
37 par ArrayS umFuture ( X ) ;
38 }
39 }
40 }

3.2.2 Streams
- Create ’StreamStudentAnalytics’ project
- Create ’src/Student.java’ file (https://gist.github.com/tsonkk/2d7c5b53294b4e516e65bec4c8d4fea8)
1 public class Student {
2 int id ;
3 boolean isCurrent ;

14
 [email protected]

4 double age ;
5 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/14f724b04a553dba56a0e2160d20e129)

1 import java . util . ArrayList ;
2 import java . util . List ;
3 import java . util . Random ;
4 import java . util . stream . Stream ;
5
6 public class Main {
7 private static double seqIteration ( Student [] studentArray ) {
8 long startTime = System . nanoTime () ;
9 List < Student > activeStudents = new ArrayList < Student >() ;
10 for ( Student s : studentArray ) {
11 if ( s . isCurrent == true ) {
12 activeStudents . add ( s ) ;
13 }
14 }
15 long ageSum = 0;
16 for ( Student a : activeStudents ) {
17 ageSum += a . age ;
18 }
19 double retVal = ( double ) ageSum / ( double ) activeStudents . size () ;
20 long timeInNanos = System . nanoTime () - startTime ;
21 printResults ( " seqIteration " , timeInNanos , retVal ) ;
22 return retVal ;
23 }
24 private static double parStream ( Student [] studentArray ) {
25 long startTime = System . nanoTime () ;
26 double retVal = Stream . of ( studentArray )
27 . parallel ()
28 . filter ( s -> s . isCurrent == true )
29 . mapToDouble ( a -> a . age )
30 . average ()
31 . getAsDouble () ;
32 long timeInNanos = System . nanoTime () - startTime ;
33 printResults ( " parStream " , timeInNanos , retVal ) ;
34 return retVal ;
35 }
36 private static void printResults ( String name , long timeInNanos , double avg ) {
37 System . out . printf ( " % s \ t completed in %8.3 f milliseconds , with average = %8.5 f \ n " , name ,
timeInNanos / 1 e6 , avg ) ;
38 }
39 public static void main ( String [] args ) {
40 final Random myRand = new Random () ;
41 // initialization
42 int n = 10 _000_000 ;
43 Student [] studentArray = new Student [ n ];
44 for ( int i = 0; i < n ; i ++) {
45 studentArray [ i ] = new Student () ;
46 studentArray [ i ]. id = i + 1;
47 studentArray [ i ]. isCurrent = true ;
48 studentArray [ i ]. age = myRand . nextInt (5) + 18; // [18 -22]
49 }
50 // run test
51 for ( int numRun = 1; numRun <= 5; numRun ++) {
52 System . out . printf ( " Run % d \ n " , numRun ) ;
53 seqIteration ( studentArray ) ;
54 parStream ( studentArray ) ;
55 }
56 }
57 }

15
 [email protected]

3.3 Loop Parallelism

- Create ’ForallMatrixMultiplication’ project
- Download the ’pcdp-core-0.0.4-SNAPSHOT.jar’ file from: https://drive.google.com/file/d/1703G0YNr1kPXlY7iW5GkkfEkEm3
view?usp=sharing, and copy it into ’libs’ folder

- Right-click project / Build Path / Configure Build Path...

• Libraries tab: Classpath / Add JARs...

• Browse to the ’libs/pcdp-core-0.0.4-SNAPSHOT.jar’ file

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/c8614f6f0645479584aa61ea61ab419b)

1 import java . util . Random ;
2 import static edu . rice . pcdp . PCDP . forseq2d ;
3 import static edu . rice . pcdp . PCDP . forall2d ;
4
5 public class Main {
6 private static void seqMatrixMultiply ( double [][] A , double [][] B , double [][] C , int n ) {
7 long startTime = System . nanoTime () ;
8 /* for ( int i = 0; i < n ; i ++) {
9 for ( int j = 0; j < n ; j ++) {
10 C [ i ][ j ] = 0;
11 for ( int k = 0; k < n ; k ++) {
12 C [ i ][ j ] += A [ i ][ k ] * B [ k ][ j ];
13 }
14 }
15 } */
16 forseq2d (0 , n -1 , 0 , n -1 , (i , j ) -> {
17 C [ i ][ j ] = 0;
18 for ( int k = 0; k < n ; k ++) {
19 C [ i ][ j ] += A [ i ][ k ] * B [ k ][ j ];
20 }
21 }) ;
22 long timeInNanos = System . nanoTime () - startTime ;
23 printResults ( " seqMatrixMultiply " , timeInNanos , C [n -1][ n -1]) ;
24 }
25 private static void parMatrixMultiply ( double [][] A , double [][] B , double [][] C , int n ) {
26 long startTime = System . nanoTime () ;
27 forall2d (0 , n -1 , 0 , n -1 , (i , j ) -> {
28 C [ i ][ j ] = 0;
29 for ( int k = 0; k < n ; k ++) {
30 C [ i ][ j ] += A [ i ][ k ] * B [ k ][ j ];
31 }
32 }) ;
33 long timeInNanos = System . nanoTime () - startTime ;
34 printResults ( " parMatrixMultiply " , timeInNanos , C [n -1][ n -1]) ;
35 }
36 private static void printResults ( String name , long timeInNanos , double sum ) {

16
 [email protected]

37 System . out . printf ( " % s completed in %8.3 f milliseconds , with C [n -1][ n -1] = %8.5 f \ n " , name
, timeInNanos / 1 e6 , sum ) ;
38 }
39 public static void main ( String [] args ) {
40 final Random myRand = new Random () ;
41 // initialization
42 int n = 512;
43 double [][] A = new double [ n ][ n ];
44 double [][] B = new double [ n ][ n ];
45 double [][] C = new double [ n ][ n ];
46 /* for ( int i = 0; i < n ; i ++) {
47 for ( int j = 0; j < n ; j ++) {
48 A [ i ][ j ] = myRand . nextInt ( n ) ;
49 B [ i ][ j ] = myRand . nextInt ( n ) ;
50 C [ i ][ j ] = myRand . nextInt ( n ) ;
51 }
52 } */
53 forseq2d (0 , n -1 , 0 , n -1 , (i , j ) -> {
54 A [ i ][ j ] = myRand . nextInt ( n ) ;
55 B [ i ][ j ] = myRand . nextInt ( n ) ;
56 C [ i ][ j ] = myRand . nextInt ( n ) ;
57 }) ;
58 // run test
59 for ( int numRun = 1; numRun <= 5; numRun ++) {
60 System . out . printf ( " Run % d \ n " , numRun ) ;
61 seq Matrix Multiply (A , B , C , n ) ;
62 par Matrix Multiply (A , B , C , n ) ;
63 }
64 }
65 }

4 Distributed Programming
4.1 Client-Server
4.1.1 Unicast Communication (one-to-one)
4.1.1.1 OneServer - OneClient

- Create ’UnicastEchoServer’ project

- Create ’src/EchoServer.java’ file (https://gist.github.com/tsonkk/63992896b24da316ffb1fcfd8a0c895a)
1 import java . io . BufferedReader ;
2 import java . io . IOException ;
3 import java . io . InputStreamReader ;
4 import java . io . PrintWriter ;
5 import java . net . ServerSocket ;
6 import java . net . Socket ;

17
 [email protected]

7
8 public class EchoServer {
9 public void start ( int port ) throws IOException {
10 ServerSocket serverSocket = new ServerSocket ( port ) ;
11 System . out . println ( " Server started : " + serverSocket ) ;
12 oneClient ( serverSocket ) ;
13 serverSocket . close () ;
14 }
15 private void oneClient ( ServerSocket serverSocket ) throws IOException {
16 Socket clientSocket = serverSocket . accept () ;
17 PrintWriter out = new PrintWriter ( clientSocket . getOutputStream () , true ) ;
18 BufferedReader in = new BufferedReader ( new InputStreamReader ( clientSocket . getInputStream () )
);
19 String msg = " " ;
20 while (! " quit " . equals ( msg ) ) {
21 // receive from client
22 msg = in . readLine () . trim () ;
23 System . out . println ( clientSocket + " sent : " + msg ) ;
24 // send to client
25 out . println ( msg ) ;
26 }
27 in . close () ;
28 out . close () ;
29 clientSocket . close () ;
30 }
31 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/bcdfa45feb97e21444720fd69ee28933)

1 import java . io . IOException ;
2
3 public class Main {
4 public static void main ( String [] args ) throws IOException {
5 EchoServer server = new EchoServer () ;
6 server . start (7777) ;
7 }
8 }

- Create ’UnicastEchoClient’ project

- Create ’src/EchoClient.java’ file (https://gist.github.com/tsonkk/6854521fbb80b68bc928617387f263f1)
1 import java . io . BufferedReader ;
2 import java . io . IOException ;
3 import java . io . InputStreamReader ;
4 import java . io . PrintWriter ;
5 import java . net . Socket ;
6 import java . util . Scanner ;
7
8 public class EchoClient {
9 public void startConnection ( String ip , int port ) throws IOException {
10 Socket clientSocket = new Socket ( ip , port ) ;
11 System . out . println ( " Client started : " + clientSocket ) ;
12 PrintWriter out = new PrintWriter ( clientSocket . getOutputStream () , true ) ;
13 BufferedReader in = new BufferedReader ( new InputStreamReader ( clientSocket . getInputStream () )
);
14 Scanner sc = new Scanner ( System . in ) ;
15 String msg = " " ;
16 while (! " quit " . equals ( msg ) ) {
17 // read from console
18 System . out . print ( " Enter a message : " ) ;
19 msg = sc . nextLine () . trim () ;
20 // sent to server
21 out . println ( msg ) ;
22 // receive from server
23 String response = in . readLine () ;
24 System . out . println ( " Server response : " + response ) ;
25 }
26 sc . close () ;
27 in . close () ;

18
 [email protected]

28 out . close () ;
29 clientSocket . close () ;
30 }
31 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/072a7606b1764cacb3b0a1f7abf4df2a)

1 import java . io . IOException ;
2
3 public class Main {
4 public static void main ( String [] args ) throws IOException {
5 EchoClient client = new EchoClient () ;
6 client . startConnection ( " 127.0.0.1 " , 7777) ;
7 }
8 }

4.1.1.2 OneServer - MultiClients

- Create ’UnicastEchoServer/src/EchoClientHandler.java’ file (https://gist.github.com/tsonkk/a50a49c8b0d6ee39aa950e3e1908ba

1 import java . io . BufferedReader ;
2 import java . io . IOException ;
3 import java . io . InputStreamReader ;
4 import java . io . PrintWriter ;
5 import java . net . Socket ;
6
7 public class Ec hoClientHandler implements Runnable {
8 private Socket clientSocket ;
9
10 public Ech oClientHandler ( Socket socket ) {
11 this . clientSocket = socket ;
12 }
13 @Override
14 public void run () {
15 try {
16 PrintWriter out = new PrintWriter ( clientSocket . getOutputStream () , true ) ;
17 BufferedReader in = new BufferedReader ( new InputStreamReader ( clientSocket . getInputStream
() ) ) ;
18 String msg = " " ;
19 while (! " quit " . equals ( msg ) ) {
20 // receive from client
21 msg = in . readLine () . trim () ;
22 System . out . println ( clientSocket + " sent : " + msg ) ;
23 // send to client
24 out . println ( msg ) ;
25 }
26 in . close () ;

19
 [email protected]

27 out . close () ;
28 clientSocket . close () ;
29 } catch ( IOException ex ) {
30 ex . printStackTrace () ;
31 }
32 }
33 }

- Update ’UnicastEchoServer/src/EchoServer.java’ file (https://gist.github.com/tsonkk/f3e8d942e47c9ab61402b7d1a6d964bd)

1 ...
2 public class EchoServer {
3 public void start ( int port ) throws IOException {
4 ...
5 // oneClient ( serverSocket ) ;
6 multiClient ( serverSocket ) ;
7 ...
8 }
9 ...
10 private void multiClient ( ServerSocket serverSocket ) throws IOException {
11 while ( true ) {
12 Socket clientSocket = serverSocket . accept () ;
13 new Thread ( new EchoClientHandler ( clientSocket ) ) . start () ;
14 }
15 }
16 }

4.1.2 Broadcast Communication (one-to-all)

4.1.2.1 OneIP - MultiPorts

- Create ’BroadcastMessageServer’ project

20
 [email protected]

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/e877baa9f70f8444c4403712e61f1f4a)

1 import java . net . DatagramPacket ;
2 import java . net . DatagramSocket ;
3 import java . net . InetAddress ;
4 import java . util . ArrayList ;
5 import java . util . Arrays ;
6 import java . util . List ;
7
8 public class Main {
9 public static void main ( String [] args ) throws Exception {
10 DatagramSocket dgSocket = new DatagramSocket () ;
11 System . out . println ( " Server started with port = " + dgSocket . getLocalPort () ) ;
12 String msg = " This is a message from server " ;
13 send1 ( dgSocket , msg ) ;
14 dgSocket . close () ;
15 }
16 private static void send1 ( DatagramSocket dgSocket , String msg ) throws Exception {
17 List < Integer > ports = new ArrayList < Integer >( Arrays . asList (7777 , 7778 , 7779) ) ;
18 for ( int port : ports ) {
19 DatagramPacket dgPacket = new DatagramPacket ( msg . getBytes () , msg . length () , InetAddress .
getByName ( " 127.0.0.1 " ) , port ) ;
20 dgSocket . send ( dgPacket ) ; // send message to clients
21 System . out . println ( " Server sent a message to client with port = " + port ) ;
22 }
23 }
24 }

- Create ’BroadcastMessageClient’ project

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/895aa89116ce6fea216f6b857ede8903)
1 import java . net . DatagramPacket ;
2 import java . net . DatagramSocket ;
3
4 public class Main {
5 public static void main ( String [] args ) throws Exception {
6 int port = 7777;
7 DatagramSocket dgSocket = new DatagramSocket ( port ) ;
8 System . out . println ( " Client started with port = " + dgSocket . getLocalPort () ) ;
9 byte [] buf = new byte [1024];
10 DatagramPacket dgPacket = new DatagramPacket ( buf , 1024) ;
11 dgSocket . receive ( dgPacket ) ; // wait for message from server
12 String msg = new String ( dgPacket . getData () , 0 , dgPacket . getLength () ) ;
13 System . out . println ( msg ) ;
14 dgSocket . close () ;
15 }
16 }

- Run 2 clients with port = 7777 | 7778 first, and then run server

21
 [email protected]

4.1.2.2 MultiIPs - OnePort

- Update ’BroadcastMessageServer/src/Main.java’ file (https://gist.github.com/tsonkk/22e08bca00593755644e809ed8f42a03)

1 ...
2 public class Main {
3 public static void main ( String [] args ) throws Exception {
4 ...
5 // send1 ( dgSocket , msg ) ;
6 send2 ( dgSocket , msg ) ;
7 ...
8 }
9 ...
10 private static void send2 ( DatagramSocket dgSocket , String msg ) throws Exception {
11 List < String > ips = new ArrayList < String >( Arrays . asList ( " 192.168.1.2 " , " 192.168.1.3 " , "
192.168.1.4 " ) ) ;
12 for ( String ip : ips ) {
13 DatagramPacket dgPacket = new DatagramPacket ( msg . getBytes () , msg . length () , InetAddress .
getByName ( ip ) , 7777) ;
14 dgSocket . send ( dgPacket ) ; // send message to clients
15 System . out . println ( " Server sent a message to client with IP = " + ip ) ;
16 }
17 }
18 }

- Run 2 clients with IP = 192.168.1.2 | 192.168.1.3 first, and then run server

4.1.3 Remote Method Invocation

- Create ’RmiAdderShared’ project
- Create ’src/IAdder.java’ file (https://gist.github.com/tsonkk/5ed55769b69b12ceff8597168bb75faa)
1 import java . rmi . Remote ;
2 import java . rmi . RemoteException ;
3
4 public interface IAdder extends Remote {
5 public int add ( int x , int y ) throws RemoteException ;
6 }

- Create ’RmiAdderServer’ project

- Right-click project / Build Path / Configure Build Path...
• Libraries tab: Classpath / Add Class Folder...

22
 [email protected]

• check ’RmiAdderShared/bin’ folder

- Create ’src/RAdder.java’ file (https://gist.github.com/tsonkk/07e99d4a08778cb2376ce073cfd72aa3)
1 import java . rmi . RemoteException ;
2
3 public class RAdder implements IAdder {
4 @Override
5 public int add ( int x , int y ) throws RemoteException {
6 System . out . println ( " x = " + x + " and y = " + y ) ;
7 return x + y ;
8 }
9 }

- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/29b0ca6627bd8ec18f5f9fda43a7c3b6)

1 import java . rmi . Naming ;
2 import java . rmi . registry . LocateRegistry ;
3 import java . rmi . server . UnicastRemoteObject ;
4
5 public class Main {
6 public static void main ( String [] args ) throws Exception {
7 String host = " 127.0.0.1 " ;
8 int port = 1099;
9 System . out . println ( " Server started ... " ) ;
10 RAdder adder = new RAdder () ; // remote object
11 IAdder iAdder = ( IAdder ) UnicastRemoteObject . exportObject ( adder , port ) ; // exporting the
remote object to the stub object
12 LocateRegistry . createRegistry ( port ) ; // creates a registry to which stubs can be bound by
server and discovered by clients
13 Naming . rebind ( " rmi :// " + host + " : " + port + " / adder " , iAdder ) ; // binding the stub object
14 System . out . println ( " Registed adder ... " ) ;
15 }
16 }

- Create ’RmiAdderClient’ project

- Right-click project / Build Path / Configure Build Path...
• Libraries tab: Classpath / Add Class Folder...
• check ’RmiAdderShared/bin’ folder
- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/8a76bf4fde5acddf2f44bc3088a99226)
1 import java . rmi . Naming ;
2 import java . util . Scanner ;
3
4 public class Main {
5 public static void main ( String [] args ) throws Exception {
6 Scanner sc = new Scanner ( System . in ) ;
7 System . out . print ( " Enter x : " ) ;
8 int x = sc . nextInt () ;
9 System . out . print ( " Enter y : " ) ;
10 int y = sc . nextInt () ;
11 sc . close () ;
12 String host = " 127.0.0.1 " ;
13 int port = 1099;
14 IAdder iAdder = ( IAdder ) Naming . lookup ( " rmi :// " + host + " : " + port + " / adder " ) ; // looking
up the stub object
15 System . out . println ( x + " + " + y + " = " + iAdder . add (x , y ) ) ;
16 }
17 }

23
 [email protected]

4.2 Message Passing Interface

4.2.1 Getting Started
- Download and unzip the MPJ Express Software (version 0.44 - updated: 18-04-2015) from: http://mpjexpress.org/
download.php

- Create ’MpiHello’ project

- Copy the ’$MPJ-v0 44 PATH$/lib/mpj.jar’ file into ’libs’ folder
- Right-click project / Build Path / Configure Build Path...
• Libraries tab: Classpath / Add JARs...
• Browse to the ’libs/mpj.jar’ file
- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/a2e7dd4d0d119ae085ef8e9b460e6cef )
1 import mpi . MPI ;
2
3 public class Main {
4 public static void main ( String [] args ) {
5 MPI . Init ( args ) ;
6 int myrank = MPI . COMM_WORLD . Rank () ;
7 System . out . println ( " Hello MPI from [ R " + myrank + " ] " ) ;
8 MPI . Finalize () ;
9 }
10 }

- Right-click ’src/Main.java’ file / Run As / Run Configurations...

• Filter text: Java Application / New launch configuration
• Name: MpiHello
• Environment tab / Add
– Name: MPJ HOME
– Value: Variables...
∗ Edit Variables... / New...
· Name: MPJ HOME
· Value: browse to $MPJ-v0 44 PATH$
∗ Choose a variable: MPJ HOME
• Arguments tab:
– VM arguments: -jar ${MPJ HOME}/lib/starter.jar

24
 [email protected]

4.2.2 Point-to-Point Communication

- Create ’MpiSendReceive’ project
- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/8183c529ae4d87fb203d059781ffb74d)
1 import mpi . MPI ;
2
3 public class Main {
4 public static void main ( String [] args ) {
5 MPI . Init ( args ) ;
6 int myrank = MPI . COMM_WORLD . Rank () ;
7 if ( myrank == 0) {
8 String [] msg = { " abcd " };
9 MPI . COMM_WORLD . Send ( msg , 0 , msg . length , MPI . OBJECT , 1 , 7777) ; // 7777 is the tag of once
send - receive
10 System . out . println ( " [ R " + myrank + " ] sent : " + msg [0]) ;
11 int [] x = { 99 };
12 MPI . COMM_WORLD . Send (x , 0 , 1 , MPI . INT , 2 , 7778) ; // 7778 is the tag of once send - receive
13 System . out . println ( " [ R " + myrank + " ] sent : " + x [0]) ;
14 } else if ( myrank == 1) {
15 String [] msg = new String [1];
16 MPI . COMM_WORLD . Recv ( msg , 0 , msg . length , MPI . OBJECT , 0 , 7777) ;
17 System . out . println ( " [ R " + myrank + " ] received : " + msg [0]) ;
18 } else if ( myrank == 2) {
19 int [] x = new int [1];
20 MPI . COMM_WORLD . Recv (x , 0 , 1 , MPI . INT , 0 , 7778) ;
21 System . out . println ( " [ R " + myrank + " ] received : " + x [0]) ;
22 } else {
23 System . out . println ( " [ R " + myrank + " ] " ) ;
24 }
25 MPI . Finalize () ;
26 }
27 }

4.2.3 Collective Communication

- Create ’MpiBcastReduceGather’ project
- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/a09ceb108aa186309fe5c9b256262f2d)
1 import mpi . MPI ;
2
3 public class Main {
4 public static void main ( String [] args ) {
5 MPI . Init ( args ) ;
6 int myrank = MPI . COMM_WORLD . Rank () ;
7 // broadcast
8 int [] x = new int [1];
9 if ( myrank == 0) {
10 x [0] = 99;
11 }
12 MPI . COMM_WORLD . Bcast (x , 0 , 1 , MPI . INT , 0) ; // broadcast integer from R0
13 System . out . println ( " [ R " + myrank + " ]: " + x [0]) ;
14 // reduce

25
 [email protected]

15 int [] y = { myrank };
16 int [] z = new int [1];
17 MPI . COMM_WORLD . Reduce (y , 0 , z , 0 , 1 , MPI . INT , MPI . SUM , 2) ; // reduce sum of ranks to R2
18 if ( myrank == 2) {
19 System . out . println ( " [ R " + myrank + " }: " + z [0]) ;
20 }
21 // gather
22 int [] yy = { myrank };
23 int [] zz = new int [ MPI . COMM_WORLD . Size () ];
24 MPI . COMM_WORLD . Gather ( yy , 0 , 1 , MPI . INT , zz , 0 , 1 , MPI . INT , 1) ; // gather ranks to R1
25 if ( myrank == 1) {
26 System . out . print ( " [ R " + myrank + " ]: " ) ;
27 for ( int ii : zz ) {
28 System . out . print ( ii + " " ) ;
29 }
30 }
31 MPI . Finalize () ;
32 }
33 }

4.2.4 Reciprocal ArraySum

- Create ’MpiReciprocalArraySum’ project
- Create ’src/Main.java’ file (https://gist.github.com/tsonkk/87b350682fdeacc84aaf4ec1955f8d62)
1 import mpi . MPI ;
2
3 public class Main {
4 public static void main ( String [] args ) {
5 // init data
6 int n = 10 _000_000 ;
7 double [] X = new double [ n ];
8 for ( int i = 0; i < n ; i ++) {
9 X [ i ] = ( i + 1) ;
10 }
11 // mpi
12 MPI . Init ( args ) ;
13 int myrank = MPI . COMM_WORLD . Rank () ;
14 int size = MPI . COMM_WORLD . Size () ;
15 // broadcast
16 MPI . COMM_WORLD . Bcast (X , 0 , X . length , MPI . DOUBLE , 0) ; // broadcast X from R0
17 // System . out . printf ("[ R " + myrank + "]: [%.0 f - %.0 f ]\ n " , X [0] , X [ X . length - 1]) ; // for
DEBUG
18 // calculate sum ( from , to ) on all ranks
19 int from = ( myrank == 0) ? 0 : (( X . length / size ) * myrank ) ;
20 int to = ( myrank == size - 1) ? ( X . length - 1) : (( X . length / size ) * ( myrank + 1) - 1) ;
21 double sum = 0;
22 for ( int i = from ; i <= to ; i ++) {
23 sum += 1 / X [ i ];
24 }
25 // System . out . printf ("[ R " + myrank + "]: %8.5 f \ n " , sum ) ; // for DEBUG
26 double [] sumOnRank = { sum };

26
 [email protected]

27 // reduce
28 double [] sumReduce = new double [1];
29 MPI . COMM_WORLD . Reduce ( sumOnRank , 0 , sumReduce , 0 , 1 , MPI . DOUBLE , MPI . SUM , 0) ; // reduce
sumOnRank to R0
30 if ( myrank == 0) {
31 System . out . printf ( " [ R " + myrank + " ]: %8.5 f \ n " , sumReduce [0]) ; // for DEBUG
32 }
33 // gather
34 double [] sumGather = new double [ size ];
35 MPI . COMM_WORLD . Gather ( sumOnRank , 0 , 1 , MPI . DOUBLE , sumGather , 0 , 1 , MPI . DOUBLE , 1) ; //
gather sumOnRank to R1
36 double sumTotal = 0;
37 for ( double ii : sumGather ) {
38 sumTotal += ii ;
39 }
40 if ( myrank == 1) {
41 System . out . printf ( " [ R " + myrank + " ]: %8.5 f \ n " , sumTotal ) ; // for DEBUG
42 }
43 MPI . Finalize () ;
44 }
45 }

5 Exercises
5.1 Exercise Deadlock
- Preventing deadlocks for the Dining Philosophers problem.

- Update ’DiningPhilosophers/src/Main.java’ file

1 public class Main {
2 public static void main ( String [] args ) {
3 final int n = 5;
4 // create forks
5 Object [] forks = new Object [ n ];
6 for ( int i = 0; i < forks . length ; i ++) {
7 forks [ i ] = new Object () ;
8 }
9 // create philosophers

27
 [email protected]

10 Philosopher [] philosophers = new Philosopher [ n ];

11 // TODO resolve deadlock
12 }
13 }

5.2 Exercise Futures

- Counting the number of primes less than or equal to the given integer N , for example, N = 10.000.000.

- Create ’ForkJoinPrimeCountingFuture’ project

- Create ’src/MyUtil.java’ file
1 public class MyUtil {
2 static boolean isPrime ( int x ) {
3 if ( x < 2) return false ;
4 for ( int i = 2; i <= Math . sqrt ( x ) ; i ++) {
5 if ( x % i == 0) {
6 return false ;
7 }
8 }
9 return true ;
10 }
11 }

- Create ’src/CountPrimes.java’ file

1 import java . util . concurrent . RecursiveTask ;
2
3 public class CountPrimes extends RecursiveTask < Integer > {
4 static int S E Q UENTIAL_THRESHOLD = 1000;
5 int lo , hi ;
6
7 public CountPrimes ( int l , int h ) {
8 lo = l ;
9 hi = h ;
10 }
11 @Override
12 protected Integer compute () {
13 if ( hi - lo <= SEQUENTIAL_THRESHOLD ) {
14 // TODO
15 } else {
16 // TODO
17 }
18 }
19 }

- Create ’src/Main.java’ file

28
 [email protected]

1 import java . util . concurrent . ForkJoinPool ;

2
3 public class Main {
4 private static double seqArraySum ( int n ) {
5 long startTime = System . nanoTime () ;
6 int count = 0;
7 for ( int i = 0; i <= n ; i ++) {
8 if ( MyUtil . isPrime ( i ) ) {
9 count ++;
10 }
11 }
12 long timeNanos = System . nanoTime () - startTime ;
13 printResults ( " seqArraySum " , timeNanos , count ) ;
14 return count ;
15 }
16 private static double parArraySumFuture ( int n ) {
17 long startTime = System . nanoTime () ;
18 // TODO
19 long timeNanos = System . nanoTime () - startTime ;
20 printResults ( " parArraySum " , timeNanos , count ) ;
21 return count ;
22 }
23 private static void printResults ( String name , long timeInNanos , int count ) {
24 System . out . printf ( " % s completed in %8.3 f milliseconds , with count = % d \ n " , name ,
timeInNanos / 1 e6 , count ) ;
25 }
26 public static void main ( String [] args ) {
27 // initialization
28 int n = 10 _000_000 ; // 10 _000_019
29 // run test
30 for ( int numRun = 1; numRun <= 5; numRun ++) {
31 System . out . printf ( " Run % d \ n " , numRun ) ;
32 seqArraySum ( n ) ;
33 par ArrayS umFuture ( n ) ;
34 }
35 }
36 }

5.3 Exercise Streams

- Counting the number of active books and finding the min, max, and average price of active books in the given
N books, for example, N = 100.000.000. Note that the activation and price of the books are randomly generated
in the range [10–20].

- Create ’StreamBookAnalytics’ project

- Create ’src/Student.java’ file
1 public class Book {

29
 [email protected]

2 int id ;
3 boolean isActive ;
4 int price ;
5 }

- Create ’src/Main.java’ file

1 import java . util . ArrayList ;
2 import java . util . List ;
3 import java . util . Random ;
4 import java . util . function . Supplier ;
5 import java . util . stream . IntStream ;
6 import java . util . stream . Stream ;
7
8 public class Main {
9 private static void seqIteration ( Book [] bookArray ) {
10 long startTime = System . nanoTime () ;
11 List < Book > activeBooks = new ArrayList < Book >() ;
12 for ( Book b : bookArray ) {
13 if ( b . isActive == true ) {
14 activeBooks . add ( b ) ;
15 }
16 }
17 int count = activeBooks . size () ;
18 int min = Integer . MAX_VALUE , max = Integer . MIN_VALUE ;
19 long sum = 0;
20 for ( Book a : activeBooks ) {
21 if ( a . price < min ) {
22 min = a . price ;
23 }
24 if ( a . price > max ) {
25 max = a . price ;
26 }
27 sum += a . price ;
28 }
29 double avg = sum * 1.0 / count ;
30 long timeInNanos = System . nanoTime () - startTime ;
31 printResults ( " seqIteration " , timeInNanos , count , min , max , avg ) ;
32 }
33 private static void parStream ( Book [] bookArray ) {
34 long startTime = System . nanoTime () ;
35 // TODO
36 long timeInNanos = System . nanoTime () - startTime ;
37 printResults ( " parStream " , timeInNanos , count , min , max , avg ) ;
38 }
39 private static void printResults ( String name , long timeInNanos , int count , int min , int max ,
double avg ) {
40 System . out . printf ( " % s \ t completed in %8.3 f milliseconds , with count = %d , min = %d , max =
%d , average = %8.5 f \ n " , name , timeInNanos / 1 e6 , count , min , max , avg ) ;
41 }
42 public static void main ( String [] args ) {
43 final Random myRand = new Random () ;
44 // initialization
45 int n = 100 _000_000 ;
46 Book [] bookArray = new Book [ n ];
47 for ( int i = 0; i < n ; i ++) {
48 bookArray [ i ] = new Book () ;
49 bookArray [ i ]. id = i + 1;
50 bookArray [ i ]. isActive = myRand . nextBoolean () ;
51 bookArray [ i ]. price = myRand . nextInt (11) + 10; // [10 -20]
52 }
53 // run test
54 for ( int numRun = 1; numRun <= 5; numRun ++) {
55 System . out . printf ( " Run % d \ n " , numRun ) ;
56 seqIteration ( bookArray ) ;
57 parStream ( bookArray ) ;
58 }
59 }
60 }

30
 [email protected]

5.4 Exercise OneServer - MultiClients

- The client sends a message to the server, and the server reverses this message and sends it back to the client.
Note that the server is able to serve multiple clients simultaneously.

- Create ’UnicastReverseServer’ project

- Create ’src/ReverseClientHandler.java’ file
1 import java . io . BufferedReader ;
2 import java . io . IOException ;
3 import java . io . InputStreamReader ;
4 import java . io . PrintWriter ;
5 import java . net . Socket ;
6
7 public class Re v erseClientHandler implements Runnable {
8 private Socket clientSocket ;
9
10 public Re v e r s e ClientHandler ( Socket socket ) {
11 this . clientSocket = socket ;
12 }
13 @Override
14 public void run () {
15 try {
16 PrintWriter out = new PrintWriter ( clientSocket . getOutputStream () , true ) ;
17 BufferedReader in = new BufferedReader ( new InputStreamReader ( clientSocket . getInputStream
() ) ) ;
18 String msg = " " ;
19 while (! " quit " . equals ( msg ) ) {
20 // receive from client
21 msg = in . readLine () . trim () ;
22 System . out . println ( clientSocket + " sent : " + msg ) ;

31
 [email protected]

23 // send to client
24 // TODO
25 }
26 in . close () ;
27 out . close () ;
28 clientSocket . close () ;
29 } catch ( IOException ex ) {
30 ex . printStackTrace () ;
31 }
32 }
33 }

- Create ’src/ReverseServer.java’ file

1 import java . io . IOException ;
2 import java . net . ServerSocket ;
3 import java . net . Socket ;
4
5 public class ReverseServer {
6 public void start ( int port ) throws IOException {
7 ServerSocket serverSocket = new ServerSocket ( port ) ;
8 System . out . println ( " Server started : " + serverSocket ) ;
9 multiClient ( serverSocket ) ;
10 serverSocket . close () ;
11 }
12 private void multiClient ( ServerSocket serverSocket ) throws IOException {
13 while ( true ) {
14 Socket clientSocket = serverSocket . accept () ;
15 new Thread ( new ReverseClientHandler ( clientSocket ) ) . start () ;
16 }
17 }
18 }

- Create ’src/Main.java’ file

1 import java . io . IOException ;
2
3 public class Main {
4 public static void main ( String [] args ) throws IOException {
5 ReverseServer server = new ReverseServer () ;
6 server . start (7777) ;
7 }
8 }

- Create ’UnicastReverseClient’ project

- Create ’src/ReverseClient.java’ file
1 import java . io . BufferedReader ;
2 import java . io . IOException ;
3 import java . io . InputStreamReader ;
4 import java . io . PrintWriter ;
5 import java . net . Socket ;
6 import java . util . Scanner ;
7
8 public class ReverseClient {
9 public void startConnection ( String ip , int port ) throws IOException {
10 Socket clientSocket = new Socket ( ip , port ) ;
11 System . out . println ( " Client started : " + clientSocket ) ;
12 PrintWriter out = new PrintWriter ( clientSocket . getOutputStream () , true ) ;
13 BufferedReader in = new BufferedReader ( new InputStreamReader ( clientSocket . getInputStream () )
);
14 Scanner sc = new Scanner ( System . in ) ;
15 String msg = " " ;
16 while (! " quit " . equals ( msg ) ) {
17 // read from console
18 System . out . print ( " Enter a message : " ) ;
19 msg = sc . nextLine () . trim () ;
20 // sent to server
21 out . println ( msg ) ;

32
 [email protected]

22 // receive from server

23 String response = in . readLine () ;
24 System . out . println ( " Server response : " + response ) ;
25 }
26 sc . close () ;
27 in . close () ;
28 out . close () ;
29 clientSocket . close () ;
30 }
31 }

- Create ’src/Main.java’ file

1 import java . io . IOException ;
2
3 public class Main {
4 public static void main ( String [] args ) throws IOException {
5 ReverseClient client = new ReverseClient () ;
6 client . startConnection ( " 127.0.0.1 " , 7777) ;
7 }
8 }

5.5 Exercise Remote Method Invocation

- The server will host an object that can execute methods such as addition, subtraction, multiplication, and
division of two integers. The client can invoke the methods of this object remotely.

- Create ’RmiCalculatorShared’ project

- Create ’src/ICalculator.java’ file
1 import java . rmi . Remote ;
2 import java . rmi . RemoteException ;
3
4 public interface ICalculator extends Remote {
5 public int add ( int x , int y ) throws RemoteException ;
6 public int sub ( int x , int y ) throws RemoteException ;
7 public int mul ( int x , int y ) throws RemoteException ;
8 public double div ( int x , int y ) throws RemoteException ;
9 }

- Create ’RmiCalculatorServer’ project

- Create ’src/RCalculator.java’ file

33
 [email protected]

1 import java . rmi . RemoteException ;

2
3 public class RCalculator implements ICalculator {
4 // TODO
5 }

- Create ’src/Main.java’ file

1 import java . rmi . Naming ;
2 import java . rmi . registry . LocateRegistry ;
3 import java . rmi . server . UnicastRemoteObject ;
4
5 public class Main {
6 public static void main ( String [] args ) throws Exception {
7 String host = " 127.0.0.1 " ;
8 int port = 1099;
9 System . out . println ( " Server started ... " ) ;
10 // TODO
11 System . out . println ( " Registed calculator ... " ) ;
12 }
13 }

- Create ’RmiCalculatorClient’ project

- Create ’src/Main.java’ file
1 import java . rmi . Naming ;
2 import java . util . Scanner ;
3
4 public class Main {
5 public static void main ( String [] args ) throws Exception {
6 Scanner sc = new Scanner ( System . in ) ;
7 System . out . print ( " Enter x : " ) ;
8 int x = sc . nextInt () ;
9 System . out . print ( " Enter y : " ) ;
10 int y = sc . nextInt () ;
11 sc . close () ;
12 String host = " 127.0.0.1 " ;
13 int port = 1099;
14 // TODO
15 }
16 }

5.6 Exercise Message Passing Interface

- Counting the number of primes less than or equal to the given integer N , for example, N = 10.000.000.

- Create ’MpiPrimeCounting’ project

- Create ’src/MyUtil.java’ file
1 public class MyUtil {
2 static boolean isPrime ( int x ) {
3 if ( x < 2) return false ;
4 for ( int i = 2; i <= Math . sqrt ( x ) ; i ++) {
5 if ( x % i == 0) {
6 return false ;
7 }
8 }
9 return true ;

34
 [email protected]

10 }
11 }

- Create ’src/Main.java’ file

1 import mpi . MPI ;
2
3 public class Main {
4 public static void main ( String [] args ) {
5 // init data
6 int n = 10 _000_000 ; // 10 _000_019
7 // mpi
8 MPI . Init ( args ) ;
9 int myrank = MPI . COMM_WORLD . Rank () ;
10 int size = MPI . COMM_WORLD . Size () ;
11 // TODO
12 MPI . Finalize () ;
13 }
14 }

Acknowledgement
Many thanks to Prof. Vivek Sarkar ([email protected]), Rice University, for teaching the great specialization:

• Concurrent Programming in Java

• Parallel Programming in Java

• Distributed Programming in Java

35