Today's Material: - Medians & Order Statistics - Ch. 9

Today’s Material
• Medians & Order Statistics – Ch. 9
1
Selection: Problem Definition
• Given a sequence of numbers a1, a2, a3, …aN
and integer “i”, 1 <= i <= N, compute the ith
smallest element
• Minimum is when k = 1, maximum is when k = N

• Median is a special case where i = N/2
• Selection looks like a very mundane problem

• But it is such a basic question that it arises in
many places in practice
– Give examples…
2
Brute Force Solution
• There is an obvious brute-force solution
• Just sort the numbers in ascending order and
return the ith element of the array
• Takes O(nlogn)—Time to sort the numbers
• Can we do better?
– There is a deterministic O(n) algorithm, but it is
very complicated and not very practical
– However, there is a simple randomized algorithm,
whose expected running time is O(n)
• We will only look at this randomized algorithm--next
3
Randomized Algorithms – An Intro
• A randomized algorithm is one that incorporates
a random number generator
• Studies in recent years because many of the
practical algorithms make use of randomization
• There are 2 classes of randomized algorithms
– Monte Carlo Algorithms
• May make an error in its output, but presumably the
probability of this happening is very small
– Las Vegas Algorithms
• Always produces the correct answer, but there is a small
probability that the algorithm takes longer than it should
– With Monte Carlo algorithms randomization affects
the result, with Las Vegas it affects the running time
4
A Simple Monte-Carlo Algorithm
• Problem: Given a number N, is N prime?
– Important for cryptography
• Randomized Monte-Carlo Algorithm based on a

Result by Fermat:
– Guess a random number A, 0 < A < N
– If (AN-1 mod N) ≠ 1, then Output “N is not prime”
– Otherwise, Output “N is (probably) prime”
• N is prime with high probability but not 100%

• Can repeat steps 1-3 to make error probability
close to 0 5
A Las Vegas Randomized Selection
• As we mentioned, there is an O(n) expected-
case randomized Las-Vegas algorithm for
Selection
– Always produces the correct answer, but with low
probability it might take longer than O(n)
• Idea is based on a modification of QuickSort:

– Assume that the array A is indexed A[1..n]
– Consider the Partition() procedure in QuickSort
• Randomly choose a pivot x, and permute the elements of A
into two nonempty sublists A[1..q] of elements < x and
A[q+1..n] of elements >= x
– See page 154 of CLRS
• Assume Partition() returns the index “q”
6
Partition Algorithm
int Partition(int A[], int N){
if (N<=1) return 0;
int pivot = A[0]; // Pivot is the first element
int i=1, j=N-1;
while (1){
while (A[j]>pivot) j--; // Move j
while (A[i]<pivot && i<j) i++; // Move i
if (i>=j) break;
Swap(&A[i], &A[j]);
i++; j--;
} //end-while
Swap(&A[j], &A[0]); // Restore the pivot
return j; // return the index of the pivot

} //end-Partition
7
A Las Vegas Randomized Selection
• Observe that there are “q” elements <= pivot,
and hence the rank of the pivot is q
• If i==q then return A[q];
• If i < q then we select the ith smallest element
from the left sublist, A[1..q]
• If i > q then we recurse on the right sublist.

– Because q elements have already been eliminated,
we select the (i-q)th smallest element from the
right sublist
8
Randomized Selection: Pseudocode
// Assumes 1<= i <=N

Select(A[1..N], i){
if (N==1) return A[1];
int q = Partition(A[1..N], N);
if (i == q) return A[q];
if (i < q) return Select(A[1..q], i);
else return Select(A[q+1..N], i-q);
} //end-Select
9
Randomized Selection: C Code
// Assumes 1<= i <=N

int Select(int A[], int i, int N){
if (N==1) return A[0];
int q = Partition(A, N);
if (i == q+1) return A[q];
else if (i <= q) return Select(A, i, q);
// We have eliminated q+1 elements
else return Select(&A[q+1], i-(q+1), N-(q+1));
} //end-Select
10
Running Time - 1
• Because the algorithm is randomized, we
analyze its “expected” time complexity
– Where the expectation is taken over all possible
choices of the random pivot element
• Let T(n) denote the expected case running

time of the algorithm on a list of size “n”
– Our analysis is with respect to the worst-case in “i”
– That is, since we do not know what “i” is, we make
the worst case assumption that whenever we
partition the list, the ith smallest element occurs on
the side having greater number of elements
– Partitioning procedure takes O(n) – See CLRS
11
Running Time - 2
• There are “n” possible choices for the pivot
• Each is equally likely with probability 1/n
• If “x” is the kth smallest element of the list, then
we create two sublists of size “k” and “n-k”
• If we assume that we recurse on the larger side of
the two sublists, then we get
1 n 1
– T(n) <= [  T (max(k , n  k ))]  n
n k 1
– Basically, the recurrence can be simplified to:
2 n 1
– T(n) <= [  T (k )]  n
n k n / 2 12
Running Time - 3
• Then an induction argument is used to show
that T(n) <= c*n for some appropriately chosen
constant c
• After working through the induction proof
(see page 189 in CLRS), we arrive at the
condition
– c*(3n/4 – ½) + n < c*n
– This is satisfied for any c >= 4
– This technique of setting up an induction with an
unknown parameter, and then determining the
conditions on the parameter is known as
“constructive proof”
13
Deterministic Selection
• Once we find the median of the medians,
partition the array using the medians of the
medians
• Then run the algorithm on the partitioned
array recursively
• Basically, we want to make partitioning

deterministic by finding the median of the
medians so that the array is partitioned into
almost 2 equal halves
14
Deterministic Selection
• (1) Divide the elements into roughly n/5
groups, each of size 5
• (2) Compute the median of each group (by any
method you like)
• (3) Compute the median of these n/5 group
medians
• How do you implement step (3)?
– You call deterministic selection recursively
– Since the list is of smaller size, it will eventually
terminate
– Why groups of 5?
• You need an odd number for median computation
• 3 does not work. The smallest odd number greater than 3 is
5. But any other bigger odd number (7, 9, ..) would do too.
15

Today's Material: - Medians & Order Statistics - Ch. 9

Uploaded by

Copyright:

Available Formats

Today's Material: - Medians & Order Statistics - Ch. 9

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Today's Material: - Medians & Order Statistics - Ch. 9

Uploaded by

Copyright:

Available Formats

Today’s Material

• Medians & Order Statistics – Ch. 9

• Minimum is when k = 1, maximum is when k = N

• Selection looks like a very mundane problem

• Randomized Monte-Carlo Algorithm based on a

• N is prime with high probability but not 100%

• Idea is based on a modification of QuickSort:

Swap(&A[j], &A[0]); // Restore the pivot

return j; // return the index of the pivot

• If i > q then we recurse on the right sublist.

// Assumes 1<= i <=N

// Assumes 1<= i <=N

• Let T(n) denote the expected case running

• Basically, we want to make partitioning

You might also like