IEEE TRANSMXIONS ON MEDICAL IMAGING, VOL. 13, NO.

4, DECEMBER 1994 601

Accelerated Image Reconstruction Using

Ordered Subsets of Projection Data
H. Malcolm Hudson and Richard S . Larkin

Abstract- We define ordered subset processing for standard fast Fourier transforms. Related approaches to the solution of
algorithms (such as Expectation Maximization, EM) for image linear systems have been used in tomography (see Section IV).
restoration from projections. Ordered subsets methods group With data acquired in time order, sequential processing is
projection data into an ordered sequence of subsets (or blocks).
An iteration of ordered subsets EM is defined as a single pass also an option. General approaches to recursive estimation in
through all the subsets, in each subset using the current es- processing sequences of images are discussed by Green and
timate to initialize application of EM with that data subset. Titterington [7]. Titterington [8] has provided a recursive EM
This approach is similar in concept to block-Kaczmarz methods algorithm for sequential acquisition of data.
introduced by Eggermont et al. [l]for iterative reconstruction.
The purpose of this paper is to introduce and assess perfor-
Simultaneous iterative reconstruction (SIRT) and multiplicative
algebraic reconstruction(MART) techniques are well known spe- mance of the OS-EM algorithm and a regularized form (GP in
cial cases. Ordered subsets EM (OS-EM) provides a restoration Section V). We aim to show the acceleration of convergence
imposing a natural positivity condition and with close links to attained with OS. Section I1 defines OS-EM. Section I11
the EM algorithm. OS-EM is applicable in both single photon discusses choice of subsets and order of processing. Section
(SPECT) and positron emission tomography (PET). In simulation IV provides a parallel with iterative methods in transmission
studies in SPECT, the OS-EM algorithm provides an order-
of-magnitude acceleration over EM, with restoration quality tomography, particularly MART; EM is a simultaneous form
maintained. of OS-EM. Section V contains simulation study description
and results. Section VI provides discussion. The Appendix
contains a proof of convergence of OS-EM to a feasible
I. INTRODUCTION solution with exact projection data.

T HE application of Expectation Maximization (EM) al-

gorithms in emission tomography by Shepp and Vardi
[2] has led to the introduction of many related techniques.
11. ORDEREDSUBSETSEM
A general procedure for sequential processing of data is
These include a number of important Bayesian (or equivalently proposed. While it may be applied with any iterative algorithm,
penalized likelihood) adaptions of EM. Recent papers have we describe the application with the EM algorithm in SPECT
emphasized the quality of the reconstruction offered by these for concreteness.
algorithms. See Hebert and Leahy [3] and Green 141 for some Shepp and Vardi’s EM algorithm is implemented by it-
recent Bayesian developments, and Chomboy et al. [5] for an erations (until convergence) requiring projection and back-
evaluation of the benefits of EM in single photon emission projection calculations, as described below. An initial estimate
tomography (SPECT). of image activity is provided to begin this sequence and the
While quality of reconstruction is good, the application of calculations require specification of a projection matrix of
EM is computer intensive and convergence slow, even with weights (often probabilities) providing the link between the
standard acceleration techniques (Kaufman [ 6 ] ) .We provide unobserved image and the observed projection counts.
here an ordered subset (OS) algorithm that processes the data With OS-EM, the projection data is grouped in ordered
in subsets (blocks) within each iteration and shows that this subsets. The OS level is defined as the number of these subsets.
procedure accelerates convergence by a factor proportional to The standard EM algorithm (i.e., projection followed by back-
the number of subsets. In SPECT, the sequential processing of projection) is then applied to each of the subsets in tum, using
ordered subsets is very natural, as projection data is collected the rows of the design matrix corresponding to these ray sums.
separately for each projection angle (as a camera rotates The resulting reconstruction becomes the starting value for use
around the patient in SPECT); counts on single projections with the next subset.
can form successive subsets. We define an iteration of OS-EM as a single pass through all
Computational benefit from such “divide and conquer” the specified subsets. Further iterations may be performed by
processing may be anticipated, as in applications in sorting and passing through the same ordered subsets, using as a starting
- -
point the reconstruction provided by the previous iteration.
Manuscript received February 24, 1993; revised November 28, 1993. H. M.
Hudson was supported by Australian Research Council grants. The Associate with (and exhaustive) each Os-
Editor responsible for coordinating the review of this paper and recommending EM iteration will have a similar computation time to One
its publication was Y. Censor. standard EM iteration.
The authors are with the Department of Statistics, Macquarie University,
NSW 13, Australia. Photon recordings on gamma cameras are binned to provide
E E E Log Number 9406863. counts yt on detectors indexed by t. These recordings are the
0278-0062/94$04.00 0 1994 IEEE
602 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 13, NO. 4, DECEMBER 1994

result of emitter activity (modeled as a Poisson point process) Notes:

within a region. This activity is assumed to be uniform within 1) We shall refer to this procedure as ordered subsets
the pixels of a grid imposed on the region; the expected EM (OS-EM). Should algorithms other than EM be of
number of photon emissions from pixel j is denoted by xi. interest, the backprojection step would be replaced by
Define the image to be the vector x = {xj : j = 1, . . . , J } . the corresponding procedure for that algorithm. OS-EM
Specify individual pixel and detector elements (or rays) by includes standard EM as a particular case when a single
subscripts j and t, respectively. Weights atj represent the subset includes all projection data.
probability that an emission from pixel j is recorded at t. 2) In backprojection, (2), the scaling of the image matches
Then, detector counts are Poisson distributed with expected the activity expected to be recorded on the projection
values p = Ey = Ax,where A is the projection matrix. See and the aggregate count recorded on the projection, i.e.,
Shepp and Vardi [2] and Chomboy ef al. [5] for details of
appropriate determination of the weights making allowance C(C a
tj
,.
;. = yt-
for photon attenuation and scatter correction. j tESi t€Si
Let Po be a prespecified starting image, e.g., uniform. However, initial scaling has no effect on the result of
Denote by 2" the estimate of 2 after iteration m. Let any subsequent EM iteration.
SI,S2, .. . ,S, denote the chosen detector subsets in the order 3) If the divisor in (2) is zero, set x:" = xi.
selected. The algorithm is then as follows: 4) The difference between Vardi-Shepp and ordered subsets
1) m = 0,P" initialized, positive EM seems similar, conceptually, to the difference be-
2) Repeat until convergence of Pm tween Jacobi and Gauss-Seidel algorithms for solution
of linear equations.
a) x1 = ~ , = m m + I
b) For subsets i = 1, .. . , 7~ SUBSETSAND ORDER
111. SELECTING
project. Calculate expected values for cumu- In SPECT, subsets may correspond naturally to groups
lated counts y as of projections. While the simulations conducted subsequently
use subsets corresponding to processing of the projections in
opposing pairs, other choices may be considered. For example,
with three-headed camera systems, in which three projections
separated by 120° are recorded simultaneously, it may be
useful to use subsets containing three or six projections. We
for detectors t E Si. believe it is advantageous to select subsets in a balanced way
backproject. Calculate so that pixel activity contributes equally to any subset.
Since PET collects all tube counts simultaneously, ordered
subset EM can best be applied after full collection of counts.
The tubes may then be rebinned in parallel families defining
projections and our algorithm applied. OS-EM in PET will
for pixels j = 1,. . . ,J . then provide the same computational advantages in reducing
c) 2" = p+l, iterations as in SPECT.
The order in which projections are processed is arbitrary,
Examples: though it may be advantageous to the quality of the recon-
1) Nonoverlapping subsets: With n subsets each of T struction provided to choose a special order. For example, one
detectors per projection, set SI = { 1 , 2 , . . . ,T } ,SZ = might encourage substantial new information to be introduced
{T + 1,.. . ,2T},. . . ,S, = {(n-l)T+l,. . . , nT}, and as quickly as possible by choosing first the projection corre-
continue to cycle in turn through each of these subsets. sponding to the direction of greatest variability in the image,
Data used in each subiteration comprises counts of all a second projection perpendicular to the first, and third and
photon scintillations recorded on the first, second, . . ., fourth projections midway between these, and so on.
last projection specified. The restoration at stage (i - 1)
is modified to incorporate data acquired on projection i. IV. RELATIONSHIP
WITH MART
2) Cumulative subsets: Altematively, set 5'1 = There are evident similarities between ordered subset EM
(1 ,..., T } , S 2 = (1,..., 2") ,...,S, = (1 ,...,nT}. and the multiplicative algebraic reconstruction technique
Again, cycle in tum through each of these subsets. (MART) of Gordon, Bender, and H e r " [9], and related
Backprojected data from all projections already methods. However, ordered subsets EM differs in its detail
processed is combined to form the current restoration and is intimately related to the EM algorithm, unlike the other
of the image. This type of approach was proposed in a approaches.
more general setting by Titterhgton [SI. Darroch and Ratcliff [lo], Censor and Segman [ l l ] , and
3 ) Standard EM: With n projections of T detectors, em- Iusem and Teboulle [ 121 provide general block-iterative forms
ploying a single subset S1 = { 1,. . . ,nT} provides of the MART algorithm. OS-EM can also be regarded as
standard EM. a block iterative method. Convergence properties of Censor
HUDSON AND LARKIN: ACCELERATED IMAGE RECONSTRUCTION 603

and Segman’s generalization of MART is contained in their block iterative methods, we conclude this section by providing
Theorem 1. For linear equations for which there exists a an argument suggesting that these distinct limit points cannot
feasible (nonnegative) exact solution, mild conditions on the be very different in many SPECT and PET applications.
weights (Section 11) and starting value suffice for convergence OS-EM resolves the linear system determined by A through
of block-iterative MART to the maximum entropy solution. a partition A I , .. . , A,, as above. In the geometry applying to
Block-iterative estimates may be contrasted with simultane- SPECT and PET systems (associated with projection symme-
ous iterative reconstruction techniques, introduced by Gilbert tries), this partition has special orthogonality properties.
[ 131. Simultaneous techniques include all equations relating Subsets should be selected so that CtESi atj is independent
projection data to parameters simultaneously, whereas MART of i, so that pixel activity contributes equally to any subset.
solves the same equations by introducing subsets of the data In SPECT, with subsets comprised of equal numbers of
in subiterations (see Censor [14]). EM can be regarded as a projections, this condition is satisfied if there is no attenuation
simultaneous form of OS-EM. or if the subsets chosen provide this balance. We may then
EM solves (consistent) nonlinear normal equations derived assume CtES, atj = 1 without essential loss of generality.
by maximizing the Poisson likelihood function of the data or Suppose also that the matrix ADAT is block diagonal for
similar regularized equations. OS-EM successively increases any diagonal matrix D of positive elements. In particular,
components of the likelihood corresponding to successive data for D = diag(xi), where xi is the current estimate, assume
subsets. AkDAT = 0 for k # i, with i , k E (1,.. . ,n}.
In consistent linear systems (including exact projections), Then, subiteration i of OS-EM has no effect on fitted values
assume there exist feasible solutions to the projection equa- for any projection k # i, since on substituting for xi+’,
tions y = A z . Maximum entropy and maximum likelihood according to (2), we obtain
solutions then satisfy this linear system. OS-EM or block
iterative methods provide solutions based on a partition AT = &(Xi+’ - xi) = AkDATZ = 0,
[AT,. . . ,A:].
OS-EM iterates towards simultaneous nonnegative solution where z is the vector with components (yt - p f ) / p i , for
of the equations A i z = yi, the equations for the projections t E Si. Hence, the Poisson likelihood function for all count
defining block z, for i = 1,.. . ,n. We prove, in the Appendix, data following subiteration i is (apart from a constant)
that under simple conditions on the weights, OS-EM will
converge to a feasible solution of the linear system. L(y, z i + l ) = C { y t log p f + l - pf+l)
EM is known to exhibit linear local convergence with a
= {yt 1ogpf+l - p f + l }
rate determined the maximum eigenvalue (spectral norm) of
k tESk
the iteration matrix I - D H , where D is a diagonal matrix
whose components are the solution provided by the iterations = L(y, xi) + {yt log pf+l - p f + l }
and H is the Hessian of the likelihood function of the full tES,
data set. (See Green [15] who proves all eigenvalues of this - { Y t 1% Pf - P i } .
matrix lie in the interval [0,1).) In certain cases (strong subset tES,
balance), we can compute the local rate of convergence of
OS-EM explicitly. The second equality assumes nonoverlapping subsets. The
Strong subset balance requires equality of the Hessian third equality follows since the subiteration affects only fitted
(equivalently, Fisher-information) matrices of each data sub- values for subset Si. But the final two terms of the right hand
set’s likelihood function, i.e., H I = . . . = H , and H = nH1. side provide the increase in the likelihood function of the data
Strong subset balance implies that each subset contains equal subset { y t , t E S i } , resulting from subiteration i 1. This +
information about image activity parameters. Since OS-EM is subiteration applies one standard EM iteration for the model
formed by n EM iterations, it is readily verified that conver- p i = Aix to this data subset, and hence has the EM property
gence is again linear, with iteration matrix [I-nDH,] . . . [ I - of increasing the likelihood function of this data subset so
nDH1] = [I- DHIn. Strong subset balance provides an that the sum of these terms is positive. In the circumstances
iteration matrix whose eigenvalues are precisely the n-th above, this implies that the likelihood function of the full set
power of the previous eigenvalues. The largest eigenvalue of projection data is also increased and OS-EM increases this
of this matrix is therefore very small in comparison to the likelihood function within each subiteration. By adapting an
largest eigenvalue of EM’S iteration matrix. With strong subset argument of Vardi, Shepp, and Kaufman [16], this implies
balance, convergence of OS-EM will therefore imply linear convergence of the OS-EM algorithm.
convergence at a geometrically improved rate (with exponent The orthogonality condition above cannot apply exactly in
the number of subsets). tomography, but our experience is that it is adequate as an
The results above for exact projection data provide some approximation. In particular, it is rare to observe a decrease in
confidence in the convergence of the algorithm in ideal cir- the likelihood function in any subiteration and we have never
cumstances. With noisy data though, inconsistent equations observed a decrease in likelihood over a complete iteration
result. The results of the Appendix are not applicable. While it of OS-EM. An argument given in Barnett er al. [17] for a
seems likely from our experiments that OS-EM cycles through similar orthogonality condition based on the geometry of the
a number of distinct limit points, as with MART and many detectors is applicable here, too.
604 IEEE TRANSACTIONS ON MEDICAL IMAGING. VOL. 13, NO. 4, DECEMBER 1994

~~~~ ~~~~~~

Fig. 1. Computer simulated chest cross-section: (a) chest phantom emitter activity; (b) attenuation map; (c) sinogram.

v. SIMULATION STUDY The variants of OS-EM and OS-GP used in the simulation
To study the properties of ordered subsets, we conducted are distinguished by a trailing number indicating the OS level
simulation studies employing computer generated projection (number of subsets). Levels considered were 1 (standard EM
data based on a model ("chest phantom") for activity. Gen- or OSL), 2, 4, 8, 16, and 32. Note that all variants of OS-EM
eration incorporated attenuation and Poisson noise. Sixty-four take equal time to compute a single iteration.
projections were generated over 360". Counts were recorded in Ordering of subsets was designed to introduce independent
64 bins per projection. Counts recorded on all projections to- information about the image in successive subsets. The se-
talled approximately 410000. Fig. 1 shows the chest phantom quence used with OS-EM level 32 introduced projection pairs
activity and attenuation map and the simulated projection data in the order 0", 90", 45", 135", 22.5", 112.5", 67.5", etc.
(sinogram). Chest phantom activity is concentrated in a ring of The reconstruction was scaled after each iteration, so that
high activity ("myocardium"). There are two regions of very total expected counts agreed with total counts (a property of
low activity ("lungs") with otherwise uniform low activity the ML solution). This step is unnecessary; it has no effect
within an elliptical body region, of cross section 40 x 32 on qualitative appearance of the reconstruction or on the
cm. The activity in myocardium, background, and lungs were subsequent iterations, as rescaling by any constant multiplier
specified to be in the ratio 8: 1:O. Attenuation coefficients were has no effect on EM. It was applied to provide consistency
0.03/cm in low activity ("lung") regions, 0.12/cm elsewhere of scaling with ML, so that criteria used to compare solutions
within the body ellipse, and O.OO/cm outside the body. were not affected by simple scaling effects.
Ordered subsets were applied with two algorithms, provid- The scaling may be conducted as follows. The image P
ing OS-EM and OS-GP algorithms. OS-EM is the adaption obtained at step 2(c) of the OS-EM definition in Section I1
of Shepp-Vardi EM described in Section 11. OS-GP is the OS is replaced by cP, where c = (ctgt)/(c, .i,a,), with
adaption of Green's one-step-late (OSL) reconstruction. U , = Et ut,. Projection weights ut, were precalculated

GP provides MAP estimation based on Gibbs priors, for for computational efficiency in our code. As a consequence,
which a penalized likelihood criterion is maximized. This the weights U were available and scaling was a trivial
criterion is operation.
Chi-square and MSE measures of error were then calculated.
Chi-square (otherwise known as deviance) is defined as G =
2 c [ y t l o g ( y t / p t ) - ( y t - p t ) ] , where { ~ t are
} the fitted
projections. Chi-square measures discrepancy between fitted
where and 0 are parameters of the procedure and 4 is projections provided by a reconstruction and the counts on all
a log-cosh function which penalizes discrepancies between projections; MSE, defined as c(PJ - ~,)*/5 ,
measures an
pixel neighbors s, T in a manner determined by the fixed average discrepancy between a reconstruction and the chest
weights w. Maximizing the criterion function L* balances two phantom image. Note that, for given data y, G differs from
objectives: increasing the likelihood function (the first sum the Poisson likelihood only by a fixed constant. Therefore, chi-
of the expression), and reducing the roughness penalty (the square decrease ensures a corresponding likelihood function
second sum). increase in the context of this study.
GP was specified with and defined as in Green [4], with Fig. 2 compares the reconstructions after a single iteration
parameters p = 0.006, ~7 = 2.00. These parameter values of each of the OS-EM variants. It can be seen clearly that
had been established as suitable for standard Gibbs prior higher OS levels provide better definition for small numbers
reconstructions of chest phantom data. of iterations, levels 16 and 32 providing low MSE after a
OS-EM was defined by (1) and (2), using nonoverlapping single iteration.
subsets. Results for cumulative subsets are not reported here Fig. 3 compares the reconstructions after matched iterations
because of the similarity of its results with those of standard of each of the OS-EM variants. The reconstructions have very
EM. similar chi-square and mean square error and are visually
HUDSON AND LARKIN: ACCELERATED IMAGE RECONSTRUCTION 605

Fig. 2. OS-EM reconstructions after one iteration, by level: 1, 2, and 4 (a)-@); 8, 16, and 32 (d)-(f).

Fig. 4 shows two cross-sections, one centrally through the h,

I
ring of high activity (myocardium), the other through the
regions of low activity (lungs). A single EM iteration provides
poor estimates of high or low activity. OS-EM-32, after one
I
iteration, closely matches the true activity section (solid line) i
and the reconstruction from 32 iterations of standard EM.
C,,f\, *,- / Q 1-
606 IEEE TRANSACTIONS ON MEDICAL IMAGING. VOL. 13, NO. 4, DECEMBER 1994

X X
40000 0
x m
X X
30000 m m
X X
m m
X
X m
20000
m
X
Y

B
S B
d
U
5 10000
s 9000 53
E 8000 \
0
7000
6000

5000 5000 1M)OO 5000 loow

Chi Square Chi Square
4000
Fig. 6. Mean square error versus chi-square for OS-EM level 1 ( x ) and
3000 level 2 (+).

with OS-EM-32 is greater than with OS-EM-1 or OS-EM-

2000
8. Similarly, OS-GP-32 converges to a limit different from
I I I I the OS-GP-1 limit. However, the OS-GP-8 limit and OS-GP-
1 5 10 50 16 limit (not shown, to enhance clarity) are, for all practical
Iterations purposes, identical with the OS-GP-1 limit whose MSE is far
superior to that of OS-EM- 1.
Fig. 5. Chi-square (G) versus iteration (log scale) by OS-EM level: OS-EM
level 1 (0);level 8 (A); level 16 (x); level 32 (+). All simulations were run on an 80486 processor running
at 33 Mhz and programmed in Turbo Pascal 5.5. A single
iteration of any of the variants took no more than 40 s.
suggests there is an effective limit to the degree of subdivision
that is desirable. VI. DISCUSSION
Fig. 6 plots mean squared error versus chi-square for 128 The simulation results above clearly indicate ordered subsets
iterations of OS-EM-1 and OS-EM-2 (also for OS-GP-1 and (OS) methods provided an order improvement in speed of
OS-GP-2). Note that every OS-EM-1 iteration reduces chi- execution, in this example. To achieve a reconstruction with
square, so that the iterations commence on the right of the a given maximum level of error, the number of iterations
graph. It can be seen that OS-EM-2 closely matches every required were inversely proportional to the level of OS. As
second iteration of OS-EM-1. This pattem applies at all levels projection data was subdivided into more subsets, accelerated
of OS-EM, and also with OS-GP iterations. reduction in chi-square occured up to a critical number of
Now, examine reconstruction quality after large numbers iterations. Beyond this critical number, the noise artifact of ML
of iterations. Fig. 6 displays a turnaround in MSE, caused by solutions is magnified by OS-EM (but not by OS-GP). This
an increasing noise artifact in EM reconstructions. This is a well known noise artifact motivates stopping short iterations
well known phenomenon when the EM algorithm is applied of Vardi-Shepp EM.
to noisy projection data. GP effectively eliminates this noise Rather than stopping short EM iterations to obtain optimal
artifact; this entails a slight sacrifice in fit to observed data. mean square error, the same MSE is obtainable with small
Fig. 7 compares the reconstruction after matched iterations numbers of OS iterations.
of OS-GP. Again, very similar high definition results are The results reported are typical of those from many other
obtained from high levels of OS-GP. It should be noted though simulation studies we have conducted. Further clinical studies
that, due to one step late calculation overhead, OS-GP-32 are ongoing in collaboration with the Department of Nuclear
computation time per iteration is twice that of OS-GP-1 (GP). Medicine, Royal Prince Alfred Hospital (see Hudson, Hutton,
Nevertheless, a substantial acceleration remains. and Larkin [18]).
Fig. 8 illustrates this acceleration. Again, OS, at levels 32 It seems clear that, except with noise-free data, OS-EM
and 16, provides better fit to projection data after one or two converges to a non-ML solution. The questions of whether
iterations than the OSL procedure after 32 iterations. The MSE OS-EM necessarily converges, and if so, to what limit, remain
of OS-GP-32 after 1 iteration (MSE = 62.5) is less than the open. Empirically, it appears that OS-EM follows a cycle
MSE for GP after anything up to 32 iterations (MSE = 68.7 between different limit images depending on the last projection
after 32 iterations). processed. Such behavior would match the theoretical proper-
Fig. 9 displays the limit points of OSEM and OSGP algo- ties of the ART algorithms applicable with linear systems. Our
rithms by plots of MSE and chi-square at selected iterations. simulations suggest that, in real scale applications, the different
Again, successive iterations reduce chi square, so iterations limit images are similar, and indeed, with moderate numbers
commence on the right of each graph. The noise artifact of subsets, indistinguishable.
 HUDSON AND LARKIN: ACCELERATED IMAGE RECONSTRUnION 607

_ _ _ _ _ _ _ ~ ~ ~ ~

Fig. 7. OS-GP iterations after matched iterations. Reconstructions are: (a) level 1, 32 iterations; (b) level 2, 16 iterations; (c) level 4, 8 iterations; . .;
(0 level 32, 1 iteration.

MAXIMUM LIKELIHOOD MAXIMUM A POSTERIORI

+
+
i
+
+
A
+
A
+ B A0
O
\ 8 8
c

i
-
@ +
\ 0 @ +
0 @+
\ 0
+
@+
P
ZOW 3000 4000 5ow 2000 3000 4WO 5oW
Chi Square Chi Square

Fig. 9. Convergence of OS-EM and OS-GP. Mean square error versus

chi-square at selected iterations: level 1, iterations 16, 32, 64,128 (0); level
8, iterations 2, 4, 8, . . . , 128 (A);level 32, iterations 1, 2, 4, . . . , 128 (+).

I I I I
Without such adaptions, the most appropriate level of
50 100 500 1000 data subdivision depends on a number of factors. These
Computation Time
are attenuation density, subset balance, and level of noise
in projection data.
Fig. 8. Chi-square versus computation time for 50 iterations of OS-GP.
Levels are: 1 (0);8 (+); 16 (x); 32 (0). With no attenuation, use of individual projections as subsets
is inefficient, as opposite pairs of projections provide estimates
of the same ray sum and should be included together in the
If an ML solution is required, a composite method may same subset.
be defined in which a single iteration comprises successive Subset balance is a second factor. In our simulations, subset
application of OS-EM at a sequence of levels descending to imbalance (variability in the probability of a pixel emission
level 1. For example, one iteration of the composite method being detected in different subsets) will be greatest for subsets
* could employ levels 32, 16, 8, 4, 2, and 1. Our computational composed of an individual projection, but balance will improve
experience is that this algorithm increases the likelihood as the number of subsets is reduced.
function at each iteration, and hence convergence to an ML With 64 projections of data available, both factors motivated
solution is attained. At the same time, overall computations our use of at most 32 data subdivisions. With fewer subdivi-
are greatly reduced. sions, slightly better likelihood values are eventually obtained,
608 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 13, NO. 4, DECEMBER 1994

as observed in the comparison of OS-EM-32 and OS-EM-I6 Lemma (Count Preservation):

in Fig. 5. The discrepancy increases with level of statistical 1) For k 2 0, Zzj"" = CtESk yt.
noise in projections, so that with low counts, we prefer to use 2) (A2) implies CtESkpf = xj" and CtESk yt = xCj+,
less data subdivision, e.g., four projections per subset instead independent of k 2 1.
of two. 3) Given y > 0, pf > 0 for all t, and (A2), the deviance
The OS method shows great promise for practical use.
OS-EM offers the benefits of EM (quality of restoration)
without the computational burden that may have hindered its
acceptance. An order reduction in computations will assist in
making real time processing of SPECT and PET data viable Proofi
with current technologies.
As a single iteration of OS-EM-32 provides a reasonable
reconstruction, it is even possible to process projection data
immediately as it is acquired. As data collection is often
time consuming, this effectively removes all postprocessing
overhead in providing reconstructions. Such an approach may
find use in medical studies of dynamic systems. = Yt.
Ordered subset EM is also suited to an arbitrary sequence of tESk
projections, as might be obtained, for example, when patient
movement is detected during the acquisition of data, as the co-
efficients represented in the design probability matrix, A , can
be computed in sequence, as required (see Fulton et al. [ 193).
While our discussion has been limited to ordered subset
variants of the EM algorithm, OS methods can be readily
applied with algorithms other than EM. The basic approach =e x ; .
advocated is flexible and readily modified for use with other
restoration algorithms and for real time restorations with other
The third equality follows substituting z* for xk in the
data collection schemes (camera technologies).
above.
~PENDIX The final inequality follows from the inequality logx 2
PROOF OF CONVERGENCE OF OS-EM WITH 1 - x-', for x > 0.
SUBSET BALANCEAND EXACTPROJECTIONS Proposition 1: Under assumptions (Al) and (A2), with
{ x k } defined in (3), { L k } is nondecreasing and bounded
Assumptions above.
(AI): Assume the existence of some feasible solution x* > Proofi Let
0 to the linear system y = Ax. We assume all atj 2 0, and Ak = Lk+l - Lk
that for each t, at least one of these coefficients is strictly
positive. By setting appropriate components of x to zero, we = x; log(x;+l/z;)
may reduce the parameter set when yt = 0 for any index t , so
without loss of generality assume y > 0.
Under assumption (AI), we may define iterations ( x k }by:
2 0 > 0;

(3)

where, at each iteration k, some subset SI, of the (finite) index

set of the data has been specified.
(A2) Subset Balance: Require that, for each subset S; used Here, the third equality uses the definition of OS-EM, the first
in (3), &Si atj = 1, for each j.
inequality follows by application of Jensen's inequality, noting
assumption (A2), and the final inequality applies the Lemma.
Let Lk := L ( x k ; x * where
)
Lk is bounded above by 0, applying the inequality logx 2
X* 1 - x-' to (4), again noting the count preservation.
L ( x ;x*) = - x; log 2. (4) From Proposition 1, { L k }converges, so A k + 0. But, this
xcj
implies the deviance CtESk ytlog(yt/p;) -+ 0, by (5). This
Note that Lk is well defined since x k > 0, x* 2 0, on defining is sufficient to prove convergence of OS-EM, as provided by
0 log 0 = 0. the result:
HUDSON AND LARKIN: ACCELERATED IMAGE RECONSTRUCTION 609

Proposition 2: With assumptions (Al), (A2), and as L ( i , i )= 0. By the continuity of L, i is the only point of
algorithm (3) accumulation of x k , implying convergence to i.
For any subset Ti used infinitely often (“i.0.”) in the Corollary: The OS-EM algorithm with subsets selected
sequence {Sk}, pt -+ yt for t E Ti (i.e., convergence of from a fixed stock of exhaustive (but possibly overlapping)
fitted values occurs for t E Ti) along any subsequence subsets of the index set of the data by cyclic control (as in
of iterations using T; exclusively. Section 11) or by almost cyclic control converges, under (Al)
If subsets are selected from a stock {TI,. . . , T,} in such and (A2), to a feasible solution of the full linear system.
a way (“nearly cyclic control”) that 3N for which the
ACKNOWLEDGMENT
set { s k , . . . , S k + N } contains (for any k ) all members of
the stock, then x k + 5,where i is a feasible solution Brian Hutton, and colleagues in the Department of Nuclear
of the linear system for {yt : t E T = U:=”=,;}. Medicine, Royal Prince Alfred Hospital, have assisted in this
Proof: Consider the subsequence K1 of integers, indicat- formulation and added much to our understanding of SPECT.
ing iterations using a particular subset, say TI, occurring i.0. Our thanks to Jeanne Young who introduced us to the links
Then, by the remark following Proposition 1, the deviance for with ART and SIRT. Also to Adrian Baddeley, Jun Ma, and
this subset, CtETl yt log(yt/pt) -+ 0 for k E K1. But, the Victor Solo for helpful discussions. This paper was prepared in
condition for equality in ( 5 ) , and continuity of the deviance part while Hudson was a visitor to the Division of Mathematics
function, together imply p; -+ yt for k E K1, for all t E TI. Statistics, CSIRO.
We now show p t -+ ys for k E K1, for all s E T , i.e., REFERENCES
convergence occurs for all the data subsets.
Assume subsets sk are selected from the stock { T I ,. . . , P. P. B. Eggermont, G. T. Herman, and A. Lent, “Iterative algo-
rithms for large partitioned linear systems, with applications to image
Tn}, and each element of the stock is selected i.0. Then, by reconstruction,” Linear Algebra and Its Applicat., vol. 40, pp. 37-67,
the argument just given, p t -+ yt, for all t E Ti,along any 1981.
subsequence using Ti exclusively. Hence, for k l E K1 and L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction
for emission tomography,” IEEE Trans. Med. Imag., vol. MI-2, pp.
s E Ti, for i E {2,. ..,n} 113-122, 1982.
T. Hebert and R. Leahy, “A generalized EM algorithm for 3-D Bayesian
reconstruction from Poisson data using Gibbs priors,” IEEE Trans. Med.
Imag., vol. 8, pp. 194-202, 1989.
P. J. Green, “Bayesian reconstruction from emission tomography data
j
using a modified EM algorithm,” IEEE Trans. Med. Imag., vol. 9, pp.
ki -1 84-93, 1990.
E. S. Chomboy, C. J. Chen, M. I. Miller, T. R. Miller, and D. L. Snyder,
‘‘An evaluation of maximum likelihood reconstruction for SPECT,”
IEEE Trans. Med. Imag., vol. 9, pp. 99-1 10, 1990.
k*-1 L. Kaufman, “Implementing and accelerating the EM algorithm for
Positron Emission Tomography,” IEEE Trans. Med. Imag., vol. MI-6,
pp. 37-51, 1987.
P. J. Green and D. M. Titterington, “Recursive methods in image
processing,” in Proc. 46th Session ISI, 1990.
D.M. Titterington, “Recursive parameter estimation using incomplete
j data,” J. Royal Sratist. Soc., B , vol. 46, pp. 257-267, 1984.
.k -1 R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction
techniques (ART) for three-dimensional electron microscopy and X-ray
photography,” J. Theor. Biol., vol. 29, pp. 471481, 1970.
J. N. Darroch and D. Ratcliff, “Generalized iterative scaling for log-
linear models,” Ann. Math. Srarisr., vol. 43, pp. 147G1480, 1972.
Y. Censor and J. Segman, “On block-iterative entropy maximization,”
where k; is the largest integer less than or equal to k1 such J. Inform. Optimization Sci., vol. 8, pp. 275-291, 1987.
that Sk, = T;. Since all subsets occur i.o., the first term A. N. Iusem and M. Teboulle, “A primal-dual iterative algorithm for a
maximum likelihood estimation problem,” Computational Statist. and
approaches 0 as kl-and hence ki-approaches CO. Assuming Data Analysis, vol. 14, pp. 443456, 1992.
nearly cyclic control, there are at most N terms in the sum P. Gilbert, “Iterative methods for the three-dimensional reconstruction
of logarithms, and each term approaches 0 since pt yt, for
-+
of an object from its projections,” J . Theor. Biol., vol. 36, pp. 105-1 17,
1972.
all t E sk. Hence, the RHS of the inequality approaches 0 Y. Censor, “Finite series-expansion reconstruction methods,” Proc.
along subsequence K I ,completing the proof that pt -+ ys for IEEE, vol. 71, no. 3, pp. 409419, 1983.
P. J. Green, “On use of the EM algorithm for penalized likelihood
IC E K1, for all s E Ti. estimation,” J. Royal Statist. Soc., B , vol. 52, no. 3, pp. 443-452, 1990.
Consider a point of accumulation i of the subsequence Y. Vardi, L. A. Shepp, and L. Kaufman, “A statistical model for Positron
{xk : k E Kl}. Since every member of the sequence belongs Emission Tomography (with discussion),” J . Amer. Starisr. Soc., vol.
80, pp. 8-20, 1985.
to the closed bounded region { E : x 2 0, x3 = 1x;}, G. Bamett, S. Crowe, M. Hudson, P. Leung, K. Notodiputro, R.
i exists and belongs to this region, and is feasible. There Proudfoot, and J. Sims, “The use of small scale prototypes in image
exists a subsequence of K1 along which x k + 2 . Then, along reconstructions from projections,” J . Applied Statist., vol. 16, pp.
this subsequence pt + lim Cj u,jx? = Cj a s j i j , and by 223-242, 1989.
H. M. Hudson, B. F. Hutton, and R. Larkin, “Accelerated EM recon-
the result above, this limit along subsequence K1 is ys for struction using ordered subsets,” J . Nucl. Med., vol. 33, (abs), p. 960,
s E T . Hence, the limit i is a solution of the linear system 1992.
R. R. Fulton, B. F. Hutton, M. Braun, B. Ardekani, and R. S. Larkin,
for t E T . But then, limk-oo L(z’,,h)-which exists by “Use of 3-D reconstruction to correct for patient motion in SPECT,”
Proposition 1-is the limit along the subsequence, evaluated Phys. Med. Biol., vol. 39, no. 3, pp. 563-574, 1994.