Brain Network Analysis 1st Edition Moo

K. Chung
Visit to download the full and correct content document:
https://textbookfull.com/product/brain-network-analysis-1st-edition-moo-k-chung/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Fundamentals of Brain Network Analysis 1st Edition

Bullmore

https://textbookfull.com/product/fundamentals-of-brain-network-
analysis-1st-edition-bullmore/

Social Network Analysis Applied to Team Sports Analysis

1st Edition Filipe Manuel Clemente

https://textbookfull.com/product/social-network-analysis-applied-
to-team-sports-analysis-1st-edition-filipe-manuel-clemente/

Heterogeneous Information Network Analysis and

Applications 1st Edition Chuan Shi

https://textbookfull.com/product/heterogeneous-information-
network-analysis-and-applications-1st-edition-chuan-shi/

Overtime Moo U 5 1st Edition Kat Mizera Heart Eyes

Press

https://textbookfull.com/product/overtime-moo-u-5-1st-edition-
kat-mizera-heart-eyes-press/
Social Network Analysis in Second Language Research 1st
Edition Kennedy Terry

https://textbookfull.com/product/social-network-analysis-in-
second-language-research-1st-edition-kennedy-terry/

Brain Source Localization Using EEG Signal Analysis 1st

Edition Munsif Ali Jatoi

https://textbookfull.com/product/brain-source-localization-using-
eeg-signal-analysis-1st-edition-munsif-ali-jatoi/

Microarray Bioprinting Technology Fundamentals and

Practices 1st Edition Moo-Yeal Lee (Eds.)

https://textbookfull.com/product/microarray-bioprinting-
technology-fundamentals-and-practices-1st-edition-moo-yeal-lee-
eds/

Social Network Analysis and Law Enforcement:

Applications for Intelligence Analysis Morgan Burcher

https://textbookfull.com/product/social-network-analysis-and-law-
enforcement-applications-for-intelligence-analysis-morgan-
burcher/

Social Network Analysis: Interdisciplinary Approaches

and Case Studies 1st Edition Xiaoming Fu

https://textbookfull.com/product/social-network-analysis-
interdisciplinary-approaches-and-case-studies-1st-edition-
xiaoming-fu/
 Brain Network Analysis

This tutorial reference serves as a coherent overview of various statistical and math-
ematical approaches used in brain network analysis, where modeling the complex
structures and functions of the human brain often poses many unique computational
and statistical challenges. This book fills a gap as a textbook for graduate students while
simultaneously articulating important and technically challenging topics. Whereas most
available books are graph theory centric, this text introduces techniques arising from
graph theory and expands to include other advanced models in its discussion on
network science, regression, and algebraic topology. Links are included to the sample
data and codes used in generating the book’s results and figures, helping empower
methodological understanding in a manner immediately usable to both researchers and
students.
M o o K . C h u n g is an Associate Professor in the Department of Biostatistics and
Medical Informatics at the University of Wisconsin–Madison and is also affiliated with
the Department of Statistics and Waisman Laboratory for Brain Imaging and Behavior.
He has received the Vilas Associate Award for his research in applied topology to
medical imaging, the Editor’s Award for best paper published in the Journal of Speech,
Language, and Hearing Research for a paper that analyzed computed tomography (CT)
images, and a National Institutes of Health (NIH) Brain Initiative Award for work
on persistent homological brain network analysis. He has written numerous papers in
computational neuroimaging and two previous books on computation on brain image
analysis.
Brain Network Analysis

MOO K. CHUNG
University of Wisconsin–Madison
 University Printing House, Cambridge CB2 8BS, United Kingdom
One Liberty Plaza, 20th Floor, New York, NY 10006, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New Delhi – 110025, India
79 Anson Road, #06-04/06, Singapore 079906

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of
education, learning, and research at the highest international levels of excellence.

www.cambridge.org
Information on this title: www.cambridge.org/9781107184862
DOI: 10.1017/9781316882610
© Moo K. Chung 2019
This publication is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.
First published 2019
Printed in the United Kingdom by TJ International Ltd., Padstow, Cornwall
A catalogue record for this publication is available from the British Library.
Library of Congress Cataloging-in-Publication Data
Names: Chung, Moo K., author.
Title: Brain network analysis / Moo K. Chung.
Description: Cambridge, United Kingdom ; New York, NY : Cambridge University
Press, 2019. | Includes bibliographical references and index.
Identifiers: LCCN 2019000330 | ISBN 9781107184862 (hardback : alk. paper)
Subjects: | MESH: Connectome–methods | Brain–physiology | Nerve
Net–physiology | Neural Pathways–physiology | Models, Neurological |
Mathematical Computing
Classification: LCC QP376 | NLM WL 141.5.B6 | DDC 612.8/2–dc23
LC record available at https://lccn.loc.gov/2019000330
ISBN 978-1-107-18486-2 Hardback
Cambridge University Press has no responsibility for the persistence or accuracy of
URLs for external or third-party internet websites referred to in this publication
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.
 To my parents
for their endless love
 Contents

Preface page xi

1 Statistical Preliminary 1
1.1 General Linear Models 1
1.2 Logistic Regression 6
1.3 Random Fields 9
1.4 Statistical Inference on Fields 16
2 Brain Network Nodes and Edges 27
2.1 Brain Templates 27
2.2 Brain Parcellations 28
2.3 Deterministic Connectivity 34
2.4 Probabilistic Connectivity 46
2.5 Parcellation-Free Brain Network 50
2.6 Structural Covariates 55
3 Graph Theory 61
3.1 Trees and Graphs 61
3.2 Minimum Spanning Trees 62
3.3 Node Degree 65
3.4 Shortest Path Length 70
3.5 Clustering Coefficient 71
3.6 Small-Worldness 72
3.7 Fractal Dimension 73
4 Correlation Networks 76
4.1 Pearson Correlations 76
4.2 Partial Correlations 78
4.3 Averaging Correlations 79

vii
viii Contents

4.4 Correlation as Metric 85

4.5 Statistical Inference on Correlations 87
4.6 Cosine Series Representation 89
4.7 Correlating Functional Signals 103
4.8 Thresholding Correlation Networks 106
5 Big Brain Network Data 108
5.1 Big Data 108
5.2 Sparsity 112
5.3 Hierarchy 114
5.4 Computing Large Correlation Matrices 120
5.5 Online Algorithms 123
6 Network Simulations 129
6.1 Multivariate Normal Distributions 129
6.2 Multivariate Linear Models 136
6.3 Mixed Effects Models 143
6.4 Simulating Dependent Images 149
6.5 Dependent Correlation Networks 152
7 Persistent Homology 156
7.1 Simplicial Homology 157
7.2 Morse Filtrations 163
7.3 Graph Filtrations 168
7.4 Betti Plots 173
8 Diffusions on Graphs 180
8.1 Diffusion as a Cauchy Problem 180
8.2 Finite Difference Method 184
8.3 Laplacian on Planner Graphs 188
8.4 Graph Laplacian 189
8.5 Fiedler Vectors 193
8.6 Heat Kernel Smoothing on Graphs 196
8.7 Laplace Equation 204
9 Sparse Networks 207
9.1 Why Sparse Models? 207
9.2 Sparse Likelihood 210
9.3 Sparse Correlation Network 213
9.4 Partial Correlation Network 222
10 Brain Network Distances 226
10.1 Matrix Norms 227
 Contents ix

10.2 Bottleneck Distance 229

10.3 Gromov–Hausdorff Distance 231
10.4 Kolmogorov–Smirnov Distance 233
10.5 Performance Analysis 236
10.6 Comparisons on Modules 238
10.7 Hypernetworks 241
11 Combinatorial Inferences for Networks 246
11.1 Permutation Test 246
11.2 Exact Combinatorial Inference 253
11.3 Bootstrap 263
12 Series Expansion of Connectivity Matrices 269
12.1 Spectral Decomposition 269
12.2 Iterative Residual Fitting 271
12.3 Spectral Decomposition with Different Bases 278
12.4 Spectral Permutation 279
12.5 Karhunen–Loève Expansion 280
12.6 Vandermonde Matrix Expansion 283
12.7 The Space of Positive Definite Symmetric Matrices 287
13 Dynamic Network Models 292
13.1 Dynamic Causal Model 293
13.2 Dynamic Time Series Models 295
13.3 Persistent Homological Dynamic Network Model 298

Bibliography 302
Index 326
 Preface

Brain network analysis is an emerging field that utilizes various noninvasive

brain imaging modalities such as magnetic resonance imaging (MRI), func-
tional MRI (fMRI), positron emission tomography (PET), diffusion tensor
imaging (DTI), and electroencephalography (EEG) in mapping out the four-
dimensional (4D) spatiotemporal dynamics of the human brain networks in
both normal and clinical populations at the macroscopic level. There has been
substantial progress in the past decade on this topic. A major challenge in
the field is caused by the massive amount of nonstandard high-dimensional
network data that are difficult to analyze using available standard techniques.
This requires new computational approaches and solutions.
The main goals of this book are to provide a coherent overview of various
statistical and mathematical approaches used in brain network analysis to
a wide range of researchers and students, and to articulate important yet
technically challenging topics further. It is hoped that the book presents the
coherent mathematical treatment of underlying methods. The book is mainly
focused on methodological issues beyond widely used graph theory–based
approaches. We wish to provide methodological understanding in a manner
immediately usable to researchers and students. Concepts and methods are
illustrated with brain imaging applications and examples. Some of the brain
network data sets along with MATLAB and R codes used in the book can
be downloaded from the author’s website. The web links are provided in
appropriate places. By making some of the data and codes available, we tried
to make the book more accessible to a wide range of readers.
Although I am indebted to many colleagues and students in writing this
book, I would particularly like to thank the following individuals, in no
particular order. Richard Davidson, Andrew Alexander, Seth Pollak, Hill
Goldsmith of the University of Wisconsin–Madison; David Zald of Vanderbilt
University; and Benjamin Lahey of the University of Chicago provided various

xi
xii Preface

brain imaging data used in illustrating the methods. Hyekyoung Lee of Seoul
National University and Yuan Wang of University of South Carolina helped me
write chapters related to persistent homology and topological distances. Her-
nando Ombao of the King Abdullah University of Science and Technology and
Dustin Pluta of the University of California–Irvine helped me write chapters
related to the dynamic network models. Andrey Gritsenko of the University of
Wisconsin–Madison performed some of basic image processing on the resting-
state fMRI from Human Connectome Project data and helped compile the
list of Automatic Anatomical Labeling (AAL) parcellation. Although most
figures are produced by myself using MATLAB, some figures are generated
by my current and former students, postdocs, and colleagues. Such figures
are identified in figure captions and the proper credits are given. I am also
indebted to Fred Boehm of the University of Wisconsin–Madison and Feng
Liu of Harvard University for proofreading a few chapters.
 1
Statistical Preliminary

This chapter covers the basic statistical methods that are mostly used in univari-
ate voxel-level approaches. However, these basic methods are equally useful
in brain network analysis as well. Most of network modeling techniques are
based on the voxel-level methods. Readers familiar with univariate statistical
methods can skip this chapter.

1.1 General Linear Models

General linear models (GLM) have been widely used in brain imaging and
network studies. The GLM is a very flexible and general statistical framework
encompassing a wide variety of fixed-effect models such as multiple regres-
sions, the analysis of variance (ANOVA), the multivariate analysis of variance
(MANOVA), the analysis of covariance (ANCOVA), and the multivariate
analysis of covariance (MANCOVA) (Timm and Mieczkowski, 1997). More
complex multilevel or hierarchical models such as the mixed-effects models
and structural equation models (SEM) are also viewed as special cases of
general linear models.
GLM provides a framework for testing various associations and hypotheses
while accounting for nuisance covariates in the model in a straightforward
fashion. The effect of age, sex, brain size, and possibly IQ may have severe
confounding effects on the final outcome of many brain network studies.
Older populations’ reduced functional activation could be the consequence of
age-related atrophy of neural systems (Mather et al., 2004). Brain volumes
are significantly larger for children with autism 12 years old and younger
compared with normally developing children (Aylward et al., 1999). Therefore,
it is desirable to account for various confounding factors such as age and
sex. This can be done using GLM automatically. The parameters of GLM are

1
2 Statistical Preliminary

mainly estimated by the least squares estimation and have been implemented
in many statistical packages such as R1 (Pinehiro and Bates, 2002), statistical
parametric mapping (SPM)2 and fMRI-STAT.3
We assume there are n subjects. Let yi be the response variable at a node
or edge, which is mainly coming from images and xi = (xi1, · · · ,xip ) to
be the variables of interest and zi = (zi1, · · · ,zik ) to be nuisance variables
corresponding to the ith subject. Then we have GLM
yi = zi λ + xi β + i ,
where λ = (λ1, · · · ,λk ) and β = (β1, · · · ,βp ) are unknown parameter
vectors to be estimated. We assume  to be the usual zero mean Gaussian
noise.
The significance of the variable of interests xi is determined by testing the
null hypothesis
H0 : β = 0 vs. H1 : β = 0.
The fit of the reduced model corresponding to β = 0, i.e.,
yi = zi λ, (1.1)
is measured by the sum of the squared errors (SSE):

n
SSE0 = (yi − zi
λ0 )2,
i=1

where λ0 is the least squares estimation obtained from the reduced model. The
reduced model (1.1) can be written in a matrix form
⎛ ⎞ ⎛ ⎞⎛ ⎞
y1 z11 · · · z1k λ1
⎜ .. ⎟ ⎜ .. .. .. ⎟ ⎜ .. ⎟ .
⎝ . ⎠=⎝ . . . ⎠⎝ . ⎠
yn zn1 ··· znk λn
y Z λ

By multiplying Z on the both sides, we obtain

Z y = Z Zλ.
Now the matrix Z Z is a full rank and can be invertible if n ≥ k, i.e., there are
more subjects than the number of parameters. The matrix equation then can be
solved by performing a matrix inversion

λ0 = (Z Z)−1 Z y.
1 www.r-project.org
2 www.fil.ion.ucl.ac.uk/spm
3 www.math.mcgill.ca/keith/fmristat
 1.1 General Linear Models 3

Similarly the fit of the full model corresponding to β = 0, i.e.,

yi = zi λ + xi β

is measured by

n
SSE1 = λ1 − xi 
(yi − zi β 1 )2 ,
i=1

where λ1 and β 1 are the least squares estimation from the full model. The full
model can be written in a matrix form by concatenating the row vectors zi and
xi into a larger row vector (zi ,xi ), and the column vectors λ and β into a larger
column vector (λ,β  ) , i.e.,
 
λ
yi = (zi ,xi ) .
β
Then the parameters of the full model can be estimated in the least squares
fashion. Note that

n
SSE1 = min (yi − zi λ1 − xi β 1 )2
λ1,β 1
i=1

n
≤ min (yi − zi λ0 )2 = SSE0 .
λ0
i=1

So the larger the value of SSE0 − SSE1 , more significant the contribution of
the coefficients β is. Under the assumption of the null hypothesis H0 , the test
statistic is the ratio
(SSE0 − SSE1 )/p
F = ∼ Fp,n−p−k . (1.2)
SSE0 /(n − p − k)
The larger the F value, it is more unlikely to accept H0 .

1.1.1 T-Statistic
When p = 1, the test statistic F is distributed as F1,n−1−k , which is the square
of the student t-distribution with n − 1 − k degrees of freedom, i.e., tn−1−k
2 . In
this case, it is better to use t-statistic. The advantage of using the t-statistic is
that the test statistic can provide the direction of the group difference that the
F -statistic cannot provide.
Let
c = (0, · · · ,0 ,1, 0, · · · ,0)
k p−1
4 Statistical Preliminary

be the contrast vector of size k + p. The incorporation of the contrast vector

makes the algebraic derivation straightforward. Consider testing the signifi-
cance of H0 : β1 = 0. The least squares estimation of β1 can be written as
 

λ

β1 = c  .
β
Under the assumption i ∼ N (0,σ 2 ),
1 = β1 .
Eβ
Further, the variance
   −1

λ
1 = cV
Vβ c = σ 2 c [ZX] ZX c.

β
Thus, the unbiased estimator of σ 2 is given by
SSE1 /(n − 1 − k).
We plug this estimator into σ 2 . Then the test statistic under the null
hypothesis is
1
β
T = ∼ tn−1−k .
Vβ1

1.1.2 R-Square
The R-square of a model explains the proportion of variability in measurement
that is accounted by the model. Sometime R-square is called the coefficient
of determination and it is given as the square of a correlation coefficient for a
very simple model. For a linear model involving the response variable yi , the
total sum of squares (SST) measures total total variation in response yi and is
defined as
n
SST = (yi − ȳ)2,
i=1

where ȳ is the sample mean of yi .

On the other hand, SSE measures the amount of variability in yi that is not
explained by the model. Note that SSE is the minimum of the sum of squared
residual of any linear model, SSE is always smaller than SST. Therefore, the
amount of variability explained by the model is SST-SSE. The proportion of
variability explained by the model is then
SST − SSE
R2 = ,
SST
 1.1 General Linear Models 5

which is the coefficient of determination. The R-square ranges between 0 and

1 and the value larger than 0.5 is usually considered significant.

1.1.3 Sum of T-Statistics

Often there is a situation such as a meta-analysis, where we have to sum the
t-statistic images or networks (Chung et al., 2017b). Note that a t-statistic for
large degrees of freedom (above 30) is very close to standard normal, i.e.,
N (0,1). For n identically distributed possibly dependent t-statistics t 1, · · · ,t n ,

the variance of sum nj=1 t j is approximately given by (Billingsley, 1995)
⎛ ⎞

n 
V⎝ tj ⎠ ≈ n + E(t i t j ),
j =1 i=j

Figure 1.1 (a)–(c) t-statistic results of group difference between maltreated

children and normal controls for three different connectivity methods (Chung
et al., 2017b). Only the connections at the p-value less than 0.01 (uncorrected) are
shown. (d) The three t-statistic maps are aggregated to form a single t-statistic.
6 Statistical Preliminary

where E(t i t j ) is the correlation between t i and t j . We used the fact Et j = 0.

Then, we have the aggregated t-statistic given by
n j
j =1 t
T =  ∼ N (0,1).
n + i=j E(t i t j )

If the statistics t j are all independent, since t j are close to standard normal,
E(t i t j ) ≈ 0. The dependency increases the variance estimate and reduces
the aggregated t-statistic value. Unfortunately, it is difficult to estimate the
correlations directly since only one t-statistic map is available for each t j .
E(t i t j ) can be empirically estimated by computing correlations over the entries
of t-statistic maps t i and t j (see Figure 1.1).

1.2 Logistic Regression

Logistic regression is useful for setting up a probabilistic model on the strength
of connectivity and performing classification (Subasi and Ercelebi, 2005).
Suppose k regressors X1, · · · ,Xk are given. These are both imaging and
nonimaging biomarkers such as gender, age, education level, and memory test
score. Let xi1, · · · ,xik denote the measurements for the ith subject. Let the
response variable Yi be the probability of connection modeled as a Bernoulli
random variable with parameter πi , i.e.,
Yi ∼ Bernoulli(πi ).
Yi = 0,1 indicates the edge connected (assigned number 1) or disconnected
(assigned number 0) respectively. πi is then the likelihood (probability) of the
edge connected, i.e., πi = P (Yi = 1).
Now consider linear model
Yi = x
i β + i , (1.3)
where x 
i = (1,xi1, · · · ,xik ) and β = (β0, · · · ,βk ). We may assume

Ei = 0, Vj = σ 2 .
However, linear model (1.3) is no longer appropriate since
EYj = πi = x
i β

but xi β may not be in the range [0,1]. The inconsistency is caused by trying
to match continuous variables xij to categorical variable Yi directly. To address
this problem, we introduce the logistic regression function g:
 1.2 Logistic Regression 7

exp(x
i βi )
πi = g(xi ) = . (1.4)
1 + exp(x
i βi )
Using the logit function, we can write (1.4) as
πi
logit(πi ) = log = x
i βi .
1 − πi

1.2.1 Maximum Likelihood Estimation

The unknown parameters β are estimated via the maximum likelihood estima-
tion (MLE) over n subjects at each edge. The likelihood function is

n
y
L(β|y1, · · · ,yn ) = πi i (1 − πi )1−yi
i=1
 yi  1−yi

n
exp(x 
n
i βi ) 1
= .
i=1
1 + exp(x
i βi ) i=1
1 + exp(x
i β)

The loglikelihood function is given by


n
log L(β) = const. + yi log πi + (1 − yi ) log(1 − πi )
i=1
n
= const. + yi x
i β + log(1 − πi )
i=1

and its maximum is obtained when

∂ log L(β) 
n
= xi (yi − πi ) = 0.
∂β
i=1

In simplifying the expression, we used the following identities

∂πi
= πi (1 − πi )
∂β0
and
∂πi
= xi πi (1 − πi ).
∂β1
Since the logistic regression function π is in complicated form, the maximum
is obtained numerically. Define the information matrix I (β) to be

∂ 2 log L(β) 
n
I (β) = − − πi (1 − πi )xi x
i .
∂β  ∂β
i=1
8 Statistical Preliminary

Then the Newton–Raphson algorithm is used to find the MLE in an iterative

fashion. Starting with an arbitrary initial vector β 0 , we estimate iteratively
∂ log L(β) j
β j +1 = β j + I (β j )−1 (β ).
∂β
Many computational packages such as R and MATLAB have the logistic
regression model fitting procedure.
Although we do not have the explicit formulas for the MLE, using the
asymptotic normality of the MLE, the distributions of the estimators can be
 is
approximately determined. For large sample size n, the distribution of β
approximately multivariate normal with means β with the covariance matrix
)−1 .
I (β

1.2.2 Best Model Selection

Consider following full model:
logit(πi ) = β0 + β1 x1 + β2 x2 + · · · + βp xp .
Let β (1) = (β0, · · · ,βq ) and β (2) = (βq+1, · · · ,βp ) . The parameter β (1)
corresponds to the parameters of the reduced model. Then we are interested in
testing
H0 : β (2) = 0.
Define the deviance D of a model as D = −2 log L( π ), which is distributed
2
asymptotically as χn−p−1 . Let 
π (p) and 
π (q) be the estimated success proba-
bilities for the full and reduced models, and let Dp and Dq be the associated
deviances. Then the log-likelihood ratio statistic for testing β (2) = 0 is
π (p) ) − log L(
2[log L( π (q) )] = Dq − Dp ∼ χp−q
2
.

1.2.3 Logistic Discriminant Analysis

Discriminant analysis resulting from the estimated logistic model is called the
logistic discrimination. We classify the ith subject according to a classification
rule. The simplest rule is to assign the ith subject as group 1:
P (Yi = 1) > P (Yi = 0).
This statement is equivalent to πi > 1/2. Depending on the bias and the error
of the estimation, the value 1/2 can be adjusted. For the fitted logistic model,
we classify the ith subject as group 1 if x 
i βi > 0 and as 0 if xi βi < 0.

The plane xi β = 0 is the classification boundary that separates two groups.
 1.3 Random Fields 9

The performance of classification technique is measured by the error rate γ ,

the overall probability of misclassification. The cross-validation is used to
estimate the error rate. This is done by randomly partitioning the data into
the training and the testing sets. In the leave-one-out scheme, the training set
consists of n−1 subjects, while the testing set consists of one subject. Suppose
the ith subject is taken as the test set. Then using the training set, we determine
the logistic model. Using the predicted model, we test if the ith subject is
correctly classified. The error rate obtained in this fashion is denoted as e−i .
Note that e−i = 0 if the subject is classified correctly while e−i = 1 if the
subject is misclassified. The leave-one-out error rate is then given by
1
n
γ =
 e−i .
n
i=1
To formally test the statistical significance of the discriminant power, we
use Press’s Q statistic (Hair et al., 1998), which is given by
n(2γ − 1)2 ∼ χ12 .
Press’s Q statistic is asymptotically distributed as χ 2 with one degree of
freedom.

1.3 Random Fields

At the voxel level, it is often necessary to model measurements at each voxel as
a random field. For instance, the deformation field of warping a brain to another
brain is often modeled as a continuous random field (Chung et al., 2001b).
The generalization of a continuous stochastic process defined in R to a higher
dimensional abstract space is called a random field. For an introduction to
random fields, see (Yaglom, 1987; Dougherty, 1999; Adler and Taylor, 2007).
In the random field theory as introduced in (Worsley, 1994; Worsley et al.,
1996b), measurement Y at voxel position x ∈ M is modeled as
Y (x) = μ(x) + (x),
where μ is the unknown functional signal to be estimated and  is the
measurement error, which is modeled as a random variable at each fixed x.
Then the collection of random variables {(x) : x ∈ M} is called a stochastic
process or random field. The more precise measure-theoretic definition can
be found in (Adler and Taylor, 2007). Random field modeling can be done
beyond the usual Euclidean space to curved cortical and subcortical manifolds
(Joshi, 1998; Chung et al., 2003a). Most of concepts in random fields are the
continuous generalization of random vectors.
10 Statistical Preliminary

Definition 1.1 Given a probability space, a random field T (x) defined in Rn

is a function such that for every fixed x ∈ Rn , T (x) is a random variable on
the probability space.
Definition 1.2 The covariance function R(x,y) of a random field T is
defined as
  
R(x,y) = E T (x) − ET (x) T (y) − ET (y) .
If the joint distribution of T at points x1, · · · ,xm
 
P T (x1 ) ≤ z1, · · · ,T (xm ) ≤ zm
is invariant under the translation
(x1, · · · ,xm ) → (x1 + τ, · · · ,xm + τ ),
T is said to be stationary or homogeneous.
For a stationary random field T , its covariance function is
R(x,y) = f (x − y)
for some function f . A special case of stationary fields is an isotropic field,
which requires the covariance function to be rotation invariant, i.e.,
R(x,y) = f (|x − y|)
for some function f (Yaglom, 1987).

1.3.1 Gaussian Fields

The most important class of random fields is Gaussian fields. A more rigorous
treatment can be found in Adler and Taylor (2007). Let us start defining a
multivariate normal distribution from a Gaussian random variable.
Definition 1.3 A random vector T = (T1, · · · ,Tm ) is multivariate normal if
m
i=1 ci Ti is Gaussian for every possible ci ∈ R.

Then a Gaussian random field can be defined from a multivariate normal

distribution.
Definition 1.4 A random field T is a Gaussian random field if T (x1 ), · · · ,
T (xm ) are multivariate normal for every (x1, · · · ,xm ) ∈ Rm .
An equivalent definition to Definition 1.4 is as follows. T is a Gaussian random
field if the finite joint distribution
P (T (x1 ) ≤ z1, · · · ,T (xm ) ≤ zm )
is a multivariate normal for every (x1, · · · ,xm ).
 1.3 Random Fields 11

T is a mean zero Gaussian field if ET (x) = 0 for all x. Because any mean
zero multivariate normal distribution can be completely characterized by its
covariance matrix, a mean zero Gaussian random field T can be similarly
determined by its covariance function R. Two fields T and S are independent
if T (x) and S(y) are independent for every x and y. For mean zero Gaussian
fields T and S, they are independent if and only if the covariance function
 
R(x,y) = E T (x)T (y)

vanishes for all x and y, which is a very strong assumption.

The Gaussian white noise is a Gaussian random field with the Dirac-delta
function δ as the covariance function. Note the Dirac delta function is defined
as δ(x) = ∞ if x = 0 and δ(x) = 0 if x = 0. Further, δ(x) = 1. Numerically
we can simulate the Dirac-delta function as the limit of the sequence of
Gaussian kernel Kσ when σ → ∞. The Gaussian white noise is simulated
as an independent and identical Gaussian random variable at each voxel.

1.3.2 Derivative of Gaussian Fields

Suppose G is a collection of Gaussian random fields. For given X,Y ∈ G, we
have c1 X + c2 Y ∈ G again for all c1 and c2 . Therefore, G forms an infinite-
dimensional vector space. Any linear combination of Gaussian fields is again
a Gaussian field. We can show that the derivatives of Gaussian fields are also
Gaussian. To see this, we define mean-square convergence.

Definition 1.5 A sequence of random fields Th , indexed by h, converges to T

as h → 0 in mean-square if
 2
lim ETh − T  = 0.
h→0

We will denote the mean-square convergence using the usual limit notation:

lim Th = T .
h→0

The convergence in mean square implies the convergence in mean. This can be
seen from
 2  2  2
ETh − T  = V Th − T + E|Th − T | .

Now let Th → T in mean square. Each term in the right-hand side should also
converge to zero, proving the statement.
Now we define the derivative of the field in the mean-square sense as
dT (x) T (x + h) − T (x)
= lim .
dx h→0 h
12 Statistical Preliminary

If T (x) and T (x + h) are Gaussian, T (x + h) − T (x) is again Gaussian. Thus,

the limit on the right-hand side is also Gaussian. If R is the covariance function
of the mean zero Gaussian field T , the covariance function of its derivative field
is given by
 dT (x) dT (y)  ∂ 2 R(x,y)
E = .
dx dy ∂x∂y

1.3.3 Integration of Gaussian Fields

The integration of Gaussian fields is also Gaussian. To see this, define the
integration of a random field as the limit of Riemann sum. Let ∪ni=1 Mi be a
partition of M, i.e.,
M = ∪ni=1 Mi and Mi ∩ Mj = ∅ if i = j .
Let xi ∈ Mi and μ(Mi ) be the volume of Mi . Then we define the integration
of field T as
n
T (x) dx = lim T (xi )μ(Mi ),
M n→∞
i=1

where the limit is taken as n → ∞ and μ(Mj ) → 0 for all j . When we

integrate a Gaussian field, it is the limit of a linear combination of Gaussian
random variables so it is again a Gaussian random variable. In general, any
linear operation on Gaussian fields will result in Gaussian fields with different
covariance structures.
We can use a collection of Gaussian fields to construct χ 2 -, t-, F -fields
(Worsley, 1994; Worsley et al., 1996b, 2004; Cao and Worsley, 1999a). The
χ 2 -field with m degrees of freedom is defined as

m
T (x) = Xi2 (x),
i=1

where X1, · · · ,Xm are independent, identically distributed Gaussian fields

with zero mean and unit variance. Similarly, we can define t and F fields as
well as Hotelling’s T 2 field (Thompson et al., 1997; Collins et al., 1998; Joshi,
1998; Cao and Worsley, 1999a; Gaser et al., 1999).

1.3.4 Simulating Gaussian Fields

We show how to simulate smooth Gaussian fields by performing Gaussian
kernel smoothing on white noise. This is perhaps the easiest way of simulating
Gaussian fields.
 1.3 Random Fields 13

White noise is defined as a random field whose covariance function is

proportional to the Dirac-delta function δ, i.e.,

R(x,y) ∝ δ(x − y).

For instance, we may take

R(x,y) = lim Kσ (x − y),

σ →0

the limit of the usual isotropic Gaussian kernel. White noise is usually
characterized via generalized functions. One example of white noise is the
generalized derivative of Brownian motion (Wiener process) called Gaussian
white noise.

Definition 1.6 Brownian motion (Wiener process) B(x),x ∈ R+ is a zero

mean Gaussian field with covariance function

RB (x,y) = min(x,y).

Following Definition 1.6, we have VB(x) = x. The increments of Wiener

processes in nonoverlapping intervals are independent identically distributed
(i.i.d.) Gaussian. Further, the paths of the Wiener process are continous while
they are not differentiable (Øksendal, 2010). Higher-dimensional Brownian
motion can be generalized by taking each component of vector fields to be
i.i.d. Brownian motion.
Although the path of the Wiener process is not differentiable, we can define
the generalized derivative via integration by parts with a smooth function f
called a test function in the following way
x dB(y) x f (y)
f (x)B(x) = f (y) dy + B(y) dy.
0 dy 0 dy
Taking the expectation on both sides, we have
x dB(y)
f (y)E dy = 0.
0 dy
It should be true for all smooth f so E dB(y)
dy = 0. Further, it can be shown that
the covariance function of process

dB(y)/dy ∝ δ(x − y).

The Gaussian white noise can be used to construct smooth Gaussian random
fields of the form
dB(x)
X(x) = K ∗ W (x) = K ∗ ,
dx
14 Statistical Preliminary

where K is a Gaussian kernel and W is the generalized derivative of Brownian

motion. Since Brownian motion is a zero-mean Gaussian process, X(x) is
obviously a zero-mean field with the covariance function
RX (x,y) = E[K ∗ W (x)K ∗ W (y)] (1.5)
∝ K(x − z)K(y − z) dz. (1.6)

The case when K is an isotropic Gaussian kernel was investigated by Sieg-

mund and Worsley with respect to optimal filtering in scale space (Siegmund
and Worsley, 1996).
In numerical implementation, we use the discrete white Gaussian noise,
which is simply a Gaussian random variable.
Example 1.1 Let w be a discrete version of white Gaussian noise given by

m
w(x) = Zi δ(x − xi ),
i=1

where i.i.d. Zi ∼ N (0,σw2 ). Note that


m
K ∗ w(x) = Zi K(x − xi ). (1.7)
i=1

The collection of random variables K ∗ w(y1 ), · · · ,K ∗ w(yl ) forms a

multivariate normal at arbitrary points y1, · · · ,yl . Hence, the field K ∗ w(x)
is a Gaussian field.
The covariance function of the field (1.7) is given by

m
R(x,y) = E(Zi Zj )K(x − xi )K(y − xj ) (1.8)
i,j =1
 m
= σw2 K(x − xi )K(y − xi ). (1.9)
i=1

As usual, we may take K to be a Gaussian kernel. Let us simulate some

Gaussian fields.
Example 1.2 Gaussian white noise is generated using w ∼ N (0,0.42 ),
which is shown in the top-left of Figure 1.2. With a Gaussian kernel with
bandwidth 1, iteratively smoother versions of Gaussian random fields are
constructed by
K=inline(’exp(-(x.ˆ2+y.ˆ2)/2/sigmaˆ2)ˆ{\top})
[dx,dy]=meshgrid([-10:10])
 1.3 Random Fields 15

sigma=1
K=K(sigma,dx,dy)/sum(sum(K(sigma,dx,dy)))

w=normrnd(0,0.4,101,101)
smooth_w=w
for i=1:10
figure; imagesc(smooth_w)
smooth_w=conv2(smooth_w,K,’same’)
end;

Figure 1.2 shows one, four, and nine iterations.

Figure 1.2 Gaussian random field simulation. Starting with Gaussian white noise
N (0,0.42 ) (top-left), we iteratively apply Gaussian kernel smoothing one, four,
and nine times with bandwidth σ = 1.
Another random document with
no related content on Scribd:
 The strength of our political organizations arises from their
development as social bodies; many of the hardest workers in their
ranks are neither office-holders nor yet paid henchmen, but merely
members who have gradually learned to identify their fortunes with
the party whose hall they have come to regard as the head-quarters
in which to spend the most agreeable of their leisure moments.
Under the American system it is impossible for a man to accomplish
anything by himself; he must associate himself with others, and they
must throw their weight together. This is just what the social
functions of the political clubs enable their members to do. The great
and rich society clubs are composed of men who are not apt to take
much interest in politics anyhow, and never act as a body. The great
effect produced by a social organization for political purposes is
shown by the career of the Union League Club; and equally striking
proof can be seen by every man who attends a ward meeting. There
is thus, however much to be regretted it may be, a constant
tendency towards the concentration of political power in the hands of
those men who by taste and education are fitted to enjoy the social
side of the various political organizations.

THE LIQUOR-SELLER IN POLITICS.

It is this that gives the liquor-sellers their enormous influence in
politics. Preparatory to the general election of 1884, there were held
in the various districts of New York ten hundred and seven primaries
and political conventions of all parties, and of these no less than six
hundred and thirty-three took place in liquor-saloons,—a showing
that leaves small ground for wonder at the low average grade of the
nominees. The reason for such a condition of things is perfectly
evident: it is because the liquor-saloons are places of social resort
for the same men who turn the local political organizations into social
clubs. Bar-tenders form perhaps the nearest approach to a leisure
class that we have at present on this side of the water. Naturally they
are on semi-intimate terms with all who frequent their houses. There
is no place where more gossip is talked than in bar-rooms, and much
of this gossip is about politics,—that is, the politics of the ward, not of
the nation. The tariff and the silver question may be alluded to and
civil-service reform may be incidentally damned, but the real interest
comes in discussing the doings of the men with whom they are
personally acquainted: why Billy so-and-so, the alderman, has
quarrelled with his former chief supporter; whether “old man X” has
really managed to fix the delegates to a given convention; the reason
why one faction bolted at the last primary; and if it is true that a great
down-town boss who has an intimate friend of opposite political faith
running in an up-town district has forced the managers of his own
party to put up a man of straw against him. The bar-keeper is a man
of much local power, and is, of course, hail-fellow-well-met with his
visitors, as he and they can be of mutual assistance to one another.
Even if of different politics, their feelings towards each other are
influenced purely by personal considerations; and, indeed, this is
true of most of the smaller bosses as regards their dealings among
themselves, for, as one of them once remarked to me with enigmatic
truthfulness, “there are no politics in politics” of the lower sort—
which, being interpreted, means that a professional politician is much
less apt to be swayed by the fact of a man’s being a Democrat or a
Republican than he is by his being a personal friend or foe. The
liquor-saloons thus become the social head-quarters of the little
knots or cliques of men who take most interest in local political
affairs; and by an easy transition they become the political head-
quarters when the time for preparing for the elections arrives; and, of
course, the good-will of the owners of the places is thereby
propitiated,—an important point with men striving to control every
vote possible.
The local political clubs also become to a certain extent mutual
benefit associations. The men in them become pretty intimate with
one another; and in the event of one becoming ill, or from any other
cause thrown out of employment, his fellow-members will very often
combine to assist him through his troubles, and quite large sums are
frequently raised for such a purpose. Of course, this forms an
additional bond among the members, who become closely knit
together by ties of companionship, self-interest, and mutual
interdependence. Very many members of these associations come
into them without any thought of advancing their own fortunes; they
work very hard for their party, or rather for the local body bearing the
party name, but they do it quite disinterestedly, and from a feeling
akin to that which we often see make other men devote their time
and money to advancing the interests of a yacht club or racing
stable, although no immediate benefit can result therefrom to
themselves. One such man I now call to mind who is by no means
well off, and is neither an office-seeker nor an office-holder, but who
regularly every year spends about fifty dollars at election time for the
success of the party, or rather the wing of the party, to which he
belongs. He has a personal pride in seeing his pet candidates rolling
up large majorities. Men of this stamp also naturally feel most
enthusiasm for, or animosity against, the minor candidates with
whom they are themselves acquainted. The names at the head of
the ticket do not, to their minds, stand out with much individuality;
and while such names usually command the normal party support,
yet very often there is an infinitely keener rivalry among the smaller
politicians over candidates for local offices. I remember, in 1880, a
very ardent Democratic ward club, many of the members of which in
the heat of a contest for an assemblyman cooly swapped off quite a
number of votes for President in consideration of votes given to their
candidate for the State Legislature; and in 1885, in my own district, a
local Republican club that had a member running for alderman,
performed a precisely similar feat in relation to their party’s candidate
for governor. A Tammany State Senator openly announced in a
public speech that it was of vastly more importance to Tammany to
have one of her own men Mayor of New York than it was to have a
Democratic President of the United States. Very many of the leaders
of the rival organizations, who lack the boldness to make such a
frankly cynical avowal of what their party feeling really amounts to,
yet in practice, both as regards mayor and as regards all other local
offices which are politically or pecuniarily of importance, act exactly
on the theory enunciated by the Tammany statesman; and, as a
consequence, in every great election not only is it necessary to have
the mass of the voters waked up to the importance of the principles
that are at stake, but, unfortunately, it is also necessary to see that
the powerful local leaders are convinced that it will be to their own
interest to be faithful to the party ticket. Often there will be intense
rivalry between two associations or two minor bosses; and one may
take up and the other oppose the cause of a candidate with an
earnestness and hearty good-will arising by no means from any
feeling for the man himself, but from the desire to score a triumph
over the opposition. It not unfrequently happens that a perfectly good
man, who would not knowingly suffer the least impropriety in the
conduct of his canvass, is supported in some one district by a little
knot of politicians of shady character, who have nothing in common
with him at all, but who wish to beat a rival body that is opposing
him, and who do not for a moment hesitate to use every device, from
bribery down, to accomplish their ends. A curious incident of this sort
came to my knowledge while happening to inquire how a certain
man became a Republican. It occurred a good many years ago, and
thanks to our election laws it could not now be repeated in all its
details; but affairs similar in kind occur at every election. I may
preface it by stating that the man referred to, whom we will call X,
ended by pushing himself up in the world, thanks to his own industry
and integrity, and is now a well-to-do private citizen and as good a
fellow as anyone would wish to see. But at the time spoken of he
was a young laborer, of Irish birth, working for his livelihood on the
docks and associating with his Irish and American fellows. The
district where he lived was overwhelmingly Democratic, and the
contests were generally merely factional. One small politician, a
saloonkeeper named Larry, who had a great deal of influence, used
to enlist on election day, by pay and other compensation, the
services of the gang of young fellows to which X belonged. On one
occasion he failed to reward them for their work, and in other ways
treated them so shabbily as to make them very angry, more
especially X, who was their leader. There was no way to pay Larry
off until the next election; but they determined to break his influence
utterly then, and as the best method for doing this they decided to
“vote as far away from him” as possible, or, in other words, to strain
every nerve to secure the election of all the candidates most
opposed to those whom Larry favored. After due consultation, it was
thought that this could be most surely done by supporting the
Republican ticket. Most of the other bodies of young laborers, or,
indeed, of young roughs, made common cause with X and his
friends. Everything was kept very quiet until election day, neither
Larry nor the few Republicans having an inkling of what was going
on. It was a rough district, and usually the Republican booths were
broken up and their ballot-distributers driven off early in the day; but
on this occasion, to the speechless astonishment of everybody,
things went just the other way. The Republican ballots were
distributed most actively, the opposing workers were bribed,
persuaded, or frightened away, all means fair and foul were tried,
and finally there was almost a riot,—the outcome being that the
Republicans actually obtained a majority in a district where they had
never before polled ten per cent. of the total vote. Such a
phenomenon attracted the attention of the big Republican leaders,
who after some inquiry found it was due to X. To show their gratitude
and to secure so useful an ally permanently (for this was before the
days of civil-service reform), they procured him a lucrative place in
the New York Post-office; and he, in turn, being a man of natural
parts, at once seized the opportunity, set to work to correct the
defects of his early education, and is now what I have described him
to be.

BOSS METHODS.
A politician who becomes an influential local leader or boss is, of
course, always one with a genuine talent for intrigue and
organization. He owes much of his power to the rewards he is able
to dispense. Not only does he procure for his supporters positions in
the service of the State or city,—as in the custom-house, sheriff’s
office, etc.,—but he is also able to procure positions for many on
horse railroads, the elevated roads, quarry works, etc. Great
corporations are peculiarly subject to the attacks of demagogues,
and they find it much to their interest to be on good terms with the
leader in each district who controls the vote of the Assemblyman and
Alderman; and therefore the former is pretty sure that a letter of
recommendation from him on behalf of any applicant for work will
receive most favorable consideration. The leader is also continually
helping his henchmen out of difficulties, pecuniary and otherwise; he
lends them a dollar or two now and then, helps out, when possible,
such of their kinsmen as get into the clutches of the law, gets a hold
over such of them as have done wrong and are afraid of being
exposed, and learns to mix judicious bullying with the rendering of
service.
But, in addition to all this, the boss owes very much of his
commanding influence to his social relations with various bodies of
his constituents; and it is his work as well as his pleasure to keep up
these relations. No débutante during her first winter in society has a
more exacting round of social duties to perform than has a prominent
ward politician. In every ward there are numerous organizations,
primarily social in character, but capable of being turned to good
account politically. The Amalgamated Hack-drivers’ Union, the
Hibernian Republican Club, the West Side Young Democrats, the
Jefferson C. Mullin Picnic Association,—there are twenty such
bodies as these in every district, and with, at any rate, the master
spirits in each and all it is necessary for the boss to keep on terms of
intimate and, indeed, rather boisterous friendship. When the
Jefferson C. Mullin society goes on a picnic, the average citizen
scrupulously avoids its neighborhood; but the boss goes, perhaps
with his wife, and, moreover, enjoys himself heartily, and is hail-
fellow-well-met with the rest of the picnickers, who, by the way, may
be by no means bad fellows; and when election day comes round,
the latter, in return, no matter to what party they may nominally
belong, enthusiastically support their friend and guest, on social, not
political, grounds. The boss knows every man in his district who can
control any number of votes: an influential saloon-keeper, the owner
of a large livery stable, the leader among a set of horse-car drivers, a
foreman in a machine-shop who has a taste for politics,—with all
alike he keeps up constant and friendly relations. Of course this fact
does not of itself make the boss a bad man; there are several such I
could point out who are ten times over better fellows than are the
mild-mannered scholars of timorous virtue who criticise them. But on
the whole the qualities tending to make a man a successful local
political leader under our present conditions are not apt to be
qualities that make him serve the public honestly or disinterestedly;
and in the lower wards, where there is a large vicious population, the
condition of politics is often fairly appalling, and the boss of the
dominant party is generally a man of grossly immoral public and
private character, as anyone can satisfy himself by examining the
testimony taken by the last two or three legislative committees that
have investigated the affairs of New York city. In some of these
wards many of the social organizations with which the leaders are
obliged to keep on good terms are composed of criminals, or of the
relatives and associates of criminals. The testimony mentioned
above showed some strange things. I will take at random a few
instances that occur to me at the moment. There was one case of an
assemblyman who served several terms in the Legislature, while his
private business was to carry on corrupt negotiations between the
Excise Commissioners and owners of low haunts who wished
licenses. The president of a powerful semi-political association was
by profession a burglar; the man who received the goods he stole
was an alderman. Another alderman was elected while his hair was
still short from a term in State Prison. A school trustee had been
convicted of embezzlement, and was the associate of criminals. A
prominent official in the Police Department was interested in
disreputable houses and gambling saloons, and was backed
politically by their proprietors.

BEATING THE MACHINE.

In the better wards the difficulty comes in drilling a little sense and
energy into decent people: they either do not care to combine or else
refuse to learn how. In one district we did at one time and for a
considerable period get control of affairs and elect a set of almost
ideal delegates and candidates to the various nominating and
legislative bodies, and in the end took an absolutely commanding
although temporary position in State and even in national politics.
This was done by the efforts of some twenty or thirty young fellows
who devoted a large part of their time to thoroughly organizing and
getting out the respectable vote. The moving spirits were all active,
energetic men, with common sense, whose motives were perfectly
disinterested. Some went in from principle; others, doubtless, from
good-fellowship or sheer love of the excitement always attendant
upon a political struggle. Our success was due to our absolute
freedom from caste spirit. Among our chief workers were a Columbia
College professor, a crack oarsman from the same institution, an
Irish quarryman, a master carpenter, a rich young merchant, the
owner of a small cigar store, the editor of a little German newspaper,
and a couple of employees from the post-office and custom-house,
who worked directly against their own seeming interests. One of our
important committees was composed of a prominent member of a
Jewish synagogue, of the son of a noted Presbyterian clergyman,
and of a young Catholic lawyer. We won some quite remarkable
triumphs, for the first time in New York politics carrying primaries
against the machine, and as the result of our most successful
struggle completely revolutionizing the State Convention held to
send delegates to the National Republican Convention of 1884, and
returning to that body, for the first and only time it was ever done, a
solid delegation of Independent Republicans. This was done,
however, by sheer hard work on the part of a score or so of men; the
mass of our good citizens, even after the victories which they had
assisted in winning, understood nothing about how they were won.
Many of them actually objected to organizing, apparently having a
confused idea that we could always win by what one of their number
called a “spontaneous uprising,” to which a quiet young fellow in our
camp grimly responded that he had done a good deal of political
work in his day, but that he never in his life had worked so hard and
so long as he did to get up the “spontaneous” movement in which we
were then engaged.

CONCLUSIONS.
In conclusion, it may be accepted as a fact, however unpleasant,
that if steady work and much attention to detail are required, ordinary
citizens, to whom participation in politics is merely a disagreeable
duty, will always be beaten by the organized army of politicians to
whom it is both duty, business, and pleasure, and who are knit
together and to outsiders by their social relations. On the other hand,
average citizens do take a spasmodic interest in public affairs; and
we should therefore so shape our governmental system that the
action required by the voters should be as simple and direct as
possible, and should not need to be taken any more often than is
necessary. Governmental power should be concentrated in the
hands of a very few men, who would be so conspicuous that no
citizen could help knowing all about them; and the elections should
not come too frequently. Not one decent voter in ten will take the
trouble annually to inform himself as to the character of the host of
petty candidates to be balloted for, but he will be sure to know all
about the mayor, comptroller, etc. It is not to his credit that we can
only rely, and that without much certainty, upon his taking a
spasmodic interest in the government that affects his own well-being;
but such is the case, and accordingly we ought, as far as possible, to
have a system requiring on his part intermittent and not sustained
action.

FOOTNOTES:
[9] The Century, November, 1886.
[10] Since succeeded every year or two by some other anti-
Tammany Democratic organization or organizations.
[11] All this has been changed, vastly for the better, by the
ballot reform laws, under which the State distributes the printed
ballots; and elections are now much more honest than formerly.
[12] As a consequence of our investigation the committee, of
which I was chairman, succeeded in securing the enactment of
laws which abolished these enormous salaries.
 VII
SIX YEARS OF CIVIL SERVICE REFORM[13]

No question of internal administration is so important to the United

States as the question of Civil Service reform, because the spoils
system, which can only be supplanted through the agencies which
have found expression in the act creating the Civil Service
Commission, has been for seventy years the most potent of all the
forces tending to bring about the degradation of our politics. No
republic can permanently endure when its politics are corrupt and
base; and the spoils system, the application in political life of the
degrading doctrine that to the victor belong the spoils, produces
corruption and degradation. The man who is in politics for the offices
might just as well be in politics for the money he can get for his vote,
so far as the general good is concerned. When the then Vice-
President of the United States, Mr. Hendricks, said that he “wished to
take the boys in out of the cold to warm their toes,” thereby meaning
that he wished to distribute offices among the more active heelers, to
the rapturous enthusiasm of the latter, he uttered a sentiment which
was morally on the same plane with a wish to give “the boys” five
dollars apiece all around for their votes, and fifty dollars apiece when
they showed themselves sufficiently active in bullying, bribing, and
cajoling other voters. Such a sentiment should bar any man from
public life, and will bar him whenever the people grow to realize that
the worst enemies of the Republic are the demagogue and the
corruptionist. The spoils-monger and spoils-seeker invariably breed
the bribe-taker and bribe-giver, the embezzler of public funds and the
corrupter of voters. Civil Service reform is not merely a movement to
better the public service. It achieves this end too; but its main
purpose is to raise the tone of public life, and it is in this direction that
its effects have been of incalculable good to the whole community.
For six years, from May, 1889, to May, 1895, I was a member of
the National Civil Service Commission, and it seems to me to be of
interest to show exactly what has been done to advance the law and
what to hinder its advancement during these six years, and who
have been the more prominent among its friends and foes. I wish to
tell “the adventures of Philip on his way through the world,” and show
who robbed him, who helped him, and who passed him by. It would
take too long to give the names of all our friends, and it is not worth
while to more than allude to most of our foes and to most of those
who were indifferent to us; but a few of the names should be
preserved and some record made of the fights that have been fought
and won and of the way in which, by fits and starts, and with more
than one set-back, the general advance has been made.
Of the Commission itself little need be said. When I took office the
only Commissioner was Mr. Charles Lyman, of Connecticut, who
resigned when I did. Honorable Hugh S. Thompson, ex-Governor of
South Carolina, was made Commissioner at the same time that I
was, and after serving for three years resigned. He was succeeded
by Mr. George D. Johnston, of Louisiana, who was removed by the
President in November, 1893, being replaced by Mr. John R. Procter,
the former State Geologist of Kentucky, who is still serving. The
Commission has never varied a hand’s-breadth from its course
throughout this time; and Messrs. Thompson, Procter, Lyman, and
myself were always a unit in all important questions of policy and
principle. Our aim was always to procure the extension of the
classified service as rapidly as possible, and to see that the law was
administered thoroughly and fairly. The Commission does not have
the power that it should, and in many instances there have been
violations or evasions of the law in particular bureaus or departments
which the Commission was not able to prevent. In every case,
however, we made a resolute fight, and gave the widest publicity to
the wrong-doing. Often, even where we have been unable to win the
actual fight in which we were engaged, the fact of our having made
it, and the further fact that we were ready to repeat it on provocation,
has put a complete stop to the repetition of the offence. As a
consequence, while there have been plenty of violations and
evasions of the law, yet their proportion was really very small, taking
into account the extent of the service. In the aggregate it is doubtful if
one per cent. of all the employees have been dismissed for political
reasons. In other words, where under the spoils system a hundred
men would have been turned out, under the Civil Service Law, as
administered under our supervision, ninety-nine men were kept in.
In the administration of the law very much depends upon the
Commission. Good heads of departments and bureaus will
administer it well anyhow; but not only the bad men, but also the
large class of men who are weak rather than bad, are sure to
administer the law poorly unless kept well up to the mark. The public
should exercise a most careful scrutiny over the appointment and
over the acts of Civil Service Commissioners, for there is no office
the effectiveness of which depends so much upon the way in which
the man himself chooses to construe his duties. A Commissioner
can keep within the letter of the law and do his routine work and yet
accomplish absolutely nothing in the way of securing the observance
of the law. The Commission, to do useful work, must be fearless and
vigilant. It must actively interfere whenever wrong is done, and must
take all the steps that can be taken to secure the punishment of the
wrong-doer and to protect the employee threatened with molestation.
This course was consistently followed by the Commission
throughout my connection with it. I was myself a Republican from the
North. Messrs. Thompson and Procter were from the South, and
were both Democrats who had served in the Confederate armies;
and it would be impossible for anyone to desire as associates two
public men with higher ideals of duty, or more resolute in their
adherence to those ideals. It is unnecessary to say that in all our
dealings there was no single instance wherein the politics of any
person or the political significance of any action was so much as
taken into account in any case that arose. The force of the
Commission itself was all chosen through the competitive
examinations, and included men of every party and from every
section of the country; and I do not believe that in any public or
private office of the size it would be possible to find a more honest,
efficient, and coherent body of workers.
From the beginning of the present system each President of the
United States has been its friend, but no President has been a
radical Civil Service reformer. Presidents Arthur, Harrison, and
Cleveland have all desired to see the service extended, and to see
the law well administered. No one of them has felt willing or able to
do all that the reformers asked, or to pay much heed to their wishes
save as regards that portion of the service to which the law actually
applied. Each has been a sincere party man, who has felt strongly
on such questions as those of the tariff, of finance, and of our foreign
policy, and each has been obliged to conform more or less closely to
the wishes of his party associates and fellow party leaders; and, of
course, these party leaders, and the party politicians generally,
wished the offices to be distributed as they had been ever since
Andrew Jackson became President. In consequence the offices
outside the protection of the law have still been treated, under every
administration, as patronage, to be disposed of in the interest of the
dominant party. An occasional exception was made here and there.
The postmaster at New York, a Republican, was retained by
President Cleveland in his first administration, and the postmaster of
Charleston, a Democrat, was retained by President Harrison; but,
with altogether insignificant exceptions, the great bulk of the non-
classified places have been changed for political reasons by each
administration, the office-holders politically opposed to the
administration being supplanted or succeeded by political adherents
of the administration.
Where the change has been complete it does not matter much
whether it was made rapidly or slowly. Thus, the fourth-class
postmasterships were looted more rapidly under the administration
of President Harrison than under that of President Cleveland, and
the consular service more rapidly under President Cleveland than
under President Harrison; but the final result was the same in both
cases. Indeed, I think that the brutality which accompanied the
greater speed was in some ways of service to the country, for it
directed attention to the iniquity and folly of the system, and
emphasized, in the minds of decent citizens, the fact that
appointments and removals for political reasons in places where the
duties are wholly non-political cannot be defended by any man who
looks at public affairs from the proper standpoint.
The advance has been made purely on two lines, that is, by better
enforcement of the law, and by inclusion under the law, or under
some system similar in its operations, of a portion of the service
previously administered in accordance with the spoils theory. Under
President Arthur the first classification was made, which included
14,000 places. Under President Cleveland, during his first term, the
limits of the classified service were extended by the inclusion of 7000
additional places. During President Harrison’s term the limit was
extended by the inclusion of about eight thousand places; and
hitherto during President Cleveland’s second term, by the inclusion
of some six thousand places; in addition to which the natural growth
of the service has been such that the total number of offices now
classified is over forty thousand. Moreover, the Secretary of the Navy
under President Harrison, introduced into the navy yards a system of
registration of laborers, which secures the end desired by the
Commission; and Secretary Herbert has continued this system. It
only rests, however, upon the will of the Secretary of the Navy; and
as we cannot expect always to have secretaries as clear-sighted as
Messrs. Tracy and Herbert, it is most desirable that this branch of the
service should be put directly under the control of the Commission.
The Cabinet officers, though often not Civil Service reformers to
start with, usually become such before their terms of office expire.
This was true, without exception, of all the Cabinet officers with
whom I was personally brought into contact while on the
Commission. Moreover, from their position and their sense of
responsibility they are certain to refrain from violating the law
themselves and to try to secure at least a formal compliance with its
demands on the part of their subordinates. In most cases it is
necessary, however, to goad them continually to see that they do not
allow their subordinates to evade the law; and it is very difficult to get
either the President or the head of a department to punish these
subordinates when they have evaded it. There is not much open
violation of the law, because such violation can be reached through
the courts; but in the small offices and small bureaus there is often a
chance for an unscrupulous head of the office or bureau to
persecute his subordinates who are politically opposed to him into
resigning, or to trump up charges against them on which they can be
dismissed. If this is done in a sufficient number of cases, men of the
opposite political party think that it is useless to enter the
examinations; and by staying out they leave the way clear for the
offender to get precisely the men he wishes for the eligible registers.
Cases like this continually occur, and the Commission has to be
vigilant in detecting and exposing them, and in demanding their
punishment by the head of the office. The offender always, of
course, insists that he has been misunderstood, and in most cases
he can prepare quite a specious defence. As he is of the same
political faith as the head of the department, and as he is certain to
be backed by influential politicians, the head of the department is
usually loath to act against him, and, if possible, will let him off with,
at most, a warning not to repeat the offence. In some departments
this kind of evasion has never been tolerated; and where the
Commission has the force under its eye, as in the departments at
Washington, the chance of injustice is minimized. Nevertheless,
there have been considerable abuses of this kind, notably in the
custom-houses and post-offices, throughout the time I have been at
Washington. So far as the Post-Office Department was concerned
the abuses were more flagrant under President Harrison’s
Postmaster-General, Mr. Wanamaker; but in the Treasury
Department they were more flagrant under President Cleveland’s
Secretary of the Treasury, Mr. Carlisle.
Congress has control of the appropriations for the Commission,
and as it cannot do its work without an ample appropriation the
action of Congress is vital to its welfare. Many, even of the friends of
the system in the country at large, are astonishingly ignorant of who
the men are who have battled most effectively for the law and for
good government in either the Senate or the Lower House. It is not
only necessary that a man shall be good and possess the desire to
do decent things, but it is also necessary that he shall be
courageous, practical, and efficient, if his work is to amount to
anything. There is a good deal of rough-and-tumble fighting in
Congress, as there is in all our political life, and a man is entirely out
of place in it if he does not possess the virile qualities, and if he fails
to show himself ready and able to hit back when assailed. Moreover,
he must be alert, vigorous, and intelligent if he is going to make his
work count. The friends of the Civil Service Law, like the friends of all
other laws, would be in a bad way if they had to rely solely upon the
backing of the timid good. During the last six years there have been,
as there always are, a number of men in the House who believe in
the Civil Service Law, and who vote for it if they understand the
question and are present when it comes up, but who practically
count for very little one way or the other, because they are timid or
flighty, or are lacking in capacity for leadership or ability to see a
point and to put it strongly before their associates.
There is need of further legislation to perfect and extend the law
and the system; but Congress has never been willing seriously to
consider a proposition looking to this extension. Bills to provide for
the appointment of fourth-class postmasters have been introduced
by Senator Lodge and others, but have never come to anything.
Indeed, but once has a measure of this kind been reported from
committee and fought for in either House. This was in the last
session of the 53d Congress, when Senators Morgan and Lodge
introduced bills to reform the consular service. They were referred to
Senator Morgan’s Committee on Foreign Affairs, and were favorably
reported. Senator Lodge made a vigorous fight for them in the
Senate, but he received little support, and was defeated, Senator
Gorman leading the opposition.
On the other hand, efforts to repeal the law, or to destroy it by new
legislation, have uniformly been failures, and have rarely gone
beyond committee. Occasionally, in an appropriation bill or some
other measure, an amendment will be slipped through, adding forty
or fifty employees to the classified service, or providing that the law
shall not apply to them; but nothing important has ever been done in
this way. But once has there been a resolute attack made on the law
by legislation. This was in the 53d Congress, when Mr. Bynum, of
Indiana, introduced in the House, and Mr. Vilas, of Wisconsin,
pushed in the Senate, a bill to reinstate the Democratic railway mail
clerks, turned out before the classification of the railway mail service
in the early days of Mr. Harrison’s administration.
The classification of the railway mail service was ordered by
President Cleveland less than two months before the expiration of
his first term of office as President. It was impossible for the
Commission to prepare and hold the necessary examinations and
establish eligible registers prior to May 1, 1889. President Harrison
had been inaugurated on March 4th, and Postmaster-General
Wanamaker permitted the spoilsmen to take advantage of the
necessary delay and turn out half of the employees who were
Democrats, and replace them by Republicans. This was an
outrageous act, deserving the severe condemnation it received; but
it was perfectly legal. During the four years of Mr. Cleveland’s first
term a clean sweep was made of the railway mail service; the
employees who were almost all Republicans, were turned out, and
Democrats were put in their places. The result was utterly to
demoralize the efficiency of the service. It had begun to recover from
this when the change of administration took place in 1889. The time
was too short to allow of a clean sweep, but the Republicans did all
they could in two months, and turned out half of the Democrats. The
law then went into effect, and since that time there have been no
more removals for partisan purposes in that service. It has now
recovered from the demoralization into which it was thrown by the
two political revolutions, and has reached a higher standard of
efficiency than ever before. What was done by the Republicans in
this service was repeated, on a less scale, by the Democrats four
years later in reference to the classification of the small free-delivery
post-offices. This classification was ordered by President Harrison
two months before his term of office expired; but in many of the
offices it was impossible to hold examinations and prepare eligible
registers until after the inauguration of President Cleveland, and in a
number of cases the incoming postmasters, who were appointed
prior to the time when the law went into effect, took advantage of the
delay to make clean sweeps of their offices. In one of these offices,
where the men were changed in a body, the new appointees hired
the men whom they replaced, at $35 a month apiece, to teach them
their duties; in itself a sufficient comment on the folly of the spoils
system.
Mr. Bynum’s bill provided for the reinstatement of the Democrats
who were turned out by the Republicans just before the classification
of the railway mail service. Of course such a bill was a mere partisan
measure. There was no more reason for reinstating the Democrats
thus turned out than for reinstating the Republicans who had been
previously turned out that these same Democrats might get in, or for
reinstating the Republicans in the free-delivery offices who had been
turned out just before these offices were classified. If the bill had
been enacted into law it would have been a most serious blow to the
whole system, for it would have put a premium upon legislation of
the kind; and after every change of parties we should have seen the
passing of laws to reinstate masses of Republicans or Democrats,
as the case might be. This would have meant a return to the old
system under a new form of procedure. Nevertheless, Mr. Bynum’s
bill received the solid support of his party. Not a Democratic vote was
cast against it in the House, none even of the Massachusetts
Democrats being recorded against it. In the Senate it was pushed by
Mr. Vilas. By a piece of rather sharp parliamentary procedure he
nearly got it through by unanimous consent. That it failed was owing
entirely to the vigilance of Senator Lodge. Senator Vilas asked for
the passage of the bill, on the ground that it was one of small
importance, upon which his committee were agreed. When it was
read the words “classified civil service” caught Senator Lodge’s ear,
and he insisted upon an explanation. On finding out what the bill was
he at once objected to its consideration. Under this objection it could
not then be considered. If it could have been brought to a vote it
would undoubtedly have passed; but it was late in the session, the
calendars were crowded with bills, and it was impossible to get it up
in its regular order. Another effort was made, and was again
frustrated by Senator Lodge, and the bill then died a natural death.
In the final session of the 53d Congress a little incident occurred
which deserves to be related in full, not for its own importance, but
because it affords an excellent example of the numerous cases
which test the real efficiency of the friends of the reform in Congress.
It emphasizes the need of having, to watch over the interests of the
law, a man who is willing to fight, who knows the time to fight, and
who knows how to fight. The secretary of the Commission was, in
the original law of 1883, allowed a salary of $1600 a year. As the
Commission’s force and work have grown, the salary in successive
appropriation bills for the last ten years has been provided for at the
rate of $2000 a year. Many of the clerks under the secretary now
receive $1800, so that it would be of course an absurdity to reduce
him in salary below his subordinates. Scores of other officials of the
Government, including, for instance, the President’s private
secretary, the First Assistant Postmaster-General, the First Assistant
Secretary of State, etc., have had their salaries increased in
successive appropriation bills over the sum originally provided, in
precisely the same way that the salary of the secretary of the
Commission was increased. The 53d Congress was Democratic, as
was the President, Mr. Cleveland, and the secretary of the
Commission was himself a Democrat, who had been appointed to
the position by Mr. Cleveland during his first term as President. The
rules of the House provide that there shall be no increase of salary
beyond that provided in existing law in any appropriation bill. When
the appropriation for the Civil Service Commission came up in the
House, Mr. Breckinridge, of Kentucky, made the point of order that to
give $2000 to the secretary of the Commission was to increase his
salary by $400 over that provided in the original law of 1883, and
was therefore out of order. He also produced a list of twenty or thirty
other officers, including the President’s private secretary, the First
Assistant Postmaster-General, etc., whose salaries were similarly
increased. He withdrew his point of order as regards these persons,
but adhered to it as regards the secretary of the Commission. The
chairman of the Committee of the Whole, Mr. O’Neill, of
Massachusetts, sustained the point of order; and not one person
made any objection or made any fight, and the bill was put through
the House with the secretary’s salary reduced.
Now, the point of order was probably ill taken anyhow. The existing
law was and had been for ten years that the salary was $2000. But,
in any event, had there been a single Congressman alert to the
situation and willing to make a fight he could have stopped the whole
movement by at once making a similar point of order against the
President’s private secretary, against the First Assistant Postmaster-
General, the Assistant Secretary of State, and all the others
involved. The House would of course have refused to cut down the
salaries of all of these officials, and a resolute man, willing to insist
that they should all go or none, could have saved the salary of the
secretary of the Civil Service Commission. There were plenty of men
who would have done this if it had been pointed out to them; but no
one did so, and Mr. Breckinridge’s point of order was sustained, and
the salary of the secretary reduced by $400. When it got over to the
Senate, however, the Civil Service reformers had allies who needed
but little coaching. In the first place, the sub-committee of the
Committee on Appropriations, composed of Messrs. Teller, Cockrell,
and Allison, to which the Civil Service Commission section of the
Appropriation bill was referred, restored the salary to $2000; but
Senator Gorman succeeded in carrying, by a bare majority, the
Appropriations Committee against it, and it was reported to the full
Senate still at $1600. The minute it got into the full Senate, however,
Senator Lodge had a fair chance at it, and it was known that he
would receive ample support. All that he had to do was to show
clearly the absolute folly of the provision thus put in by Mr.
Breckinridge, and kept in by Mr. Gorman, and to make it evident that
he intended to fight it resolutely. The opposition collapsed at once;
the salary was put back at $2000, and the bill became a law in that
form.
Whether bad legislation shall be choked and good legislation
forwarded depends largely upon the composition of the committees
on Civil Service reform of the Senate and the Lower House. The
make-up of these committees is consequently of great importance.
They are charged with the duty of investigating complaints against
the Commission, and it is of course very important that if ever the
Commission becomes corrupt or inefficient its shortcomings should
be unsparingly exposed in Congress. On the other hand, it is equally
important that the falsity of untruthful charges advanced against it
should be made public. In the 51st, 52d, and 53d Congresses a