On the Two Different Aspects of the Representative Method: The Method of Stratified

Sampling and the Method of Purposive Selection

Author(s): Jerzy Neyman
Source: Journal of the Royal Statistical Society, Vol. 97, No. 4 (1934), pp. 558-625
Published by: Wiley for the Royal Statistical Society
Stable URL: http://www.jstor.org/stable/2342192 .
Accessed: 24/06/2014 23:59

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].

Wiley and Royal Statistical Society are collaborating with JSTOR to digitize, preserve and extend access to
Journal of the Royal Statistical Society.

http://www.jstor.org

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 558 [Part IV,

ONTHETwo DIFFERENTASPECTS METHOD:

OFTHEREPRESENTATIVE
THEMETHODOF STRATIFIEDSAMPLINGANDTHEMETHOD
OF PURPOSIVE
SELECTION.
By JERZYNEYMAN
NenckiInstitute,Soc. Sci. Lit.
Laboratory,
(Biometric
Warsaw).
Varsoviensis,
[Read beforethe Royal StatisticalSociety,June 19th,1934, the PRESIDENT,
the RT. HON. LORD MESTON of Agra and Dunottar, K.C.S.I., LL.D.,
in the Chair.]

CONTENTS.
PAGE
I. Introductory ... ... ... ... ... ... ... ... 558
II. MathematicalTheoriesunderlying theRepresentative
Method ... 561
1. The theoryof probabilitiesa posterioriand the work of R. A.
Fisher. ... ... ... ... ... ... 561
2. The choice of the estimates ... ... ... ... 563
III. DifferentAspectsof theRepresentative Method ... ... ... 567
1. The methodof randomsampling ... ... ... 567
2. The methodof purposiveselection ... ... ... 570
IV. ComparisonofthetwoMethodsofSampling . .. ... ... 573
1. The estimatesof Bowley and of Gini and Galvani ... ... 573
2. The hypothesesunderlyingboth methods and the conditions
of practicalwork... . .. ... ... ... ... ... 576
3. Numericalillustration ... ... ... ... ... 583
V. Conclusions ... ... ... ... ... ... ... ... 585
VI. Appendix ... ... ... ... ... ... ... ... ... 589

I. INTRODUCTORY.

OWING to the work of the International Statistical Institute,* and

perhaps still more to personal achievements of Professor A. L.
Bowley, the theory and the possibility of practical applications of
the representative method has attracted the attention of many
statisticians in differentcountries. Very probably this popularity
of the representativemethod is also partly due to the general crisis,
to the scarcityofmoneyand to the necessityofcarryingout statistical
investigationsconnected with social life in a somewhat hasty way.
The resultsare wanted in some fewmonths,sometimesin a fewweeks
afterthe beginningof the work,and thereis neithertime nor money
foran exhaustive research.
But I think that if practical statistics has acquired something
* See " The Report on the RepresentativeMethod in Statistics" by A.
Jensen,Bull. Inst. Intern.Stat., XXII. 1preLivr.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 Aspectsof theRepresentative
1934] Two Different Method. 559

valuable in the representative method, this is due primarily to

ProfessorA. L. Bowley, who not only was one of the firstto apply
thismethodin practice,*but also wrotea veryfundamentalmemoirt
giving the theory of the method. Since then the representative
methodhas been oftenapplied in different countriesand fordifferent
purposes.
My chief topic being the theory of the representativemethod,
I shall not go into its historyand shall not -quote the examples of
its practical application howeverimportant-unless I findthat their
consideration might be useful as an illustrationof some points of
the theory.
There are two differentaspects of the representativemethod.
One of them is called the method of random sampling and the other
the method of purposive selection. This is a division into two very
broad groups and each of these may be furthersubdivided. The
two kinds of method were discussed by A. L. Bowley in his book,
in which they are treated as it were on equal terms,as being equally
to be recommended. Much the same attitude has been expressedin
the Report of the Commission appointed by the International
Statistical Institute for the purpose of studyingthe application of
the Representative Method in Statistics.: The Report says "In
the selectionof that part of the material which is to be the object of
directinvestigation,one or the other of the followingtwo principles
can be adopted: in certaininstances it will be possible to make use
of a combinationof both principles. The one principleis character-
ized by the fact that the units whichare to be included in the sample
are selected at random. This method is onlv applicable where the
circumstancesmake it possible to give every single unit an equal
chance of inclusion in the sample. The other principle consists in
the samples being made up by purposive selectionof groups of units
which it is presumedwill give the sample the same characteristicsas
the whole. There will be especial reason forpreferring this method,
wherethe material differswith respectto compositionfromthe kind
of material which is the basis of the experienceof games of chance,
and whereit is thereforedifficultor even impossible to comply with
the aforesaid condition for the application of selection at random.
Each of these two methods has certain advantages and certain
defects.
This was published in 1926. In November of the same year
* A. L. Bowley: "Working Class Households in Reading." J.R.S.S.,
June, 1913.
t A. L. Bowley: " Measurementof the PrecisionAttained in Sampling."
Memorandumpublishedby the Int. Stat. Inst., Bull. Int. Stat.Inst.,Vol. XXII.
lre Livr.
I Bull. Int. Stat. Inst., XXII. l1reLivr. p. 376.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
560 NEYMAN-On the Two Different [Part IV,

the Italian statisticiansC. Gini and L. Galvani were faced with the
problem of the choice between the two principlesof sampling,when
theyundertookto select a sample fromthe data ofthe Italian General
Census of 1921. All the data were already worked out and
published and the original sheets containing informationabout
individual familieswere to be destroyed. In orderto make possible
any furtherresearch,the need forwhich mightbe feltin the future,
it was decided to keep for a longertime a fairlylarge sample of the
census data, amountingto about I5 per cent. of the same.
The chiefpurposeofthe workis stated by the authorsas follows: *
"To obtain a sample which would be representativeof the whole
countrywith respect to its chiefdemographic,social, economic and
geographiccharacteristics."
At the beginning of the work the original data were already
sorted by provinces, districts (circondari) and communes, and the
authors state that the easiest method of obtaining the sample was
to select data in accordance with the division of the country in
administrative units. As the purpose of the sample was among
othersto allow local comparisonsto be made in the future,the authors
expressedthe view that the selectionofthe sample,takingadministra-
tive units as elements,was the only possible one.
For various reasons,which,however,the authorsdo not describe,
it was impossibleto take as an elementofsamplingan administrative
unit smaller than a commune. They did not, however, think it
satisfactoryto use communesas units of selection because (p. 3 loc.
cit.) their large number (8,354) would make it difficultto apply the
method of purposiveselection. So finallythe authorsfixeddistricts
(circondari)to serve as units of sampling. The total numberof the
districtsin which Italy is divided amounts to 2I4. The number of
the districtsto be included in the sample was 29, that is to say,
about I3-5 per cent. of the total numberof districts.
Having thus fixed the units of selection,the authors proceed to
the choice ofthe principleofsampling: shouldit be randomsampling
or purposive selection? To solve this dilemma they calculate the
probability,7, that the mean income ofpersonsincluded in a random
sample of k 29 districts drawn from their universe of K 214
districtswill differfromits universe-valueby not more than i 5 per
cent. The approximate value of this probabilitybeing very small,
about 7t *o8, the authors decided that the principle of sampling
to choose was that of purposive selection.t
The quotation fromthe Report of the Commissionof the Inter-
* Annali di Statistica,Ser. VI. Vol. IV. p. 1. 1929.
t It may be noted,however,that the choice of the principleseems to have
been predeterminedby the previouschoice of the unit of sampling.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 561

national Statistical Institute and the choice of the principle of

sampling adopted by the Italian statisticians,suggest that the idea
of a certain equivalency of both principlesof random sampling and
purposive selection is a rather common one. As the theory of
purposiveselectionseems to have been extensivelypresentedonly in
the two papers mentioned,while that of random sampling has been
discussed probably by more than a hundred authors,it seems justi-
fiable to consider carefullythe basic assumptions underlyingthe
former. This is what I intend to do in the present paper. The
theoreticalconsiderationswill then be illustratedon practical results
obtained by Gini and Galvani, and also on resultsof anotherrecent
investigation,carried out in Warsaw, in which the representative
method was used. As a result of this discussion it may be that the
general confidencewhichhas been placed in the methodof purposive
selection will be somewhat diminished.

THEORIESUNDERLYINGTHE REPRESENTATIVE
II. MATHEMATICAL
METHOD.
1. The Theory of Probabilitiesa posteriori and theworkof R. A.
Fisher.
Obviously the problem of the representative method is par
excellencethe problemof statisticalestimation. We are interestedin
characteristicsof a certain population, say 7, which it is either
impossible or at least very difficultto study in detail, and we try to
estimate these characteristicsbasing our judgment on the sample.
Until recentlyit has been usually assumed that the accurate solution
of such a problem requires the knowledge of probabilities a priori
attached to differentadmissible hypotheses concerningthe values
of the collective characters* of the population t. Accordingly,the
memoirof A. L. Bowley may be regardedas divided into two parts.
Each questionis treatedfromtwo points ofview: (a) The population
7-is supposed to be known; the question to be answered is: what
could be the samplesfromthispopulation? (b) We knowthe sample
and are concernedwiththe probabilitiesa posteriorito be ascribed to
differenthypothesesconcerningthe population.
In sections which I classifyas (a) we are. on the safe ground of
classical theory of probability, reducible to the theory of com-
binations.t
In sections (b), however, we are met with conclusions based,
* This is a translationofthe terminology used by Bruns and Orzecki. Any
characteristicsof the populationor sample is a collectivecharacter.
t In this respect1 should like to call attentionto the remarkablepaper of
the late L. Marchpublishedin M11etron,Vol. VI. Thereis practicallyno question
of probabilitiesand many classical theoremsof this theoryare reduced to the
theoryof combinations.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
562 NEYMAN-On theTwo Different [Part IV,

interalia, on some quite arbitraryhypothesesconcerningthe proba-

bilities a prtori,and ProfessorBowley accompanies his resultswith
the followingremark: " It is to be emphasized that the inference
thus formulatedis based on assumptionsthat are difficultto verify
and which are not applicable in all cases."
However, since Bowley's book was written, an approach to
problemsof this type has been suggestedby ProfessorR. A. Fisher
which removes the difficultiesinvolved in the lack of knowledgeof
the a priori probabilitylaw.* Unfortunatelythe papers referredto
have been misunderstoodand the validity of statementsthey con-
tain formallyquestioned. This I think is due largelyto the very
condensedformof explainingideas used by R. A. Fisher,and perhaps
also to a somewhat difficultmethod of attacking the problem.
Avoidingthe necessityof appeals to the somewhatvague statements
based on probabilitiesa posteriori,Fisher's theorybecomes, I think,
the very basis of the theory of representativemethod. In Note I
in the Appendix I have describedits main lines in a way somewhat
different fromthat followedby Fisher.
The possibility of solving the problems of statistical estimation
independentlyfromany knowledgeof the a priori probabilitylaws,
discoveredby R. A. Fisher,makes it superfluousto make any appeals
to the Bayes' theorem.
The whole procedure consists really in solving the problems
whichProfessorBowley termeddirectproblems: givena hypothetical
population, to findthe distributionof certain charactersin repeated
samples. If this problem is solved, then the solution of the other
problem, which takes the place of the problem of inverse pro-
bability,can be shown to follow.
The formofthis solutionconsistsin determiningcertainintervals,
whichI propose to call the confidenceintervals(see Note I), in which
we may assume are contained the values of the estimatedcharacters
of the population, the probabilityof an errorin a statementof this
sort being equal to or less than 1 - s, where C is any number
O< C < 1, chosenin advance. The number C I call the confidence
coefficient. It is importantto note that the methods of estimating,
particularlyin the case of large samples, resultingfromthe work of
Fisher, are often preciselythe same as those which are already in
common use. Thus the new solution of the problemsof estimation
consistsmainlyin a rigorousjustificationof what has been generally
consideredcorrectmore or less on intuitivegrounds.t
* R. A. Fisher: Proc. Camb. Phil. Soc., Vol. XXVI, Part 4, Vol. XXVIII,
Part 3, and Proc. Roy. Soc., A. Vol. CXXXIX.
t I regretthat the necessarilylimitedsize ofthe paper does not allow me to
go into the details of this importantquestion. It has been largelystudied by
R. A. Fisher. His results in this respect forma theorywhich he calls the

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 563

Here I should like to quote the words of Laplace, that the theory
of probabilityis in fact but the good commonsense which is reduced
to formulm. It is able to express in exact terms what the sound
mindsfeelby a sort of instinct,sometimeswithoutbeing able to give
good reasons fortheirbeliefs.

2. The ChoiceoftheEstimates.
However, it may be observed that there remains the question
of the choice of the collectivecharactersof the samples which would
be most suitable for the purpose of the constructionof confidence
intervalsand thus forthe purposes of estimation. The requirements
with regard to these characters in practical statistics could be
formulatedas follows:
1. They must follow a frequencydistributionwhich is already
tabled or may be easily calculated.
2. The resulting confidenceintervals should be as narrow as
possible.
The firstof these requirementsis somewhat opportunistic,but
I believe as far as the practical work is concerned this condition
should be borne in mind.*
Collectivecharactersof the samples whichsatisfyboth conditions
quoted above and whichmay be used in the most common cases, are
supplied by the elegant methodof A. A. Markoff,tused by him when

Theoryof Estimation. The above-mentionedproblemsof confidenceintervals

are consideredby R. A. Fisher as somethinglike an additional chapterto the
Theory of Estimation, being perhaps of minor importance. However, I do
not agreein thisrespectwithProfessorFisher. I am inclinedto thinkthat the
importanceof his achievementsin the two fieldsis in a relationwhichis inverse
to what he thinkshimself. The solutionof the problemwhich I describedas
the problemof confidenceintervalshas been soughtby the greatestmindssince
the workofBayes I50 yearsago. Any recentbook on the theoryof probability
includes large sections concerningthis problem. These sections are crowded
with all sorts of " paradoxes," etc. The present solution means, I think,
not less than a revolutionin thetheoryof statistics. On the other hand, the
problemof the choice of estimateshas-as faras I can see-mainly a practical
importance. If this is not properlysolved (grantingthat the problemof con-
fidenceintervalshas been solved correctly)the resultingconfidenceintervals
will be unnecessarilybroad, but our statementsabout the values of estimated
collectivecharacterswill still remaincorrect. Thus I thinkthat the problems
of the choice of the estimates are rather the technical problems,which, of
course, are extremelyimportantfrom the point of view of practical work,
but the importanceof which cannot be comparedwith the importanceof the
otherresultsof R. A. Fisher,concerningthe verybasis ofthe modernstatistical
theory. These are,of course," qualifyingjudgments,"whichmay be defended
and may be attacked,but whichanyone may accept or reject,accordingto his
personalpoint of view and the perspectiveon the theoryof statistics.
* The positionis a different one if we considerthe question fromthe point
of view of the theory. Here I have to mentionthe importantpapers of R. A.
Fisher on the theoryof likelihood.
t A. A. Markoff: Calculus of Probabilities. Russian. Edition IV,
Moscow 1923. Therewas a Germaneditionof thisbook,Leipzig 1912,actually
out of print.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
564 NEYMAN-On the Two Different [Part IV,

dealing with the theoryof least squares. The method is not a new
one, but as it was published in Russian it is not generallyknown.*
This method, combined with some results of R. A. Fisher and of
E. S. Pearson concerningthe extension of " Student's " distribution
allows us to build up the theoryof diflerentaspects of representative
method to the last details.
Suppose 0 is a certain collective character of a population and
. . . Xn
Xl, X2. * * . . . . (

is the sample fromthis population. We shall say that a function

of these x's, say
0' = 0'(X1, X2,. .. . . (2)
is a " mathematicalexpectation estimate " t of 0, if the mean value
of O' in repeated samples is equal to 0. Further,we shall say that
the estimate O' is the best linear estimate of 0 if it is linear with
regardto x's, i.e.
0- + X2X2
1X1X + + Xo
+ XnX, (3)
and if its standard erroris less than the standard errorof any other
linear estimate of 0.
Of course, in using the words " best estimate " I do not mean
that the estimate defined has unequivocable advantages over all
others. This is only a convention and, as long as the definitionis
borne in mind,will not cause any misunderstanding. Still, the best
linear estimateshave some importantadvantages:
1. If n be large, their distributionpractically always follows
closelythe normallaw of frequency. This is important,as in apply-
ing the representativemethod in social and economic statistics we
are commonlydealing with very large samples.
2. In most cases they are easily found by applying Markoff's
method.
3. The same method provides us with the estimate of their
standard errors.
4. If the estimate O' of 0 is a linear estimate,and if i is the esti-
mate ofits standarderror,then,in cases whenthe sampled population
is normallydistributed,the ratio

t .(4)

follows the " Student's" distribution,which is dependent only

upon the size of the sample. This is the result due to R. A.
* I doubt, forexample,whetherit was knownto Bowley and to Gini and
Galvani when theywrotetheirpapers.
t Only the estimatesof this kind will we considerbelow.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 565

Fisher. Moreover, R. A. Fisher has provided tables giving the

values of t such that the probability of their being exceeded by
10'l/ OI/ has definite values such as *oi, 02, . . . etc. This
table * was publishedlong beforeany paper dealing withthe solution
ofthe problemofestimationindependentofthe probabilitiesa priori.
However, this solution is already contained in the table. In fact it
leads directlyto the constructionof the confidenceintervals. Sup-
pose the confidence coefficientchosen is r= 99. Obtain from
Fisher's table the value of t, say tE,correspondingto the size of the
sample we deal with and to a probability of its being exceeded by
lo' - 01/ equal to 1- = *oI. It may then be easily shown that
the confidenceinterval,correspondingto the coefficient? = *99 and
to the observedvalues of O' and V, will be given by the inequality
0 - V-t,
< 0 < O' + V-te .. (5)
5. The previous statementis rigorouslytrue if the distributionof
the x's is normal. But, as it has been experimentallyshown by
E. S. Pearson,t the above result is very approximately true for
various linear estimates by fairlyskew distributions,provided the
sample dealt with is not exceedinglysmall, say not smaller than of
I 5 individuals. Obviously,whenapplyingthe representativemethod
to social problemsthis is a limitationof no importance. In fact, if
the samples are very large, the best linear estimates follow the
normal law of frequency,and the multipliertEin the formula giving
the confidenceinterval may be found fromany table of the normal
integral.1
The above propertiesof the linear estimates make them exceed-
ingly valuable fromthe point of view of their use in applying the
representativemethod. I proceed now to the Markoffmethod of
findingthe best linear estimates.
This may be applied under the followingconditions,which are
frequentlysatisfiedin practical work.
Suppose we are dealing with k populations,

71,7r2* . . . (6)
fromwhich we may draw random samples. Let

Xi1, Xi2, Xn.. ..(7)

be a sample, St, of nj individuals randomly drawn (with replace-

ment or not) fromthe population 7ri. Let Ai be the mean of the
* R. A. Fisher: StatisticalMethodsfor Research Workers,London, 1932,
Edition IV.
t This Journal,Vol. XCVI, Part I.
I For example, Table I of the Pearson's Tables for Statisticiansand Bio-
metricians.Part I. may be used.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
566 NEYMAN-On theTwo Different [Part IV,

population Ti. We have now to make some assumption about the

variances, ai2, of the populations 7r. The actual knowledgeof these
variances is not required. But we must know numbers which are
proportionalto aj2. Thus we shall assume that
a02
ai2 pi.... (8)

0a2 being an unknownfactor,and Pi a known number.* It would

be a special case of the above conditionsif it were known that
= C2=. . -CSk
= .(9)

the commonvalue of the a's being unknown.

Suppose now we are interestedin the values of one or several
collective characters of the populations, Tri,each of them being a
linear functionof the means of these populations, say
Oj aj,Al + aj2A2+ . . . + ajkAk * * (10)

where the a's are some known coefficients. Markoffgives now the
method of findinglinear functionsof the x's determinedby samples
fromall the populations, namely,
of '= llx1 + X12X12 + * * * + jn1x1,n1 +
+ . . . . . . . . . . . . +

. ... ....... + . (11)

+ XklXkl + )1k2Xk2 + * * * + )'rnkXknk

thevalueofunknown
such,thatwhatever O:
(a) Mean O'-in repeated samples Oj.
(b) Standard errorof O'jis less than that of any other linear func-
tion, satisfying(a).
The details concerningthis method are given in Note II of the
Appendix.
It is worth consideringthe statistical meaning of the two con-
ditions (a), (b), when combined with the fact that if the number of
observationsis large,the distributionof O' in repeated samplingtends
to be, and forpractical purposes is actually normal. The condition
(a) means that the most frequentvalues of O' will be those close to
0. Therefore,if 4 is some linear functionof the x's, which does not
satisfythe condition (a), but instead the condition,
Mean 4 in repeated samples = 0 + A, (say),
then, using 4 as an estimate of 0, we should commit systematic
errors,which most frequentlywould be near .A. Such estimates as
4 are called biased.
The condition (b) assures us that when using 0"s as estimates of
* Sometimes,in special problems,even this knowledgeis not required.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 567

O's, we shall get confidenceintervals correspondingto a definite

narrowerthan those obtained using any other
confidencecoefficient,
linear estimate. In other words, using linear estimates satisfying
the conditions(a) and (b) we may be sure that we shall not commit
systematicerrors,and that the accuracy of the estimate will be the
greatest.

III. DIFFERENT ASPECTS OF THE REPRESENTATIVE METHOD.

We may now proceed to consider the two aspects of the repre-

sentative method.
1. The MethodofRandomSampling.
The method of random sampling consists, as it is known, in
taking at random elementsfromthe population which it is intended
to study. The elements compose a sample which is then studied.
The resultsformthe basis forconclusionsconcerningthe population.
The nature of the population is arbitrary. But we shall be con-
cerned with populations of inhabitants of some country,town, etc.
Let us denote this population by II. Its elements will be single
individuals, of which we shall consider a certain character x, which
may be measurable or not (i.e. an attribute). Suppose we want to
estimatethe average value of the characterx, say X, in all individuals
formingthe population II. It is obvious that in the case where x is
an attribute,which may be possessed or not by the individuals of
the population, its numericalvalue in these individuals will be 0 or 1,
and its mean value X will be the proportion of the individuals
having actually the attributex.
The method of random samplingmay be of several types:
(a) The sample, E, which we draw to estimate X is obtained by
taking at random single individuals fromthe population H. The
method of sampling may be either that with replacement or not.
This type has been called by ProfessorBowley that of unrestricted
sampling.
(b) Before drawing the random sample from the population H
this is divided into several " strata," say
Hl), rl2, *...@ Hkc . . . . . (12)
and the sample E is composed of k partial.samples, say
El) E2n * @ @ . .. .
.k (13)
each being drawn (with replacement or not) from one or other of
the strata. This method has been called by ProfessorBowley the
method of stratifiedsampling. Professor Bowley considered only
the case when the sizes, say, m's, of the partial samples are pro-

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
568 NEYMAN-Ot theTwo Different [Part IV,

portionateto the sizes of correspondingstrata. I do not thinkthat

this restrictionis necessary and shall consider the case when the
sizes of the strata, say
M'1 M'2 . . . . (14)

and the sizes of partial samples, say

Mlk . . . . . (15)
. mo
Mli1,Mli * ffi
are arbitrary.
In many practical cases the types of sampling described above
cannot be applied. Random samplingmeans the method of includ-
ing in the sample singleelementsofthe population withequal chances
for each element. Human populations are rarely spread in single
individuals. Mostly they are grouped. There are certainlyexcep-
tions. For instance, when we consider the population of insured
persons,they may appear in books of the insurance officesas single
units. This circumstancehas been used among others by A. B.
Hill,* who studied sicknessof textileworkers,using a randomsample
of persons insured in certain Approved Societies. But these cases
are rather the exceptions. The process of sampling is easier when
the population fromwhich we want a sample to be drawn is not a
population of persons who are living miles apart, but some popula-
tion of cards or sheets of paper on which are recordedthe data con-
cerningthe persons. But even in this simplifiedposition we rarely
findungroupeddata. Mostly,forinstance when we have to take a
sample fromthe general census data, these are grouped in some way
or other,and it is exceedinglydifficultto secure an equal chance for
each individual to be included in the sample. The groupingof the
general census data-for the sake of definitenesswe shall bear this
example in mind-has generallyseveral grades. The lowest grade
consistsperhaps in groupingsaccordingto lodgings: the inhabitants
of one apartmentare given a single sheet. The next groupingmay
include sheets correspondingto apartmentsin several neighbouring
houses t visited by the same officercollecting the data for the
Census. These groupsare thengroupedagain and again. Obviously
it would be practicallyimpossible to sample at random single indi-
viduals fromdata subject to such complex groupings. Thereforeit
is useful to consider some furthertypes of the random sampling
method.
(c) Suppose that the population II of M' individuals is grouped
into Mo groups. Instead of consideringthe population II we may
* A. B. Hill: Sickness amongstOperativesin Lancashire CottonSpinning
Mills, London 1930.
t This was the groupingused in the Polish General Census in 1931. The
correspondinggroups will be called " statistical districts." The number of
personsin one statisticaldistrictvaried from30 to about 500.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 569

now consideranother population, say T, having forits elementsthe

MO groups of individuals, into which the population II is divided.
Turningto the example of the Polish Census, in which the material
has been kept in bundles, containingdata fromsingle statistical dis-
tricts, it was possible to substitute the study of the population 7t
of MO = 123,383 statisticaldistricts,forthe study of the population
II of M'= 32 millionindividuals. If thereare enormousdifficulties
in samplingindividuals at random, these difficulties may be greatly
diminished when we adopt groups as the elements of sampling.
This being so, it is necessary to consider, whether and how our
originalproblemof estimatingX, the average value of the character
x of individuals formingthe population II, may be transformedinto
a problem concerningthe population of groups of individuals.
The numberwe wish to estimate is
1 M'
X (xi) .(16)

where xi means the value of the character x of the i-th individual.

in groupingthe termsof the sum on
Obviously there is no difficulty
the right-handside of the above equation so that each group of
termsrefersto a certaingroup of individuals,formingthe population
7t. Suppose that these groups contain respectively

V1, V2, . . . . (17)

individuals and that the sums of the x's correspondingto these
individuals are
U1, U2, . . . UM (18)
With this notation we shall have
M-VI + V2 + **+ VM (V) . * (19)

(xi) =-1 + U2 + uM = (uf) (say). (20)

The problem of estimatingX is now identical with the problem

of estimatingthe character of the population 7t,namely,

X __ .(21)

WTehave now to distinguishtwo different cases: (a) the number

M' of individuals formingthe population II is known, and (b) this
numberis not known.
In the firstcase the problem of estimatingX reduces itself to
that of estimatingthe sum of the u's in the numeratorof (21).
In the other case we have also to estimatethe sum of the v's in the
denominatorand, what is more,the ratio of the two sums. Owing

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
570 NEYMAN-On theTwo Different [Part IV

to the resultsof S. Bernsteinand of R. C. Geary this may be easily

done if the estimatesof both the numeratorand the denominatorin
the formulagiving X are the best linear estimates. The theoremof
S. Bernstein* applies to such estimates, and states that under
ordinaryconditionsof practical worktheirsimultaneousdistribution
is representableby a normalsurfacewithconstantseasy to calculate.
Of course thereis the limitingconditionthat the size of the sample
must be large. The result of Geary t then makes it possible to
determinethe accuracy of estimation of X by means of the ratio
of the separate estimates of the numeratorand the denominator.
Thus we see that ifit is impossibleor difficult
to organizea random
sampling of the individuals formingthe population to be studied,
the,difficulty may be overcome by sampling groups of individuals.
Here again we may distinguishthe two methodsof unrestrictedand
of stratifiedsampling. It is indisputablethat the latter has definite
advantages both fromthe point of view of the accuracy of results
and of the ease in performingthe sampling. Thereforewe shall
furtherconsider only the method of stratifiedsampling from the
population , the elementsof whichare groupsofindividualsforming
the population HI. It is worthnotingthat this formof the problem
is very general. It includes the problem of unrestrictedsampling,
as this is the special case when the number of strata k i. It
includesalso the problemof samplingindividualsfromthe population
H, as an individual may be consideredas a group,the size of which
is v Ix. We shall see furtheron that the method of stratified
samplingby groupsincludesas a special case the methodofpurposive
selection.

2. The Methodof PurposiveSelection.

ProfessorBowley did not considerin his book the above type (c)
of the method of random sampling by groups.1 When, therefore,
he speaks about the principleof random samplinghe is referring to
the sampling of individuals. Accordingto Bowley, the method of
purposive selection differsfromthat of random samplingmainly in
the circumstancesthat " in purposive selectionthe unit is an aggre-
gate, such as a whole district,and the sample is an aggregateof these
aggregates,while in random selection the unit is a person or thing,
which may or may not possess an attribute,or with which some
measurable quantity is associated. . . . Further,the fact that the
selectionis purposivevery generallyinvolves intentionaldependence
* S. Bernstein: " Sur l'extensiondu theoremelimite du calcul des prob-
abilites." Math. Ann., Bd. 97.
t R. C. Geary: "The Frequency Distributionof the Quotient of Two
Normal Variates." J.R.S.S., Vol. XCIII, Part III.
t Though he applied it in practicalwork.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 1934] Aspects of the RepresentativeMethod. 571

on correlation,the correlationbetween the quantity sought and one

or more known quantities. Consequently the most important
additional investigationin this section relates to the question how
far the precision of the measurementsis increased by correlation,
and how best an inquirycan be arrangedto maximizethe precision."
It is clear fromthis quotation that the terminologyof Professor
Bowley and that which I am using do not quite fit together. In
fact the circumstancethat the elementsof samplingare not human
individuals, but groups of these individuals, does not necessarily
involve a negation of the randomnessof the sampling. ThereforeI
have thoughtit usefulto considerthe special type ofrandomsampling
by groups, and the nature of the elements of sampling will not be
furtherconsidered as constitutingany essential differencebetween
random samplingand purposive selection.
The words purposive selection will be used to definethe method
of proceduredescribedby Bowley, Gini and Galvani. This may be
divided into two parts: (a) the method of obtainingthe sample, and
(b) the methodofestimationofsuch an average as X, describedabove.
The method of obtainingthe sample assumes that the population
H of individuals is divided into several, M, districtsformingthe
population , that the numberof individuals in each district,say vi,
is known and, moreover,that there is known for each districtthe
value of one or more numericalcharacters,which ProfessorBowley
calls " controls." There is no essential differencebetween cases
where the number of controlsis one or more, so we shall consider
only the case wherethereis one control,which we shall denote by yi
for the i-th district. We shall retain onr previous notation and
denote by ui the sum of values of x, correspondingto the i-th district
or group. Considernext, say, ij == ui/vior the mean value of the
characterx in the i-th district. The basic hypothesisof the method
of purposive selectionis that the numbersx- are correlatedwith the
control yi and that the regressionof xj on yi is linear. As we shall
have to referagain to thishypothesis,it willbe convenientto describe
it as the hypothesisH.
Assumingthat the hypothesisH is true, the method of forming
the sample consists in " purposive selection of such districtsfor
which the weightedmean
, y Y(vy)
(v)(22)
has the same value, or at least as nearlythe same as it is possible, as
it has forthe wholepopulation, say Y. It is assumed that the above
method of selection may supply a fairlyrepresentativesample, at
least withregardto the characterx. As it followsfromthe quotation
from the work of Gini and Galvani, it was also believed that by

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
572 NEYMAN-On the Two Different. [Part IV,

multiplyingthe controlsit would be possible to obtain what could

be termed a generallyrepresentativesample with regard to many
characters. Otherwisethe method of purposive selection could not
be applied to supply a sample which could be used in the futurefor
purposes not originallyanticipated.
This is the method of obtainingthe sample. As we shall easily
see, it is a special case of stratifiedrandom samplingby groups. In
fact, though the three authors think of districtsas of rather large
groups with populations attaining sometimes one million persons,
they assume that the numberM of these districtsis not very small.
In the Italian investigationit was over 200. If we consider the
values ofthe control,y, calculated foreach district,we shall certainly
findsuch districtsfor which the value of y is practically the same.
Thus the districtsmay be grouped in strata, say of the firstorder

7Yi' 7Y2' * n Yk (23)

each correspondingto a given value of y. Now each of the first

orderstrata of districtsmay be subdivided into several second order
strata, accordingto the values of v in the districts. Denote by 7yv a
stratumcontaining,say,Myvdistricts,all ofwhichhave practicallythe
same values ofthe control,y, and the same numberof individuals v.
Denote furtherby myvthe number of the districtsbelongingto 7tyv
to be includedin the sample. If the principledirectingthe selection
consists only in the fulfilmentof the condition that the weighted
mean of the controlwith v's as weights should be the same in the
sample and in the population, then it means nothingbut a random
sampling of some myvdistrictsfromeach second order stratum,the
numbersmyvbeing fixedin advance, some of them being probably
zero. This is obvious, since for purposes of keeping the weighted
mean Y'= Y = constant, two differentdistrictsbelongingto the
same second orderstratumare of equal value. Hence we select one
of them at random.*
Thus we see that the method of purposive selection consists,(a)
in dividingthe population of districtsinto second orderstrata accord-
ing to values of y and v, and (b) in selectingrandomlyfromeach
stratuma definitenumber of districts. The numbersof samplings
* It must be emphasized that the above interpretation of the method of
purposiveselectionis a necessaryone ifwe intendto treat it fromthe point of
view of the theory of probability. There is no room for probabilities,for
standarderrors,etc., wherethereis no randomvariationor randomsampling.
Now if the districtsare selectedaccordingto the correspondingvalues of the
controly and also ofthe numberofindividuals,v,theycontain,the onlypossible
variate which is left to chance is 4s. If the districtsare very large and
thereforeonly veryfew,then the majorityof second orderstrata will contain
no or onlyone district. In this case, ofcourse,the processofrandomsampling
fromsuch a stratumis an imaginaryone.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method 573

are determinedby the condition of maintenance of the weighted

average of the y. Comparing the method of purposive selection
with that of stratifiedsamplingby groups we have to bear in mind
these two special featuresof the former.

IV. COMPARISON OF THE TWO METHODS OF SAMPLING.

1. Estimatesof Bowleyand of Gini and Galvani.

Suppose now the sample is drawn and considerthe methods of
estimationof the average X. In this respectthe Italian statisticians
do not agree with Bowley, so we shall have to considertwo slightly
differentprocedures. I could not exactly follow the method pro-
posed by Professor Bowley. It is more clearly explained by the
Italian writers, but I am not certain whether they properly
understoodthe idea of Bowley. It consistsin the following:
Denote by X>, the weightedmean of values xi deduced fromthe
sample, 1; by x, the unweightedmean of the same numbers,also
deduced from the sample. Y will denote the weighted mean of
the control y, having ex hypothesiequal values for the sample and
for the population. y will denote the unweighted mean of the
controly, calculated forthe population, and finallyg the coefficient
of regressionof Tj on yi,calculated partlyfromthe sample and partly
fromthe population.
As a firstapproximationto the unknown X, X>, may be used.
But it is possible to calculate a correction,K, to be subtractedfrom
X>, so that the difference X, - K shouldbe consideredas the second
approximationto X. The correctionK is given by the formula

K = - (X -xT) + g(Y -y9) . . . (24)

As the value of X is unknown,its firstapproximation X>, may be
substitutedin its place. In this way we get as a second approxima-
tion to X the expression,say,

X'-X >,y+ (X>,-x) -g(Y -y) ...(25)

I do not know whetherthis is the method by which Bowley has
calculated the very accurate estimatesin the examples he considers
in his paper. At any rate the method as described above is incon-
sistent: even if applied to a sample includingthe whole population
and even if the fundamentalhypothesisH about the linearity of
regressionof xi on yi is exactly satisfied,it may give wrongresults

X' X .(26)
This may be shown on the followingsimple example. Suppose
VOL. XCVII. PART IV. X

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
574 NEYMAN-On theTwo Different [Part IV,

that the population 7t consistsonly of fourdistrictscharacterizedby

the values of xi, y, and vi as shown in the followingTable I.

TABLE I.

Districts. Xi. /i- vi. ui = xivi yivi.

I. 07 09 100 7 9
II. 09 09 400 36 36
III. *11 *12 100 11 12
IV. | 13 *12 900 117 108

Totals 40 *42 1500 171 165

Means x 100 y = *105 X = '114 Y = *110

values,
Owingto the fact that the controly has only two different
09 and *12,thereis no question about the hypothesisH concerning
the linearityof regression,which is certainlysatisfied. The regres-
sion line passes through the points with co-ordinates (y= -09,
x= 08) and (y= *12,x= *12). Thus the coefficient of regression
g ,j-. Assume now we have a sample fromthe above population,
which includes the whole of it and calculate the estimate X' of
X *14. We shall have
X Y.= = *114
+ (X>, -x)= -*014

-9(y- -02 -007 . .(27)

X' *121,
which is not equal to X>, = *114.
Gini and Galvani applied Bowley's method to estimate the
average rate of natural increase of the population of Italy, using a
sample of 29 out of 2I4 circondari. They obtained results which
they judged to be unsatisfactory,and theyproposed anothermethod
of estimation. This consistsin the following:
They startby findingwhat could be called the weightedregression
equation. If there are several controls,say y(l), y(2)7 y(S), the

weightedregressionequation
+ b2y(2)+-..
x = bo+ bly(-" + bsy(s) (28)
is foundby minimizingthe sum of squares
Ev(x - bo-b.1yi) -. . . - bsyicS))2 (29)

withregardto the coefficients bo,bl, b2. . . . b,. This processwould

follow fromthe ordinaryformulaeif we assumed that one district.
withthe numberof individualsvi and the mean characterxj is equiva-
lent to vi individuals, each having the same value of the character

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 1934] Aspects of the RepresentativeMethod. 575

x =J. Having noticed this, it is not necessaryto go any further

into the calculations. If there is only one control, y, then the
weighted.regression equation will be differentfromthe ordinaryone
in that it will contain weighted sample means of both Ji and yt
instead of the unweightedones, and that in the formulaof the regres-
sion coefficientwe should get weightedinstead of unweightedsums.
The weightedregressionequation is then used by Gini and Galvani
to estimatethe value of xj foreach district,whetherincluded in the
sample or not. This is done by substitutinginto the equation the
values of the control yi correspondingto each districtand in cal-
culating the value of the dependentvariable. The estimates of the
means xj thus obtained, say xi', are then used to calculate their
weightedmean

X' )
E(vixs . . . . (30)
which is consideredas an estimateof the unknownmean X.
Simple mathematicalanalysis of the situation proved (see Note
III) that this estimate is consistentwhen a special hypothesis,H',
about the linearityof regressionof xj on ygholds good, and even
that it is the best linear estimateunder an additional condition,Hj,
concerningthe variationof the Ti in strata correspondingto different
fixedvalues of y and v.
The hypothesisH' consistsin the assumptionthat the regression
of x on y is linear not only if we considerthe whole population 7t of
the districts,but also if we consider only districtscomposed of a
fixednumberof individuals. It is seen that the hypothesisH' is a
still more limitingthan the hypothesisH.
The other condition,H1, is as follows. Consider a stratum, ',
definedby the values y= y' and v v' and consider the districts
belongingto this stratum. Let

X1,~
X27X.. . (31)
be the values of the means x correspondingto these districts. The
hypothesis,say H1, under whichthe estimate of X proposed by Gini
and Galvani is the best linear estimate,consists in the assumption
that the standard deviation, say a' of the xj correspondingto the
stratum r' may be presentedby the formula

=_
a/W . . . . ..(32)

a being a constant,independentof the fixedvalue of v v'. This

hypothesiswould be justifiableifthe population ofeach districtcould
be consideredas a random sample of the whole population H. In
fact, then the standard deviation of means, xj correspondingto

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
576 NEYMAN-On the Two Different [Part IV,

districtshaving theirpopulation equal to v would be proportionalto

vq-. The population of a single districtis certainlynot a random
sample fromthe population of the country,so the estimate of Gini
and Galvani is not the best linear estimate-at least in most cases.
Having got so far we may considerwhetherand to what extent
thereis justificationforthe principleof choosingthe sample so that
the weightedmean of the controlin the sample should be equal to
the weightedmean of the population. The proper criterionto use
in judging seems to be the standard error of the estimate of X'.
This is given by a function(see Note III) which,cceterisparibus,has
smallervalues whenthe weightedsample mean of the controlis equal
to its populationvalue, and whenthe sum of weightsE(v), calculated
for the sample, has the greatestpossible value. Thus the principle
of purposive selectionis justified. The analysis carriedout in Note
III suggestsalso that if the numberof districtsto be included in the
sample is fixed we should get greater accuracy by choosing larger
districts rather than smaller ones. This conclusion, however,
depends largelyupon the assumptionsmade concerningthe standard
deviations withinthe districtsand the linearityof regression.
2. The HypothesesunderlyingbothMethodsand theConditionts of
Practical Work.
We may now considerthe questions: (1) Are we likelyto findin
practice instances where the hypotheses underlyingthe method of
purposive selectionare satisfied,namely,the hypothesisH' concern-
ing the linearityof regressionand the hypothesisH1 concerningthe
variation of the charactersought withinthe strata of second order?
(2) If we find instances where these hypotheses are not satisfied
exactly, then what would be the result of our ignoringthis fact and
applying the method of purposive selection? (3) Is it possible to
get any better method than that of purposive selection? *
With regard to (1), I have no doubt that it is possible to find
instances,when the regressionof a certain characterxj on the con-
trol yj is fairlynearly linear. This may be the case especially when
one of the charactersx and y is some linear functionof the other,
say if x is the rate of natural increase of the population and y the
birth-rate. This is the example considered by Gini and Galvani.
I think, however, that this example is rather artificial. When y
is known for any district,in most cases we shall probably have all
the necessarydata to enable us to computethe x withoutany appeal
to the representativemethod. In other cases, however, when the
connectionbetween the character sought and the possible control
is not so straightforward, I think it is rather dangerous to assume
* I.e. a methodwhichwould not lose its propertyof being consistentwhen
the hypothesisH' is not satisfied.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 577

any definitehypothesisconcerningthe shape of the regressionline.

I have worked out the regressionof the mean income xi of people
inhabitingdifferentcircondarlon the firstof the controlsused by
Gini and Galvani, i.e. the birth-rate,yi. The figuresI and II give
respectivelythe approximate spot diagram of the correlationtable
of those characters,and the graph of the weighted regressionline
of xj and y,. It is to be rememberedthat the data concernthe whole
population, and thus the graph representsthe " true " regression
line. This is far frombeing straight. It is difficult,of course, to
judge how oftenwe shall meet in practice considerabledivergencies
fromlinearity. I think,however,that it is rathersafer to assume
that the linearity is not present in general and to consider the
position when the hypothesisH' is not satisfied.
The hypothesisH1 is probably never satisfied.
With regardto (2): Note III showsthat the estimateof Gini and
Galvani generallyceases to be unbiased when we can no longermake
any assumption about the shape of the regressionline of x on y.
It may be kept consistentonly by adjusting in a very special manner
the numbers of districtsselected from single second order strata.
In fact the consistencyrequires that the number of districts,say
m' to be selected froma stratumcontainingaltogetherM' districts,
should satisfythe condition
ml' Z(v) forthe sample
M E(v) forthe population (33)

Any departurefromthis rulemay introducesomebias in the estimate.

With regard to (3): There is no essential difficultyin applying
Markow's methodto findthe best unbiased estimatesof the average
X determinedfroma sample obtained by the method of stratified
samplingby groups. This has been done in full detail in my Polish
publication (there is an English summary)* concerningthe theory
of the representativemnethod. The principle of stratifying,i.e. of
the division of the originalpopulation of districtsinto strata, does
not affectthe method of obtaining the estimate. In any case, and
whatever the variances of the xj within the strata, the best linear
estimateof X is always the same.
I shall returnhere to variables introducedpreviously and shall
use
uv= vzxt . . . . . . . (34)
instead of x. Suppose that in m' samplings from a stratum con-
taining M' districts,we obtained m' different
values of u. Denote
by ut their arithmeticmean. Then the product M'1i will be the
* J. Neyman: An OutlineoftheTheoryand PracticeofRepresentative Method,
A..plied in Social Research. InstituteforSocial Problems.Warsaw. 1933.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
578 NEYMAN-On theTwo Different [Part IV,

FIG.1.

c~~~~~~

~~~ 0 0 0

'2 0 SBIRH SAT

RIEGIRESS.0 0 F0 0 0N

0 0 0 0 0

H~~~ FIG. 0

21ZI-.+0 0 03-6
0 0 _0
F,'lf-1818 0717 01
0 00 0 0 B0F0T.00000

SPOTS CRRES 000 0 000YICT FOMNGTESAPE

0 00*0 0000~]P .0 0.. 0 0

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 579

estimate of the sum of the u's for the whole stratum. Summing
these estimates for all strata, we get the best estimate of the sum
of u's forthe whole population. To get an estimate of X it remains
only to divide the estimate of the sum of u's by the sum of v's,
which may be known or may be estimated by the same method.
Thus the finalestimate of X say X" is either

x" :(M'ii) . .(35)

if the v's are knownforevery district,or in the othercase

X" = S(M u) . . . . .
(36)

where -v means the arithmetic mean of v's, calculated from the

sample separatelyfor each stratum.
The consistencyof the estimates E(M'ft) and E(M%'v)does not
depend upon any arbitrary hypothesis concerning the sampled
population. The only condition,which must be satisfiedis that the
sample should contain districtsfrom every stratum. So we may
safely apply these estimates, whatever the properties of single
strata and irrespectiveof variations of it's and v's withinthe strata.
But the standard errorsof the two estimates do depend both upon
the variability of the characters of districtswithin the strata and
upon the relationshipof numbers m' and M'. It is known that
the formulagiving the variance, say a2, of the estimate E(M'ft) is
as follows:
CY
= jm M-1
j CI ***. (37)

where mi and Mi referto the i-th stratum,Gj2 is the variance of the

u's in the i-th stratumand the summationE extendsover all strata.
The dependence of a2 upon the aj2 is obvious. If we succeed in
dividing the population rcinto strata which would be very homo-
geneous with regard to the character u of the districts,aj2 will be
small and so will be a2. It is also obvious that by increasing the
nunmbers, mi, of districtsto be selected from the strata we shall
also improvethe accuracy of the estimate. By taking mni- M the
accuracy will be absolute, but then we shall have an exhaustive
enquiry. It will probably be necessaryto assume that the actual
conditionsof the researchfix a certainnumber,say
MO= (mi).(38)
of districtsto be selected fromthe population. Our problem will
then consist in distributingthe total number of samplings among
singlestrata so as to have the minimumpossible value of a2.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
580 NEYMAN-On the Two Different [Part IV,

Simple calculations show that the variance (37) may be written

in the form
M-2
- 0- (MS,2) + mM,im MiSi))

MKo
EMi Si E(lM+&S,)) (39)

whereSj2 stands for Miai2/(M, - 1). We see that only the middle
term of the right-handside depends upon the values of the m's.
The othertermsremain constant whatever the system of m's, pro-
vided their sum, mO, remains unchanged. Thus the method of
diminishingthe value of a2 consistsin diminishingthe middle term
of the right-handside of (39). This has its minimumvalue, zero,
when the numbersmi are proportionalto the productsMiSi. Thus
if it is possible to estimate the variances aj2 of the u's within any
given stratum,the most favourable system of mi's is not that for
which the mi are proportionalto the Mi. Denote the three terms
of the right-handside of (39) respectivelyby A, B and -C. If we
assume that the m,'s are proportionalto Mi, then we shall findthat
the termB = C and the variance a2 is reducedto

a2= M- MzO(MS,2) - A . . (40)

If, however, mi are proportionalto MiSi, then the positive term

B in (39) vanishes and we get
G2 = A -C .(41)
whichis the optimumvalue of a2.
If the researchis carried out with regard to several highlycor-
related characters of groups formingthe elements of sampling,
then by means of a preliminaryenquiry it is possible to estimate
the numbers Si, which, if calculated for the differentcharacters
sought,would be also correlated. Hence we could then by a proper
choice of the numbersmi if not reduce the middle termofthe right-
hand side of (39) to zero, then at least diminishit sensibly.
Such was the case in the Warsaw enquiry already referredto,
carried out by the Institute for Social Problems. The purpose of
this enquiry was to describe the structureof the workingclass in
Poland, accordingto differentcharacters,such as the age distribu-
tion of males and females,whethermarriedor single,the distribution
of the number of childrenin families,etc., and this separately for
three differentcategories of workers. Obviously all characters of
the elements of sampling sought are highly correlated with the
number of workersin each element. As there are in Poland large

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 581

districtswhere the percentage of workersis negligible and others

wherethey are numerous,the numbersSi calculated forthe different
characterssought varied fromstratum to stratum in broad limits.
Accordingly,an adjustment of numbers mi was made in order to
diminishvariances of the estimates.
The necessityof these adjustmentsis not difficultto appreciate.
One feels intuitivelythat it would be unreasonable to include in
the sample, equal percentagesof statistical districtsfromtwo strata
A and B in one of which,A, the percentageof workers,amounts to
say 6o per cent. and in the other, B, to 5 per cent. It may even
be assumed that in such cases it would be advisable to omit totally
the stratumB. However,I do not thinkit is reallyalways advisable,
since the total numberof workersin the stratumB may be some-
times equal to or even larger than those in stratum A, and the
structureof familyconditionsin both strata may be very different.
Of course this sort of researchis a ratherspecial one. In many
cases the characterssought are not likely to be highly correlated.
In othercases-as in the work of Gini and Galvani-it is impossible
to state at the time of sampling which charactersof the elements
of sampling will be the matter of research. Any adjustments of
the numbers,mi, are then impossible, since a wrongfadjustment
may give to a2 a value largerthan that correspondingto the system
of proportionalsampling. The best we can do is to sample pro-
portionatelyto the sizes of strata.*
Thus the principlethat the numbersmi should be proportional
to Mi, suggested by Professor Bowley, is just the best that one
could advise in the most general case.
Up to this point I have considered the possibility of reducing
the value of a2 by adjusting properlythe numbersm, of samplings
fromdifferent strata. I assumed, in fact,that the districtsforming
the elementsof sampling and theirtotal number mo to be included
in the sample are fixed. Now I shall suppose that the districtsare
not fixedexcept that their size will not be very different, and that
all that is known is that the sample should include a certain per-
centage of districts,whatever be their kind.
In other words, I intend to considerthe situation in which we
decide to includein the sample some, e.g. Io per cent. of the popula-
tion, and are consideringthe question what should be our " dis-
tricts," formingthe elements of sampling: whether they should
include about, say 200 or about 20,000 persons, etc.
I wish to call attentionto the fact, that the ratios mi/Mibeing
fixedin some way or other,the value of a2 (see (37)) depends upon
* It is to be rememberedthat " the size of the stratum" is the number,
Mi, of its elements,not the numberof individuals.
x2

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
582 NEYMAN-On the Two Different [Part IV,

the productsM,S,2= M-2a 2/(M,-1), or practicallyupon the

products Mii2 , and may be influencedby a proper choice of the
elementof sampling. In fact, if we consider two different systems
of division of a stratum into larger and smaller districts,then the
values of %6'scorrespondingto several smaller districts forminga
largerone, will be verygenerallypositivelycorrelated. As the result
of this the value of M,a,2, correspondingto a subdivision of strata
into smaller districts,will be less than that correspondingto a sub-
division into larger districts. This point may be illustratedon an
extreme case. Suppose, for instance, that X representsthe pro-
portion of agricultural workers aged 20 to 2I. Then for every
individual of the population x will have the value x 1 if this
individual is an agriculturalworker aged 20 to 2I, and x = 0 in
all other cases. If now we consider as elements of sampling the
statistical districtsincluding 50 inhabitants,then in a stratum we
may have (in the most unfavourablecase) one half of the districts
composed only of agriculturalworkersat the fixedage, thus having
u = 50, while in the otherhalf of the districtu =0. The standard
deviation ai would be 25. On the other hand, if the districtswere
to include not 50 persons,but, say, 500, the maximumpossible value
of ai would be tenfold,250. The term MG,2 in this second case
would be ten times largerthan in the former. Of course it may be
argued that taking larger districtswe decrease the chance of their
being extremely differentiated. This is certainly so, but on the
other hand I think it extremelyprobable that the products MJSi2
calculated for districts including tens of thousands or hundreds
of thousands of people must be expected to be incomparablylarger
than those calculated for the districtsincludingon the average two
or three hundredpeople. And this for the majority of imaginable
characterswhichcould be the matterof statisticalresearch.*
The effectof choosingsmallerunits of samplingmay be roughly
illustratedon another example of a game of chance, in which the
probabilityof a gain is equal to 2. Suppose we dispose of a sum
of Lioo forthe game, which we may eitherbet at once or divide in
a hundred separate bettings. In the firstcase it is obviously im-
possible to predict the result. In the other case, however,we may
* I do not knowwhetherthesewerethe reasonsforwhichGini and Galvani
expressedthe view that the resultsof theirsamplingwould have been much
betterif the methodof selectionadopted were that of stratifiedsampling,and
ifthe elementofsamplingwerea commulne. The reasonsfornot applyingthis
method seems to be that " nobody could under-appreciatethe difficulty in a
stratificationof the communessimultaneouslywith regard to different char-
acters." (Page 6, loc. cit.). I think,however,that a stratification
assuming
the 214 circondarias strata,each containingabout 40 communes,whichmight
be consideredas elementsofsampling,would be quite sufficient. Of coursethe
resultswould be probablystill betterif the elementsof samplingwere smaller
than a comimune.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 583

be prettycertain that the gain or loss will not exceed some Li 5 or

?20.
Similarly, if we want to obtain a representativesample, say
amounting to I 5 per cent. of the population, it is much safer to
make, say, 3,000 samplings of small units rather than 30 of larger
ones, and this is-probablytrue, whateverthe stratification.
3. NumericalIllustration.
It may be perhaps usefulto considera simple numericalexample
showing the effecton the accuracy of the method of purposive
selection of non-linearityof regressionof the character sought on
the control.
We shall considerthe result of sampling fromfour populations,
in one of which the weightedregressionof x on y is linear, and in
three otherswhereit is showingdifferent degrees of deviation from
linearity. All fourpopulations are divided into three strata accord-
ing to the values of the control y= - 1, y_ 0 and y= + 1.
Each stratum contains three districts. The construction of the
population, say n, with linear weighted regression is shown in
Table II.
TABLE II.

Y= -1 Y = O. y= + 1.

No. of u*. v. No. of u- v* No. of Ui. v-.

District i. District i. District i.

1 -17 1 4 1 3 7 20 3
2 -18 2 5 0 2 8 18 2
3 -19 3 6 -1 1 9 16 1
Totals -54 6 0 6 54. 6
Means x-1) -9 - J (0) = 0 - (1) 9

As in the actual calculationswe have to use the productsxi=Uv i

I have omittedthe values of the x's and have given the values of
the u's instead. It is easy to see that the population values X1 =
Y= 0. The weightedaverages of the x's in each array are given
at the bottom,namely - 9, 0, + 9, and it is seen that the regression
is linear.
The populations 72, 73 and 74 may be obtained fromthe popu-
lation n; so easily that it is not necessaryto describethemin special
tables. The population n2 iS obtained by keeping the strata cor-
respondingto y- 1 and y + 1 unchanged and by adding to
each value of ui in the stratum y= 0 the same number,6. As a
resultof thisthe weightedmean of xi, say x(O) in the middle stratum
will be raised to x(O) 3 and the regressionwill cease to be linear.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
584 NEYMAN-On theTwo Diffierent [Part IV,

X will now have the value X2= 1. The population 73 will be

obtainedfromthe populationn2 in the same way as thiswas obtained
fromthe populationn,. Similarly,the population74 will be obtained
fromn3 by the same operation. The values of the weighted mean
of x's in the stratumy 0 and in the populations.willbe as follows:

?3(0) 6, X3-2, .(42)

.x(O) = 9, X = 3.
I then considered all possible samples from these populations,
subject to the conditions: (a) E(v) = 7, i.e. the numberofindividuals
in the sample (not the numberof elementsof the sample) is fixedin
advance, and (b) the sample weightedmean of the controly should
be equal to its population value Y= 0. The details of the results
obtained are given in the followingTable III:

TABLE III.
Populations. TT2. T3. IT4. All popul.

Districts. A' = X'. A'. A'. A'. A".

1, 2, 6, 7 -2-29 -2-43 -2-57 -2-71 *25

1, 2, 6, 8, 9 - *29 - 43 - 57 - 71 -*25
3, 6, 7 00 - 14 - *29 - 43 00
3, 6, 8, 9 2-00 1-86 1-71 1-57 -50
2, 4, 8 *14 00 - 14 -*28 *17
2, 5, 6, 8 - *14 *57 1-29 2-00 -08
1, 4, 5, 9 *00 *71 1-43 2-14 -08

Here X' and X" mean the estimatesof X, (i) obtained by method
proposed by Gini and Galvani, and (ii) calculated fromthe formula
(35). A' = X'-X and A" -X" - X represent the errors of
these estimates. It willbe seen that the estimateX" gives generally
better results. But this is not an essential point in the example,
as it is easy to constructanotherin which the estimate X' would be
the better. In fact, the accuracy of X" is connected with the
variabilityofthe u's withinthe strata. If in singlestratacorrespond-
ing to different values of y, the variation of the u's is very large,
then the results obtained by using X" would not be very good.
The comparisonbetweentwo methodscould perhaps be worked out
arithmeticallyif we were to considersecond orderstrata. But this
would extend the example to the point of losing its illustrative
properties.
What is importantto note is that the results obtained by using
X' get worse and worse with the departure fromthe linearity of
regression. This last circumstance does not affectthe accuracy
of X" at all. On the otherhand, a change in the values of aj would
affectX".

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 585

V. CONCLUSIONS.

Let us now turn to the question,which I raised at the beginning

of the paper, whetherthe idea of a certain equivalency of the two
aspects of the representativemethod is really justified. We shall
have to considerboth the theoryand the practical resultsobtained
by bothmethods. ProfessorBowley,who was firstto give thetheory
of the methcd of purposive selection,has not, I believe, used it in
practice. The most important research, known to me, by which
the representativemethod was used, is the New Survey of London
Life and Labour. It has been directed by Bowley, who chose the
methodofrandomsamplingby groups. This is, I think,an example
of the intuitionto which Laplace referred.
The Italian statisticians,who applied the method of purposive
selection of very few (29) and very large districtswith populations
from about 30,000 to about i million persons, did not find their
results to be satisfactory. The comparison between the sample
and the whole country showed, in fact, that though the average
values of seven controlsused are in a satisfactoryagreement,the
agreement of average values of other characters,which were not
used as controls,is often poor. The agreementof other statistics
besides the means, such as the frequencydistributions,etc., is still
worse. This applies also to the charactersused as controls. The
statementof the above facts is followedin the paper by Gini and
Galvanii by general considerations concerning the concept of a
representativesample. They question whetherit is possible to give
any precise sense to the words " a generallyrepresentativesample."
I think it is, and I agree also that an exhaustive enquiry is
the only method which can give absolutely true results. How-
ever,the need fora representativemethodis an urgentone and man,y
enquirieswould be impossibleif we were not able to use this method.
In fact we are oftenforcedto apply sampling for general purposes,
so as to get a " generallyrepresentative sample, which mightbe
used fora varietyof different purposes.
If there are difficulties
in definingthe "generally representative
sample," I think it is possible to definewhat should be termed a
representative methodofsamrplingand a consistent methodofestimation.
These I think may be definedaccurately as follows. I should use
these wordswithregardto the methodof samplingand to the method
of estimation,if they make possible an estimate of the accuracy of
the results obtained in the sense of the new formof the problemof
estimation,irrespectively of theuinknown propertiesof thepopulation
studied. Thus, if we are interestedin a collective characterX of a
populationn and use methodsofsamplingand ofestimation,allowing

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
586 NEYMAN-On the Two Different [Part IV,

us to ascribe to every possible sample, X, a confidenceinterval

X1 (E), X2 (Y) such that the frequencyof errorsin the statements
X1(E) < X < X2(E) . . * * * (43)
does not exceed the limit 1 - e prescribedin advance, whateverthe
unknownpropertiesof the population, I should call the method of
sampling representativeand the method of estimation consistent.
We have seen that the method of random sampling allows a con-
sistent estimate of the average X whatever the properties of the
population. Chocsing properlythe elements of sampling we may
deal with large samples, for which the frequency distributionof
the best linear estimates is practically normal, and there are no
in calculatingthe confidenceintervals. Thus the method
difficulties
of randomstratifiedsamplingmay be called a representativemethod
in the sense of the word I am using. This, of course,does not mean
that we -shall always get correct results when using this method.
On the contrary,erroneousjudgmentsof the form(43) must happen,
but it is known how oftenthey will happen in the long run: their
probabilityis equal to .
On the other hand, the consistencyof the estimate suggested
by Gini and Galvani, based upon a purposely selected sample, de-
pends upon hypotheseswhich it is impossible to test except by an
extensiveenquiry.
If these hypothesesare not satisfied,which I think is a rather
general case, we are not able to appreciate the accuracy of the
resultsobtained. Thus thisis not what I should call a representative
methcd. Of course it may give sometimesperfectresults,but these
will be due ratherto the uncontrollableintuitionof the investigator
and good luck than to the method itself. Even if the underlying
hypotheses are satisfied,we have to rememberthat the elements
of samplingwhich it is possible to use when applying the purposive
selective method, must be very few in number and very large in
size. Consequently I think that when using this method we are
very much in the position of a gambler,bettingat one time gioo.
For the above reasons I have advised the Polish Institute for
Social Problems to use the method of random stratifiedsampling
by groups when carryingout the enquiry on the structureof Polish
workers.*
I23,383 elements
Polandwas dividedintoII3 strata,containing
of sampling (statistical districts). The average number of persons
within an elementof sampling was about 250 persons. There were
* The resultsofthis enquiryare to be foundin the publicationof J. Piekal-
kiewicz: Rapport sur les recherchesconcernantla structurede la population
representative.InstituteforSocial Problems,
ouvriereenPologneselonla methode
Warsaw, 1934.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 1934] Method.
Aspectsof theRepresentative 587

considerable variations from stratum to stratum. The random

stratifiedsample contained altogether I,621 elements, thus about
IP24 per cent. of the whole population. I am not yet able to state
how accurate are the resultsobtained, as the respectivedata of the
General Census are not yet published. All that was possible in
testingtheiraccuracywas to comparethe age distributionof workers
foundin the whole sample with the age distributioncomputed from
a minorsample of 235 elementsselected foran introductoryenquiry

FIG III.
250

200 - / LARGER SAMPLE OF 162i ELEMENTS.

~SMALLEIR SAMPLE OF 235 ELEMENTS.

15 20 25 50 35 +0 +5 50 55 60 65 70 75 80
AGE

whichaimed at testingthe variabilitywithinthe strata. The results

are presented in Figure III and in Table IV, and seem to be
satisfactory. However, even if through the chances of sampling
they bad been bad, I thinkI was justifiedin advising the methodof
stratifiedsamplingby groups,because I was able to calculate that
(with the probabilityof an errorequal to *oI) the errorof actuarial
calculations, based upon the tables which were computed as the
result of enquiry,could not exceed 4-5 per cent.
The method of stratifiedsampling by groups has been recently
used by Professor0. Anderson,*whodirectedan enquiryintothefarm-
* Bull. de Statistique,publ. DirectionGen. de Statistiquede Bulgarie,No. 8,
1934.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
588 NEYMAN-On the Two Different [Part IV,

ing conditionsin Bulgaria. The process of gettingthe sample with

whichhe was faced was a more difficult one, as this was not a sample
of sheetsof paper containingthe necessaryinformation, but a sample
of villages fromwhich it was necessaryto collect the originaldata.
In fact the enquiryin question was a substitutefora generalagricul-
tural census. The element of the sampling was a village. The
total number of about 5,000 villages was divided into 28 strata.
Out of each stratum 2 per cent. of the villages were selected to
formthe sample. There is only one detail in this enquiry which
I am not certain is justifiable. When selecting the villages
from single strata special attention was paid to selecting villages
which according to the last General Census in 1926 showed a dis-

TABLE IV.
Age Distributionof Polish Workers.
Males.

Age. Larger Sample. Smaller Sample.

15-19 ... ... ... 148 141

20-24 ... ... ... 199 213
25-29 ... ... 178 176
30-34 ... ... ... 122 130
35-39 ... ... ... 82 85
40-44 ... ... 67 69
45-49 ... .. 58 54
50-54 ... ... 49 44
55-59 ... ... ... 39 34
60-64 ... ... ... 28 23
65-69 ... ... ... 18 19
70-74. ... ... 9 7
75-79 .. 4 4
Totals ... ... 1001 999

tributionof diflerentcharactersof farms,similarto that in the whole

stratum. I think that the variability of farmsand villages is also
a character of their population which may be of interest. This
character, however, if the effortsof Bulgarian investigatorswere
successful,would be biased in the sample.
The final conclusion which both the theoretical considerations
and the above examples suggest is that the only method which
can be advised for general use is the method of stratifiedrandom
sampling. If the conditions of the practical work allow, then the
elementsof the samplingshould be individuals. Otherwisewe may
sample groups,which,however,should be as small as possible. The
examples of enquiriesin London, in Bulgaria, and in Poland show
that random sampling by groups does not present unsurmountable
difficulties.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 589

There are instances when we may select individuals purposely

with great success. Such is, for instance, the case when we are
interestedin regressionof some variate y on x, in which case the
selection of individuals with values of x varyingwithinbroad limits
wouldgiveus moreprecision. But thesecases are ratherexceptional.*

VI. APPENDIX.
Note I.
Suppose we are taking samples, E, from some population 7t.
We are interestedin a certaincollectivecharacterof this population,
say 0. Denote by x a collective character of the sample E and
suppose that we have been able to deduce its frequencydistribution,
say p(x10), in repeated samples and that this is dependent on the
unknowncollectivecharacter,0, of the population 7.
The collective characters I am speaking about are arbitrary.
The position may be illustrated, for instance, by supposing that
the collective character0 is the proportionof a certain type of in-
dividuals in the population 7t, and x the proportion of the same
type of individuals in the sample. The distributionof x is then a
binomial, dependingupon the value of 0.
Denote now by cp(O)the unknown probability distribution a
priori of 0. Suppose that the general conditions of sampling and
the propertiesof the collective characters 0 and x definecertain
values which these characters may possess. In the example I
mentionedabove, 0, the proportionof individuals of the given type
in the population may be any number between 0 and 1. On the
other hand, x, the proportionof these individuals in the sample,
say of n, could have values of the form k7/n,k being an integer
o< k <n.
The new form of the problem of estimation of the collective
character 0 may be stated as follows: given any positive number
? < 1, to associate with any possible value of x an interval

01(X) < 02(X). * . . . . (1)

such that if we accept the rule of stating that the unknown value
of the collectivecharacter0 is contained withinthe limits
O1(x') < 0 < 02(x'). (2)
every time the actual sampling provides us with the value x= x',
the probabilityof our being wrongis less than or at most equal to
1 - c, and this whatever the probability law a priori, y(O).
The value of ?, chosen in a quite arbitrarymanner, I propose
* Interestingremarksin thisrespectare to be foundin the excellentbook of
M. Ezekiel: Methodsof CorrelationAnalysis (1930).

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
590 NEYMAN-Onl theTwo Different [Part IV,

to call the " confidencecoefficient." If we choose, for instance,

? = *99 and findforeverypossible x the intervals[01(x),02(x)]having
the propertiesdefined,we could roughly describe the position by
saying that we have 99 per cent. confidencein the fact that 0 is
contained between 0_(x) and 02(x). The numbers 0_(x) and 02(x)
are what R. A. Fisher calls the fiducial limits of 0. Since the word
"fiducial" has been associated with the concept of "fiducial pro-

FIG. IV

o~~~~~~~~~~~~~~~~~~~~~~~~~(

,~~~~x (0 ,

0-c~ ~ ~ ~~~~A

bability" which has caused the misunderstandingsI have already

referredto, and which in reality cannot be distinguishedfromthe
ordinaryconcept of probability,I preferto avoid the term and call
the intervals [01(x), 02(x)] the confidenceintervals, corresponding
to the confidencecoefficientc. The solution of the problem thus
stated is an immediateone.
Considerthe plane on which the rectangularaxes of coordinates
OX and 00 are drawn. Fix any possible value of 0, say 0 -O', and
findon the straightline
0-=' . (3)

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 591

an interval,say x1(0') to x2(0'), having the propertythat in cases

when O' is the truevalue ofthe collectivecharacter0, the probability,
say P(0'), of having fromthe sample x withinthe limits
`:
X1(0') `< X X2(0) .(4)

is larger than or at least equal to ?. Obviously the limits x1(0')

and x2(0') may be always found,and this generallyin many different
ways. If the variate x is a continuous one, then the limits xl(0')
and x2(0') may be fixedso as to have rigorously

P(0') = . . . . . (5)
In the case, however, when the variate x is not continuous, for
instance if it followsthe binomial law of frequency,we should have
only
P (0) > ? . . " . . . '(6)
For the sake of definiteness,we shall assume that the interval
[x1(0'), x2(0')] is chosen to be the shortestpossible satisfyingthe
condition (6). Such an interval will be called the interval of ac-
ceptance, correspondingto the chosen value of ?. Suppose now we
have foundthe intervalsof acceptance, correspondingto all possible
values of 0. Now join all left-handside boundaries of the intervals
of acceptance by a continuousline, which may be a smooth curve
or a polygon. Denote this line by LL. Another line, say 11,will
join the right-handside boundaries of the intervals of acceptance.
The two lines LL and 11will be boundaries of a certainbelt, which I
shall call the confidencebelt, CB.
Consider now the points, say A with coordinates (x10), thus
representingcombinations of all possible values of x and 0. The
confidencebelt CB as definedabove has the fundamentalproperty
that whatever the probability law a priori cp(O),the probabilityof
having the point A inside of the confidencebelt is equal to or larger
than the chosenvalue ofthe confidencecoefficient. This probability
may be representedeitherby means of integralsor by means of the
sums extendingover all possible positions of the point A inside CB,
according to the properties of the variates x and 0, which may be
continuous or not. Thus if pUB is the probability under consider-
ation, we should have eitherthe expression

PCB ffc(0)p(xI0)dxd0, . . . . (7)

CB
or
X<X2(O)

PCB -p(O) E p(x0), .(8)

0 X,(0)<X

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 592 NEYMAN-On the Two Different [Part IV,

or
X2(0)

PCB =2 p(O)fp(xlO)dx,.(9)
0 Xi(a)
or finally
PCB fp(O) E P(xO)dO, . . . . (10)
x,(O)

all summations and integrationsextending over the area of CB.

It will be noticedthat the case of 0 and x being proportionsof certain
individuals in the population and in the sample could lead us either
to the formula(8) or to the formula(10) according to the assump-
tions about the population sampled, which may be finiteor infinite.
Whatever the case may be and whicheverof the four formulae
may accurately correspondto the actual conditionsof the problem,
the summationsand the integrationsmay be executed in the same
order. We sum first(or integrate) for differentvalues of x, that
is forthe values containedin the intervalof acceptance, correspond-
ing to a fixedvalue of 0. This gives us the probabilityP(0), which
owing to the special choice of the interval of acceptance is > .
Substitutingc in the formulagivingPCB forthe integralor sum with
regardto x, we get either

PCB = f(0)P(0)d0 > fcP(0)dO= * . (11)

or
PCB = E;(0)P(0) > CZY(O) = . . . . (12)
0 0
where the integrationor summationwith regard to 0 extends over
all pessible value of this character. Thus the respective integral
or sum of p(O)is equal to one. The above formulae completethe proof
of the fundamentalproperty of the confidencebelt. It is to be
noted that if x is continuousand the intervalsof acceptance are so
chosen that P(0) ? for every 0, then, whatever the unknown dis-

tributiona priori of 0, we should have

PCB -?.(13)
The constructionof the confidencebelt is quite independent
of any arbitraryassumption concerningthe values of 0. If the
confidencebelt is constructed,we may affirmthat the point A will
lie inside of the belt. This statement may be erroneous,but the
probabilityof the erroris eitherequal to or less than 1 - s-thus is
as small as desired.
The solutionof the problemof estimationconsistsin constructing
the confidencebelt and in affirming that the point A, representing
the combination of some possible value of x with some possible
value of 0, will lie inside of the belt. When observationprovides

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 1934] Aspects of the RepresentativeMethod. 593

us with the value of x x', we shall consider this as additional,

and this time accurate, informationabout the position of the point
A, and shall combine it with the previous (uncertain) statement
that A lies in-sideCB. We shall draw the line parallel to the axis
of 0 and correspondingto the equation
x =x'. (14)
and findits intersectionswith the boundaries 11and LL of the con-
fidencebelt. Denote by 01(x') and 02(x') the ordinatesofthese points
of intersections. The interval 01(x'), 02(x') will be the confidence
interval correspondingto x= x'. Stating that
01(X') < 0 < 02(X') (15)

every time the observationgives us x = x' we may be wrong. This

will happen only if the point A happens to be outside the confidence
belt, but the probabilityof this last fact is equal to 1 -.

Note II. The MarhoffMethodand MarkoffTheoremon Least Squares.

The importance of the work of Markoif concerningthe best
linear estimatesconsists,I think,chieflyin a clear statementof the
problem. The subsequent theoryis a matter of easy algebra.
Suppose we have a sample of n values
X1, X2, . . . Xn*)

each being randomlydrawnfromsome populationTs(i 1, 2, .. . n).

Denote by Ai and ai the mean and the standard deviation in the
population nj. Suppose furtherthat it is known that
Ai = ailp1 + aj2p2 + . * . + aisp, . * * (2)
where the p's are some unknown parameters, and the a's known
coefficients
and where s < n. Consider now the problemof finding
* The theorydeveloped by Fisher runs on somewhat different lines. It
applies only to the case just describedwhen we know the distributionof the
collective characterx depending upon only one unknown character. The
methodI am usingseemsto have the advantage that it allows an easy general-
ization to the case wherethereare manyunknownparametersin the frequency
distributionof several variates describingthe resultsof sampling,whilewe are
interestedonlyin the value ofsome of them. My methodof approachseemsto
have also the advantagethat startingwiththe calculationsdependingexplicitly
upon the unknownprobabilitylaw a prioriit shows exactly how this depend-
ence is being eliminated. This theoryhas been partlyset out in my lectures
at the UniversityofWarsaw and is now the chieftopic of mylecturesdelivered
at UniversityCollege. It is hoped it will be soon published as an issue of a
lecture seriesdeliveredat the Departmentof Applied Statisticsat University
College,London. The case when 0 and x are proportionsof some individuals
in the populationand in the sample is discussed in the paper by C. J. Cloper
and E. S. Pearson (now in print)givingsomeinteresting remarkson the relation-
ship betweenthe conceptsof confidenceintervalsand the probabilitiesa pos-
terioriin the sense of Bayes. This paper containsalso graphswitha systemof
confidencebelts,corresponding to different
sizes of the samplesand to different
confidencecoefficients.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
594 NEYMAN'-On theTwo Diffrrent [Part IV,

the best linear estimate of a collective characterof the populations

definedby the equation
0 = blpl + b2p2+ .+ bsps (3)
the b's being known coefficients.
The method of the solution follows at once fromthe statement
of the problem. It will be convenientto use the notation E(x) for
the mean value of any variate x. Denote by 0' a linear function
of x's:
0'-= (?,ixi) . . . . . . (4)
Our problem consistsin determiningthe X's so that both
E(0') 0.
o . (5)
and
a2o, E(O' - 0)2 = minimum . . . . (6)
The condition(5) leads to the followingequations whichwe reach by
taking into account (2) and (3):
E(O')= Y>sE(x) 0 .(7)
or
P1Y2(?jajj) + P27(,iai2) + * + psY_(?jaj,)
plbL +P2b2 + * +pb8 . . . (8)
This equality should hold whatever be the unknown parameters
. . . Ps
Pr' P2.
This is possible only if we have
1(,jail)= b,
(a2) b2(9)

I (Xiais)= bs
As there are s linear equations with regard to n > s unknown co-
efficientsX we may generally make a choice among all possible
systemssatisfying(9) in orderto satisfy(6).
Using the known formulafor the variance of a linear function
of n variables xi, we may writethe condition(6) in the form
G20, _ Z(X,i2G,2) + 2=E(XjXjaia,gri) ***(10)

The values of ? satisfying(9) and minimising(10) may be easily

found provided we have sufficientinformation concerning the
cs's and the correlationcoefficientsrij. The case considered by
Markofiis whenit is knownthat ri3=0 , and
2 *2 (11)

Pi being a knownnumberand G2 some unknownconstant. In this

case the function(10) to minimizemay be writtenin the form
,a2z i .
,v20, (12)\

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 595

The solutionof minimizing(12), or, whichis the same, of minimizing

the sum

>(Ps
)*2** * * * * (13)

under the conditionof having (9) satisfied,is now a straightforward

one. It requires,however,that among n equations (2) connecting
the means Ai with the parametersp there should be at least s in-
dependentequations. If this conditionis satisfied,then the solution
of the whole problemis given by the followingtheoremof MIarkoft.*
The best linear estimnate
of 0 is obtained by substitutingin (3)
insteadofparametersp, linearfunctionsof thex's say

q1 q2,. . . (14)
found byminimizingthesumn
of squares

S =~ (Xi - ailql -ai2q2 - * * aisqs)2Pi * (15)

withreyardto theq's consideredas independentvariables.

The second part of this very important theorem gives us an
estimate of the variance of 0'. Denote by p2 the estimate of a20'
and by SO the minimumvalue ofS obtained from(15) by substituting
the functions(14) for the q's. Then, according to the result of
Markoft
2 ~~~~O~
2
v- - s >(Pi) *e * * **(16)

If it wereknownthat P1 =P2 = . . .-1, then the ratio S/(n - s)

would be the estimate of the value of the variance a2, common to
all x's. Consideringthe denominatorin (16) we recognizein n - s
the number of degrees of freedom-the concept introduced by
R. A. Fisher-equal to the number of observations, n, minus the
numberof independentparameters.
The above results concern the case when the variables (1) are
independent. If their dependency arises from the fact that they
are characters of individuals randomly drawn from limited popu-
lations, then the above results may be easily adapted to this case.
In particular the method of Markoffapplies when we are dealing
with stratifiedsampling.
Consider, for example, the problem of estimating the sum
0 = SE(u) of some numbersu associated with the elementsof some
stratifiedpopulation 7r. Denote by Tci the i-th stratum (i 1,
2, . . . k), by Mi- the numberof its elements,by vi the mean of
* ActuallyMarkoffformulatedhis theoremin a slightlydifferent form,but
this difference
is of no importance,and the formhere chosenseems to be more
convenient.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
596 NEYMAN-On theTwo Different [Part IV,

the u's correspondingto the elementsof 71 and by oi theirstandard

deviation. The lettersuij, x;, and mi will denote respectivelythe
value of u correspondingto the j-th element of 7ti, the value of u
correspondingto the j-th of the elements selected from7i to form
the sample and the number of the elements of 7tj selected for the
sample. We may writenow

0 .(Mit) *(17)

To findthe best linear estimateof 0 we write

k mi
?' E z xijX) *....(18)
i=1 j=1

? which would
and then try to findsuch values of the coefficients
satisfythe conditions
E(O') 0. (19)

whateveris the value of 0 and of the f7'sand

E(O'- 0)2= minimum . . . . (20)

Owing to (17) and to the obvious fact that

E(xi3) =. i *(21)

the condition (19) transformsitselfinto the following

z tvt(E xjj - Mi) = O. . ...(22)

=1 j=1l

This shows that the necessary and sufficientconditions which the

?,'smust satisfyin orderto have (19) satisfiedwhateverthe unknown
propertiesof the population, will be the following:
'fli
E s(Xj) = Mi (i= 1, 2, . . . 1c) ...(23)
J 1
This being fixed,considerthe condition (20).
Straightforwardalgebra gives for the left-handside of (20) the
expression, say,

i, - {i (2T(jj i\.2 + M - l (Be-Xi)2)} (24)

where Xistands forthe mean value of ?ij calculated from (23), that
is,
X Mt . . . (25)
M?,

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 597

We see now that o20, is minimizedby a systemof X's satisfying(23)

in which,whateverj 1, 2, . . mi

Xi Mt.. (26)

It will be noticed that this result holds good not only whatever
be the unknown vii,but also whatever the standard deviations ac.
Thus if we denote by jT the mean value xij forany stratum,then the
function
6' E (M) . (27)
i=l1

is the best linearestimateof 0 whatever

thepropertiesofthepopulation.
The familiarformulafor the variance of O' is easily obtained from
(24) and (25):
k M. M,a,2
G20, :. M I -M, mt Mj__ . (28)

The estimate O' is, of course, the one which could be suggested
on purely intuitive grounds. The advantage of using Markoff's
method consists (i) in avoiding biased estimates, which may be
sometimes used when their choice is based on intuition only,
and (ii) in findingthe best linear estimates. It would be rather
difficultto fulfilthis last conditionon intuitivegrounds only.
Note III. The consistencyand theefficiency
of theestimateof C. Gini
and L. Galvani.
Consider a population 7r of districtsdivided into second order
strata, 7CYV,accordingto the values of the controly and the number
v of individuals in the districts. Thus any districtin the stratum
7yv contains the same number of individuals v and the value of the
control correspondingto each districtis also the same y. Denote
by My, and myvthe total numberof districtscontainedin ny and the
numberof them to be included in the sample. The lettersuyviand
xyj will denote the values of the character sought, x, associated
respectivelywith the i-th districtof the stratum7yvand with the
i-th districtout of the myvof them, which have been selected from
this stratum. The lettersuyvand xyVwill denote the means of uyv
and xyvicorrespondingto the stratumnyvand to the partial sample
of districts,drawn from this stratum. The standard deviation of
uyvjwill be denoted by oyv. Finally, X and Y will denote the
weightedmeans of the charactersought x and of the control,calcu-
lated forthe whole population n. The sample weightedmeans will
be denoted by X, and Yy. Denote furtherby
W == =(MyVv) .(1)
yv

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
598 NEYMAN-On theTwo Different [Part IV,

We shall have
X - Y(Myvuy * .
-(2)
yv
I
Y l2'(MYvy) .(3)
_ yV

- (4)
E(MYVV)
y v
YE(rnyvvy)

SEY(MAYV *****. (5)

yv

'Using this notation, we shall now considerthe necessary condition

which must be satisfiedby x and y in the case when the estimate of
Gini and Galvani is consistent. This conditionis more easily found
when we considerthe ideal case wherethe sampleweightedmean Ye
is exactly equal to its population value Y. In this case the estimate
of Gini and Galvani reduces itselfto X., Thus we shall consider
the conditionsunder which the mean of XX in repeated samples is
equal to X whenever
Yy = Y.. . . . . . .(6)
We suppose that the numbersmvvare fixedin some way or otherin
order to satisfy(6), and consider the mean value of X>, which we
should get from all possible samples, correspondingto the fixed
values of myv. We shall have

E(Xyn) Y vgu X_
E(X~) V
YY,( ) -
Xe (say). . . . (7)
y V
Now we wish to have
X= X,.(8)
whenever(6) is satisfied.
The equations (6) and (8) may be writtenin the followingform
YM2YVv(y Y) 0
O .(9) .
y v
E-ny,,v(uyv -X) = 0, .(10)
yv

and it is easily seen that if it is requiredthat (10) holds good whenever

the numbers myvsatisfy(9), it is necessarythat

uyv-X = A(y -Y).(11)

A being an absolute constant,independentof y and v.
The necessary (and obviously also sufficient)condition (11) of
the consistencyof the estimate of Gini and Galvani is a rather
peculiar one. It is to be noticed that it is more limitingthan the

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Method.
Aspectsof theRepresentative 599

hypothesisH mentionedin the text (p. 571). In fact (11) means

that the regressionof x on y should be linear not only if we consider
the whole population of districts,but also if we consider the part
of this population containing only districts with a fixed number
of individuals v. Furthermore,'the regressionlines corresponding
to differentv's should be the same. Denote the hypothesisthat
these conditionsare fulfilledby H'. Obviously if H' is true,thenH
is also true,but not inversely.
It is perhaps worthnoticingthat X, may be a consistentestimate
of X fora special systemof the rn's. In fact denote

(myv) = w .(12)
y v

and subtract (2) from(7):

Xy - = E7( vuyvt wYV--wYV) *(13) *

It is easily seen that if forany y and v

myv W const., (14)

then (13) vanishes and X, becomes an unbiased estimateof X what-

ever the propertiesof the population. However, it may be noticed
that the fulfilmentof (14) means the rejection of the principle of
purposive selection.
Assume now that the hypothesisH' is true and find the best
linear estimate of X correspondingto any systemof mn's not neces-
sarilysatisfyingthe condition(6). As the values xyvi, correspond-
xy.-
ing to two districtsdrawn fromthe same stratumny are correlated,
it is impossibleto apply the theoremof Markoffat once. We shall
do so later on. However, we must start by followingthe general
method. Denotingby O' the linear functionof x's required,we shall
have
0' =:. YY.(>y1:i) .(15)
yv .

It must satisfythe conditions

E(o') X (16)
and if a2o, stands forthe variance of 0',
A2o mt%inimum. (17)
Since the values of the controlare knownforall districts,we may
change the originof co-ordinatesand assume that Y = 0.
According to the assumed hypothesis, H', (see (11)) we have
then
E(xy,-) _ uy, = X + Ay . . . . (18)

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
600 NEYMAN-On theTwo Different [Part IV,

where X and A play the role of the unknown parameters,p, con-

sidered in the theoryof the Markoifmethod. Therefore
E(O') = SEY(kyvj(X+ Ay))
yv i

XY,YY(:kYVi)+ AE{yYEN(;ky,j)}** (19)

y vi y vi

The values of the X's must be so chosen that this last expression
should be identicallyequal to X whateverthe unknownvalue of A.
Thus we should have, say

fa-(Pv) 1.(20)
yv i
= ?
Y- (YYG(Xyvz)) (21)
U
y v

These are the conditionsof the consistencyofthe estimateO'. Now

consider the condition of its being a best linear estimate. Taking
into account the correlationbetween xyvland xyvj,the variance of
O' may be writtenin the form,which is easy to check

a20, >2
E{ G2 v E(X2yvi) - 2_ LY(ytXY'Y))}J . (22)

We proceedto minimizethis expressionwith regardto all systems

of the X's satisfying(20) and (21). For this purpose we shall equate
to zero the derivativesofthe function,say,
F= 2o - 2ccf- 2f,. (23)
a and f being some coefficients
to be determinedfrom(20) and (21).
We have

2 ,i
yvf 1 - y}=o
P :2
- (Xy j) )YV)-oX
y (24)
Uxvi myv

The above equation may be writtenin the followingform

m.g
MY := EY (Iyv) + (OC+ 3Y))(Myv-1), * * (25)
j=l
which shows that whatever i 1, 2, . . myv,the controly and v
being fixed,Xyvihas a constant value, say yv. This value may be
obtained from(25), in termsof ocand 3

ymv (O + 3Y) MY -
m (26)
myv yv

Substitutingthis value into (20) and (21) we should be able to

calculate ocand f and then to find xyvfrom (26). However, it is
easier to get the result by using the Markof,theorem.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 601

The resultswhich are already obtained justifyus in writing

o' = =(YVE(XYVt))
yv i
.(27)
Y.(XYv.nY.XYV)
yv
and thus to treat O' as a linear functionof the variables x,vwhich,
being associated with samplings from differentstrata, are totally
independent. The mean value of xyvis the same as of xyv,equal
to utyv.The variance of xl,, say S2Y,, is connectedwith the variance
and is as follows:
of xyv1
m m
S2 (
a2? (say). (28)
Thus the original problem of finding a linear function O' of
dependentvariables xyvisatisfying(20) and (21) is now reduced to
that of findinga linear function (27) of independentvariates xyv
satisfyingthe same conditions. This may be done by applying the
Markofitheorem. However,we shall have to assume some additional
hypothesesconcerningthevariances U2Yv Giniand Galvani assumed
in theirpaper a hypothesis(p. 63) which,I think,in my terminology
and notation could be expressed by the equality
-2yv=a2 - constant. (29)
in assuming this hypothesisand in finding
There is no difficulty
the best linear estimate of X directlyfrom the Markofftheorem.
Comparing(28) and (29) we see that the role of the " weights," P,
involved in the Markofftheoremis now played by the ratios, say,

pV _ qxyv(Myv1) (30)

Thus the best linear estimate of X is found by minimizingthe

sum of squares
r~
S E{(Xyv ~ q2Y
ql- q2)2 mYb(MYV 1) (31)

with regardto q1 and q2 consideredas independent variables. The

best linear estimate sought,O', will be the solutionforql, minimizing
(31). It is easily seen that it is not the one whichhas been suggested
by Gini and Galvani, which results from minimizingthe sum of
squares, say
s' -{(Xyv-l- q2y)2vMyv * * * (32)
y v

The estimateof Gini and Galvani becomes the best linearestimate

when we assume the hypothesis,which will be denoted by H1, that
the variancesS2yvare inverselyproportionalto vmy,. This could be

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 602 NEYMAN-On theTwoDifferent [Part IV,
true, for instance, if the districtscontained in each stratum were
samples of v individuals drawn randomly from the population IH.
In this case we should have

92 Y- (say). (33)

Additionalassumptions,that ?2y is independentof y, and is equal

to say, and that each myv 1 will reduce the formula(28) to
a2,

672
S2YV -, (34)

which will lead to minimizing (32) when getting the best linear
estimate of X. I think it would be exceedingly difficultto find
instancesin social,vital or economicstatisticsin whichthe hypothesis
H1 would be true. However, it may be true in some engineering
problems.
I want to emphasize that the general problem of estimationof
any given collectivecharacter0 of a population 7 must be considered
from two differentpoints of view: (i) Given a sample from the
population, obtained in some known manner, what arithmetical
procedure will give us an unbiased and a most accurate estimate
of 0 ? (ii) What method of samplingwill give samples, allowingthe
most accurate estimates? These two aspects of the problem may
be traced in any theoreticalresearch concerningthe representative
method. Oftenthe solutions proposed are based only on intuition
and requiretheoreticaljustification. The principleof the purposive
selectionmethod,advisingselectionofsamples such that the weighted
sample means of the controls should be equal to their population
values, is an intuitive solution of the problem (ii). The methods
whichhave been proposedto estimatethe unknownweightedpopula-
tion mean X are the solutionsof the problemof kind (i). In the first
part of the present Note I have considered the conditions under
which the intuitivesolutions of the problem (i) are justified. Now
I shall proceed to consider whether, and if so then under what
conditions,is justifiedthe solution of the problemof kind (ii).
To do so I shall assume that the hypothesisH' is satisfiedand
shall consider the variance of the best linear estimate, O', of the
unknownweightedmean X. Its expressionwill involvethe numbers
MYVof districtsselected for the sample fromeach stratum7ry. It
will be possible to see what-system of these numbers myvwould
minimizethe value of the variance a20' of O'. We shall see that under
certain, rather limiting,conditions,the system of myvminimizing
a20' will be the system for which the sample weightedmean of the
controlis equal to its population value.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspectsof theRepresentative
Method. 603

We shall consider the question in its full generalityand shall

make no assumptions about the variances, G2y, of the character
sought within the second order strata. This will lead to the ex-
pression (28) for the variances of numbers xy,. The best linear
estimate O' of X will be obtained by minimizingthe sum of squares
S --{(XYV ql q2y)2QYv} * * * (35)
y v

with regard to q, and q2. The solution is easily obtained and is

given by the formula
0'. ( .YXYV). .(36)
yv
where
(Qyvy2)
-yE(QYy) Q 37
- (3)
_(Qyvy2)y(Qyv) (yy(Qyvy)2 y

The variance of (36) is given by the familiarformula

2 = (38)
(2E(,j32yS)yv).

This, owing to (37) and (28) reduces itselfto the following

1 (yy(Qyvy))2
a20o -f1+ (9
=20 Qyv) E-(QYVy2)y_(QYV)- (y(Qyvy))2J
Consideringthis formula,we see that it will provide a small
value of a20' if we succeed in minimizingsay

I YT I I . . . . .(40)
SE(QYVY)I

and at the same time maximize YY(Qyv). The formerexpression

(40) may be consideredas a weightedmean of the control,calculated
for the sample, where the Qy, are playing the role of the weights.
Rememberingthat by a proper choice of the originof co-ordinates,
we have reduced the value of the population weightedmean

y EE(M1v ) .(41)

to zero, we see that the above result would justifythe principleof

purposive selection,if the new weightsQyvwere proportionalto the
weightsused in (4); thus if we had

Qyv== yv(yv- 1) - m11v .

(42)
(Myv- myv)a22yv C

C being a constant. Solving (42) with regardto C we get

C Myv-1 yv .. (43)

It is easily seen that the right-handside of (43) may be constant

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 604 NEYMAN-On the Two Different [Part IV,

when meyv=1 and if the values of a2yvare independentof y and in-

verselyproportionedto v; thus when the hypothesisH1 is satisfied.
Obviouslythis is not the only conditionunderwhich (43) is constant,
but it is difficultto formulateany other hypothesiswhich would
concernthe unknownvalues of a2yv
The analysis of the formula(39) mightbe carrieda littlefurther,
but the considerablesize of the presentNote and the resultsalready
obtained suggestthat this may be superfluous.
The conclusionswhich were obtained above may be summed up
as follows:
(a) The estimate of Gini and Galvani is unbiased only if the
very limitinghypothesisH' is satisfied.
(b) This estimate is the best linear estimate when another still
more limitinghypothesisH1 is satisfied. This hypothesisconsists
in the assumptionthat the variation of x between districtsincluded
in each second order stratum,7Y., depends only upon the value of
v and is such as could arise if the districtswere random samples from
the population studied. This conditionis hardly ever satisfied.
(c) If the hypothesisH' is true, then the principleof purposive
selection of districtsso as to keep the value of the sample weighted
mean of the controlequal to its population value, is justifiedwhen
the hypothesisH1 is true. The dependence of the weightedmean
on the systemof weightsbeing only slight,it may be assumed that
this principle is approximatelysatisfiedeven if the hypothesisH1
is not exact. Other conclusions which may be considered as cor-
rectionsto the principleof purposive selection,may be drawn from
the formula(39).
Whetherit is likely that the hypothesesH' and H1 are justified
and whetherthe divergenciesin this respect will seriouslyinfluence
the accuracy of the results of the application of the purposive
selectionmethod,must be consideredin any special case.
It will be usefulto finishthis Note by a numericalillustrationof
the assumptions involved in the hypothesesH, H', and H1. The
followingthree tables give data concerningthree populations 71,
72' and 73 of districts. Each population is subdivided into two first
order strata, according to the values of the control y= - 1 and
y= + 1. Each firstorder stratum is in its turn subdivided into
two second order strata accordingto the size of the districtsv= 1
and v= 6. (Obviously the units in which the numberv of persons
included in a districtis measured, is of no importance. One unit
may be, forinstance,io,ooo individuals.) The Tables I, II, III, give
the values of the character x for each districtin a stratum (these,
according to notation used in this Note, are denoted by ut), their
arithmeticmean ui,and theirvariance a2.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 605

TABLE I.

Population n1.

y= -1. y= +1.

v=1. v = 6. v = 1. v = 6.
Stratum I. Stratum II. Stratum III. Stratum IV.

ul =-4 ul =-2 ul =-2 ul =0

U2 = -1 U2 = O-
?u2 = u2 = 2
U3= +2 U32= 4_
u =-1 u =-1 =~~~~f
u +1 u = +I
0r2 =6 U2 =1 2 =6 U2 I

TABLE II.
Population72
Y=-1 y =+ 1

v=1. v =6. v=1. v = 6.

Stratum I. Stratum II. Stratum III. Stratum IV.

=u =- 44 u= -2
U2 =0 U2 = -1 U2 =+2 U2 =+1
- -=U3= +2 U3 +4
ib =-1~~~f =
=- I = +1 uf =: +1
a2 =Q'1 =1 a2 = Q1 = 6 u2 = Q'1 = I a2 Q 1= 6

TABLE III.
Population7C3.
y=-1. y=+1.

v=1. v = 6. v=1. v = 6.
Stratum I. Stratum II. Stratum III. Stratum IV.

u1= U2 U3= -2 U1 = -6 It1 = U2 =u 3= 0 u I= O

U4 =U5 =U6= 0 U2 = -3 U4 = U5= U6= 2 IU2 = 3
-43 = 0 U3 =6

U =-1 U =-3 U +1 I =3
0r2 = 1 2= 6 U2 1 u2 = 6

It will be seen that forall threepopulationsX = Y = 0. Owing

to the fact that y has only two differentvalues, the regressionof
x on y is linear in all populations, and thus all of them satisfythe
hypothesisH. The populationsn3 and 72 satisfyalso the hypothesis
H'. In fact, the means ii in the second order strata do not depend
upon the value of v. It is not so in the population 7r3. Here in
strata correspondingto v 1 the regressioncoefficientof x on y is
VOL. XCVII. PART IV. Y

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
606 NEYMAN-On the Two Different [Part IV,

equal to A1 1 and in strata correspondingto v 6, to A2 3

Thus the population 3 does not satisfythe hypothesisH'.
The hypothesisH1 is satisfiedonly in the population T- as here
the variances a are inverselyproportionalto the v's.
Consequently,whatever be the numbers,say mi, i2, i3, M4 of
districtsselected forthe sample fromthe four strata of the popula-
tions T1 and T2 so that the sample weighted mean of the controly
is equal to zero, the estimate of Gini and Galvani will have its mean
in repeated samples equal to X_ 0. If it were mn= M2= M3
m4 1, then in the case of the population 71 the estimate of Gini
and Galvani would be the best linear estimate. Its variance is
equal to 3. When sampling in the same way fromthe population
7r2 the estimate of Gini and Galvani would not be the best linear
estimate. In fact its variance would be equal to J. On the other
hand, the best linear estimate, which could be derived from the
formuloe(36) and (37), namely,
, S(Q) .. . . (44)
I

would have the variance -`,as previously.

As the hypothesisH' is not satisfiedin the population 3, it is
possible to findsuch a system of the m's that the estimate of Gini
and Galvani will have its mean value in repeated samples not equal
to X= 0. Owing to the exceptional symmetryof the population
3, there will be many systemsof m's by which the estimate of Gini
and Galvani will be consistent. However, let us considerthe system
nl = 6, 2 = 0, M3 = 0, r4= 1. Obviouslythe sampleweighted
mean of the control Y2 - 0. There will be only three possible
samples correspondingto the fixed system of the in's depending
upon the choice of the districtin the fourthstratum. The mean
of the estimates of Gini and Galvani, calculated for these three
samples will be
(6) (- 1) + (6)(3)
X-= 1 (45)
12
and is not equal to X= 0. Thus for this system of the in's the
estimate of Gini and Galvani is not consistent. It is easily seen
that if there were more districtsin each stratum the number of
systems,for which the estimate of Gini and Galvani would not be
consistent, would be increased. It would be also considerably
largerif the structureof the population 73 were not symmetrical.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] Aspects of the RepresentativeMethod. 607

DISCUSSION ON DR. NEYMAN'S PAPER

PROFESSOR BOWLEY: There are some who appear to pride

themselveson theirabsence of knowledgeof mathematics. I never
understood why it should be a matter of pride. I do not think,
however,that thereare very many who now hold that mathematics
is not properly appropriate to the study of statistical problems.
This paper will, when it is thoroughlystudied, do very much to
remove any remainingdoubt that the mathematical approach is of
fundamentalimportance. Sampling is at the very root of a great
deal of statistical investigationand, as Dr. Neyman points out, of
increasinguse and applicability; it is thereforeof the firstpractical
importanceto decide what is the best method of sampling-best in
the sense that the best use willbe made ofthe resourcesat the disposal
of the investigator,and that thereshall be-if the two are consistent
-a minimumexpenditureof time.
This paper of Dr. Neyman's will be found to answer most of the
questions which relate to the setting out of an investigationby
sample. One of the thingsthat is so interestingin it is the analysis
of the problems. There is not one perfectmethod of sampling; the
method depends upon the nature of the material which is available
or whichcan be obtained. To me a new suggestionin the classifica-
tion is this stratifiedrandom sampling of groups. In the analysis
to which he has referredin much too favourable terms,I had dis-
tinguishedin the ways he named certain methods of sampling,but
in effectI realizethat it is preciselythe methodwhichhe has discussed
that I have been drivenby circumstancesto use or to recommend.
In the Survey of London, the unit was not the family or the
person,but the house-the unitwhichwas providedby the directories
we had. In the recommendationI had the honour to make with
Mr. Robertson to the Governmentof India, of sampling on a very
large scale, the unitsuggestedis thevillage. In therecommendations,
whichhad no effect,whichI made withregardto the 1931 Population
Census, of a method which I had in fact made use of on the 1911
figures,the unit was the householder's schedule, and I suggested
that if one took one householdin fivehundredthroughoutthe country
we could deduce some of the results of the Census whichnow, after
three years, we are still waiting for. But that, I think, is not
necessarilythe method appropriateto all problems,nor do I under-
stand Dr. Neyman to recommendit universally.
I am surprisedthat he thought that when in 1925 I- examined
the problemofrepresentativesamplingforthe InternationalInstitute
of Statistics,I gave equal importanceto that method,as I definedit,
and to others. Certainly I thought I damned it with very faint
praise at the end of the summaryof my report. I agree that it is
difficultto formulate,difficult to carryout, and I stillthinkthat it is
very difficultto get a good estimate of the precision of the result,
except in ratherunusual cases.
The second problem, after discussingthe material and defining
the best method of sampling, is to get a definiteestimate of the
precisionof the samplingin the sense that when one has a result,one

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
608 Discussion [Part IV,

knowsthatit can be trusted,withoutdefiningthat term,to i per cent.

or i shilling,or whateverit may be; and it is partly because of that
necessitythat a common method, partly intuitive,of choosing the
sample fromthe obvious-of taking the mode ratherthan anything
else-is hopelessly faulty, because not only is there difficultyin
obtainingan average, but thereis no means that I know of obtaining
precision.
This methodof stratification, so far as it differsin precisionfrom
purely random sampling, gives an improvementin precision. If,
in fact, one has made a stratifiedselection and writes down the
statementof the precisionas if it had been purelyrandom,then one
is on the safe side. I have myselfgenerallybeen content to let it
go at that fortwo reasons: (1) that it is verydifficult to measurethe
additional precision due to stratificationin ordinarymaterial-at
any rate it is a very lengthybusiness; and (2) (of quite a different
nature) I have had to explain and try to justify the methods of
samplingto non-technicalreaders,and thereforeI have been obliged
to leave out a great deal that Dr. Neyman would have put in. But
I have endeavouredto be on the safe side, and what I have neglected
I know to be unimportant.
This process of stratificationcan be thought of or applied in
various ways. In London we took the simplest way, taking one
house in thirty,forty,or fifty,rightthroughthe directory,so that
the houses we examined, if plotted on a map of London, would be
regularlydistributedin proportionto the density of the area and
every region would be included. But in that way, I thinkit must
appeal to everyonewho studiesthe problem.
A new point that comes out in the paper is that a selectionof the
units to be examined for one purpose is not necessarilythe best for
other purposes. As the selection has generally to be made once
and forall, that becomes an importantconsiderationin selectingthe
method to pursue. I am glad that the general recommendation
forthis kind of purpose is this random stratifiedsampling.
To give an example in this case in London, if we take a house,
we take one, two, or three families. The familyis not selected at
random, and within the family the persons are not selected at
random. If co-existencein one case gives positive correlationin one
instance,it may be negativein another,and similarlywith relation-
ship betweenpersonsin the family. The effectof this is difficult to
estimate. My point, however, would be that we must secure a
selectionthat would minimizethe influenceof these unknowncorre-
lations, and one method would be to make our samples sufficiently
large to make them unimpaired by the most unfavourable hypo-
thesis.
AfterDr. Neyman's very courteous referencesto my work on
the subject, it is somewhat ungratefulthat I feel it my duty to
criticizethe theoryofprobabilitiesin Section II, part 1, and I am very
glad ProfessorFisher is present,as it is his work that Dr. Neyman
has accepted and incorporated. I am not certain whetherto ask
for an explanation or to cast a doubt. It is suggestedin the paper
that the work is difficultto follow and I may be one of those who

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 609

have been misled by it. I can only say I have read it at the time
it appeared and since, and I read Dr. Neyman's elucidation of it
yesterdaywith great care. I am referringto Dr. Neyman's confi-
dence limits. I am not at all sure that the " confidence" is not a
" confidencetrick." Put in a simple formI thinkthe methodis as
follows:-Given that in a sample of I,000 taken at random, there
are i in io with the definedquality, and given that the population
fromwhichthe sample was drawncontainedan'yproportionbetween
I20 and 8o per thousand, then the chance of such an occurrenceis
less than one in twenty(approx.). Actual figures,of course,do not
matter. That margin between 120 and 8o per thousand in the
assumed population is shown on the vertical of the confidencebelt
in the veryilluminatinggraphs whichDr. Neyman has given. Does
that reallytake us any further? Do we know morethan was known
to Todhunter? Does it take us beyondKarl Pearson and Edgeworth?
Does it reallylead us towards what we need-the chance that in the
universe which we are sampling the proportion is within these
certain limits? I thinkit does not. I thinkwe are in the position
of knowing that eitheran improbable event has occurred or the
proportionin the population is withinthe limits. To balance these
things we must make an estimate and forma judgment as to the
likelihoodof the proportionin the universe-the very thing that -is
supposed to be eliminated. I do not say that we are making crude
judgments that everythingis equal throughoutthe possible range,
but I thinkwe are makingsome assumptionor we have not got any
further. I do not know that I have expressed my thoughtsquite
accurately,but it is not a thing that has occurredto me for the
I have feltsince the method
firsttimethisevening; it is the difficulty
was firstpropounded. The statementofthe theoryis not convincing,
and untilI am convincedI am doubtfulofits validity.
I regretthatin openingup that subject I have distractedattention
fromDr. Neyman's paper, but since he has made that an integral
part of his paper, I thinkit a properoccasion on whichto make this
kind of statement.
With referenceto my formula,quoted in Section IV, equation 24,
I must admit that the originalpassage is obscure. In Dr. Neyman's
notationand withonly one control,my estimatewould be *
X1 _ X-X + (rax - rayryx)ax. aaIaa.

where the a's are the weights attached to the x's in the weighted
average. The second termis zero, if thereis no correlationbetween
the weights and the divergencies,e's, from the linear regression
equation.t In other cases, K should be regardedas attached to the
errortermnegatively,since the weightedaverage of the e is not zero,
but -K. The formulais then " consistent." I am not, however,
* The completeformulain the notationI used may be written
X = X?,+ {ux. RxlRll . oaIi/a}
where Rx= raxrau'rav . . . r I pay . . rvPav I

t Thereis considerablecorrelationin Table I in Dr. Neyman'spaper.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
610 Discussion [Part IV,

at all sure that the particularhypothesesunderlyingmy treatment

are the best; it was in some way pioneer work, and I should have
been astonishedif no improvementshould have been made in course
of time.
I wish to propose a hearty vote of thanks to Dr. Neyman for
his veryimportantpaper.

DR. E. S. PEARSON: I have great pleasure in seconding this

vote of thanks to Dr. Neyman and welcominghim here among us
to-day. I thinkwe are all veryglad that as one of our Fellows from
abroad he has been able to take the opportunityof being in England
to read a paper beforeus.
I should like to tryto expressin a fewwordswhat appears to me
to be the essentialcontributionto statisticalmethodthat Dr. Neyman
has made in this paper and in otherwork that he has done. In the
past thirtyor fortyyears the developmentof mathematicalstatistics
has been an extremelyrapid one; it has been associated with all
the excitementof discovery,the discoveryof the power of new tools
in the solution of a great variety of problems. But in this rapid
progressintuitionwas sometimesat fault; the tools used were not
always the best tools, nor was it always very clear what was the
meaningof these tools, nor why one tool should be used ratherthan
another. In the last few years there has been a determinedeffort
to clear away some of this uncertaintyfromour statisticalreasoning.
The processis not complete; it is still to somneextentin the stage of
controversyand discussion, but there are fundamentalsthat are
emergingsurely and steadily.
In this process many of us owe a great deal to ProfessorR. A.
Fisher for the stimulus we have gained from wrestlingwith the
ideas he has put forward. If I purposelyuse the word " wrestling,"
ProfessorFisher will, I think, take no exception when I add that
the stimulusis all the greaterbecause it has been necessaryto wrestle.
Stimulatedby theseideas, as he has franklyadmitted,Dr. Neyman
has brought a very real contributionof his own into the field of
statistical inference. For example, although in the present paper
it may be regarded as only a side issue of the main subject, the
approach to the problemof estimationoutlined on pp. 563-567 and
in Appendix I is somethingof very great interest. This conception
of the problem of estimationis not exactly ProfessorFisher's con-
ception,but it seems to me that some of the interestlies in just those
points where there are differences. I do not, however, think that
this is the rightplace to discuss the doubts regardingconfidenceor
fiducial intervals raised by Professor Bowley; they need to be
cleared up, but that perhaps can best be done with pencil and paper
at a table.
Returningto the main subject of the paper, I think the chief
emphasis lies on the importance of logical planning in any investi-
gation; the particular problem considered is that of estimating
certain characteristicsof a heterogeneouspopulation froma limited
sample. In doing that we have to consider,if we can, how best to
take a sample, how to obtain our estimate, and what measure of

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 611

reliability to place on that estimate. To answer these questions

certainassumptionsregardingthe unknownpopulationare necessary,
and it is importantto employa methodof samplingand of estimation
which will reduce these unavoidable assumptions to a minimum,
and at the same timeto make perfectlycleartheirpreciseimplication.
I do not think that this problem can be solved in any way except
by the introductionof mathematics. Of course mathematicsalone
are not adequate, but I believe the highestlevel of statisticalcrafts-
manship is only reached when, in planning an investigation,we
attempt to formulatein precise, and thereforemathematical,terms
the frameworkof hypothesison which our final inferenceis to be
based. It is towardsthis level that Dr. Neymanis pointingthe way.
The special problemof the paper and otherinvestigationsof this
kind has been to estimatethe average value of some characterin the
population. This is the formof samplingproblemthat has presented
itselfto many investigators.
I would like,however,beforeI sit down to suggestthat there are
many cases in whicha method of representativesamplingis needed,
wherethe average is not goingto be sufficient;whereit is necessary,
in fact, to estimate in some way the nature and degree of hetero-
geneity in the population. This is perhaps not an easy problem,
as we have firstto determinewhat is the most appropriatemeasure
of heterogeneityin a particularcase.
Let me illustratethis by shiftingfromhuman populations to a
population of bricks in a kiln. Owing to the arrangementof firing,
the quality of the bricks varies very considerablyfromone part of
the kiln to another. We measurevarious charactersof those bricks,
one being their strength; this though of less importancein itselfis
correlated with importantpropertiesof weathering. What do we
want to know about the batch of bricksfromthe kiln? The precise
average strengthis of much less importance than the uniformity
and in particular the lowest strengthswhich may be met. In
determiningwhat procedureof samplingto employ,we have firstto
decide what index or indices ofuniformity will be of mostvalue to us,
and then to decide by what rules of sampling and calculation the
most reliable estimatesof these indices can be obtained.
Anotherillustrationis that of fertilizerswhich may be sold in
bags, the bags being drawn froma large silo in which the material
is stored. Since a time elementis presentin the fillingof the silo,
there may be lack of uniformityin the quality of material. Here I
think the percentageconstituentsof the standard fertilizersare laid
down by law; for example, it may be that the nitrogencontent
must not be less than, say, I2 per cent. The producerwants, there-
fore,to be sure that he will not be summoned because. the quality
in one bag is foundto be below that level. It is importantforhim
to get a measure of the average nitrogen content, but it is also
necessary for him to have some sampling scheme which will give
him reasonable assurance that he is not sendingout bags in which
the content is less than a certain amount, and also that there are
none in which it is too high. He must find some representative
method of testing.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
612 Discussion [Part IV,

A final case is that of cement. There the ordinarytest is to

take portionsof the cementfromdifferent parts of the bulked mass,
mix and quarterthem,and finallysubmita small sample to chemical
analysis. The resultwill give an idea of average quality but nothing
more. Yet what is of most importanceto the user,because it affects
his techniquein the mixingand settingof concrete,is the uniformity
of the material. Again, therefore,some method of representative
samplingis wanted whichwill give some idea of uniformity, not only
of average.
These are problems of the future,but I believe of not so very
far distant a future. In solving them a method of representative
samplingshould be planned, based on a sound statisticalframework.
This does not mean,of course,that the mathematicswill be presented
to the manufactureror British Standards Institution, any more
than they were presented by Dr. Neyman to the Polish Institute
for Social Problems, but I hope they will be there underneathas a
foundation.

DR. ISSERLIS said that it was related of Sylvesterthat while he

was Savilian Professorof Geometry at the Universityof Oxford,
when he had discoveredsome reconditeresult,he would walk out of
his rooms, dressed adequately or otherwise,buttonhole the first
milkmanor postmanhe met in the street,and hold on to that button
until his victim confessedthat he had been convinced by Sylvester
of the truth of the theory he had discovered. Dr. Neyman had
combined in one paper views on the philosophical foundationsof
statistical method with an exposition of an important technical
problem. Each of them would have sufficedfora paper, but it was
his choice to treatthe two in one.
ProfessorBowley,who was an expertin both, and who had been a
teacherofa generationbothin the theoryand applicationofstatistics,
and who had been particularlythe exponentofthistechnicalproblem,
had referredonly verybrieflyto the second subject, whichhappened
to come firstin the paper.
. Dr. Isserlis feltthat it was necessaryfor someone to face up to
the task of trying,withoutthe use of any mathematics,to say what
it was that was foundto be puzzling in this kind of philosophy,and
to see where they stood. The Society had had experiencein that
very room of people who were more fortunatethan the mathema-
ticians. The economists,menlikeMr.Hawtrey,who wereaccustomed
in their scientificlife to do withoutthe shorthandof a special tech-
nique such as mathematics, came and spoke to their colleagues,
who were experts in finance and economics, without the use of
technical language, and some of their friendssat aghast at such
miastery,whereas their colleagues like Sir Alfred Flux and Mr.
Macrostylapped it up like milk.
He would like brieflyto referto the thingthat was worryinghim,
and he would try to make it clear without mathematics. There
was a classical theoryof probability,based on certaindefinitionsand
experiences,whichtold how to measureand expressone's lack of full
convictionin certain matters. There were two things: one might

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
 1934] on Dr. Neyman's Paper. 613

know what was the state of affairsin a general population, and ask
what was one likely to get in particular cases? One might be
convinced,for instance, that pennies such as were provided by the
Mint were fairlysymmetrical,and on the basis of that it mightbe
said that the theoretical probability of heads was so and so, say
50 per cent. It mightthen be said, supposinga coin was tossed IOO
times, what should one expect? Should there be surpriseif there
were only 3o heads, or iftherewere go heads, or should one expect to
get about 50? Withoutany mathematicaltechniqueit was perhaps
sufficient thereto say that there was an a prioriprobabilityof one-
half.
But there was the converse problem. A coin had been tossed
ioo times and fallen heads iOO times. What kind of a universe of
coins had it come from? Had it come fromthat kind of universe
of coin in which the side with the head was more likely to show up
than the side withthe tail?
The classical theory provided us with a method which, when
limitedin its application to the fieldforwhich it was intended,was
perfectlylegitimate. According to this theory, if we knew the
a priori probabilities,that the penny was an Epsom Downs penny,
or that it was an ordinaryMintpenny,equally likelyto fall heads or
tails, or if we knew that it was a penny three times more likely to
fall heads than tails,-then if 70 heads had been observed in Ioo
trials, we could say what were the respectiveprobabilitiesthat the
pennywas an ordinarypenny or an Epsom Downs penny.
The criticismswhich had been made of the so-called theory of
inverse probability and of Bernoulli's theorem had always rested
not on the accuracy of the theoryitself,but on the correctnessof its
application, because in most cases these things were not known
a prioriat all.
Given the actual probabilityin the universe,it was possible to
make probabilitystatementsabout the sort of thing one was likely
to get in a sample. These probabilitystatementsusually provideda
measure forthe probabilitythat a certaininequality should be true.
Referringto Equation No. 4 in Dr. Neyman's Appendix I, in that
equation x was somethingthat belonged to the observedsample, and
it was imaginedtherethat we knew for the momenatthe particular
propertyof the universe.
His own criticismof that particular equation, and of the whole
structureplaced thereon,was that it was nothingnew; it was not a
departure from the various earlier attempts that had been made.
What was actually done was this: A particular0 was chosen; an
inequality was writtendown, and more values of 0 were chosen.
In each of these inequalities, c, which occurred on the right-hand
side of the equation, occurredin the definitionof inequality on the
left side, and had been left out of the equation by Dr. Neyman.
When all the pointswhicharose fromall the inequalitieswhich could
be got by consideringall permissiblevalues of 0 had been marked,it
would be found that we were no furtherthan the man who said,
" Let us suppose that the a priori probability in the universe is
distributedin a particular way. Let us suppose that it fulfilsa
y2

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
614 Discussion [Part IV,

certain law." Some had tried to avoid the difficultiesby saying,

" We can get general results not by assuming that the a priori
probabilitysatisfiesa certainlaw, but merelyby assumingthat it is
continuous, whichmeant that if the chance of a pennybeing exactly
symmetricalwere so and so, the chance of the penny being nearly
symmetricalwould be nearlythat. As a matteroffactthat assump-
tion was equivalent to the absence of gaps among the points on
Dr. Neyman's curves; othershad triedto followout the consequences
of assuming that the a priori probability was continuous near a
certainpoint. All these attemptswereratherbeating about the bush
because the problem was incorrectlystated. When we said that
if the probability in the universe were p, then the probability
of a certain sample would be x, we were specifyingthe probability
of a certain inequality. It was a matter of elementaryalgebra to
start from that and to say that the inequality so specified,which
said that X must be between certain limits in terms of 0 and
?, led to another inequality which said that given x, 0 must lie
within certain limits also dependent upon x and ?, and that there
was a probabilityfor that. The philosophical idea at the bottom
of that was rather difficultbecause we were not now speaking of a
probability,but of the probabilityof a probability. We measured
the probabilityofthe truthofthe statementthat a certaininequality
had a particularprobability. A hundredyears ago mathematicians
tried to sum an infinitenumber of terms in a series, and talked
about the ratio of two qualities which ultimatelyvanished. They
happened to be good mathematicians and to have a very sound
intuition,and most of theirresultswere correctand had ultimately
survived. We had learned that they were occasionally led into a
morass, and we said, "Mathematicians cannot performan infinite
numberof operations,but can make precise statementsabout certain
inequalities," and it was time that people recognised that while
certainprobabilitiescould not be evaluated,correctstatementsoftype
P(P(x)) could usefullybe made. It was possible to go on to higher
things, and talk about probabilities of the third or higher orders
and still remain in the region of the old subject. The principleto
follow was that entitiesshould not be multipliedbeyond necessity.
This was an argumentagainst belts of confidenceand so on, if, as a
matter of fact,'they only expressed probabilities of statements of
the same kind as those which had been made in the past. Dr.
Isserlis felt-thatperhaps he had been wrong,and that he ought to
have followedthe lead set by ProfessorBowley and not trench on
mathematics. To try and state mathematicseitherwithout chalk
or with a minimum of chalk was perhaps a hopeless task. He
apologized if he had made himselfin any way unintelligible; if
somethinghe had meant to say had emerged,he must be satisfied.

PROFESSORFIShIERsaid that the problem of samplingplayed an

importantpart in AgriculturalResearch. It was, indeed,in Experi-
mental Agriculturethat an adequate technique, bringing out the
differentaspects of the sampling problem, and displaying compre-
hensivelyexactly how these differentaspects were interrelated,was

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 615

firstdeveloped. In the luminous account which Dr. Neyman had

given of the samplingtechnique,as applied to economic researches,
which had itself,perhaps, been influencedby his persona,lexperience
in AgriculturalScience, one ofthe featureswhich had interestedhim
most had been the parallelismbetween the processes he advocated
(and the reasons he gave forthem) on the one hand, with the corre-
spondingprocesses and reasons which had been developed by agri-
culturalresearchworkersin this country.
His own contactwiththe subject had been gained at Rothamsted,
where he had the pleasure of collaborating with a succession of
brilliantplant physiologists,under whom, and especially under Drs.
Maskell and Clapham, the techniquewas graduallyperfected. As in
agriculturalsamplingtheoreticalconsiderationswereat theirsimplest,
the logical connectionbetweenthe means employedand theinferences
which mightvalidly be drawn were conspicuouslyclear; it might
thus be usefulif he gave an outlineof the hierachyof fivesuccessive
subdivisions used in the sampling of an agricultural experiment.
Exactly the same problemsdiscussedby Dr. Neymancould be simply
illustratedin this manner.
The smallestunitthat need be considered,the unit ofmeasurement,
as it mightbe called,consisted,in the case ofa cereal crop,of,perhaps,
io inches or 25 centimetresmeasured along a drill row. Again, it
mightconsistof a singleplant, as with potatoes or sugar beet. For
simplicityhe would adhere to the cereal crop. A numberof units of
measurements,usually four, fixed in relative position, but not
necessarily adjacent, constituted a sampling unit, which would,
therefore, containin all one metrelengthofdrillrow,taken,however,
in practice,fromfourdifferent rows. Since the parts of a sampling
unit were fixedin a relativeposition,the positionsof all were deter-
mined simultaneouslyby a singleact of randomsampling,i.e. by the
choice, by a physicallyrandom process, of the particular sampling
unit used fromamong all those available in the sampli,ngarea. Two
or moresamplingunitswereobtainedin this way fromeach sampling
area,each beinglocated independentlyby a freshact ofrandomization.
It was essential that there should be at least two independently
located samplingunits in each samplingarea, since it was fromthe
differences betweenthese,or the variances among them,if theywere
more than two, that the error of sampling was estimated. The
variance among the units of measurementwithinthe same sampling
unit served a different and subsidiarypurpose. It was essential to
the studyofwhat structureor size the samplingunit shouldhave, and
by analysing the variance within and among sampling units, one
could ensure that the samplingunits were so chosen as to give the
maximum precisionin returnfor the labour expended. But once
their size and structurewere chosen, this analysis could throw no
furtherlight on the interpretationof the experimentalresults. The
errorof random sampling,on the otherhand, should be ascertained
with high precisionfromevery experimentto which the sampling
methodwas applied, foron it one relied forjudging of the numberof
samplingunitswhich could with advantage be taken fromthe grow-
ing crop. A usual and convenientnumberwas 32 for each experi-

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
616 Discussion [Part IV,

mental plot. The plot would, therefore,either constitute a single

sampling area yielding32 sample units, or, perhaps, be subdivided
into quarterseach yielding8, or at mostinto sixteenthseach yielding
2 samplingunits.
Beforeproceedingto the highermembersofthe hierachy,it might
be usefulto indicate a sociologicalparallel. The samplingunit might
be thoughtof as a family(or as a house, or as a registrationdistrict).
The samplingarea mightbe thoughtof as a stratumof such families
when they were stratifiedwith respect,say, to earnings. The plot
mightbe thoughtof as all the familiesof a given occupational group
in a given area, irrespectiveof theirearnings. Then the subdivision
of the agriculturalplot into sampling areas played the same part in
increasingthe precisionof the ultimateestimatesas the stratification
of an occupational group according to their earnings. They were,
however,ultimatelyconcernedto comparethe agriculturalplot with
otherplots whichhad receiveddifferent agriculturaltreatments,just
as one mightbe concernedto compare the morbidityof an occupa-
tional group with that of otheroccupational groups. The sampling
was not an essentialpart of this comparison,but only a convenient
means of measurement,which-onewas concerned,in the firstplace,
to make sufficiently precise.
In an agriculturalexperimentdesignedto compare,say 6 different
treatments,48 plots mightbe assignedto the experiment,and, after
dividingthe experimentalarea into 8 compactblocks,each containing
6 plots, these 6 plots should be assigned, strictlyat random,to the
6 experimentaltreatments. This processofexperimentalrandomiza-
tioncould not,unfortunately, be imitatedin sociologicalenquiries. If
it could, morethan was knownwould certainlybe knownabout cause
and effectin humanaffairs. But withinthislimitationthe experiment
was strictlyparallel to one involvinga comparisonof 6 occupational
groupsin, say, 8 differenttowns. In a well-designedexperiment,how-
ever,the mathematicsweresimplified, and all anxietywas avoided in
respect to different systemsof weighting. Dr. Neyman advocated,
wisely,in his opinion,thesystemwhichhe ascribedto Markoff, though
this was in essencethe systemof Gauss. It mustbe rememberedthat
if the variancesfromthe different populationswerenot,on a plausible
expectation,to be consideredequal, one seldomhad priorknowledge
or experimentalevidence sufficient to make the Pi of Dr. Neyman's
equation (8) properly speaking known numbers. This seemed to
Dr. Fishera real difficulty,ifone wishedto speak ofthe methodas the
best possible, though it was no obstacle if, as reasonable beings,
statisticianswere oontentthat it should be a good or valid method.
The subdivisioninto blocks made clear the fact that samplingerror
was notthe onlykindoferrorwhichhad to be considered. Ultimately
the validity of the equations must depend upon the concordanceof
the evidencefromthe different blocks.
It would be expected that he should commenton those applica-
tions ofinductivelogic whichconstitutedso illuminatingand refresh-
ing an aspect of the evening's paper. All realized that problemsof
mathematical logic underlay all inferences from observational
material. They werewidelyconscious,too, that morethan 15o years

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 617

of disputationbetween the pros and cons of inverseprobabilityhad

left the subject only more befogged by doubt and frustration.
Recently,however,some researchworkers,workingin the apparently
abstract realms of the theory of estimation,and the logical bases of
tests of significance,had become increasinglyconfidentthat, when
properly stated, rigorously exact, though, of course, uncertain
inferencesmightbe drawn fromobservationalor experimentaldata.
In a word,the confidenceofthe advocates ofinverseprobabilitycould
be confirmed,that valid conclusionsof the kind sought could, some-
times, be drawn with assurance, while the arbitraryassumptions
upon which fromthe time of Laplace onwards such inferenceshad
been supported could be rejected as unnecessary. The particular
aspect of this work, of which Dr. Neyman's paper was a notable
illustration,was the deductionof what Dr. Fisher had called fiducial
probability. Dr. Neyman did not use this term,whichhe suggested
had been misunderstood,but he used instead the term " confidence
coefficient." Dr. Fisher thoughtDr. Neyman must be mistakenin
thinking the term fiducial probability had led to any misunder-
standing; he had not come upon any signs of it in the literature.
When Dr. Neyman said " it really cannot be distinguishedfromthe
ordinaryconcept of probability,"Dr. Fisher agreed with him; and
that seemed to him a reason forcallingit a probabilityratherthan a
coefficient. He qualifiedit fromthe firstwith the wordfiducial to
show that it was a probability inferredby the fiducial method of
reasoning,then unfamiliar,and not by the classical methodofinverse
probability. Dr. Neyman qualifiedit withthe wordconfidence. The
meaningwas evidentlythe same, and he did not wish to deny that
confidencecould be used adjectivally. They were all too familiar
with it, as Professor Bowley had reminded them, in the phrase
" confidencetrick." Still fiducial was, perhaps, on purely formal
grounds,the better adjective.
Dr. Neyman,as he had explained, differedfromDr. Fisher in the
relative importancehe attached to the two stages in which he had
attempted to develop a theory of estimation,independentlyof all
assumptionsas to probabilitya priori,namely,the earlier approach
throughthe notions of likelihood and quantity of information,as
comparedwith the later developmentof the notion of fiducialprob-
ability. This differencewas not entirelyone of perspective. Dr.
Fisher'sown applicationsoffiducialprobabilityhad been severelyand
deliberatelylimited. He had hoped, indeed, that the ingenuityof
later writerswould findmeans of extendingits application to cases
about whichhe was still in doubt, but some limitationsseemed to be
essential. Those who had followed the earlier parts of the story
would have no difficultyin perceiving these, but there might
be pitfalls for those who interested themselves only in the later
chapters. In particular,he would apply the fiducial argument,or
ratherwouldclaimunique validity* forits results,only in those cases
* Naturally,no rigorouslydemonstrablestatements,such as these are, can
fail to be true. They can, however, only convey the truth to those who
apprehendtheir exact meaning; in the case of fiducialstatementsbased on
estimatesthis meaningmust include a specificationof the processof
inefficient

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
618 Discussion [Part IV,

for which the problem of estimation proper had been completely

solved, i.e. eitherwhen there existed a statistic of the kind called
sufficient,which in itself contained the whole of the information
suppliedby the data, or when,thoughtherewas no sufficient statistic,
yet the whole of the informationcould be utilized in the form of
ancillaryinformation. Both these cases were fortunatelyof common
occurrence,but the limitationseemed to be a necessaryone, if they
were to avoid drawing fromthe same body of data statements of
fiducialprobabilitywhich were in apparent contradiction.
Dr. Neyman claimed to have generalizedthe argumentof fiducial
probability,and he had everyreason to be proud of the line of argu-
ment he had developed for its perfectclarity. The generalization
was a wide and very handsome one, but it had been erected at
considerableexpense,and it was perhaps as well to count the cost.
The firstitemto whichhe would call attentionwas the loss ofunique-
ness in the result,and the consequent danger of apparentlycontra-
dictoryinferences.
In the second place, Dr. Fisher had limited his application to
continuousdistributions,hoping,with more confidencein this case,
that the limitationmightlater be removed. Dr. Neyman removed
this limitation,but at the expense of replacinginferencesthat stated
the exact value of the fiducial probability by inequalities, which
assertedthat it was not less than some assignedvalue. This also was
somewhata wide departure,forit raised the question whetherexact
statementsof probabilitywere really impossible,and if they were,
whetherthe inequalityarrivedat was really the closest inequality to
be derived by a valid argumentfromthe data.
Thirdly,Dr. Neyman proposed to extend the fiducial argument
fromcases wheretherewas onlya singleunknownparameter,to cases
in which there were several. Here, again, there might be serious
difficultiesin respect to the mutual consistency of the different
inferencesto be drawn; for, with a single parameter,it could be
shown that all the inferencesmightbe summarizedin a single prob-
ability distributionforthat parameter,and that, forthis reason, all
weremutuallyconsistent; but it had not yet been shownthat when
the parameterswere more than one any such equivalent frequency
distributioncould be established.
Dr. Fisher said that here he ought to point out that Dr. Neyman
did him too much honour in ascribing to him the establishmentof
" Student's " distribution. It was " Student" himselfwho took the
reallynovel step,whichhad in factrevolutionizedthe theoryoferrors.
He showed,in the particularcase he treated,that it was possible to
finda quantity,which was known to them as " Student's t," having
a frequencydistributionindependentof all unknownparameters,and
being at the same time expressibleas a functionof one only of these
parameters,togetherwith other quantities directlyobservable,and,
estimationemployed.But thisprocessis knownto omit,or suppress, partof
theinformation suppliedby the sample. The statementsbased on inefficient
so longas theyare understood
estimatesare true,therefore, not to be the
wholetruth. Statementsbased on sufficient estimatesare freefromthis
drawback, and mayclaima uniquevalidity.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 619

therefore,knownwith exactitude. That, as it seemed to Dr. Fisher,

constitutedthe real revolution. All that he had added to it was to
" studentize" a number of analogous problems,and to exploit the
logical advantages of the position to which he showed the way.
It was the more essential that he should make this clear since
"Student " was himselffartoo modesta man to claim, perhaps even
to believe, how much he had done for the advance of statistical
theory.
possible
The criticismas to the mutual consistencyofthe different
inferencesdid not affectthe value of Dr. Neyman's advice on the
samplingproblem. He stressedit here only because it was just this
question of consistencywhich had led a succession of mathematical
writersto rejectthe theoryofinverseprobability,and he had no wish
to see fiducialprobabilityfollowthe same course.

The followingcontributionwas received fromPROFESSOR OSKAR

ANDERSON afterthe Meeting:-
Dr. Neyman has referredto the enquiryinto farmingconditions
in Bulgaria. As I am responsibleforthe scientificmethodapplied in
it and for the main lines of its organization(the technicalexecution
rested with Dr. Stefanoff, and, of course,with the Chief of the Bul-
garian Statistical Office,Dr. Kiranoff),I would like to be permitted
to make a fewremarks.
The process of fixingthe villages to be examined was indeed, in
the case ofthe BulgarianEnquiry, a difficult one and took a relatively
long time. Our opinionwas that, forour type of selection,thisis the
centralproblemand the key of the whole work. We began by pick-
ing out all villages in which some special formof farmingwas highly
developed (such as tobacco, roses,rice, silkworms,grape vines, etc.).
These formed I3 groups containing about i,ooo villages. The
remaining4,000 villages of Bulgaria, forminga much more homo-
geneous mass, were divided into 5 climaticregions,and each of these
in its turn into 3 physical regions-villages on the mountain slopes,
villages in the foothillsand villages of the plain. The uniformityof
the compositionof each group was checked. We then chose for our
sample some villages of each group in such a mannerthat they con-
tained about one-fiftieth of the farmsof the group,whichin theirturn
gave, forthe census data of 1927, the same distributionof the size of
estates (in I5 categories!) as that of the whole groupfromwhichthey
were taken. This preliminarychoice was then carefullychecked by
comparingthe means of the characteristicsof the sample with the
means forthe whole of Bulgaria for all data providedby thecensusof
1927. It was found,aftersome correctionsin the firstchoice, that
in at least I8 directionsout of I9 the agreementbecame very satis-
factory*) This is confirmedby I9 tables and diagrams in the
Bulgarian Bulletinde Statistique,Number 8, 1934, a copy of which is
now in the Tihra.rv of this societv. We tried also to reach a more or
* The onlv exceptionis the area of artificialmeadow-entirely a quantite
nggligeablein Bulgaria. Some slightdiscrepanciesin the area of vineyardscan
be easily explainedby the existenceofvineyardsin urbanareas (Varna, Plevna,
etc.).

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
620 Discussion Part IV,

less equal distributionof the selected'villages on the territoryof

Bulgaria.
Dr. Neyman findsthat there is only one detail in our Enquiry
about which he is not certainwhetherit is justifiable: viz. the pur-
posive mannerof selectingthe villages fromthe above 28 " strata,"
instead ofpurelyrandomsampling. He thinks" That thevariability
offarmsand villagesis also a characteroftheirpopulationwhichmay
be of interest." This character,however,would be biased in the
sample.
My replyis:
(1) Our chiefobject was to determinea great numberof general
means for the whole of Bulgaria (of 1934) and not to measure the
variabilityof any characteristic,and
(2) It can be shown that, owing to (a) the very small scale of
Bulgarian farming,(b) the uniformityof each of the 28 groups,and
(c) the relativelylarge numberof villages in the selection(100), and
(d) the approximatelylinear character of the correlationbetween
the size of farms, and most of their other characteristics the
variabilityof the farms would not be sensiblybiased in a sample
containingin any case more than i8,ooo farms; and as to the vari-
ability of the villages,it is already well knownto us.
Without going into mathematics,let us consider the following
simpleexample. Let our problembe to determinethe average length
ofthe rightarm ofthosepresentin the Annual Meetingofthe Society.
Leaving stratificationon one side as an unnecessarycomplicationof
our example, the method of random sampling would consist in a
selection by chance of,say, io membersof the audience, measuring
their right arms and calculating the mean. But suppose that we
know beforehanid that the mean lengthof the leftarms of the whole
group presentis, say, 28 inches: if then we again select io members
of the audience whose left arm is of or about this length, then
Dr. Neyman will agree that it is most probable that the new mean
lengthof rightarms (estimatedby a " purposive selection") will be
more accurate than the formermean (estimatedon a purelyrandom
sample). But, if our selectionwere based, e.g. on the mean colour of
ties worn by membersof the audience, the resultwould be much less
useful. In the Bulgarian Enquiry, I think,the analogy was much
closer to control by lengthof leftarm than to controlby colour of
ties.
Why did we use purposive selection and not random sampling,
which I also personallyprefer? The reasons are very simple time
and money. Given our verydetailed programme,the shorttime and
the limited qualified personnelavailable in Bulgaria, it was by no
means possible to extend the enquiry over more than ioo villages.
The work had to be finishedin one month,beforethe beginningof
farmingoperations,and, of course,we could not drive our relatively
small staffover mountains,throughforests,water and snow in early
springand on Bulgarian roads to, say, i,ooo villages,whichwould be
necessaryin the case of a purelyrandom sampling of them, even if
stratified.
The Bulgarian Statistical Officehopes to publishthe resultsof our

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 621

enquiryat the end of this year. The reportwill containa theoretical

introductionand all possible controlson the generallinesindicatedby
the formulaeof Prof. Bowley, and also of Dr. Neyman.
To conclude: I fullyagree with Dr. Neyman that with random
samplingthe statisticianis on firmer
groundand feelsmore.confidence
than withpurposiveselection. But I differ fromhim in thinkingthat
thereare occasions wherethe latteris both moreeconomicaland more
exact. The conditions for a good purposive selection are, firstly,
that the object of the enquiry should possess some qualities similar
to those of the Bulgarian or Danish * example, and secondly,that
all the circumstancesof the enquirymust be very carefullyplanned
and checked. I agree that both are far frombeing always possible;
and in any case, in my opinion,the firstthingwhich is needed fora
successfulissue is by no means " good luck," as Dr. Neyman says,
but a perfectknowledgeofthe object and sound statisticalreasoning.

DR. NEYMAN, in reply: I am most gratefulto all those who have

taken part in the discussion of my paper. Extensive discussion is
very useful,not only to listenersand readers,but also to the author,
because it shows him what is properlydone in his paper and where
he is at fault.
The present discussion has shown, I think, (i) that my criticism
against the methodofpurposiveselectionwas sufficiently convincing,
and (ii) that the sectionsconcernedwiththe confidenceintervalsand
the problem of estimation were not. Out of the four eminent
statisticianswho have honouredmy paper by discussingit, therewas
only one who defendedthe methodof purposiveselection. And then
this was not a defenceof the method itself,but ratherof a separate
inquiryin which a mixtureof the two methodsof stratifiedsampling
by groups, and of purposive selection was used. ProfessorBowley
stated that he distrustedthe method of purposive selection,even in
1925, when he prepared his Report to the International Statistical
Institute.
Therefore,as far as the main subject of my paper is concerned,
I have to argue only with ProfessorAnderson,and I am glad that
the argument will concern only details, not my main thesis. In
fact, his statementthat, " with random sampling,the statisticianis
on firmerground and feels more confidencethan with purposive
selection," shows that we are in perfect agreement on the main
point. Our agreementextendseven furtherthan ProfessorAnderson
seems to think,as I agree with him that in certaincases the method
of purposive selection may be applied with great success. I even
suggested a class of problemsin which this is the case (see the last
paragraph of the main text of the paper, page 589). But we are
relying on good luck if we apply the method without sufficient
evidence of its validity.
It is a very interestingfact that, in Bulgarian conditions,the
regressionsof many charactersof farmson theirsize are linear. In
* See Adolph Jensen," Purposive Selection," Journ.Roy. Stat. Soc., Vol.
XCI, pp. 541-547.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
622 Discussion [Part IV,

Poland we have found* that many regressions,for instance the

regressionof the gross income on the size and on the outlay, cannot
be representedby a plane, and that the partial regressionof the
income on the outlay depends very much upon the size of the farms.
Probably the general farmingconditions in the two countries are
very different. The application of ProfessorAnderson's method in
Poland would lead to biased estimatesof the means sought.
With regard to the problem of determiningthe average length
of the right arm of those present at the meeting,which has been
quoted by ProfessorAnderson as an instance in which the method
of purposive selectionshould be applied, I should like to notice that
it is a special case of the class of the problemsindicated in the last
sentence of my paper (page 589) as suitable for the application of
this method. So here again we are in agreement. I cannot agree,
however,that the accuracy of an unbiased estimatewill be increased
if we purposely select individuals with the length of the left arm
approximatelyequal to its average. The solution of this question
follows directlyfromthe formulae,already fully discussed in text-
books,t to the effectthat the greater accuracy is obtained (i) by
minimisingthe differencebetween the sample and the population
means of the control,and (ii) by maximizingthe sample standard
deviation of the control.
* See: (1) W. Pytkowski: The Dependenceof theIncome in small Farms
upon theirArea, the Outlay and the Capital investedin Cows. Bibljoteka
Pulawska, Warsaw, 1932.
(2) K. Iwaszkiewicz: " La rentabilitede l'etendue,du fonds de roulement
etdu capital investien vachesdans les petitesexploitationsrurales." Kwartalnik
Statystyczny,t. IX, Fascicule 1, 1932, Warsaw.
(3) M. Iwaszkiewicz: " Recherchesstatistiquessur la rentabilitedes engrais
artificielsdans les petitesexploitationsrurales." Kwartalnik Statystyczny,
t. X, Fascicule 2-3, 1933, Warsaw.
All these publicationsare to be foundin " Statistica"-the collectionsof the
papers prepared in the Biometric Laboratory, Nencki Institute and in the
StatisticalLaboratory,CentralCollegeof Agriculture,Warsaw.
t See, for instance (1) M. Ezekiel: " Methodsof CorrelationAnalysis,"
New York, 1930,pp. 252-255.
(2) R. A. Fisher: " StatisticalMethodsforResearchWorkers,"London, 1932,
nn.115-117.
Finally, the solution may be obtained from a suitable adjustment of
formulae(36), (37) and (39) in the Note III of the Appendix,p. 603.
The mentionedformulaeare:-
Y' = 4+ a(X -

and 2)2)
/L2= av~2(1-r2)(1 ? (X 2
-
wherex and meanthesamplemeansofthetwovariables,uX2 and a52 the sample
variances,X the knownpopulationmean of x, Y'-the estimateof the popula-
tion mean of y, p2-the estimate of the variance of the same, a-the sample
regressioncoefficientof y on x and r-the sample correlationcoefficient.As
the differenceI X - J is practicallyneverzero,in orderto attain the greatest
accuracyofthe estimateY', it is advisable to tryto minimizeI X - J keeping
ax as large as possible. In order to do so, it is necessary to include in the
sample individualsboth with very large values of x and with very small ones.
- will be small and gxlarge,and 2 will approach its
Then the differenceIX -
minimumvalue Oy2(1- r2)
n -2

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 623

I did not expect that the sections of my paper dealing with the
new formof the problemof estimationwould play so large a part in
the discussion. It is at least gratifyingthat the criticismswere so
divergentthat one of the speakers could say that everythingin it
is doubtful,and another that it is nothing new. I considered it
necessaryto include these sectionsin mypaper, as otherwiseit would
not be complete,and it would be justifiableto ask why I am not
troublingto considerthe problems which ProfessorBowley termed
the inverseproblems.
Detailed commentson all questionsraised in the discussionon the
confidenceintervalswould requiretoo much space. In fact, to clear
up the matter entirely,a separate publication is needed. As this
is in preparation,I shall limit myselfonly to one or two remarks,
which may clear away certainobvious misunderstandings.
It has been suggested in the discussion that I used the term
"confidence coefficient " insteadof the term " fiducialprobability."
This is certainly a misunderstanding. The term confidence co-
efficientis not synonymousto the term probability. It means an
arbitrarilychosen value of the probability of our being rightwhen
applying a certain rule of behaviour. The relation of the concept
of the confidencecoefficient to.that of probabilitymay be compared
to the relation betweenthe concepts of the " price " and " money"
(this,ifwe accept the definitionofthe " price" as " a certainamount
of money which has been fixed by the merchant . . ."). Perhaps
a still better comparison is provided by the terms " rate of
interest" and " money." The analogy here is less superficialthan
one would expect. Banks are workingat a certainrate of interest,
which is being fixedonce fora longerperiod,and just this constancy
led to the introductionof the term " rate of interest," The validity
of probability statements in the new form of the problem of
estimation,whichhas been here so extensivelydiscussed,depends on
the permanentuse of a systemof confidenceintervals. This system
as a whole (not separate intervals) correspondsto a fixedprobability
that our predictionsare correct, and certainly there is a definite
advantage in having a special term to denote this value of the
probability. It would allow us, for example, to use the convenient
expressions like the following: the seed-testingstation in X is
workingwith the confidencecoefficient -95,etc.
Another important misunderstanding,which I think it useful
to clear up now, is contained in the following remarks of Pro-
fessor Bowley concerningthe theory of the confidenceintervals:-
" (a) Does it really lead us towards what we need the chance that,
in the universe which we are sampling, the proportionis within
these certainlimits? I thinkit does not; (b) I thinkwe are in the
position of knowingthat eitheran improbable event has occurred,
or the proportionin the population is withinthe limits."
I have marked the two sentences with letters (a) and (b) as I
shall have to commenton them separately.
The sentence(a) containsthe statementof the problemof estima-
tion in the formof Bayes. Simple algebra shows that the solution
of this problem must depend upon the probability law a priori.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
624 Discussion [Part IV,

Therefore,if all we need consists in "the chance that, in the

universe which we are sampling, the proportion is within given
limits," we certainlycannot go any furtherthan it is already known.
In so far as we keep to the old formof the problem,any further
progress is impossible. It would be possible only if the previous
writers on the subject had been wrong. The present progress is
connected with the fact that, instead of the mathematical problem
stated in the sentence (a), we are solving some other mathematical
problem, say (oc), which (i) has a solution independent of any
arbitraryassumptions concerningthe probabilitylaw a priori, and
(ii) may form a basis for the practical work of a statistician con-
cernedwith problemsof estimation.
Now what is the differencebetween the problems (a) and (oc)?
Both of them are dealing with probabilities,but these probabilities
apply to differentevents. In the problem (a) we ask about the
probability that a character of the sampled population lies within
certain limits, while in the problem (oc),we are interestedin the
probability of committingan error when applying constantly a
certain rule of behaviour. The formerprobability proved to be
dependent on the probabilitiesa priori. On the other hand, it was
possible to invent such systems of statements about the values of
population characters that the probability of being wrong in
these statementsis a fixed one, whatever the probabilities a priori
(the law of which may be continuous or not, without restriction).
This circumstance,that in the problem of confidenceintervals the
probabilitystatementsconcernthe resultsof our behaviour, not the
populations and that they relate to this given rule of behaviour,
not to the propertiesof samples to which this rule is being applied,
is very important.
Next, let me remark on the sentence (b) in the criticism of
ProfessorBowley. I shall simplifythe position, assuming that we
want to determinethe proportion,p, of black balls in a bag, and for
this purpose we intendto draw as many as n = 3 balls (withreplace-
ment-for simplicity). The number of black balls in the sample
will be denoted by X, which may have the values 0, 1, 2, 3. The
sentence (b) of ProfessorBowley assumes that we actually have the
sample drawn and that, for instance, X = 3. Having got so far,
we certainlycannot tell much about the probabilityof p having any
definitevalue. In fact, all we can say is that eitheran improbable
fact occurred,or p has a rather large value. But what is done in
the methodof confidenceintervalsis different. We start our reason-
ing, as it were,beforedrawingthe sample. Or again, we assume that
this is not the only one sample with which we shall have to deal.
We notice that the sample may provide us only with one out of the
four possibilities X1 -0, X2= 1, X3 2 and X4 3. Having
noticed this,we fixa rule as follows:-
If in the sample which we shall draw, X will have the value
X = 0, then we shall state that 0 < p < Ii'"
If X-1 , ,, P < 7r2
,, ,, ,,
X= 3, ) 4 < p <

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions
1934] on Dr. Neyman's Paper. 625

We are aware that the statement which we shall make, in

applyingthis rule to the result of actual sampling,may be wrongor
may be true. We calculate the probability,P, that the statement
will be a true one, and try to arrange the system of values of the
's so as to have P > 95, whatever the probabilitylaw a prori.
In this way, having a sample and knowingthat X 3, we cannot
tell any more about the value ofp than ProfessorBowley has stated.
On the other hand, making sta-tementsfollowingthe rules set out
above, we know something important about the results of these
statements: the probabilitythat we shall be wrongis then < 05.
The last misunderstandingI should like to clear concerns the
recognitionof the merits of the work of " Student." I certainly
recognize and appreciate it very much. If I call a distribution
" Student's " distribution,it means clearlythat I attributeits dis-
coveryto " Student," and not to anybodyelse. This does not prevent
me fromrecognizingand appreciatingthe work of ProfessorFisher
concerningthe same distribution.
I am very gratefulto all present at the meeting for the kind
receptionof my paper.

This content downloaded from 194.29.185.251 on Tue, 24 Jun 2014 23:59:44 PM

All use subject to JSTOR Terms and Conditions