Evidence-based Orthodontics

How sample size influences research outcomes

Jorge Faber1, Lilian Martins Fonseca2

DOI: http://dx.doi.org/10.1590/2176-9451.19.4.027-029.ebo

Sample size calculation is part of the early stages of conducting an epidemiological, clinical or lab study. In preparing a
scientific paper, there are ethical and methodological indications for its use. Two investigations conducted with the same
methodology and achieving equivalent results, but different only in terms of sample size, may point the researcher in
different directions when it comes to making clinical decisions. Therefore, ideally, samples should not be small and, con-
trary to what one might think, should not be excessive. The aim of this paper is to discuss in clinical language the main
implications of the sample size when interpreting a study.

Keywords: Sample calculation. Sample size. Clinical trial. Methodology. Scientific evidence.

O cálculo amostral faz parte dos estágios iniciais de realização de um estudo epidemiológico, clínico ou laboratorial. Há
indicações éticas e metodológicas para o seu emprego na elaboração de um trabalho científico. Duas pesquisas, realiza-
das com a mesma metodologia obtendo resultados equivalentes, e que diferem apenas no tamanho da amostra, podem
apontar para diferentes direções no processo de tomada de decisão clínica. Portanto, as amostras estudadas idealmente não
devem ser pequenas e, ao contrário do que pode-se pensar, não devem ser excessivas. O objetivo desse artigo é discutir,
numa linguagem clínica, as principais implicações do tamanho das amostras na interpretação de um estudo.

Palavras-chave: Cálculo amostral. Tamanho da amostra. Ensaios clínicos. Metodologia. Evidência científica

In recent years a growing concern has overwhelmed FACTORS THAT AFFECT SAMPLE SIZE
the scientific community in the healthcare area: Sam- The purpose of estimating the appropriate sample
ple size calculation. Although at first blush it may seem size is to produce studies capable of detecting clini-
like an overriding concern over methodological issues, cally relevant differences. Bearing this point in mind,
notably to clinicians, such concern is utterly justifiable. there are different formulas to calculate sample size.2,3
This issue is of paramount importance. These formulas comprise several aspects which are
Samples should not be either too big or too small listed below. Most sample size calculators available
since both have limitations that can compromise on the web have limited validity because they use a
the conclusions drawn from the studies. Too small single formula — which is usually not divulged — to
a sample may prevent the findings from being ex- generate sample sizes for the studies.
trapolated, whereas too large a sample may amplify The first aspect is the type of variable being stud-
the detection of differences, emphasizing statistical ied. For example, it should be determined if the
differences that are not clinically relevant.1 We will variable is categorical like the Angle classification
discuss in this article the major impacts of sample (Class I, II or III), or continuous like the length of
size on orthodontic studies. the dental arch (usually measured in millimeters).

How to cite this article: Faber J, Fonseca LM. How sample size influences research Submitted: May 20, 2014 - Revised and accepted: June 02, 2014
outcomes. Dental Press J Orthod. 2014 July-Aug;19(4):27-9. DOI: http://dx.doi.
org/10.1590/2176-9451.19.4.027-029.ebo Contact address: Jorge Faber
Brasília Shopping, SCN Q. 5 – bloco A – sala 408
Adjunct professor, Department of Orthodontics, University of Brasília.
1 Brasília/DF — Brazil
Invited Professor, Department of Orthodontics, University of Brasília.
2 CEP: 70.715-900
E-mail: [email protected]

© 2014 Dental Press Journal of Orthodontics 27 Dental Press J Orthod. 2014 July-Aug;19(4):27-9
Evidence-based Orthodontics How sample size influences research outcomes

It is then necessary to determine the relation- uncomfortable has the potential to improve treatment
ship between the groups that will be evaluated and of Class II malocclusions. The researcher wishes to
the statistical analysis that will be employed. Are we compare the new functional device with the Herbst
going to evaluate groups that are independent, i.e., appliance. Patients will be randomly assigned to each
the measurements of one group do not influence the group. The researcher is not aware, but we are, that
other? Are they dependent groups like the measure- s/he needs 60 subjects (30 patients in each group) to
ments taken before and after treatment? Are we go- ensure sufficient power to be able to extrapolate the
ing to use a split-mouth design, whereby treatment statistical analysis results to the overall population.
is performed on one quadrant and a different thera- In  other words, so that we can feel confident that
py on another quadrant? Will we be using t-test or these results will serve as a parameter on which to
chi-square test? All these questions lead to different base the proposed treatment. Furthermore, we also
sample size calculation formulas. know, although the researcher does not, that this new
Subsequently, we have to answer the question con- therapy is less effective than the traditional method.
cerning which results we envisage if a standard treat- However, the researcher used only 15 patients
ment is performed. What is the mean value or the ex- in each group. The results of the study showed that
pected ratio? The answer to this question is usually ob- the new device is inferior to conventional treatment.
tained from the literature or by means of pilot studies. What are the implications?
It is also important to determine what is the The first is that using a sample smaller than the
smallest magnitude of the effect and the extent to ideal increases the chance of assuming as true a
which it is clinically relevant. For example, how false premise. Thus, chances are that the proposed
many degrees of difference in the ANB angle can be device has no disadvantage compared to tradition-
considered relevant? It is vital that we address this is- al therapy. Furthermore, it is assumed that people
sue. The smaller the difference that we wish to iden- were subjected to a study, and had to undergo in
tify, the greater the number of cases in a study. If re- vain all additional suffering associated with the
searchers wish to detect a difference as small as 0.1° therapy, given that the goals of the study were not
in an ANB angle, they will probably need thousands achieved. In addition, financial and time resources
of patients in their study. If this value rises to 1°, the were squandered since ultimately it will contribute
number of cases required falls drastically. absolutely nothing to improve clinical practice or
Finally, it is essential that the researcher deter- quality of life. The situation becomes even worse if
mine the level of significance and the type II error, the research involves public funding: A total waste
which is the probability of not rejecting the null hy- of taxpayer money.
pothesis, although the hypothesis is actually false,
which the study will accept as reasonable. PROBLEMS WITH VERY LARGE SAMPLES
With this information in hand, we will apply the There is a widespread belief that large samples
appropriate formula according to the study design in are ideal for research or statistical analysis. Howev-
question, and determine the sample size. Today, this er, this is not always true. Using the above example
calculation is typically carried out with the aid of a as a case study, very large samples that exceed the
computer program. For example, Pocock’s formula2 value estimated by sample size calculation present
for continuous variables is frequently used in our spe- different hurdles.
cialty. It is used in studies where one wishes to exam- The first is ethical. Should a study be performed
ine the difference between data means with normal with more patients than necessary? This means that
distribution and equal-size, independent groups. more people than needed are exposed to the new
therapy. Potentially, this implies increased hassle
PROBLEMS WITH VERY SMALL SAMPLES and risk. Obviously the problem is compounded if
Try to envision the following scenario. A re- the new protocol is inferior to the traditional meth-
searcher conducts a study on patients who are be- od: More patients are involved in a new, uncomfort-
ing treated with a new device which although very able therapy that yields inferior results.

© 2014 Dental Press Journal of Orthodontics 28 Dental Press J Orthod. 2014 July-Aug;19(4):27-9
Faber J, Fonseca LM Evidence-based Orthodontics

The second obstacle is that the use of a larger num- CONCLUSIONS

ber of cases can also involve more financial and human In designing a study, sample size calculation is
resources than necessary to obtain the desired response. important for methodological and ethical reasons, as
In addition to these factors, there is another note- well as for reasons of human and financial resources.
worthy issue that has to do with statistics. Statistical When reading an article, the reader should be on the
tests were developed to handle samples, not popula- alert to ascertain that the study they are reading was
tions. When numerous cases are included in the sta- subjected to sample size calculation. In the absence
tistics, analysis power is substantially increased. This of this calculation, the findings of the study should
implies an exaggerated tendency to reject null hy- be interpreted with caution.
potheses with clinically negligible differences. What An appropriate sample renders the research
is insignificant becomes significant. Thus, a potential more efficient: Data generated are reliable, re-
statistically significant difference in the ANB angle source investment is as limited as possible, while
of 0.1° between the groups cited in the previous ex- conforming to ethical principles. The use of sam-
ample would obviously produce no clinical difference ple size calculation directly influences research
in the effects of wearing an appliance. findings. Very small samples undermine the in-
When very large samples are available in a retrospec- ternal and external validity of a study. Very large
tive study, the researcher needs first to collect subsam- samples tend to transform small differences into
ples randomly, and only then perform the statistical test. statistically significant differences — even when
If it is a prospective study, the researcher should collect they are clinically insignificant. As a result, both
only what is necessary, and include a few more indi- researchers and clinicians are misguided, which
viduals to compensate for subjects that leave the study. may lead to failure in treatment decisions.

REFERENCES

1. Altman DG. Practical Statistics for Medical Research. London, UK;

Chapman & Hall; 1991.
2. Pandis N, Polychronopoulou A, Eliades T. Sample size estimation: an
overview with applications to orthodontic clinical trial designs. Am J
Orthod Dentofacial Orthop. 2011 Oct,140(4):e141-e146.
3. Hajian-Tilaki K. Sample size estimation in diagnostic test studies of
biomedical informatics. J Biomed Inform. 2014 Feb 26. pii: S1532-
0464(14)00050-1. doi: 10.1016/j.jbi.2014.02.013. [Epub ahead of print]

© 2014 Dental Press Journal of Orthodontics 29 Dental Press J Orthod. 2014 July-Aug;19(4):27-9