Contents

10 Chi 10.1 10.2 10.3 10.4 Square Tests Introduction . . . . . . . . . . . . . . . . . . . . . . . The Chi Square Distribution . . . . . . . . . . . . . Goodness of Fit Test . . . . . . . . . . . . . . . . . . Chi Square Test for Independence . . . . . . . . . . 10.4.1 Statistical Relationships and Association . . . 10.4.2 A Test for Independence . . . . . . . . . . . . 10.4.3 Notation for the Test of Independence . . . . 10.4.4 Reporting Chi Square Tests for Independence 10.4.5 Test of Independence from an SPSS Program 10.4.6 Summary . . . . . . . . . . . . . . . . . . . . 10.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . 703 . 703 . 704 . 709 . 731 . 732 . 734 . 740 . 750 . 752 . 763 . 764

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

702

Chapter 10

Chi Square Tests

The statistical inference of the last three chapters has concentrated on statistics such as the mean and the proportion. These summary statistics have been used to obtain interval estimates and test hypotheses concerning population parameters. This chapter changes the approach to inferential statistics somewhat by examining whole distributions, and the relationship between two distributions. In doing this, the data is not summarized into a single measure such as the mean, standard deviation or proportion. The whole distribution of the variable is examined, and inferences concerning the nature of the distribution are obtained. In this chapter, these inferences are drawn using the chi square distribution and the chi square test. The rst type of chi square test is the goodness of t test. This is a test which makes a statement or claim concerning the nature of the distribution for the whole population. The data in the sample is examined in order to see whether this distribution is consistent with the hypothesized distribution of the population or not. One way in which the chi square goodness of t test can be used is to examine how closely a sample matches a population. In Chapter 7, the representativeness of a sample was discussed in Examples ?? through ??. At that point, hypothesis testing had not yet been discussed, and there was no test for how well the characteristics of a sample matched the characteristics of a population. In this chapter, the chi square goodness of t test can be used to provide a test for the representativeness of a sample. The second type of chi square test which will be examined is the chi

703

Chi-Square Tests

704

square test for independence of two variables. This test begins with a cross classication table of the type examined in Section 6.2 of Chapter 6. There these tables were used to illustrate conditional probabilities, and the independence or dependence of particular events. In Chapter 6, the issue of the independence or dependence of the variables as a whole could not be examined except by considering all possible combinations of events, and testing for the independence of each pair of these events. In this chapter, the concept of independence and dependence will be extended from events to variables. The chi square test of independence allows the researcher to determine whether variables are independent of each other or whether there is a pattern of dependence between them. If there is a dependence, the researcher can claim that the two variables have a statistical relationship with each other. For example, a researcher might wish to know how the opinions of supporters of dierent political parties vary with respect to issues such as taxation, immigration, or social welfare. A table of the distribution of the political preferences of respondents cross classied by the opinions of respondents, obtained from a sample, can be used to test whether there is some relationship between political preferences and opinions more generally. The chi square tests in this chapter are among the most useful and most widely used tests in statistics. The assumptions on which these tests are based are minimal, although a certain minimum sample size is usually required. The variables which are being examined can be measured at any level, nominal, ordinal, interval, or ratio. The tests can thus be used in most circumstances. While these tests may not provide as much information as some of the tests examined so far, their ease of use and their wide applicability makes them extremely worthwhile tests. In order to lay a basis for these tests, a short discussion of the chi square distribution and table is required. This is contained in the following section. Section 10.3 examines the chi square goodness of t test, and Section 10.4 presents a chi square test for independence of two variables.

10.2

The Chi Square Distribution

The chi square distribution is a theoretical or mathematical distribution which has wide applicability in statistical work. The term chi square (pronounced with a hard ch) is used because the Greek letter is used to dene this distribution. It will be seen that the elements on which this dis-

Chi-Square Tests

705

tribution is based are squared, so that the symbol 2 is used to denote the distribution. An example of the chi squared distribution is given in Figure 10.1. Along the horizontal axis is the 2 value. The minimum possible value for a 2 variable is 0, but there is no maximum value. The vertical axis is the probability, or probability density, associated with each value of 2 . The curve reaches a peak not far above 0, and then declines slowly as the 2 value increases, so that the curve is asymmetric. As with the distributions introduced earlier, as larger 2 values are obtained, the curve is asymptotic to the horizontal axis, always approaching it, but never quite touching the axis. Each 2 distribution has a degree of freedom associated with it, so that there are many dierent chi squared distributions. The chi squared distributions for each of 1 through 30 degrees of freedom, along with the distributions for 40, 50 , . . . , 100 degrees of freedom, are given in Appendix ??. The 2 distribution for 5 degrees of freedom is given in Figure 10.1. The total area under the whole 2 curve is equal to 1. The shaded area in this gure shows the right 0.05 of the area under the distribution, beginning at 2 = 11.070. You will nd this value in the table of Appendix ?? in the fth row (5 df) and the column headed 0.05. The signicance levels are given across the top of the 2 table and the degrees of freedom are given by the various rows of the table. The chi square table is thus quite easy to read. All you need is the degree of freedom and the signicance level of the test. Then the critical 2 value can be read directly from the table. The only limitation is that you are restricted to using the signicance levels and degrees of freedom shown in the table. If you need a dierent level of signicance, you could try interpolating between the values in the table. The Chi Square Statistic. The 2 statistic appears quite dierent from the other statistics which have been used in the previous hypotheses tests. It also appears to bear little resemblance to the theoretical chi square distribution just described. For both the goodness of t test and the test of independence, the chi square statistic is the same. For both of these tests, all the categories into which the data have been divided are used. The data obtained from the sample are referred to as the observed numbers of cases. These are the frequencies of occurrence for each category into which the data have been

Chi-Square Tests

706

Figure 10.1: 2 Distribution with 5 Degrees of Freedom grouped. In the chi square tests, the null hypothesis makes a statement concerning how many cases are to be expected in each category if this hypothesis is correct. The chi square test is based on the dierence between the observed and the expected values for each category. The chi square statistic is dened as =
i 2

(Oi Ei )2 Ei

where Oi is the observed number of cases in category i, and Ei is the expected number of cases in category i. This chi square statistic is obtained by calculating the dierence between the observed number of cases and the expected number of cases in each category. This dierence is squared and divided by the expected number of cases in that category. These values are then added for all the categories, and the total is referred to as the chi squared value.

Chi-Square Tests

707

Chi Square Calculation Each entry in the summation can be referred to as The observed minus the expected, squared, divided by the expected. The chi square value for the test as a whole is The sum of the observed minus the expected, squared, divided by the expected.

The null hypothesis is a particular claim concerning how the data is distributed. More will be said about the construction of the null hypothesis later. The null and alternative hypotheses for each chi square test can be stated as H0 : Oi = Ei H1 : Oi = Ei If the claim made in the null hypothesis is true, the observed and the expected values are close to each other and Oi Ei is small for each category. When the observed data does not conform to what has been expected on the basis of the null hypothesis, the dierence between the observed and expected values, Oi Ei , is large. The chi square statistic is thus small when the null hypothesis is true, and large when the null hypothesis is not true. Exactly how large the 2 value must be in order to be considered large enough to reject the null hypothesis, can be determined from the level of signicance and the chi sqaure table in Appendix ??. A general formula for determining the degrees of freedom is not given at this stage, because this diers for the two types of chi square tests. In each type of test though, the degrees of freedom is based on the number of categories which are used in the calculation of the statistic. The chi square statistic, along with the chi square distribution, allow the researcher to determine whether the data is distributed as claimed. If the chi square statistic is large enough to reject H0 , then the sample provides evidence that the distribution is not as claimed in H0 . If the chi square statistic is not so large, then the researcher may have insucient evidence to reject the claim made in the null hypothesis. The following paragraphs briey describe the process by which the chi square distribution is obtained. You need not read the following page, and

Chi-Square Tests

708

can go directly to the goodness of t test in Section 10.3. But if you do read the following paragraphs, you should be able to develop a better understanding of the 2 distribution. Derivation of the Distribution. In mathematical terms, the 2 variable is the sum of the squares of a set of normally distributed variables. Imagine a standardized normal distribution for a variable Z with mean 0 and standard deviation 1. Suppose that a particular value Z1 is randomly selected from this distribution. Then suppose another value Z2 is selected from the same standardized normal distribution. If there are d degrees of freedom, then let this process continue until d dierent Z values are selected from this distribution. The 2 variable is dened as the sum of the squares of these Z values. That is,
2 2 2 2 2 = Z1 + Z2 + Z3 + + Zd

Suppose this process of independent selection of d dierent values is repeated many times. The variable 2 will vary, because each random selection from the normal distribution will be dierent. Note that this variable is a continuous variable since each of the Z values is continuous. This sum of squares of d normally distributed variables has a distribution which is called the 2 distribution with d degrees of freedom. It can be shown mathematically that the mean of this 2 distribution is d and the standard deviation is 2d. An intuitive idea of the general shape of the distribution can also be obtained by considering this sum of squares. Since 2 is the sum of a set of squared values, it can never be negative. The minimum chi squared value would be obtained if each Z = 0 so that 2 would also be 0. There is no upper limit to the 2 value. If all the Z values were quite large, then 2 would also be large. But note that this is not too likely to happen. Since large Z values are distant from the mean of a normal distribution, these large Z values have relatively low probabilities of occurrence, also implying that the probability of obtaining a large 2 value is low. Also note that as there are more degrees of freedom, there are more squared Z values, and this means larger 2 values. Further, since most values under the normal curve are quite close to the center of the distribution, within 1 or 2 standard deviations of center, the values of 2 also tend to be concentrated around the mean of the 2 distribution. The chi square distribution thus has the bulk of the cases near the center of the distribution,

Chi-Square Tests

709

but it is skewed to the right. It has a lower limit at 0, and declines as 2 increases to the right, in the general shape of Figure 10.1. While the above provides a general description of the theoretical 2 distribution of Appendix ??, this may seem to be quite dierent than the chi square statistic: (Oi Ei )2 2 = Ei i The theoretical distribution is a continuous distribution, but the 2 statistic is obtained in a discrete manner, on the basis of discrete dierences between the observed and expected values. As will be seen on page 717, the probabilities of obtaining the various possible values of the chi square statistic can be approximated with the theoretical 2 distribution of Appendix ??. The condition in which the chi square statistic is approximated by the theoretical chi square distribution, is that the sample size is reasonably large. The rule concerning the meaning of large sample size is dierent for the 2 test than it is for the approximations which have been used so far in this textbook. The rough rule of thumb concerning sample size is that there should be at least 5 expected cases per category. Some modications of this condition will be discussed in the following sections.

10.3

Goodness of Fit Test

This section begins with a short sketch of the method used to conduct a chi square goodness of t test. An example of such a test is then provided, followed by a more detailed discussion of the methods used. Finally, there is a variety of examples used to illustrate some of the common types of goodness of t tests. The chi square goodness of t test begins by hypothesizing that the distribution of a variable behaves in a particular manner. For example, in order to determine daily stang needs of a retail store, the manager may wish to know whether there are an equal number of customers each day of the week. To begin, an hypothesis of equal numbers of customers on each day could be assumed, and this would be the null hupothesis. A student may observe the set of grades for a class, and suspect that the professor allocated the grades on the basis of a normal distribution. Another possibility is that a researcher claims that the sample selected has a distribution which is very close to distribution of the population. While no general statement can be

Chi-Square Tests

710

provided to cover all these possibilities, what is common to all is that a claim has been made concerning the nature of the whole distribution. Suppose that a variable has a frequency distribution with k categories into which the data has been grouped. The frequencies of occurrence of the variable, for each category of the variable, are called the observed values. The manner in which the chi square goodness of t test works is to determine how many cases there would be in each category if the sample data were distributed exactly according to the claim. These are termed the expected number of cases for each category. The total of the expected number of cases is always made equal to the total of the observed number of cases. The null hypothesis is that the observed number of cases in each category is exactly equal to the expected number of cases in each category. The alternative hypothesis is that the observed and expected number of cases dier suciently to reject the null hypothesis. Let Oi is the observed number of cases in category i and Ei is the expected number of cases in each category, for each of the k categories i = 1, 2, 3, , k , into which the data has been grouped. The hypotheses are H0 : Oi = Ei H1 : Oi = Ei and the test statistic is 2 =
i

(Oi Ei )2 Ei

where the summation proceeds across all k categories, and there are k 1 degrees of freedom. Large values of this statistic lead the researcher to reject the null hypothesis, smaller values mean that the null hypothesis cannot be rejected. The chi square distribution with k 1 degrees of freedom in Appendix ?? is used to determine the critical 2 value for the test. This sketch of the test should allow you to follow the rst example of a chi square goodness of t test. After Example 10.3.1, the complete notation for a chi square goodness of t test is provided. This notation will become more meaningful after you have seen how a goodness of t test is conducted. Example 10.3.1 Test of Representativeness of a Sample of Toronto Women In Chapter 7, a sample of Toronto women from a research study was examined in Example ??. Some of this data is reproduced here in Table 10.1.

Chi-Square Tests

711

This data originally came from the article by Michael D. Smith, Sociodemographic risk factors in wife abuse: Results from a survey of Toronto women, Canadian Journal of Sociology 15 (1), 1990, page 47. Smith claimed that the sample showed a close match between the age distributions of women in the sample and all women in Toronto between the ages of 20 and 44. This is especially true in the youngest and oldest age brackets.

Age 20-24 25-34 35-44 Total

Number in Sample 103 216 171 490

Per Cent in Census 18 50 32 100

Table 10.1: Sample and Census Age Distribution of Toronto Women Using the data in Table 10.1, conduct a chi square goodness of t test to determine whether the sample does provide a good match to the known age distribution of Toronto women. Use the 0.05 level of signicance. Solution. Smiths claim is that the age distribution of women in the sample closely matches the age distribution of all women in Toronto. The alternative hypothesis is that the age distribution of the women in the sample do not match the age distribution of all Toronto women. The hypotheses for this goodness of t test are: H0 : The age distribution of respondents in the sample is the same as the age distribution of Toronto women, based on the Census. H1 : The age distribution of respondents in the sample diers from the age distribution of women in the Census. The frequencies of occurrence of women in each age group are given in the middle column of Table 10.1. These are the observed values Oi , and there are k = 3 categories into which the data has been grouped so that i = 1, 2, 3.

Chi-Square Tests

712

Suppose the age distribution of women in the sample conforms exactly with the age distribution of all Toronto women as determined from the Census. Then the expected values for each category, Ei , could be determined. With these observed and expected numbers of cases, the hypotheses can be written H0 : Oi = Ei H1 : Oi = Ei For these hypotheses, the test statistic is 2 =
i

(Oi Ei )2 Ei

where the summation proceeds across all k categories and there are d = k 1 degrees of freedom. The chi square test should always be conducted using the actual number of cases, rather than the percentages. These actual numbers for the observed cases, Oi , are given as the frequencies in the middle column of Table 10.1. The sum of this column is the sample size of n = 490. The expected number of cases for each category is obtained by taking the total of 490 cases, and determining how many of these cases there would be in each category if the null hypothesis were to be exactly true. The percentages in the last column of Table 10.1 are used to obtain these. Let the 20-24 age group be category i = 1. The 20-24 age group contains 18% of all the women, and if the null hypothesis were to be exactly correct, there would be 18% of the 490 cases in category 1. This is E1 = 490 18 = 490 0.18 = 88.2 100

For the second category 25-34, there would be 50% of the total number of cases, so that 50 E2 = 490 = 490 0.50 = 245.0 100 cases. Finally, there would be E3 = 490 32 = 490 0.32 = 156.8 100

cases in the third category, ages 35-44. Both the observed and the expected values, along with all the calculations for the 2 statistic are given as in Table 10.2. Note that the sum of

Chi-Square Tests

713

each of the observed and expected numbers of cases totals 490. The dierence between the observed and expected numbers of cases is given in the column labelled Oi Ei , with the squares of these in the column to the right. Finally, these squares of the observed minus expected numbers of cases divided by the expected number of cases is given in the last column. For example, for the rst row, these calculations are (Oi Ei )2 (103 88.2)2 14.82 219.04 = = = = 2.483. Ei 88.2 88.2 88.2 The other rows are determined in the same manner, and this table provides all the material for the chi square goodness of t test. Category 1 2 3 Total Oi 103 216 171 490 Ei 88.2 245.0 156.8 490.0 Oi Ei 14.8 -29.0 14.2 0.0 (Oi Ei )2 219.04 841.00 201.64 (Oi Ei )2 /Ei 2.483 3.433 1.286 7.202

Table 10.2: Goodness of Fit Calculations for Sample of Toronto Women The sum of the last column of the table gives the required 2 statistic. This is 2 =
3 i=1

(Oi Ei )2 Ei

= 2.483 + 3.433 + 1.286 = 7.202 The next step is to decide whether this is a large or a small 2 value. The level of signicance requested is the = 0.05 level. The number of degrees of freedom is the number of categories minus one. There are k = 3 categories into which the ages of women have been grouped, so that there are d = k 1 = 3 1 = 2 degrees of freedom. From the 0.05 column and the second row of the 2 table in Appendix ??, the critical value is 2 = 5.991. This means that there is exactly 0.05 of the area under the curve to the right of 2 = 5.991. A chi square value larger than this leads to rejection of

Chi-Square Tests

714

the null hypothesis, and a chi square value from the data which is smaller than 5.991 means that the null hypothesis cannot be rejected. The data yields a value for the chi squared statistic of 7.202 and this exceeds 5.991. Since 7.202 > 5.991 the null hypothesis can be rejected, and the research hypothesis accepted at the 0.05 level of signicance. The statement of the author that the sample is a close match of the age distribution of all women in Toronto is not exactly justied by the data presented in Table 10.1. Of course, there is a chance of Type I error, the error of rejecting a correct null hypothesis. In this case, the sampling method may generally be a good method, and it just happens that the set of women selected in this sample ends up being a bit unrepresentative of the population of all women in Toronto. If this is so, then there is Type I error. However, the chance of this is at most 0.05. Since this is quite small, the test shows that the sample is representative of all Toronto women. Additional Comments. There are a few parts of the calculations which should be noted. First, note the dierences between the observed and expected values in the middle of Table 10.2. The largest dierence is -29.0, for group 2, the 25-34 age group. This group appears to be considerably underrepresented in the sample, with there being 29 less observed than expected cases. As can be noted in the last column of the table, this group contributes 3.433 to the 2 total of 7.202, the largest single contribution. For the other two age groups, the discrepancy between the observed and expected is not so large. Each of the rst and third groups is somewhat overrepresented in the sample, compared with the Census distribution. But the smaller chi squared values for these rst and third groups, 2.483 and 1.286 (in the last column), means that these groups are not overrepresented as much as the middle aged group is underrepresented. If the sample were to be adjusted to make it more representative, the sample of 25-34 year old women should be boosted. Note also that the sum of the Oi Ei column is 0. This is because the dierences between the observed and expected numbers of cases are sometimes positive and sometimes negative, but these positives and negatives cancel if the calculations are correct. Finally, an appreciation of the meaning of the degrees of freedom can be obtained by considering how the expected values are constructed. Note that the expected values are calculated by using the percentage of cases that are in each category, in order to determine the expected numbers of cases. After the expected values for each of the rst two categories have

Chi-Square Tests

715

been determined, the number of expected cases in the third category can be determined by subtraction. That is, if there are 88.2 and 245.0 expected cases in the rst two categories, this produces a total of 88.2 + 245.0 = 333.2 expected cases. In order to have the sum of the expected cases equal 490, this means that there must be 490.0 333.2 = 156.8 cases in the third category. In terms of the degrees of freedom, this means that there are only two degrees of freedom. Once values for two of the categories have been specied, the third category is constrained by the fact that the sum of the observed and expected values must be equal. Notation for the Chi Square Goodness of Fit Test. As noted earlier, there is no general formula which can be given to cover all possible claims which could be made in a null hypothesis. In the examples which follow, you will see various types of claims, each of which required a slightly dierent method of determining the expected numbers of cases in each category. However, notation can be provided to give a more complete explanation of the test. Begin with k categories into which a frequency distribution has been organized. The frequencies of occurrence for the variable are f1 , f2 , f3 , , fk , but for purposes of carrying out the 2 goodness of t test, these are called the observed values, Oi and given the symbols O1 , O2 , O3 , , Ok . Let the sum of these frequencies or observed values be n, that is, fi = Oi = n.

The expected values for the test must also total n, so that Ei = Oi = n.

The expected numbers of cases Ei is determined on the basis of the null hypothesis. That is, some hypothesis is made concerning how the cases should be distributed. Let this hypothesis be that there is a proportion pi cases in each of the categories i, where i = 1, 2, 3, , k . Since all the cases must be accounted for, the sum of all the proportions must be 1. That is
k

pi = 1.
i=1

The null hypothesis could then be stated as

Chi-Square Tests

716

H0 : The proportion of cases in category 1 is p1 , in category 2 is p2 , . . . , in category k is pk . Alternatively, this hypothesis could be stated to be: H0 : The proportion of cases in category i is pi , i = 1, 2, 3, , k. The alternative hypothesis is that the proportions are not equal to pi as hypothesized. In Example 10.3.1 the proportions hypothesized in the null hypothesis are the percentages divided by 100. The hypotheses are H0 : p1 = 0.18, p2 = 0.50, p3 = 0.32 H1 : Proportions are not as given in H0 . The alternative hypothesis is that at least one of the proportions in the null hypothesis is not as stated there. Note that
3

pi = 0.18 + 0.50 + 0.32 = 1.

The expected cases are obtained by multiplying these pi by n. That is, for each category i, Ei = pi n. Since the sum of the pi is 1, the sum of the Ei is n. These values of Oi and Ei can now be used to determine the values of the chi squared statistic =
i=1 2 k

(Oi Ei )2 Ei

where there are d = k 1 degrees of freedom. The only assumption required for conducting the test is that each of the expected numbers of cases is reasonably large. The usual rule of thumb adopted here is that each of the expected cases should be greater than or equal to 5, that is, Ei 5, for each i = 1, 2, 3, , k. This rule may be relaxed slightly if most of these expected values exceed 5. For example, suppose there are 6 categories, and all of the categories

Chi-Square Tests

717

have more than 5 expected cases, but one category has only 4.3 expected cases. Unless the chi square value comes out quite close to the borderline of acceptance or not acceptance of the null hypothesis, this should cause little inaccuracy. The larger the expected number of cases, the more accurate the chi squared calculation will be. Also note that this condition is placed on the expected values, not the observed values. There may be no observed values in one or more categories, and this causes no diculty with the test. But there must be around 5 or more expected cases per category. If there are considerably less than this, adjacent categories may be grouped together to boost the number of expected cases. An example of this is given in Example 10.3.4. The reason why the number of expected cases must be 5 or more is as follows. As noted earlier, there are really two chi squares which have been given here. The chi square distribution is a theoretically derived mathematical distribution, based on the sums of squares of normally distributed variables. This is a continuous distribution. In contrast, the chi square statistic is a discrete statistic, based on a nite number of possible values Oi Ei . These dierences are used to compute the chi square statistic =
i=1 2 k

(Oi Ei )2 . Ei

This statistic can be shown to have what mathematicians refer to as a multinomial distribution. This distribution is a generalization of the binomial probability distribution, obtained by examining the distribution of the possible values where there are more than 2 categories. It can also be shown mathematically that the multinomial distribution can be approximated by the theoretical chi square distribution under certain conditions. The condition ordinarily specied by statisticians is that the expected cases in each category be 5 or more. This is large enough to ensure that the discrete multinomial distribution is reasonably closely approximated by the continuous chi square distribution. Several examples of dierent types of goodness of t tests now follow. If you study these examples, this should provide you with the background required to carry out most goodness of t tests. Example 10.3.2 Suicides in Saskatchewan The media often seizes on yearly changes in crime or other statistics. A large jump in the number of murders from one year to the next, or a large

Chi-Square Tests

718

decline in the the support for a particular political party may become the subject of many news reports and analysis. These statistics may be expected to show some shift from year to year, just because there is variation in any phenomenon. This question addresses this issue by looking at changes in the annual number of suicides in the province of Saskatchewan. Suicides in Saskatchewan declined by more than 13% from 1982 to 1983, was the headline in the Saskburg Times in early 1984. The article went on to interview several noted experts on suicide who gave possible reasons for the large decline. As a student who has just completed a statistics course, you come across the article and decide to check out the data. By consulting Table 25 of Saskatchewan Health, Vital Statistics, Annual Report, for various years, you nd the gures for 1978-1989. The data is given in Table 10.3. Number of Suicides in Saskatchewan 164 142 153 171 171 148 136 133 138 132 145 124

Year 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989

Table 10.3: Suicides in Saskatchewan, 1978 1989 Use the 2 test for goodness of t to test the hypothesis that the number of suicides reported for each year from 1978 to 1989 does not dier signicantly from an equal number of suicides in each year. (0.05 signicance). Based on this result, what might you conclude about the newspaper report, especially in light of the extra information for 1984 1989 that is now available to you? Also comment on any possible errors in the conclusion.

Chi-Square Tests

719

Solution. The null hypothesis is that there is the same number of suicides each year in the province. While this is obviously not exactly true, the test of this hypothesis will allow you to decide whether the actual number of suicides is enough dierent from an equal number to reject the null hypothesis. The hypotheses can be stated as follows: H0 : The distribution of the observed number of suicides for each year does not dier signicantly from an equal number of suicides in each year. H1 : The distribution of the observed number of suicides for each year diers signicantly from an equal number of suicides in each year. For purposes of conducting the test, these hypotheses translate into H0 : Oi = Ei H1 : Oi = Ei where the Ei are equal for each year, and sum to the same total number as the total of the observed number of suicides over all these years. Since we observe a total of 1757 suicides over the 12 years shown here, this means an average of 1757/12 = 146.4 suicides each year. If there were an equal number of suicides for each of the 12 years, then this would mean 146.4 each year, and this is the value of the expected number of cases for each year. The 2 test is then conducted as follows, with the calculations as shown in Table 10.4. For the 2 test, the number of degrees of freedom is the number of categories minus one. Here this is d = 12 1 = 11 degrees of freedom. For = 0.05 and 11 degrees of freedom, the 2 value from the table in Appendix ?? is 19.675. The region of rejection is all 2 values of 19.675 or more. In Figure 10.2, the chi square distribution with 11 degrees of freedom and = 0.05 is shown. The critical 2 value is 2 = 19.675, and 0.05 of the area under the curve lies to the right of this. From the set of the observed yearly number of suicides, 2 = 18.135 < 19.675. As can be seen in Figure 10.2, 2 = 18.135 is not in the critical region for this test. and this means that there is insucient evidence to reject H0 at the 0.05 level of signicance. As a result the null hypothesis that the distribution of the actual number of suicides is not any dierent than an equal number of suicides each year cannot be rejected at the 0.05 level of signicance. There is a considerable chance of Type II error here, the error of failing to reject the null hypothesis when it is not correct. It is obvious that the

Chi-Square Tests

720

Figure 10.2: Chi Square Distribution for Test of Equality in the Annual Number of Suicides number of suicides is not equal for each of these 12 years and as a result, the null hypothesis is not exactly correct. What the chi square goodness of t test shows is that the null hypothesis is also not all that incorrect. The test shows that there is little dierence between the uctuation in the number of suicides from year to year and an assumption that there was no change in the number of suicides each year. What might be concluded then is that the number of suicides has stayed much the same over these years and that whatever factors determine the suicides on a yearly basis have remained much the same over these years. A certain degree of random uctuation in the yearly total of suicides is bound to take place even if the basic factors determining the number of suicides does not change. Also note that the population of Saskatchewan changed little over these years, so that uctuations in the total population cannot be used to explain the yearly variation in suicides. As a result, not too much should be made of the drop in the number of suicides from 171 to 148, or (23/171) 100% = 13.5% over one year. The

Chi-Square Tests

721

Year 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 Total

Oi 164 142 153 171 171 148 136 133 138 132 145 124 1,757

Ei 146.4 146.4 146.4 146.4 146.4 146.4 146.4 146.4 146.4 146.4 146.4 146.4 1,756.8

(Oi Ei ) 17.6 -4.4 6.6 24.6 24.6 1.6 -10.4 -13.4 -8.4 -14.4 -1.4 -22.4 0.2

(Oi Ei )2 309.76 19.36 43.56 605.16 605.16 2.56 108.16 179.56 70.56 207.36 1.96 501.76

(Oi Ei )2 /Ei 2.116 0.132 0.298 4.134 4.134 0.017 0.739 1.227 0.482 1.416 0.013 3.427 2 = 18.135

Table 10.4: Calculations for 2 Test

totals often change from year to year even though the underlying causes do not really change. In this case then, no real explanation of changes in the number of suicides need be sought. This change is more likely just the result of random yearly uctuation. The data from 1983 to 1989 show that for these seven years for which mortality data is available in the province, the number of suicides is lower than in the early 1980s. Since the recent high point of 171 suicides of 1981 and 1982, the number of suicides has declined considerably. This evidence points in a dierent direction than indicated by the hypothesis test and may indicate that suicides peaked in the early 1980s, then declined with a new, lower number of suicides being a more normal level in the mid 1980s. Example 10.3.3 Birth Dates of Hockey Players The Toronto Globe and Mail of November 27, 1987 contained an article, written by Neil Campbell, entitled NHL career can be preconceived. In the article, Campbell claimed that the organization of hockey has turned half the boys in hockey-playing countries into second

Chi-Square Tests class citizens. The disadvantaged are those unlucky enough to have been born in the second half of the calendar year.

722

Campbell calls this the Calendar Eect, arguing that it results from the practice of age grouping of very young boys by calendar year of birth. For example, all boys 7 years old in 1991, and born in 1984, would be in the same grouping. By 1991, those boys born in the rst few months of 1984 are likely to be somewhat larger and better coordinated than those boys born in the later months of 1984. Yet all these players compete against each other. Campbell argues that this initial advantage stays with these players, and may become a permanent advantage by the time the boys are separated into elite leagues at age 9. In order to test whether this Calendar Eect exists among hockey players who are somewhat older, a statistics student collected data from the Western Hockey League (WHL) Yearbook for 1987-88. The birth dates for the players are as shown in Table 10.5. Number of Players 84 77 35 34

Quarter January to March April to June July to September October to December

Table 10.5: Distribution of Birth Dates, WHL Players, 1987-88

The results from Table 10.5 seem quite clear, and one may take this as evidence for the Calendar Eect. The proportion of players born in the rst half of the year is (84+77)/230 = 161/230 = 0.70, well over half. However, a test to determine whether the distribution in Table 10.5 diers signicantly from what would be expected if the birth dates of hockey players match those of the birth dates in the population as a whole should be conducted. In order to do this, note rst that there is a seasonal pattern to births, and this seasonal pattern should be considered to see whether it accounts for this dierence. In order to determine this, the birth dates of births in the four Western provinces of Canada was obtained for the births in the

Chi-Square Tests

723

years 1967-70. This would approximate the dates of the hockey players in the WHL in 1987-88. This data is presented in Table 10.6. Births in 1967-70 Number Proportion 97,487 104,731 103,974 97,186 0.242 0.260 0.258 0.241

Quarter January to March April to June July to September October to December

Table 10.6: Distribution of Births by Quarter, 4 Western Provinces, 1967-70

As can be seen in Table 10.6, while there is a seasonal pattern to births, it does not appear to be very dierent from an equal number of, or proportion of, births in each quarter. Also, it appears to be very dierent from the pattern of births by quarter for the hockey players. In order to conduct a test of whether the distribution of birth dates of hockey players diers from the distribution of the birth dates shown in Table 10.6, conduct a chi square test of goodness of t. Use 0.001 signicance. Solution. The hypotheses can be stated as follows: H0 : The distribution of the observed number of births by quarter does not dier signicantly from that of the distribution of births in the Western provinces in 1967-70. H1 : The distribution of the observed number of births by quarter diers signicantly from that of the distribution of births in the Western provinces in 1967-70. For the 2 test, it is necessary to determine the expected number of births in each quarter, based on H0 . In order to do this, use the proportion of births for each quarter in Table 10.6. Then multiply these proportions by the total number of players. This is the total of 230 players in Table 10.5. For example, for January to March, there are 0.242 of total births expected. This amounts to 0.242 230 = 55.6 expected births between January and March. This is done for each of the four quarters, and the results shown

Chi-Square Tests

724

as the expected Ei column in Table 10.7. The rest of the calculations in Table 10.7 are the calculations required for the 2 test. i 1 2 3 4 Total Oi 84 77 35 34 230 Ei 55.6 59.7 59.3 55.4 230.0 Table 10.7: Calculations for 2 Test For the 2 test, the number of degrees of freedom is the number of categories minus one. This is d = k 1 = 4 1 = 3 degrees of freedom. For = 0.001 and 3 degrees of freedom, the 2 value from the table is 16.266. The region of rejection is all 2 values of 16.266 or more. The value from the WHL data is 38.113 and this is well into the region of rejection of the null hypothesis. As a result the null hypothesis can be rejected at the 0.001 level of signicance. There appears to be very strong evidence of the Calendar Eect, based on the birth dates of the Western Hockey League for 1987-88. The chance of Type I error, rejecting the null hypothesis when it is really true, is extremely small. There is a probability of less than 0.001 of Type I error. Goodness of Fit to a Theoretical Distribution. Suppose that a researcher wishes to test the suitability of a model such as the normal distribution or the binomial probability distribution, as an explanation for an observed phenomenon. For example, a grade distribution for grades in a class may be observed, and someone claims that the instructor has allocated the grades on the basis of a normal distribution. The test of the goodness of t then looks at the observed number of grades in each category, determines the number that would be expected if the grades had been exactly normally distributed, and compares these with the chi square goodness of t test. In general this test is the same as the earlier goodness of t tests except for one alteration. The degrees of freedom for this test must take into account the number of parameters contained in the distribution being tted (Oi Ei ) 28.4 17.3 - 24.3 - 21.4 (Oi Ei )2 806.56 299.29 590.49 457.96 (Oi Ei )2 /Ei 14.506 5.383 9.958 8.266 2 = 38.113

Chi-Square Tests

725

to the data. The new formula for the degrees of freedom becomes d = k 1r where k is the number of categories into which the data has been grouped, and r is the number of parameters contained in the distribution which is being used as the model. In the case of the normal distribution, there are two parameters, and , so that there are d = k 1 2 = k 3 degrees of freedom when an acutal distribution is being compared with a normal distribution with the goodness of t test. If the suitability of the binomial as an explanation for the data is being requested, there are d = k 1 1 = k 2 degrees of freedom. For the binomial, there is only one parameter, the proportion of successes. Using this modication, the suitability of the normal distribution in explaining a grade distribution is examined in the following example. Example 10.3.4 Grade Distribution in Social Studies 201 The grade distribution for Social Studies 201 in the Winter 1990 semester is contained in Table 10.8. Along with the grade distribution for Social Studies 201 is the grade distribution for all the classes in the Faculty of Arts in the same semester. Social Studies 201 1990 Winter (Number) 2 7 10 15 8 1 43 68.8 12.6

Grade Less than 50 50s 60s 70s 80s 90s Total Mean Standard Deviation

All Arts (Per Cent) 8.3 15.4 24.7 30.8 17.8 3.0 100.0 68.2

Table 10.8: Grade Distributions

Chi-Square Tests

726

Use the data in Table 10.8 to conduct two chi square goodness of t tests. First test whether the model of a normal distribution of grades adequately explains the grade distribution of Social Studies 201. Then test whether the grade distribution for Social Studies 201 diers from the grade distribution for the Faculty of Arts as a whole. For each test, use the 0.20 level of signicance. Solution. For the rst test, it is necessary to determine the grade distribution that would exist if the grades had been distributed exactly as the normal distribution. The method outlined in Example 6.4.6 on pages 382386 of Chapter 6 will be used here. That is, the normal curve with mean and standard deviation the same as the actual Social Studies 201 distribution will be used. This means that the grade distribution for the normal curve with mean = 68.8 and standard deviation = 12.7 is used to determine the grade distribution if the grades were normally distributed. The Z values were determined as -1.48, -0.69, 0.09, 0.88 and 1.67 for the X values 50, 60, 70, 80 and 90, respectively. By using the normal table in Appendix ??, the areas within each range of grades were determined. These are given as the proportions of Table 10.9. These proportions were then multiplied by 43, the total of the observed values, to obtain the expected number of grades in each of the categories into which the grades have been grouped. These are given in the last column of Table 10.9. Normal Distribution of Grades Proportion (pi ) Number (Ei ) 0.0694 0.1757 0.2908 0.2747 0.1419 0.0475 1.000 3.0 7.6 12.5 11.8 6.1 2.0 43.0

Grade Less than 50 50s 60s 70s 80s 90s Total

Table 10.9: Normal Distribution of Grades

Chi-Square Tests The hypotheses can be stated as follows:

727

H0 : The distribution of the observed number of students in each grade category does not dier signicantly from what would be expected if the grade distribution were exactly normally distributed. H1 : The distribution of the observed number of students in each category diers signicantly from a normal distribution of grades. In order to apply the 2 test properly, each of the expected values should exceed 5. The less than 50 category and the 90s category both have less than 5 expected cases. In this test, the 90s have only 2 expected cases, so this category is merged with the grades in the 80s. For the grades less than 50, even though there are only 3 expected cases, these have been left in a category of their own. While this may bias the results a little, the eect should not be too great. The calculation of the 2 statistic is as given in Table 10.10. Category 1 2 3 4 5 Total Oi 2 7 10 15 9 43 Ei 3.0 7.6 12.5 11.8 8.1 43.0 (Oi Ei ) -1.0 -0.6 -2.5 3.2 0.9 0.0 (Oi Ei )2 1.00 0.36 6.25 10.24 0.81 (Oi Ei )2 /Ei 0.333 0.047 0.500 0.868 0.100 2 = 1.848

Table 10.10: Test for a Normal Distribution of Grades For this 2 test, the number of degrees of freedom is the number of categories minus one, minus the number of parameters. Here this is 5 1 2 = 2 degrees of freedom. For = 0.20 and 2 degrees of freedom, the 2 value from the table is 3.219. The region of rejection is all 2 values of 3.219 or more. The value of the chi square statistic for this test is 1.848 and this is not in the region of rejection of the null hypothesis. While the grades are not exactly normally distributed, the hypothesis that they are distributed normally cannot be rejected at the 0.20 level of signicance. For the test concerning a dierence in the grade distributions of Social

Chi-Square Tests

728

Category 1 2 3 4 5 Total

Oi 2 7 10 15 9 43

Ei 3.6 6.6 10.6 13.2 9.0 43.0

(Oi Ei ) -1.6 0.4 -0.6 1.8 0.0 0.0

(Oi Ei )2 2.56 0.16 0.36 3.24 0.00

(Oi Ei )2 /Ei 0.711 0.024 0.034 0.245 0.000 2 = 1.014

Table 10.11: Test for Dierence Between SS201 and Arts Grades

Studies 201 and the distribution for the Faculty of Arts, the hypotheses can be stated as follows: H0 : The distribution of the observed number of students in each grade category, in Social Studies 201, does not dier signicantly from what would be expected if the grade distribution exactly matched that for all Arts grades. H1 : The distribution of the observed number of students in each category diers signicantly from that of all Arts grades. To obtain the expected number of grades in each category under this null hypothesis, multiply 43 by the percentages from the distribution of grades in the Faculty of Arts. This provides the expected number of cases in each category. For example, in the less than 50 category, there would be expected to be 8.3% of 43, or 3.6 expected cases. Using the same method, and moving from the lowest to the highest grades, the expected numbers of cases for each category are 3.6, 6.6, 10.6, 13.2, 7.7 and 1.3. For the 90s, there are only 1.3 expected cases, violating the assumption of more than 5 expected cases per cell. This category is merged with the 80s so that the assumption of 5 or more cases per category is met. For the less than 50 category, the same problem emerges. However, there are 3.6 expected cases in this category, and this is only a little less than 5, so this is kept as a separate category, even though this violates the assumptions a little. The calculation of the 2 statistic is given in Table 10.11. For the 2 test, the number of degrees of freedom is the number of categories minus one. Here this is 5 1 = 4 degrees of freedom. For = 0.05

Chi-Square Tests

729

and 4 degrees of freedom, the 2 value from the table is 9.4877. The region of rejection is all 2 values of 9.4877 or more. The value from this data is 2 = 1.014, and this is not in the region of rejection of the null hypothesis. As a result, the null hypothesis that the distribution of the grades in Social Studies 201 does not dier from the distribution of all grades in the Faculty of Arts cannot be rejected. This conclusion can be made at the 0.20 level of signicance. Additional Comments. 1. Note that two dierent null hypotheses have been tested here, and neither can be rejected. One hypothesis claimed that the grades are normally distributed, and the other null hypothesis was that the grade distribution is the same as that of grades in the Faculty of Arts as a whole. Since neither of these could be rejected, there is some question concerning which of these is the correct null hypothesis. Within the method of hypothesis testing itself, there is no clear way of deciding which of these two null hypotheses is correct. Since the 2 value is smaller in the the case of the second null hypothesis, that the grades are the same as the Faculty of Arts as a whole, it could be claimed that this is the proper hypothesis, and is the best explanation of the Social Studies 201 grade distribution. However, the normal distribution also provides a reasonable explanation. The problem here is that of Type II error. In each case, the null hypothesis could not be rejected, but the null hypothesis may not be exactly correct either. Where there are two null hypotheses, both of which seem quite reasonable and neither of which can be rejected, then the uncertainty is increased. The only real solution to this problem would be to obtain more data. If the grades for Social Studies 201 over several semesters were to be combined, then a clearer picture might emerge. If a sample of grades over the course of 4 or 5 semesters were to be examined, it is likely that one or other of these two null hypotheses could be rejected. Since this data is not provided here, there is little that can be done except to regard each of these two null hypotheses as possible explanations for the distribution of Social Studies 201 grades. 2. Accepting the Null Hypothesis. In the discussion of hypothesis tests so far in the textbook, the null hypothesis has never been accepted. If the data is such that the null hypothesis cannot be rejected, then the conclusion is left at this point, without accepting the null hypothesis. The reason for this is that the level of Type II error is usually fairly considerable,

Chi-Square Tests

730

and without more evidence, most researchers feel that more proof would be required before the null hypothesis could be accepted. For the second of the two tests, 2 = 1.014 is so low that the null hypothesis might actually be accepted. On the second page of the chi square table of Appendix ??, there are entries for the chi square values such that most of the distribution lies to the right of these values. For 4 degrees of freedom, and 2 = 1.014, there is approximately 0.900 of the area under the curve to the right of this. That is, for 4 degrees of freedom, the signicance = 0.900 is associated with 2 = 1.064. This means that just over 90% of the area under the chi square distribution lies to the right of a chi square value of 1.014. Since this 2 value is so close to the left end of the distribution, this might be taken as proof that the distribution is really the same as that of the Faculty of Arts as a whole. If this is to be done, then it makes sense to reduce Type II error to a low level, and increase Type I error. Note that if a signicance level of 0.900 is selected, and the null hypothesis is rejected, there is a 0.900 chance of making Type I error. Since this is usually regarded as the more serious of the two types of error, the researcher wishes to reduce this error. But if the common hypothesis testing procedure is reversed, so that the aim is to reject the alternative hypothesis, and prove that a particular null hypothesis is correct, then the usual procedures are reversed. In order to do this, the signicance level should be a large value, so that the null hypothesis can be rejected quite easily. Only in those circumstances where the data conforms very closely with the claim of the null hypothesis, should the null hypothesis be accepted. From this discussion, it would be reasonable to accept the claim made in the second null hypothesis, that the Social Studies 201 distribution is the same as the distribution of grades for all classes in the Faculty of Arts. While this conclusion may be in error, this error is very minimal, because the actual distribution conforms so closely to the hypothesized distribution. Conclusion. The examples presented here show the wide range of possible types of goodness of t test. Any claim that is made concerning the whole distribution of a variable can be tested using the chi square goodness of t test. The only restriction on the test is that there should be approximately 5 or more expected cases per cell. Other than that, there are really no restrictions on the use of the test. The frequency distribution could be measured on a nominal, ordinal, interval or ratio scale, and could be either

Chi-Square Tests

731

discrete or continuous. All that is required is a grouping of the values of the variable into categories, and you need to know the number of observed cases which fall into eachi category. Once this is available, any claim concerning the nature of the distribution can be tested. At the same time, there are some weaknesses to the test. This test is often a rst test to check whether the frequency distribution more or less conforms to some hypothesized distribution. If it does not conform so closely, the question which emerges is how or where it does not match the distribution claimed. The chi square test does not answer this question, so that further analysis is required. For example, in the case of the sample of Toronto women in Example 10.3.1, the chi square goodness of t test showed that the sample was not exactly representative of Toronto women in age. By looking back at the observed and expected numbers of cases, it was seen that the middle aged group was underrepresented. But this latter conclusion really falls outside the limits of the chi square goodness of t test itself. The chi square test for goodness of t is a very useful and widely used test. The following section shows another, and quite dierent way, of using the chi square test and distribution.

10.4

Chi Square Test for Independence

The chi square test for independence of two variables is a test which uses a cross classication table to examine the nature of the relationship between these variables. These tables are sometimes referred to as contingency tables, and they have been discussed in this textbook as cross classication tables in connection with probability in Chapter 6. These tables show the manner in which two variables are either related or are not related to each other. The test for independence examines whether the observed pattern between the variables in the table is strong enough to show that the two variables are dependent on each other or not. While the chi square statistic and distribution are used in this test, the test is quite distinct from the test of goodness of t. The goodness of t test examines only one variable, while the test of independence is concerned with the relationship between two variables. Like the goodness of t test, the chi square test of independence is very general, and can be used with variables measured on any type of scale, nominal, ordinal, interval or ratio. The only limitation on the use of this

Chi-Square Tests

732

test is that the sample sizes must be suciently large to ensure that the expected number of cases in each category is ve or more. This rule can be modied somewhat, but as with all approximations, larger sample sizes are preferable to smaller sample sizes. There are no other limitations on the use of the test, and the chi square statistic can be used to test any contingency or cross classication table for independence of the two variables. The chi square test for independence is conducted by assuming that there is no relationship between the two variables being examined. The alternative hypothesis is that there is some relationship between the variables. The nature of statistical relationships between variables has not been systematically examined in this textbook so far. The following section contains a few introductory comments concerning the nature of statistical relationships among variables.

10.4.1

Statistical Relationships and Association

There are various types of statistical relationships which can exist among variables. Each of these types of relationship involves some form of connection or association between the variables. The connection may be a causal one, so that when one variable changes, this causes changes in another variable or variables. Other associations among variables are no less real, but the causal nature of the connection may be obscure or unknown. Other variables may be related statistically, even though there is no causal or real connection among them. A few examples of the types of statistical relationship that can exist, and how these might be interpreted follow. In this Chapter, the chi square test provides a means of testing whether or not a relationship between two variables exists. In Chapter ??, various summary measures of the extent of the association between two variables are developed. One way to consider a relationship between two variables is to imagine that one variable aects or inuences another variable. For example, in agriculture, the inuence of dierent amounts of rainfall, sunshine, temperature, fertilizer and cultivation can all aect the yield of the crop. These eects can be measured, and the manner in which each of these factors aect crop yields can be determined. This is a clear cut example of a relationship between variables. In the social sciences, experimentation of the agriculture type is usually not possible, but some quite direct relationships can be observed. Education is observed to have a positive inuence on incomes of those who are

Chi-Square Tests

733

employed in the paid labour force. Those individuals who have more years of education, on average have higher income levels. While the higher incomes are not always caused by more education, and are certainly not assured to any inidividual who obtains more years of education, this relationship does hold true in the aggregate. In terms of conditional probabilities, the probability of being in a higher income category is greater for those with more years of education than for those with fewer years of education. Studies of education and the labour force generally give these results, so that there is a relationship between the variables income and education. Both of the previous examples involve a causal, and a one directional statistical relationship. Another type of relationship that is common in the social sciences is the association between two variables. An example of this could be opinions and political party supported. In Example ?? it was observed that PC supporters in Edmonton were more likely to think that trade unions were at least partially responsible for unemployment than were Liberal party supporters. It is dicult to know exactly what the nature of the connection between these variables is, but it is clear that the two variables, political preference and opinion, are related. Perhaps those who tend to have a negative attitude toward trade unions are attracted to the program of the Conservative Party. Or it may be that Conservative Party supporters together tend to develop views that trade unions are partially responsible for unemployment. It is also possible that opinions and political preferences are developed simultaneously, so that those who come to support the PCs, at the same time develop negative views toward trade unions. Regardless of the manner in which opinions and political preferences are formed, there are often associations of the type outlined here. This association can be observed statistically, and the nature of the pattern or connection between the variables can be examined with many statistical procedures. The chi square test for independence provides a rst approach to examining this association. The chi square test of independence begins with the hypothesis of no association, or no relationship, between the two variables. In intuitive terms this means that the two variables do not inuence each other and are not connected in any way. If one variable changes in value, and this is not associated with changes in the other variable in a predictable manner, then there is no relationship between the two variables. For example, if the cross section of opinions is the same regardless of political preference, then opinions and political preference have no relationship with each other. When one variable does change in value, and this is associated with some

Chi-Square Tests

734

systematic change in values for another variable, then there is statistical dependence of the two variables. For example, shen the distribution of opinions does dier across dierent political preferences, then opinion and political preference have an association or relationship with each other. This is the alternative hypothesis which will be used in the chi square test for independence. The conditional probabilities of Chapter 6 can be extended to discuss statistical relationships. In Chapter 6, two events A and B were said to be independent of each other when the conditional probability of A, given event B, equals the probability of A. That is, P (A/B ) = P (A) or P (B/A) = P (B ). If independence of events is extended so that all possible pairs of events across two variables are independent, then the two variables can also be considered to be independent of each other. The two variables X and Y are dened as being independent variables if the probability of the occurrence of one category for variable X does not depend on which category of variable Y occurs. If the probability of occurrence of the dierent possible values of variable X depend on which category of variable Y occurs, then the two variables X and Y are dependent on each other. Based on these notions of independence and dependence, the chi square test for independence is now discussed.

10.4.2

A Test for Independence

The test for independence of X and Y begins by assuming that there is no relationship between the two variables. The alternative hypothesis states that there is some relationship between the two variables. If the two variables in the cross classication are X and Y , the hypotheses are H0 : No relationship between X and Y H1 : Some relationship between X and Y In terms of independence and dependence these hypotheses could be stated H0 : X and Y are independent H1 : X and Y are dependent

Chi-Square Tests

735

The chi square statistic used to conduct this test is the same as in the goodness of t test: (Oi Ei )2 2 = . Ei i The observed numbers of cases, Oi , are the numbers of cases in each cell of the cross classication table, representing the numbers of respondents that take on each of the various combinations of values for the two variables. The expected numbers of cases Ei are computed on a dierent basis than in the goodness of t test. Under the null hypothesis of no relationship between X and Y , the expected cases for each of the cell can be obtained from the multiplication rule of probability for independent events. The manner in which these expected cases are computed will be shown in the following example, with a general formula given later in this section. The level of signicance and the critical region under the 2 curve are obtained in the same manner as in the goodness of t test. The formula for the degrees of freedom associated with a cross classication table is also given a little later in this section. The chi square statistic computed from the observed and expected values is calculated, and if this statistic is in the region of rejection of the null hypothesis, then the assumption of no relationship between X and Y is rejected. If the chi square statistic is not in the critical region, then the null hypothesis of no relationship is not rejected. An example of a chi square test for independence is now given. The general format and formulas for the chi square test of independence are provided following Example 10.4.1. Example 10.4.1 Test for a Relationship between Sex and Class In Section 6.2.9 of Chapter 6, a survey sampling example showing a cross classication of sex by class was given. The cross classication table is presented again in Table 10.12. If variable X is the sex of the respondent, and variable Y is the social class of the respondent, use the chi square test of independence to determine if variables X and Y are independent of each other. Use the 0.05 level of signicance. Solution. The solution provided here gives quite a complete description of the procedures for conducting this test. In later tests, some of these procedures can be streamlined a little.

Chi-Square Tests

736

Y (Social Class) Upper Middle (A) Middle (B ) Working (C ) Lower (D) Total

X (Sex) Male (M ) Female (F ) 33 153 103 16 305 29 181 81 14 305

Total 62 334 184 30 610

Table 10.12: Social Class Cross Classied by Sex of Respondents

The test begins, as usual, with the statement of the null and research hypotheses. The null hypothesis states that there is no relationship between the variables X and Y , so that sex and social class are independent of each other. This means that the distribution of social class for males should be the same as the distribution of social class for females. By examining the two columns of Table 10.12, it can be seen that the distributions are not identical. The question is whether the dierences in the distributions of class between males and females are large enough to reject the hypothesis of independence. The alternative hypothesis is that sex and class are related, so that the variables X and Y are dependent. These hypotheses could be stated in any of the following forms. H0 : No relationship between sex and class H1 : Some relationship between sex and class or H0 : Sex and Class are independent H1 : Sex and Class are dependent or H0 : Oi = Ei H1 : Oi = Ei The last format is the least desirable form in which to write the hypotheses, in that it does not really say exactly what is being tested. However, each of

Chi-Square Tests

737

the null hypotheses listed above are equivalent to each other, and the last form is the one in which the test is actually carried out. That is, if there is no relationship between the two variables, or if X and Y are independent of each other, then Oi = Ei for each of the cells in the table. This is because the Ei s are computed assuming independence of the two variables. The chi square statistic used to conduct this test is the same as in the goodness of t test: (Oi Ei )2 2 = Ei i and the main problem becomes one of computing the expected number of cases for each cell of the table. The observed number of cases are the 8 entries 33, 29, 153, , 16, 14 in Table 10.12. These are repeated in Table 10.13 with the expected numbers of cases also given. X (Sex) Male (M ) Female (F ) 33 (31.0) 153 (167.0) 103 (92.0) 16 (15.0) 305 29 (31.0) 181 (167.0) 81 (92.0) 14 (15.0) 305

Y (Social Class) Upper Middle (A) Middle (B ) Working (C ) Lower (D)

Total 62 334 184 30

Total

610

Table 10.13: Observed and Expected Number of Cases for Social Class Cross Classied by Sex of Respondents The expected numbers of cases are obtained under the null hypothesis that the two variables are independent of each other. If the two variables are independent, then each pair of possible events is also independent. Take the top left cell, the males who are upper middle class. This is the combination of events A and M . In order to determine the expected number of cases for this cell, assuming independence of A and M , the probability of obtaining

Chi-Square Tests

738

this combination is determined rst. If these two events are independent of each other, the probability of the two events occuring together is P (A and M ) = P (A)P (M ). As was shown in Section 6.2.11, when two events are independent of each other, their joint probability is the product of their individual probabilities. This probability is 62 305 P (A and M ) = . 610 610 Note that this probability does not take into account the observed number of cases in this cell of the table, it uses only the information about the proportion of males and the proportion of upper middle class respondents. If being male and being upper middle class are independent of each other, the expected number of respondents in this category is the probability of nding this combination of characteristics, times the number of respondents in the whole sample. That is E (A and M ) = P (A and M ) n. where n = 610 is the sample size. This means that the above probability, multiplied by n, gives the expected number of upper middle class males. This is E (A and M ) = 305 62 610 610 610 62 305 = 610 = 31.0

This same procedure can be followed for each of the cells. For the number of middle class males, the probability of nding this combination, assuming no relationship between male and being middle class, is (334/610) (305/610). Since there are again n = 610 cases in the sample, the expected number of middle class males is E (B and M ) = 334 305 610 610 610 334 305 = 610 = 167.0

Chi-Square Tests

739

For the number of working class females, the probability of nding this combination, assuming no relationship between female and being working class, is (184/610) (305/610). Since there are again n = 610 cases in the sample, the expected number of working class females is E (B and M ) = 184 305 610 610 610 184 305 = 610 = 92.0

That is, 184 out of the 610 respondents are working class, and if sex and class are independent, there should be 184/610 of each sex who are working class. Of this proportion, 305/610 of the sample are females, so the proportion of female, working class members in the sample should be (184/610) (305/610). Since this is a proportion, this is multiplied by the number of cases, n = 610 to obtain the expected number of working class females. Note that each of these expected numbers of cases follows a pattern. For each row, the row total is multiplied by the column total, and this product is divided by the sample size. This produces a general format as follows. For each cell in the cross classication table, the expected number of cases is row total column total . E= grand total For example, the expected number of lower class females is E= 30 305 = 15.0. 610

Each of the expected numbers in the cells can be obtained in the same manner. All of the expected values are in Table 10.13. There are some short cuts which could be taken though, and these short cuts also give an idea of the degrees of freedom associated with the test. Take the rst row. Once it has been determined that there are 31.0 expected upper middle class males, this means there must be 62 31.0 = 31.0 upper middle class females, in order to preserve the total of 62 upper middle class respondents overall. Similarly, when there are 92.0 working class females to be expected, there must be 184 92.0 92.0 working class males. From this, it can be seen that once one column of expected values has been determined, in this table the second column can be determined by subtraction.

Chi-Square Tests

740

Also note that the last row of the table has a similar property. If there are expected to be 31.0, 167.0 and 92.0 males in the rst three rows, then the fourth column can be obtained by subtracting the sum of these three expected numbers from the total of 305 males. That is, the expected number of lower class males is 305 (31.0 + 167.0 + 92.0) = 305 290.0 = 15.0 The expected number of lower class females can be determined in s similar manner. This set of calculations also illustrates the number of degrees of freedom. In this table, there are only 3 degrees of freedom. Once the expected number of males in the rst three rows is determined, each of the other entries in the table is constrained because of the given row and column totals. This means that only 3 cells of the table can be freely assigned values, and once these 3 values have been assigned, all the other expected values are determined by the row and column totals. The general formula for the degrees of freedom is the number of rows minus one, times the number of columns minus 1. Here there are 4 rows and 2 columns, so that the degrees of freedom is (4 1) (2 1) = 3 1 = 3. The chi square values must still be calculated, and these are given in Table 10.14. Each entry in this table is labelled according to the cell it is in. For example, category AM is the upper left cell, that is events A and M. For = 0.05 signicance and 3 degrees of freedom, the critical chi squared value is 7.815. Since the chi square statistic from this data is 2 = 5.369 < 7.815, the null hypothesis of independence of sex and class cannot be rejected. While the male and female social class distributions dier somewhat, they do not dier enough to conclude that there is a relationship between sex and class.

10.4.3

Notation for the Test of Independence

The steps involved in carrying out the 2 test for independence of two variables are as follows. 1. State the null and research hypotheses as H0 : No relation between the two variables H1 : Some relation exists between the two variables

Chi-Square Tests

741

Category AM BM CM DM AF BF CF DF Total

Oi 33 153 103 16 29 181 81 14 610

Ei 31.0 167.0 92.0 15.0 31.0 167.0 92.0 15.0 610.0

(Oi Ei ) 2 -14 11 1 2 14 11 1 0.0

(Oi Ei )2 4 196 121 1 4 196 121 1

(Oi Ei )2 Ei

0.129 1.174 1.315 0.067 0.129 1.174 1.315 0.067 2 = 5.370

Table 10.14: Chi Square Test for Independence between Sex and Class

2. The test statistic is 2 =

(Oi Ei )2 . Ei

3. The degrees of freedom for the test is the number of row minus one times the number of columns minus 1. 4. The level of signicance for the test is selected. 5. From the 2 table, the critical value is obtained. This is based on the level of signicance and the degrees of freedom. The critical region is all values of the 2 statistic that exceed the critical value. 6. The expected number of cases for each cell of the table is obtained for each cell in the table. For each cell of the table, this is accomplished by multiplying the row total times the column total, and divided by the grand total. 7. The chi square statistic is computed by using the observed and the expected numbers of cases for each cell. For each category, subtract the observed from the expected number of cases, square this dierence, and divide this by the expected number of cases. Then sum these values for each of the cells of the table. The resulting sum is the chi square statistic.

Chi-Square Tests

742

8. If the chi square statistic exceeds the critical chi square value, reject the null hypothesis and accept the alternative hypothesis that there is a relationship between the two variables. If the chi square statistic does not exceed the critical value, then do not reject the null hypothesis that there is no relationship between the two variables. The only restriction on the use of the test is that the expected number of cases should exceed 5 in most cells of the cross classication table. One widely used rule is as follows.

Assumptions about Ei No expected case Ei should be less than 1. No more than 20% of the cells should have less than 5 expected cases.

This is the rule which SPSS prints out as part of its output when the chi square test for independence is requested. The following paragraphs discuss this test in a little more detail, and provide a general notation for the test. If you have diculty following this example, you can skip the next part and proceed to the examples. If you follow the above steps, and can understand the examples, then you should have little diculty in carrying out chi square tests of independence. In order to develop a general notation for the chi square test of independence, some new notation is necessary. Let the two variables being examined be X and Y . Suppose that the distribution for variable X has been grouped into c categories, and variable Y into r categories. Let the variable X represent the columns of the table, so that the columns are Xj , where j = 1, 2, 3, , c. The variable Y represents the rows of the table so that the rows are Y1 , Y2 , Y3 , , Yr . Next let the oberserved values be O with two subscripts, so that Oij represents the observed number of cases in row i and column j . Let the row totals be R1 , R2 , R3 , , Rr , the column totals C1 , C2 , C3 , , Cc , and the grand total is n, the sample size. All of this can be summarized in Table 10.15. The null hypothesis is that variables X and Y are independent of each other. The alternative hypothesis is that X and Y are dependent variables.

Chi-Square Tests

743

Variable Y Y1 Y2 . . . Yr Total

X1 O11 O21 . . . Or1 C1

Variable X X2 X3 O12 O22 . . . Or2 C2 O13 O23 . . . Or3 C3 . . .

Xc O1c O2c . . . Orc Cc

Total R1 R2 . . . Rr n

Table 10.15: General Notation for the 2 Test

In order to carry out the 2 test, it is necessary to compute the expected values. The expected values are computed on the basis of the null hypothesis. If the two variables X and Y are independent of each other, let the probability that X = Xj and Y = Yi be Pij . Pij = P (Yi and Xj ) = P (Yi )P (Xj ) = Ri Cj . n n

This result holds based on the principles of probability and of independence in Chapter 6. The expected number of cases in each category under the assumption of independence of X and Y is the above probability multiplied by the number of cases. That is, Pij is the probability of selecting a case which is in row i and column j , under the independence assumption. If n cases are selected, then the number of cases which would be expected to be in row i and column j would be the probability Pij multiplied by the number of cases n. Thus the expected number of cases in row i and column j is Ri Cj Ri Ci n= . Eij = Pij n = n n n These Eij values are then used along with the observed numbers of cases Oij to compute (Oi Ei )2 2 = . Ei i

Chi-Square Tests

744

The degrees of freedom are the number of rows minus one times the number of columns minus 1. Since there are r rows and c columns, the degrees of freedom are d = (r 1) (c 1). This can be seen as follows. Begin with the rst row and move across. Each of the rst c 1 columns can be assigned any values the researcher likes, but once these are assigned, the last entry is determined by the requirement that the sum of the rst row of the table be R1 . For the second row, the procedure is the same, with there being c 1 entries which could be hypothesized to equal any value, but the last entry is constrained by the requirement that the sum of this row be R2 . The same argument can be made for each of the rst r 1 rows. This means that there are (r 1) (c 1) degrees of freedom to this point. The last row adds no extra degrees of freedom, because each of the entries in the last row is constrained by the requirement that the column totals be C1 , C2 , C3 , , Cc . What the above discussion means is that there are (r 1) (c 1) cells of the table which could have values assigned to them in practically any manner the researcher wishes. But the remainder of the cells are constrained by the requirement that the proper row and column totals be produced. In terms of the construction of the 2 statistic, this means that the multinomial distribution determines the probabilities of the various possible combinations of observed and expected numbers of cases for these (r 1) (c 1) cells. This multinomial distribution is approximated by the 2 distribution with (r 1) (c 1) degrees of freedom. This distribution is constructed by summing the squares of (r 1) (c 1) normally distributed values. While all of this would have to be proved mathematically, hopefully this discussion provides an intuitive idea of the manner in which this test and distribution works. Several examples of chi square tests for independence follow. Some of the diculties of interpreting the results will be discussed in these examples. Example 10.4.2 Political Preferences and Opinions in Edmonton Study The relationship between PC and Liberal party supporters in Edmonton was examined in several examples in Chapter 9. The opinion question examines there concerned views on whether or not trade unions were responsible for unemployment. The attitude scale which was used was a 7 point ordinal scale, and this did not satisfy the assumptions that the scale

Chi-Square Tests

745

be interval or ratio level in order that the means be interpreted meaningfully. In spite of this, tests for the dierence of two means, and of two variances were conducted. However, the chi square test for independence might be a more appropriate rst test to conduct when examining this data. Since the sample sizes of the earlier tests were small, all the PCs and Liberals in the Alberta study are used for this example. Table 10.16 gives the distribution of the opinions of 214 PC supporters and 53 Liberal supporters in the Edmonton study. The opinion question asked is the same as that used earlier. That is, repondents were asked their view concerning the opinion Unemployment is high because trade unions have priced their members out of a job. Respondents gave their answers on a 7 point scale, with 1 being strongly disagree and 7 being strongly agree. Use the data in this table to test whether political preference and opinion are independent of each other or not. Use the 0.01 level of signicance. Political Preference PC Liberal 9 7 7 28 51 54 58 3 5 11 3 12 7 12

Opinion 1 2 3 4 5 6 7

Table 10.16: Distribution of Opinions of Edmonton PCs and Liberals

Solution. The null hypothesis for the test is that there is no relationship between political preference and opinion concerning whether trades unions are partly responsible for unemployment. The alternative hypothesis is that there is a relationship between these two variables. While it is a little dicult to determine whether supporters of the two dierent parties have dierent views by examing Table 10.16, it appears that the PCs are more concentrated in the 5 through 7 categories of opinion than are the Liberals. Based on this, there does appear to be a relationship between opinion and political

Chi-Square Tests preference, but this must be tested. The null and alternative hypotheses are H0 : No relation between opinion and political preference. H1 : Some relation between opinion and political preference. There are The statistic used to test these hypotheses are 2 =
i

746

(Oi Ei )2 . Ei

The observed values are in Table 10.16 and both the observed and the expected values are given in Table 10.17. The rst step in computing these is to obtain each of the row and column totals. For the rst row, there are 9 PCs and 3 Liberals for a total of 12. The rst column has 214 PCs, and the second column has 53 Liberals. Using these row and column totals, the expected numbers of cases for each category are computed. For the rst row, the expected numbers are obtained by multiplying the rst row total by the column total for the rst column and dividing by the grand total. For the rst row and rst column let the expected number of cases be E11 . E11 = 2568 12 214 = = 9.62 267 267

For the rst row and the second column, E12 = 12 53 636 = = 2.38 267 267

This could be obtained by subtracting E11 from the row total of 12, so that E12 = 12 9.62 = 2.38. For each of the next rows, the procedure is the same. For example, for the fourth row and the second column, E42 = 1643 31 53 = = 6.15 267 267

Once all the expected numbers of cases have been obtained, the next step is to compute the chi square value from all the observed and expected numbers of cases. These values are given in Table 10.18. Each of these is obtained by subtracting the observed from the expected number of cases in

Chi-Square Tests

747

Opinion 1 2 3 4 5 6 7

Political Preference PC Liberal 9 9.62 7 9.62 7 14.43 28 24.85 51 50.49 54 48.89 58 56.10 214 3 2.38 5 2.38 11 3.57 3 6.15 12 12.51 7 12.11 12 13.90 53 12 12 18 31 63 61 70

Total

267

Table 10.17: Observed and Expected Values of Opinions of Edmonton PCs and Liberals

Table 10.17, squaring this dierence and dividing by the expected number of cases. The rst entry, for example, is (9 9.62)2 0.622 = = 0.040. 9.62 9.62 Each of the entries in Table 10.18 is computed in the same manner, with the entries in this table set up to correspond to the entries in the earlier two tables. The sum of the chi square values in Table 10.18 is 2 = 28.105. There are 7 rows and 2 columns, so that there are (7 1) (2 1) = 6 1 = 6 degrees of freedom. At the = 0.01 level of signicance, the critical chi square value is 2 = 16.812, and the region of rejection of the null hypothesis is all chi square values of 16.812 or more. Since the chi square value from the data is

Chi-Square Tests

748

0.040 0.713 3.823 0.400 0.005 0.534 0.064

0.160 2.877 15.438 1.616 0.020 2.155 0.258

Table 10.18: Chi square Calculations

in the critical region, the hypothesis of no relationship between attitude and political preference can be rejected. At the 0.01 level of signicance, there is evidence of a relationship between political preference and opinion. That is, assuming that this is a random sample of Edmonton adults, and given the data here, the probability that there is no relationship between political preference and opinion is less than 0.01. Additional Comments. 1. One problem with the chi square test for this sample is that some of the cells have too few cases. The top three categories for the Liberals all have less than 5 expected cases. None of these is less than one, but 3 of the 34 cells, just over 20% of the cells, have less than 5 expected cases. Strictly speaking, this violates the rules for use of the chi square. However, the result is very signicant statistically, with the null hypothesis being rejected at the 0.01 level of signicance. The small violation of the rules in this circumstance would not aect this result much. If the assumptions are not to be violated, then the rst two groups could be grouped together. That is, if all those of opinions 1 and 2 are grouped together, there will be only 6 rows for the table, with 16 PCs observed having opinions 1 or 2, and 8 Liberals with opinions 1 or 2. If the expected cases are then recalculated, the result would be to have 12 cells in the table, and only 2 of these would have less than 5 expected cases. This would then satisfy the assumptions, with less than 20% of the cells having fewer than 5 expected cases. There would be only 5 degrees of freedom in this case, so that with = 0.01, the critical value is 2 = 15.086 from the chi square table in Appendix ??. If the 2 statistic is recalculated, it is 27.057, greater than the critical value. Thus the conclusion does not change when the rst

Chi-Square Tests

749

two categories are grouped together. At the 0.01 level of signcance, there is still strong evidence that there is a relationship between political preference and opinion for all Edmonton adults. 2. The chi square test shows that there is a relationship between the two variables. One weakness of the test though, is that the test itself says nothing concerning the nature of the relationship, other than that the relationship exists. With the chi square test for independence, there is no one directional test. However, by returning to the table, you are often able to determine the nature of the relationship. One way to do this is to compare the observed and expected cases, and examine the manner in which they dier. In Table 10.17, rst look at the dierences between observed and expected for the PCs in the rst data column. For each of the rst 3 rows of the table, there are fewer observed than expected PCs. In contrast, for the last 4 rows, and especially for rows 5-7, there are considerably more PCs observed than expected. This means that at the low values of opinion (the disagree) end of the scale, there are fewer PCs than what would be expected if there is no relationship between the two variables. In contrast, at the agree to strongly agree end of the scale (5-7), there are many more PCs than expected. This means that PCs are in agreement more than what would be expected if there was no relationship between opinion and political preference. Now examine the Liberal column in Table 10.17. There the opposite pattern prevails, with there being considerably more Liberals than expected at the disagree end of the scale. In the rst three rows, there are 2.38, 2.38 and 3.57 expected cases, but there are 3, 5 and 11 observed cases, respectively. At the disagree end of the scale, there are less observed than expected cases among the Liberals. Together these two patterns show the nature of the relationship. Liberals are more likely to disagree than expected, and Conservatives more likely to agree than expected when there is no relationship between the two variables. This means that PCs are inclined to agree or even strongly agree than trade unions have sometimes priced their members out of a job. Among Liberal party supporters, this view is much less commonly held. Some of the other measures of association examined later in this textbook are able to test for the direction of a relationship. But the chi square test by itself tests only for the existence or nonexistence of a relationship. 3. Comparing the results of the earlier tests with this test shows that the conclusions are much the same. The test for a dierence of two means, for the smaller sample in Chapter 9, was technically illegitimate because the

Chi-Square Tests

750

opinion scale was only an ordinal level scale. But the result from that test was very similar to this, showing that the PCs were considerably more in agreement on this issue, than were the Liberals. The results was statistically signicant there, as it is here. While tests for the dierence between two means in the case of an ordinal scale do not satisfy all the assumptions, they are commonly used. One rationale for using them is that they often give much the same results as does the more legitimate chi square test. The test for the dierence of two means also can be used as a one directional test, and it even gives some idea of the size of the dierence between the two means. In that sense, it may be a preferable test to conduct, even though the assumptions for conducting it are not really satised. One way to look on the chi square test for independence is to regard it as a rst test to be conducted. If you know little about the relationship between two variables, then conduct a chi square test for independence to see whether there is any relationship or not. No assumptions are require for this test, other than that the expected number of cases i each cell of the table be 5 or more, or close to this. If there is no relationship between the two variables, based on the chi square test, then other tests are unlikely to demonstrate any relationships either. If there is a relationship between the two variables, using the chi square test, then further investigation of the nature of the relationship is called for. The researcher may then spend some time closely examining the pattern of dierences between observed and expected cases in the table. These may allow the researcher to determine the nature the relationships that exist between the variables. On this basis, other statistical test, such as the test for dierence of means or proportions may be conducted.

10.4.4

Reporting Chi Square Tests for Independence

When reporting the results of chi square tests of independence, the table of expected values and the value of the chi square statistic are usually given. The expected values are not ordinarily reported, and these can easily be computed by the reader. In addition to the chi square statistic obtained from the table, it is common to report the exact signicance level, or the probability, of for the statistic. The degrees of freedom may also be reported, although this is not really necessary, since the reader can easily compute the degrees of freedom. Using these values, the nature of the relationship can be determined. The following example shows how results might be reported

Chi-Square Tests and interpreted.

751

Example 10.4.3 Attitudes to Social Spending in Newfoundland A sample of adults in Eastern and Central Newfoundland was conducted early in 1988 to examine public attitudes toward government cuts in social spending. Some of the results from this study are described in Morris Saldov, Public Attitudes to Social Spending in Newfoundland, Canadian Review of Social Policy, 26, November 1990, pages 10-14. The data in Table 10.19 comes from Table 2 of this article. Concerning this data, the author comments, Respondents who knew someone on social assistance, were more likely to feel that welfare rates were too low, ... .

Welfare Spending Too Little About Right Too Much Column Totals

Knows Someone on Social Assistance Yes No 40 16 9 65 6 13 7 26

Row Totals 46 29 16 91

2 = 11 df = 2 p < .005 Table 10.19: Knowing Someone on Social Assistance by Adequacy of Welfare Rates Implied in the test and the comments are the null and research hypothesis H0 : No relation between attitude and knowing someone on assistance H1 : Some relation between attitude and knowing someone on assistance If the null hypothesis of no relationship between attitudes and whether the respondent knows someone who receives social assistance is assumed, then the expected values and the chi square statistic can be determined in the

Chi-Square Tests

752

same manner as the previous examples. As an exercise, you should be able to conrm that the 2 statistic for this table is 11.00, to two decimal places. There are 3 rows and 2 columns so there are (3 1) (2 1) = 2 1 = 2 degrees of freedom for this table. At = 0.005 and 2 degrees of freedom, the critical value from the chi square table in Appendix ?? is 10.597. Since 2 = 11.00 > 10.597, the null hypothesis of no relationship between the two variables can be rejected. For the 0.001 level of signicance, 2 0.001,2 = 13.816. The chi square statistic does not exceed this value, so the null hypothesis cannot be rejected at the 0.001 level of signicance. Thus the signicance level is reported as being less than 0.005. From the chi square test, it is known that there is a relationship between the variables. By examining the table, the nature of the relationship can be determined. Of those respondents who know someone receiving social assistance, 40 out of 65 or (40/65) 100% = 61.5% said that there is too little welfare spending. In contrast, of those who did not know someone who received social assistance, only (6/26)% = 23.1% considered welfare spending inadequate. Some of the other cells can be examined in a similar manner. The result is that larger percentages of those who know someone on social assistance support more welfare spending than for those who do not know anyone receiving such assistance. Saldovs result shows that whether or not an individual knows someone on social assistance appears to be associated with that individuals views concerning welfare spending.

10.4.5

Test of Independence from an SPSS Program

When survey data has been entered into a computer program, it is relatively easy to obtain cross classication tables and chi square tests of independence from these tables. This section gives several examples of such tables, along with some guidelines concerning how such tables can be organized and interpreted. Since the SPSS program calculates the expected values, the 2 statistic, and the exact signicance level of the statistic, you need not carry out any calculations by hand. Instead, with these tests you can spend more time organizing and interpreting the results. Rather than providing a general description of methods to be used, some of the guidelines concerning analysis of cross classications and the chi square test of independence are introduced in the examples which follow. The SPSS program commands for obtaining the tables in this section are not given here. For those, consult an SPSS manual.

Chi-Square Tests

753

Example 10.4.4 Relationship Between Age and Attitudes toward Welfare Assistance In the Regina Labour Force Survey, respondents were asked their views concerning whether it is too easy to get welfare assistance. Table 10.20 gives a cross classication of respondents age by respondents opinion concerning welfare assistance. Notice that a four point attitude scale has been used with the categories being strongly agree to strongly disagree, but with no neutral category. The question asked was Do you strongly agree, somewhat agree, somewhat disagree, or stronlgy disagree that its too easy to get welfare assistance? In this table, age has been grouped into the categories less than 30, 30-44, 45-59, and 60 and over. Other age groupings could have been used, and this just happens to be the particular age grouping chosen for this table. Each cell in Table 10.20 has two entries. The rst entry in each cell is the observed number of cases for each combination of age and attitude. The second entry in each cell is the expected number of cases under the null hypothesis that there is no relationship between age and attitude. For example, the top left cell shows that there were 84 respondents less than age 30 who strongly agreed that it is too easy to get welfare assistance. If there had been no relationship between age and attitude, 78.9 respondents would have been expected to have this combination of characteristics. The row totals are given on the right, the column totals at the bottom, and the grand total of 753 respondents on the lower right. There were 232 respondents in the Survey under age 30, and this constitutes (232/753) 100% = 30.8% of all respondents. For the Strongly Disagree category, there were 85 respondents in total, or (85/753) 100% = 11.3% of all respondents. For the test of independence, the assumptions are the usual hypotheses, H0 that there is no relationship, and H1 that there is some relationship between age and attitude. The chi square statistic is given at the bottom of the table, to the right of Pearson. The chi square test of this section was originally developed early in the 1900s by the statistician Karl Pearson, and this statistic is sometimes named after him. For the cross classication of Table 10.20, 2 = 17.463 has been computed by the SPSS program, using the number of observed and expected cases, and the chi square formula. As an exercise, you could use these values to check that this is the proper value of the chi square statistic for this table. For this table, there are

Chi-Square Tests

754

AGE WELF

RESPONDENTS AGE by TOO EASY TO GET WELFARE ASSISTANCE?

WELF Count | Exp Val |STRONGLY SOMEWHAT SOMEWHAT STRONGLY | AGREE AGREE DISAGRE DISAGRE | 1 | 2 | 3 | 4 | AGE +--------+--------+--------+--------+ 1 | 84 | 76 | 52 | 20 | < 30 | 78.9 | 72.4 | 54.5 | 26.2 | +--------+--------+--------+--------+ 2 | 88 | 82 | 80 | 34 | 30-44 | 96.6 | 88.6 | 66.8 | 32.1 | +--------+--------+--------+--------+ 3 | 39 | 44 | 33 | 15 | 45-59 | 44.5 | 40.9 | 30.8 | 14.8 | +--------+--------+--------+--------+ 4 | 45 | 33 | 12 | 16 | 60+ | 36.0 | 33.1 | 24.9 | 12.0 | +--------+--------+--------+--------+ Column 256 235 177 85 Total 34.0% 31.2% 23.5% 11.3% Chi-Square ---------Pearson Value ----------17.46299 DF ---9 Significance -----------.04194

Row Total 232 30.8% 284 37.7% 131 17.4% 106 14.1% 753 100.0%

Table 10.20: Relation between Age and Attitude toward Welfare Assistance from SPSS Program

Chi-Square Tests

755

4 rows and 4 columns so that there are (4 1) (4 1) = 9 degrees of freedom. The chi square table in Appendix ?? for 0.05 signicance and 9 degrees of freedom gives a critical 2 value of 16.919. At 0.05 signicance, the region of rejection of H0 is all chi square values of 16.919 or more. Since 17.463 > 16.919, the chi square value from the table is in the region of rejection of the null hypothesis, and the null hypothesis of no relationship between age and attitude can be rejected. At the 0.05 level of signicance, there is evidence of a relationship between age and attitude. With the SPSS program, the exact level of signicance for the value of the chi square statistic is also given. This provides an alternative way of conducting the test, without having to use the chi sqaure table in Appendix ??. At the bottom right of the table, the exact signicance is reported to be 0.042. This means that there is exactly 0.042 of the area under the chi square curve that lies to the right of 2 = 17.463. This means that if the null hypothesis is true, the chance that the chi square value is as large as 17.463, or larger, is 0.042. Since this is a fairly small probability, the null hypothesis can be rejected. That is, at any level of signicance as low as 0.042, the null hypothesis can be rejected. At any lower level of signcance than 0.042, the null hypothesis cannot be rejected. The reader can decide whether 0.042 is low enough to allow for rejection of the hypothesis of no relationship, and acceptence of the hypothesis of some relationship. Given the conclusion that age and attitude are related to each other, the nature of the relationship can be examined in Table 10.20. The pattern of the relationship appears to be that the youngest and the oldest age groups both tend to agree that it is too easy to get welfare assistance. The age groups which have larger percentages of respondents who disagree are the 30-44 and 45-59 age groups. This can be seen by comparing the observed and expected numbers of cases in each of the cells as follows. Notice rst that for the less than 30 age group, there are more observed than expected cases in each of the strongly agree and somewhat agree categories. That is, there are more of the youngest age group who agree, than what would be expected if there was no relationship between age and attitude. For the middle two age groups, this pattern is reversed, with there being fewer observed than expected cases. For these two age groups, 3044 and 45-59, there are more observed than expected cases among the two disagree categories. Finally, for the oldest age group, there are considerably more observed than expected cases in the two agree categories. This means that older respondents were more likely to agree that it is too easy to get welfare assistance, than what would have been expected under the

Chi-Square Tests

756

assumption of no relationship between age and attitude. In summary, the pattern that appears in this table is that the middle aged groups express more disagreement, with the youngest and oldest age groups expressing more agreement. This pattern could be a consequence of the particular sample chosen, or the particular age grouping adopted here. But given these, this does appear to the pattern that exists in this table. Example 10.4.5 Dierences in Education and Attitudes Toward Welfare Spending This example shows the relationship between the education of the respondent and attitudes toward welfare spending. The attitude question used in this example is the same question as in Example 10.4.4, and this variable was cross classied by education of respondent. The complete table is given in Table 10.21. This table is read in the same manner as Table 10.20. The rst number in each cell is the observed number of respondents with each combination of characteristics. The second number is the expected number of respondents, assuming independence between education and attitude. Row, column and grand totals are also given. At the bottom of the table, it can be seen that 2 = 35.107, but with 27 degrees of freedom, this result is not very signicant statistically. The exact signicance of 0.13618 means that there is 0.13618 of the area under the 2 curve to the right of the chi square value of 35.107. Alternatively stated, if the null hypothesis of no relationship between education and attitude is true, the probability of obtaining a chi square value of 35.107 or more is 0.13618. This is not a low probability, so that the null hypothesis of no relationship between education and attitude may be correct. This null hypothesis of no such relationship cannot rejected at any signicance level below 0.136. In stating this conclusion, there are two problems, both caused by the size of the table. First, there is one cell with as few as 1.7 expected cases. This is rather low, being quite close to 1 case in that cell, and well below 5 expected cases per cell. In addition, over 20% of the cells have less than 5 expected cases per cell. The assumptions concerning the number of expected cases are not met in this table. The second problem with so many cells in the table is that it is dicult to determine any patterns of relationship which may exist in the data. Perhaps there is little relationship between the variables, but even if there is, it would be dicult to discern what the nature of this relationship would be. In order to make the table a little easier to analyze, the categories of education were

Chi-Square Tests

757

Table 10.21: Relationship between Education and Attitude

Chi-Square Tests

758

regrouped, so that there were only 4 categories. This was done by grouping adjacent categories so the new categories are (11) less than high school, (12) completed high school, (13) some postsecondary education, but no degree, and (14) having a degree. Table 10.22 contains the SPSS table for the cross classication of education by attitude, where these new education categories are used. Again the table is read in the same manner as earlier, but now all the assumptions are met and the table is considerably easier to examine. The relationships also become much clearer. If the null hypothesis is that there is no relationship between education and attitude, and the alternative hypothesis is that education and attitude are dependent variables, then the null hypothesis can be rejected at the 0.02 level of signicance. That is, the chi square statistic for this table is 19.791 and with 9 degrees of freedom, there is 0.01925 of the area under the chi square curve to the right of 2 = 19.791. This means that the null hypothesis of no relationship can be rejected at all levels of signicance down to 0.02. Since this is quite a low probability of Type I error, this is usually regarded as providing sucient evidence for a statistical relationship between two variables. The pattern of the relationship between education and attitude can also be determined from the table. For education level 13, some postsecondary education, the observed and expected numbers of cases are very close to each other. But for those who have a degree, there are considerably more observed than expected cases in each of the two disagree categories. The opposite pattern prevails for those with high school education or less. There, the observed numbers of cases generally exceeds the expected numbers of cases on the agree side. The pattern which emerges then is that respondents with degrees tend toward disagreement, and those with high school or less education tend toward agreement. Those with some postsecondary education, but not a degree fall in between these two groups on both variables. The pattern is a fairly clear one that as education level increases, there is a greater tendency for respondents to agree, and less of a tendency to disagree. By taking a little eort to organize the table in a manner which groups together respondents of similar levels of education, the relationship between education and attitude can be more clearly seen. In addition, this grouping solved the problem of the low numbers of expected cases in some of the cells of the cross classication table. Example 10.4.6 Political Preference and Attitude Concerning Indians and Metis

Chi-Square Tests

759

EDUC RESPONDENTS CURRENT EDUCATION by WELF TOO EASY TO GET WELFARE ASSISTANCE? WELF Count | Exp Val |STRONGLY SOMEWHAT SOMEWHAT STRONGLY | AGREE AGREE DISAGRE DISAGRE Row | 1 | 2 | 3 | 4 | Total EDUC --------+--------+--------+--------+--------+ 11.00 | 64 | 51 | 38 | 20 | 173 LESS THAN HIGH | 58.4 | 54.2 | 40.3 | 20.1 | 22.8% +--------+--------+--------+--------+ 12.00 | 69 | 49 | 30 | 15 | 163 HIGH | 55.0 | 51.1 | 38.0 | 18.9 | 21.5% +--------+--------+--------+--------+ 13.00 | 77 | 80 | 50 | 25 | 232 HIGH PLUS | 78.3 | 72.7 | 54.1 | 26.9 | 30.6% +--------+--------+--------+--------+ 14.00 | 46 | 58 | 59 | 28 | 191 DEGREE | 64.4 | 59.9 | 44.5 | 22.1 | 25.2% +--------+--------+--------+--------+ Column 256 238 177 88 759 Total 33.7% 31.4% 23.3% 11.6% 100.0% Chi-Square ----------Pearson Value ---------19.79131 DF ---9 18.899 Significance -----------.01925

Minimum Expected Frequency -

Table 10.22: Relation between Education (4 levels) and Attitude

Chi-Square Tests

760

Another question asked of respondents in the Regina Labour Force Survey was Now, Id like to ask you some questions about Native Canadian Indians and Metis. Again, can you tell me if you strongly agree, agree somewhat, disagree somewhat or strongly disagree with these statements. First of all, Indians and Metis deserve to be better o than they are now ... In order to determine whether there was any relationship between political preference of respondents and this attitude question, the cross classication of Table 10.23 was obtained. Political preference in that table is restricted to those who expressed a preference for either the NDP or PCs at the provincial level in Saskatchewan. The table gives the observed, expected, row, column and total numbers of cases, along with the chi square statistic for the test of independence, the degrees of freedom and the exact signicance level. If a 0.05 level of signicance is adopted, the null hypothesis of no relationship between attitude and political preference can be rejected. This is because the exact signicance printed on the table is 0.04150 < 0.05 so that 2 = 8.229 is in the right 0.05 of the area under the chi square curve for 3 degrees of freedom. By examining the cells of the table, some of the patterns of the relationship can be seen, although the relationship is not as simple as might be imagined. There are considerably more NDP supporters (125) in the strongly agree category than expected (111.4) under the assumption of no relationship between attitude and political preference. For the PCs the pattern is reversed, with fewer observed than expected PCs in the strongly agree category. But in the agree somewhat category, the situation is again reversed, with more PCs than expected, but fewer NDPers. For the two disagree categories, the observed and expected numbers of cases are quite close for supporters of both parties. In summary, the dierence between PC and NDP supporters is that NDPers tend to more strongly agree than do PCers that Indians and Metis deserve to be better o. In order to test whether there is any real dierence in the propensity of PCs and NDPers to agree or disagree, the two agree categories are grouped together, and the two disagree categories are grouped together. The fairly strong relationship of Table 10.23 disappears when the attitude categories are only agree and disagree. That is, by grouping all the agrees together, and all the disagrees together, the strength of agreement and disagreement is ignored. The new table is Table 10.24 and it can be seen there that the

Chi-Square Tests

761

AIM:INDIANS/METIS DESERVE TO BE BETTER OFF PROVINCIAL POLITICAL PREFERENCE

by

PROVINCIAL POLITICAL PREFERENCE Count | Exp Val |NDP P.C. | Row | 2 | 3 | Total AIM --------+--------+--------+ 1 | 125 | 36 | 161 STRONGLY AGREE | 111.4 | 49.6 | 33.1% +--------+--------+ 2 | 114 | 64 | 178 AGREE SOMEWHAT | 123.2 | 54.8 | 36.6% +--------+--------+ 3 | 54 | 28 | 82 DISAGREE SOMEWHA | 56.7 | 25.3 | 16.8% +--------+--------+ 4 | 44 | 22 | 66 STRONGLY DISAGRE | 45.7 | 20.3 | 13.6% +--------+--------+ Column 337 150 487 Total 69.2% 30.8% 100.0%

Chi-Square ----------Pearson

Value ---------8.22937

DF ---3 20.329

Significance -----------.04150

Minimum Expected Frequency -

Table 10.23: Relation between Political Preference and Attitude

Chi-Square Tests

762

AIM:INDIANS/METIS DESERVE TO BE BETTER OFF PROVINCIAL POLITICAL PREFERENCE

by

PROVINCIAL POLITICAL PREFERENCE Count | Exp Val |NDP P.C. | Row | 2 | 3 | Total AIM --------+--------+--------+ 1 | 239 | 100 | 339 agree | 234.6 | 104.4 | 69.6% +--------+--------+ 2 | 98 | 50 | 148 disagree | 102.4 | 45.6 | 30.4% +--------+--------+ Column 337 150 487 Total 69.2% 30.8% 100.0%

Chi-Square ----------Pearson

Value ---------.88761

DF ---1 45.585

Significance -----------.34613

Minimum Expected Frequency -

Table 10.24: Relation between Political Preference and Attitude

Chi-Square Tests

763

chi square statistic of 0.888 is associated with an area of 0.34613 to the right of it. The probability of obtaining 2 = 0.888 under the assumption of no relationship between the two variables is 0.346. Since this is a large probability, the hypothesis of no relation between political preference and attitude is not rejected. Note that in the previous table, the NDP tended to strongly agree, and the PCs tended to agree somewhat. Once all the strongly agree and somewhat agree are grouped together as agree, and the same for the disagree side, the relationship between political preference and attitude disappears.

10.4.6

Summary

The chi square test for independence is an extremely exible and useful test. The test can be used to examine the relationship between any two variables, with any types of measurement - nominal, ordinal, interval or ratio, and discrete or continuous. The only constraint on the use of the test is that there be sucient numbers of cases in each cell of the table. The rule given on page 742 that there be no cell with less than one expected case, and no more than one fth of the cells with less than 5 expected cases, is generally an adequate rule. If there are too few cases in some of the cells, then adjacent categories can usually be grouped together so that this rule can be satised. In doing this, the researcher must be careful concerning how the grouping is carried out. Examples 10.4.5 and 10.4.6 demonstrate how dierent conclusions can be obtained from the same data. In each case, dierence groupins of the data led to dierent conclusions. In the education example, a seemingly nonexistent relationship is made to appear as a relationship. In the case of the last example, there is some relationship between strength of attitude and political preference, but there is no relationship between overall agreement and disagreement and political preference. While chi square tests of independence are very exible, and can be used with any cross classication, a researcher must be careful not to either overemphasize or hide a relationship between two variables. In doing this, the chi square test itself is not the problem. The methodological problem is the diculty of deciding the proper approach to the grouping of the data. There are no strict guidelines concerning how data is to be grouped properly. In many cases, the researcher may try several groupings, and observe what happens as these groupings change. If the results change little as the groupings change, then the relationship is likely to be quite apparent. Where the

Chi-Square Tests

764

relationship seems to change as the grouping of the data changes, considerably more eort may have to be made to discern the exact nature of the relationship between the variables. If a dierent grouping of the data provides too few expected cases, there may be little that the researcher can do without obtaining more data. One test that can be used for very small sample size is Fishers Exact Test. This is not discussed in this textbook, but can be found in many statistics textbooks. This test is also provided in the SPSS program when the sample size in the cross classication table are small. In addition, some statistical programs provide corrections for continuity in the case of small sample sizes. Chi square tests for independence are most commonly used in the course of conducting research. If a researcher has a large number of variables, some of which are measured at less than the interval level, then cross classication tables and chi square tests may be conducted for many pairs of variables. Once a chi square test for independence of two variables has been conducted, this may only be the beginning of the analysis of the relationship between these variables. If there is no relationship betweeen the two variables shown by the test, the researcher must make sure that this lack of relationship really does mean no statistical relation. If the chi square test for independence provides some evidence that there is a relationship between the variables, then the researcher may want to nd some way to describe the direction, pattern and strength of this relationship. There are many possible ways of doing this, and Chapter 11 contains a number of measures such as the correlation coecient which can be used to describe a statistical relationship. When reporting research results, cross classication tables, along with the chi square statistic, and the level of signicance of the statistic may be reported. In addition, the researcher may provide some measure of association to summarize the relationship. But when examining any statistical relationship, there is no substitute for looking at the table itself, and examining how the two variables are related. Once the researcher understands the nature of the relationship between the variables, then the summary measures of the next chapter are more useful.

10.5

Conclusion

This chapter has presented two dierent types of chi square tests. The goodness of t test allows the researcher to hypothesize that a distribution

Chi-Square Tests

765

behaves in a particular manner. The chi square test is used to determine whether the observed distribution conforms to the model hypothesized by the researcher in the null hypothesis. In the test for independence, the chi square statistic and distribution are used to test whether there is a relationship between the two variables or not. For each of the two tests, the chi square statistic and distribution are the same. However, the two tests are quite dierent types of hypothesis tests, one examining only a single variable, and the other examining the relationship between two variables. This chapter completes the introduction to hypothesis testing. The general principles involved in testing hypotheses were introduced in Chapter 9. There the tests involved tests of parameters such as the mean or proportion, or the dierence in these between two populations. These are the most commonly used tests for these parameters. In this chapter, these principles of hypothesis testing were applied to whole distributions. The structure of the test is the same for each of the chi square tests, but the statistic used, and the fact that whole distributions rather than parameters are used, makes these tests appear somewhat dierent. In the remainder of this book, methods of describing relationships among variables are discussed. In doing this, tests of hypothesis are an essential part but again the tests may appear quite dierent than the tests introduced so far. In the tests which follow in Chapter 11, the principles of hypothesis testing are the same as the principles introduced in Chapter 9. However, the emphasis in Chapter 11 is placed on determining the nature of the relationship among variables.