Malhotra Mr05 PPT 18
Malhotra Mr05 PPT 18
Malhotra Mr05 PPT 18
18-1
© 2007 Prentice Hall
Chapter Outline
1) Overview
2) Basic Concept
3) Relation to Regression and ANOVA
4) Discriminant Analysis Model
5) Statistics Associated with Discriminant Analysis
6) Conducting Discriminant Analysis
i. Formulation
ii. Estimation
iii. Determination of Significance
iv. Interpretation
v. Validation
18-2
© 2007 Prentice Hall
Chapter Outline
7) Multiple Discriminant Analysis
i. Formulation
ii. Estimation
iv. Interpretation
v. Validation
18-3
© 2007 Prentice Hall
Chapter Outline
i. Estimation
v. An Illustrative Application
10) Summary
18-4
© 2007 Prentice Hall
Similarities and Differences between ANOVA,
Regression, and Discriminant Analysis
Table 18.1
Differences
Nature of the
dependent Metric Metric Categorical
variables
Nature of the
independent Categorical Metric Metric
variables
18-5
© 2007 Prentice Hall
Discriminant Analysis
Discriminant analysis is a technique for analyzing data
when the criterion or dependent variable is categorical and
the predictor or independent variables are interval in nature.
The objectives of discriminant analysis are as follows:
Development of discriminant functions, or linear
combinations of the predictor or independent variables,
which will best discriminate between the categories of the
criterion or dependent variable (groups).
Examination of whether significant differences exist among
the groups, in terms of the predictor variables.
Determination of which predictor variables contribute to
most of the intergroup differences.
Classification of cases to one of the groups based on the
values of the predictor variables.
Evaluation of the accuracy of classification.
18-6
© 2007 Prentice Hall
Discriminant Analysis
When the criterion variable has two categories, the
technique is known as two-group discriminant analysis.
When three or more categories are involved, the technique
is referred to as multiple discriminant analysis.
The main distinction is that, in the two-group case, it is
possible to derive only one discriminant function. In
multiple discriminant analysis, more than one function may
be computed. In general, with G groups and k predictors,
it is possible to estimate up to the smaller of G - 1, or k,
discriminant functions.
The first function has the highest ratio of between-groups
to within-groups sum of squares. The second function,
uncorrelated with the first, has the second highest ratio,
and so on. However, not all the functions may be
statistically significant. 18-7
© 2007 Prentice Hall
Geometric Interpretation
Fig. 18.1
X2 G1
1 1 2 2
G2
1 1 11 2
1 1 1 1 2
1
2 2
2 22
1 2
21
1 22
22
G1
G2 X1
18-8
© 2007 Prentice Hall
Discriminant Analysis Model
The discriminant analysis model involves linear combinations of
the following form:
D = b0 + b1X1 + b2X2 + b3X3 + . . . + bkXk
Where:
D = discriminant score
b 's = discriminant coefficient or weight
X 's = predictor or independent variable
The coefficients, or weights (b), are estimated so that the groups differ
as much as possible on the values of the discriminant function.
This occurs when the ratio of between-group sum of squares to within-
group sum of squares for the discriminant scores is at a maximum.
18-9
© 2007 Prentice Hall
Statistics Associated with
Discriminant Analysis
Canonical correlation. Canonical correlation measures
the extent of association between the discriminant scores
and the groups. It is a measure of association between
the single discriminant function and the set of dummy
variables that define the group membership.
Centroid. The centroid is the mean values for the
discriminant scores for a particular group. There are as
many centroids as there are groups, as there is one for
each group. The means for a group on all the functions
are the group centroids.
Classification matrix. Sometimes also called confusion
or prediction matrix, the classification matrix contains the
number of correctly classified and misclassified cases.
18-10
© 2007 Prentice Hall
Statistics Associated with
Discriminant Analysis
Discriminant function coefficients. The discriminant
function coefficients (unstandardized) are the multipliers
of variables, when the variables are in the original units
of measurement.
Discriminant scores. The unstandardized coefficients
are multiplied by the values of the variables. These
products are summed and added to the constant term to
obtain the discriminant scores.
Eigenvalue. For each discriminant function, the
Eigenvalue is the ratio of between-group to within-group
sums of squares. Large Eigenvalues imply superior
functions.
18-11
© 2007 Prentice Hall
Statistics Associated with
Discriminant Analysis
F values and their significance. These are calculated
from a one-way ANOVA, with the grouping variable
serving as the categorical independent variable. Each
predictor, in turn, serves as the metric dependent variable
in the ANOVA.
Fig. 18.2
1 1 50.2 5 8 3 43 M (2)
2 1 70.3 6 7 4 61 H (3)
3 1 62.9 7 5 6 52 H (3)
4 1 48.5 7 5 5 36 L (1)
5 1 52.7 6 6 4 55 H (3)
6 1 75.0 8 7 5 68 H (3)
7 1 46.2 5 3 3 62 M (2)
8 1 57.0 2 4 6 51 M (2)
9 1 64.1 7 5 4 57 H (3)
10 1 68.1 7 6 5 45 H (3)
11 1 73.4 6 7 5 44 H (3)
12 1 71.9 5 8 4 64 H (3)
13 1 56.2 1 8 6 54 M (2)
14 1 49.3 4 2 3 56 H (3)
15 1 62.0 5 6 2 58 H (3)
18-16
© 2007 Prentice Hall
Information on Resort Visits: Analysis
Sample
Table 18.2, cont. Annual Attitude Importance Household Age of Amount
Resort Family Toward Attached Size Head of Spent on
No. Visit Income Travel to Family Household Family
($000) Vacation Vacation
16 2 32.1 5 4 3 58 L (1)
17 2 36.2 4 3 2 55 L (1)
18 2 43.2 2 5 2 57 M (2)
19 2 50.4 5 2 4 37 M (2)
20 2 44.1 6 6 3 42 M (2)
21 2 38.3 6 6 2 45 L (1)
22 2 55.0 1 2 2 57 M (2)
23 2 46.1 3 5 3 51 L (1)
24 2 35.0 6 4 5 64 L (1)
25 2 37.3 2 7 4 54 L (1)
26 2 41.8 5 1 3 56 M (2)
27 2 57.0 8 3 2 36 M (2)
28 2 33.4 6 8 2 50 L (1)
29 2 37.5 3 2 3 48 L (1)
30 2 41.3 3 3 2 42 L (1)
18-17
© 2007 Prentice Hall
Information on Resort Visits:
Holdout Sample
Table 18.3
Annual Attitude Importance Household Age of Amount
Resort Family Toward Attached Size Head of Spent on
No. Visit Income Travel to Family Household Family
($000) Vacation Vacation
1 1 50.8 4 7 3 45 M(2)
2 1 63.6 7 4 7 55 H (3)
3 1 54.0 6 7 4 58 M(2)
4 1 45.0 5 4 3 60 M(2)
5 1 68.0 6 6 6 46 H (3)
6 1 62.1 5 6 3 56 H (3)
7 2 35.0 4 3 4 54 L (1)
8 2 49.6 5 3 5 39 L (1)
9 2 39.4 6 5 3 44 H (3)
10 2 37.0 2 6 5 51 L (1)
11 2 54.5 7 3 3 37 M(2)
12 2 38.2 2 2 3 49 L (1)
18-18
© 2007 Prentice Hall
Conducting Discriminant Analysis
Estimate the Discriminant Function
Coefficients
18-19
© 2007 Prentice Hall
Results of Two-Group Discriminant Analysis
Table 18.4
GROUP MEANS
VISIT INCOME TRAVEL VACATION HSIZE AGE
INCOME 1.00000
TRAVEL 0.19745 1.00000
VACATION 0.09148 0.08434 1.00000
HSIZE 0.08887 -0.01681 0.07046 1.00000
AGE - 0.01431 -0.19709 0.01742 -0.04301 1.00000
INCOME 0.74301
TRAVEL 0.09611
VACATION 0.23329
HSIZE 0.46911
AGE 0.20922
Structure Matrix:
Pooled within-groups correlations between discriminating variables & canonical discriminant functions
(variables ordered by size of correlation within function)
FUNC 1
INCOME 0.82202
HSIZE 0.54096
VACATION 0.34607
TRAVEL 0.21337
AGE 0.16354 Cont.
18-21
© 2007 Prentice Hall
Results of Two-Group Discriminant Analysis
Table 18.4, cont.
Group FUNC 1
1 1.29118
2 -1.29118
Classification results for cases selected for use in analysis
Predicted Group Membership
Actual Group No. of Cases 1 2
Group 1 15 12 3
80.0% 20.0%
Group 2 15 0 15
0.0% 100.0%
Percent of grouped cases correctly classified: 90.00%
Cont.
18-22
© 2007 Prentice Hall
Results of Two-Group Discriminant Analysis
Table 18.4, cont.
18-23
© 2007 Prentice Hall
Conducting Discriminant Analysis
Determine the Significance of
Discriminant Function
The null hypothesis that, in the population, the means of all
discriminant functions in all groups are equal can be
statistically tested.
In SPSS this test is based on Wilks' l . If several functions
are tested simultaneously (as in the case of multiple
discriminant analysis), the Wilks' l statistic is the product of
the univariate for each function. The significance level is
estimated based on a chi-square transformation of the
statistic.
If the null hypothesis is rejected, indicating significant
discrimination, one can proceed to interpret the results.
18-24
© 2007 Prentice Hall
Conducting Discriminant Analysis
Interpret the Results
The interpretation of the discriminant weights, or coefficients, is similar to
that in multiple regression analysis.
Given the multicollinearity in the predictor variables, there is no
unambiguous measure of the relative importance of the predictors in
discriminating between the groups.
With this caveat in mind, we can obtain some idea of the relative
importance of the variables by examining the absolute magnitude of the
standardized discriminant function coefficients.
Some idea of the relative importance of the predictors can also be obtained
by examining the structure correlations, also called canonical loadings or
discriminant loadings. These simple correlations between each predictor
and the discriminant function represent the variance that the predictor
shares with the function.
Another aid to interpreting discriminant analysis results is to develop a
Characteristic profile for each group by describing each group in terms
of the group means for the predictor variables.
18-25
© 2007 Prentice Hall
Conducting Discriminant Analysis
Assess Validity of Discriminant Analysis
Many computer programs, such as SPSS, offer a leave-one-out
cross-validation option.
The discriminant weights, estimated by using the analysis
sample, are multiplied by the values of the predictor variables in
the holdout sample to generate discriminant scores for the
cases in the holdout sample. The cases are then assigned to
groups based on their discriminant scores and an appropriate
decision rule. The hit ratio, or the percentage of cases
correctly classified, can then be determined by summing the
diagonal elements and dividing by the total number of cases.
It is helpful to compare the percentage of cases correctly
classified by discriminant analysis to the percentage that would
be obtained by chance. Classification accuracy achieved by
discriminant analysis should be at least 25% greater than that
obtained by chance.
18-26
© 2007 Prentice Hall
Results of Three-Group Discriminant Analysis
Table 18.5
Group Means
AMOUNT INCOME TRAVEL VACATION HSIZE AGE
1 38.57000 4.50000 4.70000 3.10000 50.30000
2 50.11000 4.00000 4.20000 3.40000 49.50000
3 64.97000 6.10000 5.90000 4.20000 56.00000
Total 51.21667 4.86667 4.93333 3.56667 51.93333
18-28
© 2007 Prentice Hall
Results of Three-Group Discriminant
Analysis
Table 18.5, cont.
Structure Matrix:
Pooled within-groups correlations between discriminating variables and canonical
discriminant functions (variables ordered by size of correlation within function)
FUNC 1 FUNC 2
INCOME 0.85556* -0.27833
HSIZE 0.19319* 0.07749
VACATION 0.21935 0.58829*
TRAVEL 0.14899 0.45362*
AGE 0.16576 0.34079*
Group 1 10 9 1 0
90.0% 10.0% 0.0%
Group 2 10 1 9 0
10.0% 90.0% 0.0%
Group 3 10 0 2 8
0.0% 20.0% 80.0%
Percent of grouped cases correctly classified: 86.67%
Classification results for cases not selected for use in the analysis
Predicted Group Membership
Actual Group No. of Cases 1 2 3
Group 1 4 3 1 0
75.0% 25.0% 0.0%
Group 2 4 0 3 1
0.0% 75.0% 25.0%
Group 3 4 1 0 3
25.0% 0.0% 75.0%
Percent of grouped cases correctly classified: 75.00%
18-30
© 2007 Prentice Hall
All-Groups Scattergram
Fig. 18.3
Across: Function 1
Down: Function 2
4.0
1 1
1 *1 3
23 3 *3 3
1 1 12 * 3 3
0.0 1 1 2 2
3
1 2 2
2
-4.0
* indicates a group
centroid
13
13
13 Across: Function 1
8.0 Down: Function 2
13
13 * Indicates a
13
group centroid
13
4.0 113
112 3
112233
*1 1 1 2 2 2 2 3 3 *
1 1 2 2* 223
0.0 1122 233
1122 2233
11122 223
233
-4.0 11222
1122 223
11122 233
11122 2233
1122 223
-8.0 11122 233
-8.0 -6.0 -4.0 -2.0 0.0 2.0 4.0 6.0 8.0 18-32
© 2007 Prentice Hall
Stepwise Discriminant Analysis
Stepwise discriminant analysis is analogous to stepwise
multiple regression (see Chapter 17) in that the predictors
are entered sequentially based on their ability to
discriminate between the groups.
An F ratio is calculated for each predictor by conducting a
univariate analysis of variance in which the groups are
treated as the categorical variable and the predictor as the
criterion variable.
The predictor with the highest F ratio is the first to be
selected for inclusion in the discriminant function, if it
meets certain significance and tolerance criteria.
A second predictor is added based on the highest adjusted
or partial F ratio, taking into account the predictor already
selected.
18-33
© 2007 Prentice Hall
Stepwise Discriminant Analysis
Each predictor selected is tested for retention based on its
association with other predictors selected.
The process of selection and retention is continued until all
predictors meeting the significance criteria for inclusion and
retention have been entered in the discriminant function.
The selection of the stepwise procedure is based on the
optimizing criterion adopted. The Mahalanobis
procedure is based on maximizing a generalized measure
of the distance between the two closest groups.
The order in which the variables were selected also
indicates their importance in discriminating between the
groups.
18-34
© 2007 Prentice Hall
The Logit Model
18-35
© 2007 Prentice Hall
Binary Logit Model Formulation
The probability of success may be modeled using the logit model as:
P
log = a +a X +a X +... +a X
1 - P
Or e 0 1 1 2 2 k k
P =
a X
n
log
1 - P
Or e i i
i= 0
18-36
© 2007 Prentice Hall
Model Formulation
exp( a X )
P =
i i
i =0
1 + exp(
k
a X i i )
i =0
Where:
P = Probability of success
Xi = Independent variable i
ai = parameter to be estimated.
18-37
© 2007 Prentice Hall
Properties of the Logit Model
18-38
© 2007 Prentice Hall
Estimation and Model Fit
Where,
nonmetric.
18-40
© 2007 Prentice Hall
Interpretation of Coefficients
18-41
© 2007 Prentice Hall
Explaining Brand Loyalty
Table 18.6
No. Loyalty Brand Product Shopping
1 1 4 3 5
2 1 6 4 4
3 1 5 2 4
4 1 7 5 5
5 1 6 3 4
6 1 3 4 5
7 1 5 5 5
8 1 5 4 2
9 1 7 5 4
10 1 7 6 4
11 1 6 7 2
12 1 5 6 4
13 1 7 3 3
14 1 5 1 4
15 1 7 5 5
16 0 3 1 3
17 0 4 6 2
18 0 2 5 2
19 0 5 2 4
20 0 4 1 3
21 0 3 3 4
22 0 3 4 5
23 0 3 6 3
24 0 4 4 2
25 0 6 3 6
26 0 3 6 3
27 0 4 3 2
28 0 3 5 2
29 0 5 5 3
30 0 1 3 2
18-42
© 2007 Prentice Hall
Results of Logistic Regression
Table 18.7
Model Summary
18-43
© 2007 Prentice Hall
Results of Logistic Regression
Table 18.7, cont.
Classification Table a
Predicted
Analyze>Classify>Discriminant …
18-45
© 2007 Prentice Hall
SPSS Windows: Two-group Discriminant
1. Select ANALYZE from the SPSS menu bar.
2. Click CLASSIFY and then DISCRIMINANT.
3. Move “visit” in to the GROUPING VARIABLE box.
4. Click DEFINE RANGE. Enter 1 for MINIMUM and 2 for MAXIMUM. Click
CONTINUE.
5. Move “income,” “travel,” “vacation,” “hsize,” and “age” in to the
INDEPENDENTS box.
6. Select ENTER INDEPENDENTS TOGETHER (default option)
7. Click on STATISTICS. In the pop-up window, in the DESCRIPTIVES box
check MEANS and UNIVARIATE ANOVAS. In the MATRICES box check
WITHIN-GROUP CORRELATIONS. Click CONTINUE.
8. Click CLASSIFY.... In the pop-up window in the PRIOR PROBABILITIES
box check ALL GROUPS EQUAL (default). In the DISPLAY box check
SUMMARY TABLE and LEAVE-ONE-OUT CLASSIFICATION. In the USE
COVARIANCE MATRIX box check WITHIN-GROUPS. Click CONTINUE.
9. Click OK.
18-46
© 2007 Prentice Hall
SPSS Windows: Logit Analysis
1. Select ANALYZE from the SPSS menu bar.
6. Click OK.
18-47
© 2007 Prentice Hall