6 Loglinear Models Beamer-Online PDF
6 Loglinear Models Beamer-Online PDF
6 Loglinear Models Beamer-Online PDF
Edps/Psych/Soc 589
Carolyn J. Anderson
c
Board of Trustees, University of Illinois
Fall 2018
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Outline
In this set of notes:
◮ Loglinear models for 2–way tables.
◮ Loglinear models for 3–way tables.
◮ Statistical inference & model checking.
◮ Statistical versus Practical Significance.
◮ Higher–way tables.
◮ The logit–log-linear model connection. (We’ll discuss further
connections when we cover multicategory logit models).
◮ Model building (graphical models).
◮ Modeling ordinal associations, including linear × linear
association models.
◮ Modeling approach to testing conditional independence.
◮ Sparse data, including
◮ Structural zeros
◮ Sampling zeros
◮ Effect on G 2 and X 2 .
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 2.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 4.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 5.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 6.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 7.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Statistical Independence
◮ The joint probabilities {πij } of observations falling into a cell equal
the product of the marginal probabilities,
πij = πi + π+j for all i = 1, . . . , I and j = 1, . . . , J
log(µij ) = λ + λX Y
i + λj
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 9.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 10.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Example
The observed and estimated fitted values
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 11.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
R:
summary(i.mod ← glm(count ∼ view + choice,
data=gss.data,family=poisson) )
(X2 ← sum(residuals(indep.mod,type=c("pearson"))**2))
i.mod$fitted
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 12.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
(168.78)(305.60)/(227.74)(226.49) = 1.00
(226.49)(339.66)/(305.60)(251.73) = 1.00
(168.78)(339.66)/(227.74)(251.73) = 1.00
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 13.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Independence (continued)
Fit statistics for the independence log-linear model:
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 14.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
E (counts) = µij
log(µij ) = λ + λX Y
i + λj
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 15.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 16.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 17.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Interpretation of Parameters
◮ When there are only 2 levels of the response, logit models are
preferable (fewer terms in the model). This is especially true
when we have 2 or more explanatory variables.
◮ Log-linear models are primarily used when modeling the
relationship among 2 or more categorical responses.
Odds ratios are functions of model parameters:
µ11 µ22
log(odds ratio) = log(θ(12,12) ) = log
µ12 µ21
= (λ + λX Y X Y
1 + λ1 ) + (λ + λ2 + λ2 )
−(λ + λX Y X Y
1 + λ2 ) − (λ + λ2 + λ1 )
= 0
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 19.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
(λ̂Y Y
1 − λ̂2 ) = unique value
(λ̂X X
1 − λ̂2 ) = unique value
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 20.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µ̂ij ) = α + λX Xi + λY Yj
For Dummy Coding (i.e., X1 = 0, X2 = 1 and Y1 = 0, Y2 = 1),
λ for (1, 1)
λ + λX
for (2, 1)
log(µ̂ij ) = Y
λ + +λ for (1, 2)
λ + λX + λY for (2, 2)
◮ λ, λX Y
i , and λj , are the overall and marginal effect terms (as
defined before).
◮ λXY
ij ’s
◮ Represent the association between X and Y .
◮ Reflect the departure or deviations from independence.
◮ Ensure that µij = nij
◮ Fits the data perfectly; the fitted values are exactly equal to
the observed values.
◮ Has as many unique parameters are there are cells in the table
(i.e., N = IJ), so df = 0.
◮ Called the “Saturated Model”.
◮ Is the most complex model possible for a 2–way table.
◮ Has independence as a special case (i.e., the model with
λXY
ij = 0 for all i and j).
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 22.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 24.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
We generally hope to find models that are simpler than the data
itself (simpler than the saturated model). Simpler models
“smooth” the sample data and provide more parsimonious
descriptions.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 25.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Hierarchical Models
We’ll restrict attention to hierarchical models.
◮ Hierarchical models include all lower-order terms that
comprise the the higher-order terms in the model.
◮ Is this a hierarchical model?
log(µij ) = λ + λX XY
i + λij
◮ Restrict attention to hierarchical models because
◮ We want interaction terms to represent just the association
(dependency).
◮ Without lower order terms, the statistical significance and
(substantive) interpretation of interaction terms would depend
on how variables were coded.
◮ With hierarchical models, coding doesn’t matter.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 26.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 27.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 28.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µijk ) = λ + λX Y Z XY XZ YZ XYZ
i + λj + λk + λij + λik + λjk + λijk
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 29.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Joint
Independence
(XY , Z ) (XZ , Y ) (X , YZ )
Complete Independence (X , Y , Z )
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 30.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
(Andersen, 1985)
Bad Management Good Management
Worker’s Worker’s
satisfaction satisfaction
Low High Low High
Supervisor’s Low 103 87 190 Low 59 109 168
satisfaction High 32 42 74 High 78 205 283
135 129 264 137 314 451
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 31.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 32.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Complete Independence
There are no interactions; everything is independent of everything
else.
log(µijk ) = λ + λX Y
i + λj + λk
Z
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 33.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µijk ) = λ + λX Y Z
i + λj + λk
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 34.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Joint Independence
df = IJK − 1 − (I − 1) − (J − 1) − (K − 1) − (I − 1)(J − 1)
= (IJ − 1)(K − 1)
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 35.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µijk ) = λ + λX Y Z XY
i + λj + λk + λij
The partial or conditional odds ratios for XZ given Y and the odds
ratios for YZ given X equal 1.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 36.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µijk ) = λ + λX Y Z XY
i + λj + λk + λij
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 37.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
XY λXY
e λij e i ′j′
θXY (k) = θii ′ ,jj ′ (k) = exp(λXY
ij + λXY
i ′j ′ − λXY
i ′j − λXY
ij ′ ) =
λXY
i′j
λXY
ij ′
e e
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 38.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Conditional Independence
Two variables are conditionally independent given the third
variable. e.g., the model in which Y and Z are conditionally
independent given X equals
log(µijk ) = λ + λX Y Z XY
i + λj + λk + λij + λik
XZ
df = IJK − 1 − (I − 1) − (J − 1) − (K − 1)
−(I − 1)(J − 1) − (I − 1)(K − 1)
= I (J − 1)(K − 1)
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 39.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
log(µijk ) = λ + λX Y Z XY XZ
i + λj + λk + λij + λik
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 40.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 41.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Homogeneous Association
or the “no 3–factor interaction model”.
This is a model of association; it is not an “independence” model,
but it is also not the most complex model possible.
log(µijk ) = λ + λX Y Z XY XZ YZ
i + λj + λk + λij + λik + λjk
df = IJK − 1 − (I − 1) − (J − 1) − (K − 1) − (I − 1)(J − 1)
−(I − 1)(K − 1) − (J − 1)(K − 1) = (I − 1)(J − 1)(K − 1)
The partial odds ratios are a direct function of the model parameters
Each of the partial odds ratios for 2 variables given levels of the third
variable
◮ depends only on the corresponding 2–way interaction terms.
◮ do not depend on levels of the third variable.
◮ are equal across levels of the third variable.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 43.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
df = 0
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 44.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
= (λXY XY XY XY
ij + λi ′ j ′ − λi ′ j − λij ′ )
+(λXYZ
ijk + λXYZ XYZ XYZ
i ′ j ′ k − λi ′ jk − λij ′ k )
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 45.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
3–way Association
A measure/definition of 3–way association is the ratio of partial odds
ratios (ratios of ratios of ratios),
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 46.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Joint
Independence
(XY , Z ) (XZ , Y ) (X , YZ )
Complete Independence (X , Y , Z )
Any model that lies below a given model may be a special case of the
more complex model(s).
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 47.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 48.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 49.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
2. Residuals.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 50.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
X X X (nijk − µ̂ijk )2
Pearson statistic: X 2 =
µ̂ijk
i j k
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 51.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Residuals
Local (miss)fit. A good model has small residuals.
We can use Pearson residuals
(observed − expected)
eijk = q
V̂ar(expected)
(nijk − µ̂ijk )
= p
µ̂ijk
or eijk
adjusted residual = p
(1 − hijk )
where hijk equals the leverage of cell (i, j, k).
If the model holds, then adjusted residuals ≈ N(0, 1)
Adjusted residuals suggest a lack of fit of the model
◮ When there are few cells (small N) and adjusted residuals > 2.
◮ When these are lots and lost of cells (larger N) and adjusted
residuals > 3.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 53.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
◮ df for the model (MS, MW ) equals 2 and therefore there are only 2
non-redundant residuals.
◮ df for the model (MS, MW , WS) equals 1 and therefore there is
only 1 non-redundant residual.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 54.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 55.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 56.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Model df G2 p HO ∆df ∆G 2 p
(MW , SW , MS) 1 .065 .80 — — — —
(MW , SW ) 2 71.90 < .001 λMS
ij =0 1 71.835 < .01
(MS, MW ) 2 5.39 .07 λSW
jk = 0 1 5.325 .02
(MS, WS) 2 19.71 < .001 λMW
ik =0 1 19.645 < .01
Sample size and hypothesis tests:
◮ With small samples, “reality may be much more complex than
indicated by the simplest model that passed the goodness-of-fit test.
◮ With large samples, “. . . statistically significant effects can be weak
and unimportant.”
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 57.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
(MSW )
df = 0
Summary G2 = 0
∆df = 1 ∆G 2 = .065
(MS, MW , SW )
df = 1
2
∆df = 1 G
=X.065
XX
∆G = 5.33 ∆G 2 = 19.645XXXX∆G 2 = 71.84
2
XXX
(MS, MW ) (MS, SW ) (MW , SW )
df = 2 df = 2 df = 2
G 2 =X5.39 G 2=X19.71 G 2= 71.90
XXX XXX
XX XX
XX X XX X
(MS, W ) (MW , S) (M, SW )
df = 3 df = 3 df = 3
G 2 =X35.6 G 2 = 87.79 G2 = 102.11
XX
XXX
XX
(M, S, W )
df = 4
G 2 = 118.0
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 58.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Estimate ASE
SW
λ̂low,low = .3847 .1667
λ̂SW
low,hi = .0000 .0000
λ̂SW
hi,low = .0000 .0000
SW
λ̂hi,hi = .0000 .0000
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 62.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
They seem similar. Whether they are “close” enough, that depends
on purpose or uses you’ll make of the results.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 63.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Dissimilarity Index
For a table of with
Any dimension (e.g., (I × J), (I × J × K ), etc).
Cell counts equal to ni = npi .
Fitted counts equal to µ̂i = nπ̂i .
The “Dissimilarity Index” is a summary statistic of how close the
fitted values of a model are to the data. It equals
P P
i |ni − µ̂i | |pi − π̂i |
D= = i
2n 2
Properties of D:
◮ 0 ≤ D ≤ 1.
◮ D = the proportion of sample cases that need to move to a
different cell to have the model fit perfectly.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 64.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 65.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
55.2306
D= = .039
2(715)
5.8888
D= = .004
2(715)
A large value indicates that the observed and fitted are “close”.
Worker satisfaction example:
For the model of conditional independence (MW , MS),
r = .9906
and for the model of homogeneous association
r = .9999
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 67.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Information Criteria
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 68.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 69.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
P(Mo |X )
B=
P(M1 |X )
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 70.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 71.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 72.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 73.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 75.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 77.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Since λ̂GP
women,yes = −1.3106, given EMS and marital status the
odds of PMS for women is
e −1.3106 = .2696
times the odds for men.
Alternatively, given EMS and marital status, the odds of PMS for
men is
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 78.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 79.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
(38.40)(327.49)
θ̂ME |PMS=no = = 10.62
(5.50)(204.32)
times the odds of divorce given the person did not have extramarital sex.
3–way EMP association: The partial odds ratio for marital status and
extramarital sex given the person did not have premarital sex are
10.62
= 6.03
1.76
times the partial odds ratio given the person did have premarital sex.
(We could have arrived at the same interpretation of the partial
associations by using the parameters of the log-linear model.)
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 81.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
λ̂ME
11 = 2.3960 (Divorced and had EMS),
all other λ̂ME
ij ’s equal zero.
λ̂MEP
111 = −1.7955 (Divorced, had EMS & had PMS),
all other λ̂ME
ijk ’s equal zero.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 82.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Of those who did not have PMS (k = 2 = no), the estimated odds ratio
for marital status and extramarital sex equals
exp(2.3626) = 10.62
exp(λ̂ME MEP
11 − λ̂111 ) = exp(2.3626 − 1.7955) = exp(.5671) = 1.76
q
se(λ̂ME MEP
11 − λ̂111 ) =
2 + σ2
σME MEP − 2σME ,MEP
p
= 0.14962 + 0.2622 − 2(−0.14962) = 0.84323
So. . .
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 84.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
and for the MEP partial association for those who had PMS
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 85.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 86.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 87.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 88.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
logit(πij ) = logit(πij )
P(Hi worker satisfaction|M = i , S = j)
= log
P(Lo worker satisfaction|M = i , S = j)
= log(µij2 /µij1 )
= log(µij2 ) − log(µij1 )
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 89.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
logit(πij ) = (λW W MW
2 − λ1 ) + (λi 2 − λMW SW SW
i 1 ) + (λj2 − λj1 )
= α + βiM + βjS
◮ α = (λW W
2 − λ1 ) a constant.
◮ βM = (λMW− λMW
i i2 i 1 ).
The relationship (effect) of management quality between (on)
worker job satisfaction is the same at each level of supervisor’s
job satisfaction.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 90.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
logit(πij ) = (λW W MW
2 − λ1 ) + (λi 2 − λMW SW SW
i 1 ) + (λj2 − λj1 )
= α + βiM + βjS
And. . .
βjS = (λSW SW
j2 − λj1 ).
The relationship (effect) of supervisor’s job satisfaction between
(on) worker job satisfaction is the same at each level of
management quality.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 91.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
+λEP GP MEP
jk + λkl + λijk
and the estimated parameters for the logit model using the ones from the
log-linear model. . . .
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 92.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 93.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 94.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Logit Model df G2 p
E,G,P 4 13.63 .001
GP,E 3 13.00 < .001
EG,P 3 10.75 .010
EP,G 3 .70 .873
EG,GP 2 10.33 < .001
EP,GP 2 .44 .803
EG,EP 2 .29 .865
EG,EP,GP 1 .15 .700
EGP 0 0.00 1.00
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 95.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
EP
logit(πijk ) = α + βiG + βjE + βkP + βjk ,
which is different from the logit model that we used to interpret
our log-linear model (GP, EMP), i.e.,
logit(πijk ) = α + βjE + βkP + βjk
EP
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 96.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
because. . . .
◮ When we consider the data as 8 independent Binomial
samples, the “row” margin corresponding to the total number
of observations for each Gender × EMS × PMS combination
is “ fixed.”
◮ When we fit a log-linear model to the data, we should always
include parameters to ensure that the GEP margin is fit
perfectly.
◮ If marital status is our response variable, we are not interested
in the relationship between/among Gender, EMS, and PMS,
except with respect to how they are related to marital status.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 97.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
EP
logit(πijk ) = α + βiG + βjE + βkP + βjk
is the (GEP, MEP, GM) log-linear model,
µijkl = λ + λG E P GE GP EP GEP
i + λj + λk + λij + λik + λjk + λijk
+λM GM
l + λil + λEM PM MEP
jl + λkl + λljk
λ, λG E P GE GP EP GEP
i , λj , λk , λij , λik , λjk , λijk
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 98.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
The log-linear model (GEP, MEP, GM) will have the exact same
df and fit statistics as the (EP, G ) logit model.
The estimated parameters of the logit model are equal to
differences of estimated log-linear model parameters.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 99.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Logit Loglinear
Model Model df G2 p
E,G,P EGP,ME,MG,MP 4 13.63 .001
GP,E EGP,MGP,ME 3 13.00 < .001
EG,P EGP,MEG,MP 3 10.75 .010
EP,G EGP,MEP,MG 3 .70 .873
EG,GP EGP,MEG,MGP 2 10.33 < .001
EP,GP EGP,MEP,MGP 2 .44 .803
EG,EP EGP,MEG,MEP 2 .29 .865
EG,EP,GP EGP,MEG,MEP,MGP 1 .15 .700
EGP EGPM 0 0.00 1.00
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 100.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 102.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 103.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 104.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 105.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
Backward Elimination
Stage Model G2 df Best Model
Initial (EMP,EGM,EGP,GMP) 0.15 1
1 (EMP,EGM,GMP) 0.19 2 ∗
(EMP,EGM,EGP) 0.29 2
(EMP.EGP,GMP) 0.44 2
(EGM,EGP,GMP) 10.33 2
2 (GP,EMP,EGM) 0.37 3 ∗
(EG,EMP,GMP) 0.46 3
(EP,EGM,GMP) 10.47 3
3 (EG.GM,GP,EMP) 0.76 4 ∗
(EP,GP,MP,EGM) 10.80 4
(EMP,EGM) 67.72 4
4 (GM,GP,EMP) 5.21 5 ∗
(EG,GP,EMP) 5.25 5
(EG,GM,GP,EM,EP,MP) 13.63 5
(EG,GM,EMP) 70.10 5
5 (GP,EMP) 8.15 6 ∗
(GM,GP,EM,EP,MP) 18.13 6
(GM,EMP) 83.38 6
6 (GP,EM,EP,MP) 21.07 7 ∗
(G,EMP) 83.41 7
LASSO
Least Absolute Shrinkage and Selection Operator.
A constrained regression that finds the βk s that solves:
!2
n p
1 X X
min yi − βk xki + tP(β)
βk ∈R 2n
i =1 k=0
where
◮ t is the “tuning” parameter.
◮ P(β) is the penalty p
X
P(β) = ||β||ℓ1 = |βk |
k=1
Ridge Regression
Finds the βk s that solves:
!2
n p
1 X X
min yi − βk xki + tP(β)
βk ∈R 2n
i =1 k=0
Elastic Net
Elastic net is a compromise between ridge regression and lasso.
It finds the βk s that solve:
n p
" #
1 X X
2
min (yi − βk xki ) + tPα (β)
βk ∈R 2n
i =1 k=0
where
◮ Pα (β) is the elastic-net penalty
p
1 2
X 1 2
Pα (β) = (1 − α) ||β||ℓ2 + α||β||ℓ1 = (1 − α) βk + α|βk |
2 2
k=1
Pp 1 2
◮ “Ridge regression” −→ α = 0 so Pα (β) = k=1 2 βk
◮ “lasso” −→ α = 1 so Pα (β) = |βk |
◮ If α is close to 1, it performs like LASSO but without problems
caused by extreme correlations.
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 110.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 111.1/ 112
Overivew LL2-way Parm Constraints LL3–way Inference Stat vs Practical 4+–Way Tables Logit≡Log-linear Strategies
C.J. Anderson (Illinois) Log-linear Models for Contingency Tables Fall 2018 112.1/ 112