Chapter two

Classical Linear Regression Models

Simple Linear Regression Model

1
 Lecture Plan

2.1. Concept of Regression Function

2.2. Method of Moments & Method of Least Squares
2.3. Residuals and Goodness-of-fit
2.4. Properties of OLS Estimates and Gauss-Markov Theorem
2.5. Maximum Likelihood Estimation
2.6. Confidence Intervals and Hypothesis Testing
2.7. Predictions using Simple Linear Regression Mode

2
#. What are the concept regression Analysis ?

14
 2.1.1 Definition

 Regression analysis may show cause- &- effect relationship

between X & Y;

 A change in the value of independent variable X affects or

influences (or effect or may cause) a corresponding change in the
value of dependent variable Y, holding other factors constant.

4
 Definition

o Regression analysis implies (but does not prove) causality

(interconnection) between dependent variable Y & independent
variable X.
o Regression is estimation or prediction of the average value of a
dependent variable on the basis of the fixed values of other
variables.
o In regression analysis Y is dependent variable & X is
independent variable or there are many independent variables
5
 Definition

o The statistical method that helps to formulate an algebraic

relationship between two or more variables to estimate the
values of associated parameters with the variables (alias
independent variables), given the value of another variable
(alias dependent variable) is known as regression analysis.
o Regression model or equation is the mathematical equation
that helps to predict or forecast the value of the dependent
variable based on the known independent variables.
7
 2.1.2 Use of regression analysis

#. When do we apply regression analysis?

#. What are some of the considerations we should

make when choosing statistical analysis?

9
 When do we apply regression analysis?

 To characterize the relationship (association or perhaps causality)

between dependent & independent variables by determining the
extent, direction & strength of the relationship.

 To obtain a quantitative formula or equation to describe the dependent

variable as a function of the independent variables.

 To determine which of several independent variables are important

& which are not describing or predicting a dependent variable.

10
 When do we apply regression analysis?...

o To determine the best mathematical model (or parsimonious

model) for describing the relationship between a dependent &
one or more independent variables.
o To describe quantitatively or qualitatively the relationship between
different observed variables.
o To assess the interactive effects of two or more independent variables
with regard to a dependent variable.

11
 What are some of the considerations we should
make when choosing statistical analysis?

 Some of the major considerations are

 The purpose of investigation,

 The mathematical characteristics of the variable,

 The statistical assumptions made about the variables

 How the data were collected (sampling procedures).

12
 What are some of the considerations or
assumptions that lead to a choice of analysis?

These are considerations made about the variables (continuous

or discrete) that lead to what type of models to apply to the
data.
How the data were collected (sampling method);the use of
sample characteristics to estimate population characteristics
such as mean, variance, covariance, proportion, etc. depends on
the sampling technique use.
13
2.1.3. Types of regression analysis

#. What are the types of regression

models?

14
 What are the different types of
regression models?

 There are many types of regression models; but, here

we will deal only with some three types of regression
models.
1. Simple regression model

2. Multiple regression model

3. Multivariate regression model

15
 I. Simple regression model

 Simple regression model is a statistical equation that

characterizes the relationship between a dependent variable &
only one independent variable.

16
 I. Simple regression model...

EXAMPLES:
Qd= f(P) Demand function
QS = f(P) Supply function

Normally the explained variable is designated by Y &

the explanatory variables by Xis.

17
 II. Multiple regression model

 Multiple regression model is a mathematical model that

characterizes the relationship between a dependent variable & two
or more independent variables.
 Generally,

 Using generic mathematical expression,

 If k=1, that is, there is only one X-variable, we have the simple
regression
 18
 II. Multiple regression model...

 But, If k>1, i.e. more than one x-variable, we have the

multiple regression case.
 Using specific mathematical expression,

Example: for k=4

 Where Y= demand, X1 = income, X2 = price of good, X3=

Price of related good, & X4 = taste & preferences

19
 III. What is multivariate regression model?

 Multivariate regression model is algebraic system of

equations that characterizes the relationship among
more than one dependent variable & one or more
independent variables through a set of statistical
regression models.

20
 III. What is multivariate regression model?..

 Using mathematical expression,

21
 2.1.4.Variables in regression models

 Explain the variables involved in a regression model?

 What is the significance of the stochastic term or
disturbance term?
 What are the justifications for the inclusion of the
disturbance term in a regression model?

22
 Explain the variables involved in a
regression model?

These variables are observable, unobservable & unknown parameters.

1.Observable Variables : These are the variables in which their values are
collected from the field through questionnaires, interviews & other means of
data collection mechanisms. Thus,

Yi = the ith value of the dependent variable. Y i is also called the dependent variable or the
regressand or the explained variable

Xi = the ith value of the independent variable. Xi is also called the independent
variable or the regressor or the explanatory variable

23
 Explain the variables involved in a regression model?...

2. Unobservable variables

 These are the values that will be determined from the

observations & estimated values of the data set.

 The εi is the random error term for the ith member of the
population

 εi is also called the disturbance term or the stochastic term

24
 Explain the variables involved in a
regression model?...

3. Unknown Parameters (regression coefficients)

It is the values that will be estimated from the sample data of

dependent & independent variables.

 The unknown parameters include –

25
 What is the significance of the stochastic term?

 Regression model usually contains variables; a

disturbance (stochastic) term; & parameters in a
system of structural equations.

 The stochastic disturbance term is a random variable

that typically is added to all equations of the model
other than identities or equilibrium conditions.

26
 What are the justifications for the inclusion of
the disturbance term in a regression model?

(1) Effect of omitted variables from the model

Due to a variety of reasons some variables (or factors) which might
affect the dependent variable are omitted.
EXAMPLE

Qd = f(p) Gd = α+ βpi + ε a demand equation

Factors like family size, tastes & preferences, price of other commodities, income etc.
are excluded & taken care by ε
But, due to:
Lack of knowledge on the factors that should be included
Difficulty in measurement or lack of data (usually time series data)
Some factors are random (unpredictable, e.g. flood), etc., & they may
not be included.

27
 What are the justifications for the inclusion of
the disturbance term in a regression model?...

(2) The Randomness of human behaviour

 There is an unpredictable element of randomness in human
behaviour that can be accounted by the error term.

28
 What are the justifications for the inclusion of
the disturbance term in a regression model?...

(3) Measurement Error

o Deviations of the points from the line may be due to errors of measurement
of the variables, due to the methods of collection, processing statistical
information, etc.

o The variables included in the model may be measured inaccurately & the
stochastic term is expected to account for these errors.

o Problems that arise due to the methods of data collection, processing statistical
information, etc. can be captured by the error term.

29
 What are the justifications for the inclusion of
the disturbance term in a regression model?...

4. Imperfect specification of the mathematical form of the

model

 The equations may be mis-specified in that the particular

functional forms chosen may be incorrect.

o We may have linearized a non-linear relationship or we

may have left out some equation out of the model

o Many variables are simultaneously determined by a

system containing many equations
30
 What are the justifications for the inclusion of the disturbance
term in a regression model?...

o Therefore, in order to take all these sources of error into

account, a random variable (i.e., error term, random
disturbance term or stochastic term) is included in the
regression equation.

31
Assumptions of the Classical Linear Stochastic Regression
Model

# What is the Assumptions of the Simple Linear Stochastic

Regression Model?

5
 Assumptions

1. The variable  has normal distribution with mean of 0 and a

standard deviation of σ referred to as standard error.  : N(0, σ)
Mean ( ) = 0
Therefore, Mean (y ) =  0 +  1 x
In words Mean of y is a linear function of x .The actual y for any
given x has normal distribution with mean of  0 + 1 x and a
standard deviation of standard error.

5
 Assumption

The dependent variable Yi is normally distributed.

i.e  Yi ~ N(  x i ), 2 ………………………………(2.7)

Proof:
Mean: (Y )    xi  u i 
   X i Since (u i )  0

Variance: Var (Yi )  Yi  (Yi ) 2

   X i  u i  (  X i ) 
2

 (ui ) 2

 2
(since (ui ) 2   2 )

 var( Yi )   2
……………………………………….(

5
 Assumptions

o The shape of the distribution of Yi is determined by the shape of

the distribution of Ui which is normality by assumption .
o Since , α and β being constant, they don‟t affect the distribution of
Yi.
o Furthermore, the values of the explanatory variable, Xi, are a set of
fixed values by assumption and therefore don‟t affect the shape of
the distribution of Yi.

5
 Assumptions

2. The model is linear in parameters

This is because if the parameters are non-linear it is difficult to
estimate them since their value is not known but you are given with
the data of the dependent and independent variable.
Example

Are linear in parameters

5
 Assumption…

3. Ui is a random real variable

This means that the value which u may assume in any one period
depends on chance; it may be positive, negative or zero. Every value
has a certain probability of being assumed by u in any particular
instance.

5
 Assumption

4. The variance of the random variable(U) is constant in each period

(The assumption of homoscedasticity)
For all values of X, the u‟s will show the same dispersion around
their mean.

Var(U i )  E[U i  E(U i )]2  E(U i ) 2   2 (Since E (U i )  0 ).This constant

variance is called homoscedasticity assumption and the constant variance itself
is called homoscedastic variance.
5
 Assumption

5. The random terms of different observations are independent

(The assumption of no autocorrelation)
This means the value which the random term assumed in one period
does not depend on the value which it assumed in any other period.

Cov(Ui,Uj )  E (ui u j )  0
5
 Assumption…

6. The random variable (U) is independent of the explanatory

variables.
o This means there is no correlation between the random variable
and the explanatory variable.
o If two variables are unrelated their covariance is zero.

5
39
 Assumption…

7. Explanatory variable are measured with out error.

o Since the random variable absorb the influence of omitted
variables and possible error of measurement in the dependent
variables, the regressors/explanatory variable(s) are error free.
The value of dependent variable may or may not include error of
measurement.

5
 Assumption…

8. There is no perfect Multicollinearity i.e there are no perfect

linear relationships among the explanatory variables.

o If there are more than one explanatory variables in the function,

then it is assumed that they are not perfectly correlated with each

other.

5
Method of Simple linear regression Estimation

# How can estimate the simple linear regression ?

5
Method of Simple linear regression Estimation
o Specifying the model and stating its underlying assumptions are
the first stage of any econometric application.
o The next step is the estimation of the numerical values of the
parameters of economic relationships.
o The parameters of the simple linear regression model can be
estimated by various methods. Three of the most commonly used
methods are:
1. Ordinary least square method (OLS)
2. Maximum likelihood method (MLM)
3. Method of moments (MM)
5
 1. Ordinary least square method (OLS)
o The model is called the true relationship between Y and X
because Y and X represent their respective population value are
called the true parameters since they are estimated from the
population value of Y and X.
o But, it is difficult to obtain the population value of Y and X
because of technical or economic reasons. So we are forced to
take the sample value of Y and X.
o The parameters estimated from the sample value of Y and X are
called the estimators of the true parameter and are symbolized as
α and 𝛽 .
5
 1. Ordinary least square method (OLS)

o The model is called estimated relationship between Y and X

since are estimated from the sample of Y and X and represents
the sample counterpart of the population random disturbance.

The model Yi  ˆ  ˆX i  ei , is called estimated relationship between Y and X since

ˆ and ˆ are estimated from the sample of Y and X and ei represents the sample
counterpart of the population random disturbance U i .
Estimation of  and  by least square method (OLS) or classical least square

(CLS) involves finding values for the estimates ˆ and ˆ which will minimize the
o
sum of square of the squared residuals (  ei2 ).

5
 Conti..
From the estimated relationship Yi  ˆ  ˆX i  ei , we obtain:

ei  Yi  (ˆ  ˆX i ) …………………………… (2.6)

e 2
i   (Yi  ˆ  ˆX i ) 2 ………………………. (2.7)

To find the values of ˆ and ˆ that minimize this sum, we have to partially

differentiate e 2
i with respect to ˆ and ˆ and set the partial derivatives equal to

zero.
  ei2
1.  2 (Yi  ˆ  ˆX i )  0.......... .......... .......... .......... .......... .....( 2.8)
ˆ

Rearranging this expression we will get: Y i  n  ˆX i …… (2.9)

If you divide (2.9) by „n‟ and rearrange, we get

ˆ  Y  ˆX .......... .......... .......... .......... .......... .......... .......... ....( 2.10)
46
 Conti..
 ei2
2.  2 X i (Yi  ˆ  ˆX )  0.......... .......... .......... .......... .......... (2.11)
ˆ

Note: at this point that the term in the parenthesis in equation 2.8and 2.11 is the
residual, e  Yi  ˆ  ˆX i . Hence it is possible to rewrite (2.8) and (2.11) as

 2 ei  0 and  2 X i ei  0 . It follows that;

 e  0 and  X e  0.......... .......... .......... .......... ....( 2.12)

i i i

If we rearrange equation (2.11) we obtain;

Y X
i i  ˆX i  ˆX i2 ……………………………………….(2.13)

47
 Conti..
Equation (2.9) and (2.13) are called the Normal Equations. Substituting the values
of ̂ from (2.10) to (2.13), we get:

Y Xi i  X i (Y  ˆX )  ˆX i2

 YX i  ˆXX i  ˆX i2

Y Xi i  Y X i  ˆ (X i2  XX i )

XY  nXY = ˆ ( X i2  nX 2)

XY  nXY
ˆ  ………………….(2.14)
X i2  nX 2

48
 Conti…
Equation (2.14) can be rewritten in somewhat different way as follows;
( X  X )(Y  Y )  ( XY  XY  XY  XY )

 XY  Y X  XY  nXY

 XY  nY X  nXY  nXY

( X  X )(Y  Y )  XY  n X Y              (2.15)

( X  X ) 2  X 2  nX 2                  (2.16)

Substituting (2.15) and (2.16) in (2.14), we get

( X  X )(Y  Y )
ˆ 
( X  X ) 2

49
 Conti…
Now, denoting ( X i  X ) as xi , and (Yi  Y ) as yi we get;

x y
ˆ  i 2 i ……………………………………… (2.17)
xi

The expression in (2.17) to estimate the parameter coefficient is termed is the

formula in deviation form.

50
 Estimation of a function with zero intercept
Suppose it is desired to fit the line Yi    X i  U i , subject to the restriction   0.

to estimate ˆ , the problem is put in a form of restricted minimization problem and

then Lagrange method is applied.
n
We minimize: ei2   (Yi  ˆ  ˆX i ) 2
i 1

Subject to: ˆ  0

51
 Conti…
The composite function then becomes
Z   (Yi  ˆ  ˆX i ) 2  ˆ , where  is a Lagrange multiplier.

We minimize the function with respect to ˆ , ˆ, and 

Z
 2(Yi  ˆ  ˆX i )    0        (i)
ˆ
Z
 2(Yi  ˆ  ˆX i ) ( X i )  0        (ii)
ˆ

z
 2  0                    (iii)

52
 Conti…
Substituting (iii) in (ii) and rearranging we obtain:
X i (Yi  ˆX i )  0

Yi X i  ˆX i  0
2

X Y
ˆ  i 2 i ……………………………………..(2.18)
X i
This formula involves the actual values (observations) of the variables and not their
deviation forms, as in the case of unrestricted value of ˆ .

53
Statistical Properties of Least Square Estimators

 There are various econometric methods with which we may

obtain the estimates of the parameters of economic
relationships.

 We would like to an estimate to be as close as the value of

the true population parameters i.e. to vary within only a
small range around the true parameter.

54
 Conti….
 How are we to choose among the different econometric
methods, the one that gives „good‟ estimates?

 We need some criteria for judging the „goodness‟ of an

estimate.

55
 Conti…
 Closeness‟ of the estimate to the population parameter is
measured by the mean and variance or standard deviation of
the sampling distribution of the estimates of the different
econometric methods.

56
 Cont..
 We assume the usual process of repeated sampling i.e. we
assume that we get a very large number of samples each of
size „n‟.

 we compute the estimates' from each sample, and for each

econometric method and we form their distribution.

57
 Conti….
 We next compare the mean (expected value) and the
variances of these distributions and we choose among the
alternative estimates the one whose distribution is
concentrated as close as possible around the population
parameter.

58
 PROPERTIES OF OLS ESTIMATORS

 The ideal or optimum properties that the OLS estimates

possess may be summarized by well known theorem known
as the Gauss-Markov Theorem.

 Statement of the theorem: “Given the assumptions of the

classical linear regression model, the OLS estimators, in the
class of linear and unbiased estimators, have the minimum
variance, i.e. the OLS estimators are BLUE.

59
 Conti…
 According to the this theorem, under the basic assumptions of the

classical linear regression model, the least squares estimators are

linear, unbiased and have minimum variance (i.e. are best of all linear
unbiased estimators).

 Some times the theorem referred as the BLUE theorem i.e. Best,
Linear, Unbiased Estimator. An estimator is called BLUE if:

60
A)Linear:
a linear function of the a random variable, such as, the dependent
variable Y

 ̂  K1Y1  K 2Y2  K 3Y3       K nYn

 ̂ is linear in Y

where

61
 B. Unbiasedness
Proposition: ˆ & ˆ are the unbiased estimators of the true parameters  & 

From your statistics course, you may recall that if ˆ is an estimator of  then
E(ˆ)    the amount of bias and if ˆ is the unbiased estimator of  then bias =0 i.e.

E(ˆ)    0  E(ˆ)  

In our case, ˆ & ˆ are estimators of the true parameters  &  .To show that they
are the unbiased estimators of their respective parameters means to prove that:
(ˆ )   and (ˆ )  
62
 C) Minimum variance:

• It has a minimum variance in the class of linear and

unbiased estimators. An unbiased estimator with the least
variance is known as an efficient estimator.

63
Statistical test of Significance of the OLS Estimators (First Order tests)

 After the estimation of the parameters and the determination of the least

square regression line, we need to know how „good‟ is the fit of this line

to the sample observation of Y and X, that is to say we need to measure

the dispersion of observations around the regression line.

 This knowledge is essential because the closer the observation to the line,

the better the goodness of fit, i.e. the better is the explanation of the

variations of Y by the changes in the explanatory variables.

64
 Cont.….
 We divide the available criteria into three groups: the theoretical a priori
criteria, the statistical criteria, and the econometric criteria.

 Under this section, our focus is on statistical criteria (first order tests). the two
most commonly used first order tests in econometric analysis are

A. The coefficient of determination (the square of the correlation coefficient

i.e. R2). This test is used for judging the explanatory power of the

independent variable(s).

B. B) The standard error tests of the estimators. This test is used for judging

the statistical reliability of the estimates of the regression coefficients.

65
 Tests of the ‘ Goodness of fit’ with R2
A. R2 shows the percentage of total variation of the dependent variable that
can be explained by the changes in the explanatory variable(s) included in
the model.

B. To elaborate this let‟s draw a horizontal line corresponding to the mean

value of the dependent variable (see figure„d‟ below).

By fitting the line Yˆ  ˆ 0  ˆ1 X we try to obtain the explanation of the variation of
the dependent variable Y produced by the changes of the explanatory variable X.

66
 Conti…

.Y
Y = e  Y  Yˆ
Y Y = Yˆ Yˆ  ˆ 0  ˆ1 X

= Yˆ  Y
Y.

X
Figure „d‟. Actual and estimated values of the dependent variable Y 67
 Conti……
 As can be seen from fig.(d) above, 𝑌 − 𝑌 represents measures the

variation of the sample observation value of the dependent variable

around the mean.

 However the variation in Y that can be attributed the influence of X,

(i.e. the regression line) is given by the vertical distance .

 The part of the total variation in Y about that can‟t be attributed to X

is equal to which is referred to as the residual variation.

 e= 𝑌 − 𝑌 which is referred to as the residual variation.

68
 Conti…
In summary:
ei  Yi  Yˆ = deviation of the observation Yi from the regression line.

yi  Y  Y = deviation of Y from its mean.

yˆ  Yˆ  Y = deviation of the regressed (predicted) value ( Yˆ ) from the mean.

69
 Conti…
Now, we may write the observed Y as the sum of the predicted value ( Yˆ ) and the
residual term (ei.).
Yi  Yˆ  ei
 predicted Yi

Observed Yi Re sidual

From equation (2.34) we can have the above equation but in deviation form
y  yˆ  e . By squaring and summing both sides, we obtain the following expression:

70
 Conti…
y 2  ( yˆ 2  e) 2

y 2  ( yˆ 2  ei2  2 yei)

 yi  ei2  2yˆei

2

But ŷei = e(Yˆ  Y )  e(ˆ  ˆxi  Y )

 ˆei  ˆexi  Yˆei

(but ei  0, exi  0 )

  yˆe  0 ………………………………………………(2.46)
71
 Conti….
Therefore;
yi2  
yˆ 2  ei2 ………………………………...(2.47)
 
Total Explained Un exp lained
var iation var iation var ation

OR,
Total sum of Explained sum Re sidual sum
 
square of square of square
    
TSS ESS RSS

i.e
TSS  ESS  RSS ……………………………………….(2.48)
72
 Conti…
Mathematically; the explained variation as a percentage of the total variation is
explained as:
ESS yˆ 2
 ……………………………………….(2.49)
TSS y 2

From equation (2.37) we have yˆ  ̂x . Squaring and summing both sides give us

yˆ 2  ˆ 2 x 2                        (2.50)
We can substitute (2.50) in (2.49) and obtain:
ˆ 2 x 2
ESS / TSS  …………………………………(2.51)
y 2
73
 CONTI…
 xy  x x y
2 2
  2  i2 , Since ˆ  i 2 i

  x  y xi

xy xy
 ………………………………………(2.52)
x 2 y 2

Comparing (2.52) with the formula of the correlation coefficient:

r = Cov (X,Y) / x2x2 = xy / nx2x2 = xy / ( x 2 y 2 )1/2 ………(2.53)

Squaring (2.53) will result in: r2 = ( xy )2 / ( x 2 y 2 ). ………….(2.54)

74
 Conti….
Comparing (2.52) and (2.54), we see exactly the expressions. Therefore:
xy xy
ESS/TSS  = r2
x 2 y 2

From (2.48), RSS=TSS-ESS. Hence R2 becomes;

TSS  RSS RSS e 2
R2   1  1  i2 ………………………….…………(2.55)
TSS TSS y
From equation (2.55) we can drive;
RSS  ei2  yi2 (1  R 2 )                            (2.56)

The limit of R2: The value of R2 falls between zero and one. i.e. 0  R 2  1 .
75
 Interpretation of R2

A) suppose , this means that the regression line gives a

good fit to the observed data since this line explains
90% of the total variation of the Y value around their
mean.

B) The remaining 10% of the total variation in Y is

unaccounted for by the regression line and is attributed
to the factors included in the disturbance variable

76
 B) TESTING THE SIGNIFICANCE OF OLS PARAMETERS

To test the significance of the OLS parameter estimators we need the following:
 Variance of the parameter estimators
 Unbiased estimator of  2
 The assumption of normality of the distribution of error term.
We have already derived that:
77
 Conti….
ˆ 2
 var( ˆ ) 
x 2
ˆ 2 X 2
 var(ˆ ) 
nx 2
e 2 RSS
 ˆ 2  
n2 n2
For the purpose of estimation of the parameters the assumption of normality is not
used, but we use this assumption to test the significance of the parameter estimators;
because the testing methods or procedures are based on the assumption of the
normality assumption of the disturbance term.
78
 Cont.…
Hence before we discuss on the various testing methods it is important to see
whether the parameters are normally distributed or not.
We have already assumed that the error term is normally distributed with mean zero
and variance  2 , i.e. U i ~ N ( 20) . , Similarly, we also proved that

Yi ~ N[(  x), 2 ] . Now, we want to show the following:

 2 
1. ˆ ~ N  ,
2
 x 

  2 X 2 
2. ˆ ~ N , 2 

 nx 
79
 Cont.…
To show whether ˆ and ˆ are normally distributed or not, we need to make use of
one property of normal distribution. “........ any linear function of a normally
distributed variable is itself normally distributed.”
ˆ  ki Yi  k1Y1  k 2 Y2i  ....  k n Yn

ˆ  wi Yi  w1Y1  w2 Y2i  ....  wn Yn

Since ˆ and ˆ are linear in Y, it follows that

 2    2 X 2 
ˆ ~ N  , 2  ; ˆ ~ N , 2 
 x   nx  80
 Conti…
The OLS estimates ˆ and ˆ are obtained from a sample of observations on Y and
X. Since sampling errors are inevitable in all estimates, it is necessary to apply test
of significance in order to measure the size of the error and determine the degree of
confidence in order to measure the validity of these estimates. This can be done by
using various tests. The most common ones are:
i) Standard error test ii) Student’s t-test iii) Confidence interval
81
 Conti…
All of these testing procedures reach on the same conclusion. Let us now see these
testing methods one by one.
i) Standard error test
This test helps us decide whether the estimates ˆ and ˆ are significantly different
from zero, i.e. whether the sample from which they have been estimated might have
come from a population whose true parameters are zero.   0 and / or   0 .
Formally we test the null hypothesis
H 0 :  i  0 against the alternative hypothesis H 1 :  i  0

The standard error test may be outlined as follows.

First: Compute standard error of the parameters.
82
 Conti…
SE(ˆ )  var( ˆ )

SE(ˆ )  var(ˆ )

Second: compare the standard errors with the numerical values of ˆ and ˆ .
Decision rule:
 If SE(ˆi )  12 ˆi , accept the null hypothesis and reject the alternative

hypothesis. We conclude that ˆi is statistically insignificant.

 If SE(ˆi )  12 ˆi , reject the null hypothesis and accept the alternative

hypothesis. We conclude that ˆi is statistically significant.

83
 Conti…
The acceptance or rejection of the null hypothesis has definite economic meaning.
Namely, the acceptance of the null hypothesis   0 (the slope parameter is zero)
implies that the explanatory variable to which this estimate relates does not in fact
influence the dependent variable Y and should not be included in the function, since
the conducted test provided evidence that changes in X leave Y unaffected. In other
words acceptance of H0 implies that the relation ship between Y and X is in fact
Y    (0) x   , i.e. there is no relationship between X and Y.

Numerical example: Suppose that from a sample of size n=30, we estimate the
following supply function.
Q  120  0.6 p  ei
SE : (1.7) (0.025)
84
 Conti…
Test the significance of the slope parameter at 5% level of significance using the
standard error test.
SE( ˆ )  0.025

( ˆ )  0.6

1
2 ˆ  0.3

This implies that SE(ˆi )  12 ˆi . The implication is ˆ is statistically significant at

5% level of significance.
Note: The standard error test is an approximated test (which is approximated from
the z-test and t-test) and implies a two tail test conducted at 5% level of significance.
85
 Cont.…
i) Standard error test
This test helps us decide whether the estimates ˆ and ˆ are significantly different
from zero, i.e. whether the sample from which they have been estimated might have
come from a population whose true parameters are zero.   0 and / or   0 .
Formally we test the null hypothesis
H 0 :  i  0 against the alternative hypothesis H 1 :  i  0

The standard error test may be outlined as follows.

First: Compute standard error of the parameters.

SE( ˆ )  var( ˆ )

SE(ˆ )  var(ˆ )

Second: compare the standard errors with the numerical values of ˆ and ˆ .
86
 Conti…
Decision rule:
 If SE(ˆi )  12 ˆi , accept the null hypothesis and reject the alternative

hypothesis. We conclude that ˆi is statistically insignificant.

 If SE(ˆi )  12 ˆi , reject the null hypothesis and accept the alternative

hypothesis. We conclude that ˆi is statistically significant.

87
 Conti…
The acceptance or rejection of the null hypothesis has definite economic meaning.
Namely, the acceptance of the null hypothesis   0 (the slope parameter is zero)
implies that the explanatory variable to which this estimate relates does not in fact
influence the dependent variable Y and should not be included in the function, since
the conducted test provided evidence that changes in X leave Y unaffected. In other
words acceptance of H0 implies that the relation ship between Y and X is in fact
Y    (0) x   , i.e. there is no relationship between X and Y.

Numerical example: Suppose that from a sample of size n=30, we estimate the
following supply function.
Q  120  0.6 p  ei
SE : (1.7) (0.025)

Test the significance of the slope parameter at 5% level of significance using the
standard error test. 88
 Conti…
SE ( ˆ )  0.025

( ˆ )  0.6

1
2 ˆ  0.3

This implies that SE(ˆi )  1 2 ˆi . The implication is ˆ is statistically significant at

5% level of significance.
Note: The standard error test is an approximated test (which is approximated from
the z-test and t-test) and implies a two tail test conducted at 5% level of significance.
i) Student’s t-test
Like the standard error test, this test is also important to test the significance of the
parameters. From your statistics, any variable X can be transformed into t using the
general formula:
89
 Conti..
X 
t , with n-1 degree of freedom.
sx

Where  i  value of the population mean

s x  sample estimate of the population standard deviation

( X  X ) 2
sx 
n 1
n  sample size
We can derive the t-value of the OLS estimates
ˆi   
t ˆ  
SE( ˆ ) 
 with n-k degree of freedom.
ˆ   
tˆ 
SE(ˆ ) 
90
 Conti…
Where:
SE = is standard error
k = number of parameters in the model.
Since we have two parameters in simple linear regression with intercept different
from zero, our degree of freedom is n-2. Like the standard error test we formally
test the hypothesis: H 0 :  i  0 against the alternative H 1 :  i  0 for the slope
parameter; and H0 :  0 against the alternative H1 :   0 for the intercept.

To undertake the above test we follow the following steps.

Step 1: Compute t*, which is called the computed value of t, by taking the value of
 in the null hypothesis. In our case   0 , then t* becomes:

ˆ  0 ˆ
t*  
SE( ˆ ) SE( ˆ )
91
 cont..
Step 2: Choose level of significance. Level of significance is the probability of
making „wrong‟ decision, i.e. the probability of rejecting the hypothesis when it is
actually true or the probability of committing a type I error. It is customary in
econometric research to choose the 5% or the 1% level of significance. This means
that in making our decision we allow (tolerate) five times out of a hundred to be
„wrong‟ i.e. reject the hypothesis when it is actually true.
92
 Conti…
Step 3: Check whether there is one tail test or two tail test. If the inequality sign in
the alternative hypothesis is  , then it implies a two tail test and divide the chosen
level of significance by two; decide the critical rejoin or critical value of t called tc.
But if the inequality sign is either > or < then it indicates one tail test and there is no
need to divide the chosen level of significance by two to obtain the critical value of
to from the t-table.
Example:
If we have H 0 :  i  0
against: H1 :  i  0
93
 Cont..
Then this is a two tail test. If the level of significance is 5%, divide it by two to
obtain critical value of t from the t-table.
Step 4: Obtain critical value of t, called tc at  2 and n-2 degree of freedom for two
tail test.
Step 5: Compare t* (the computed value of t) and tc (critical value of t)
 If t*> tc , reject H0 and accept H1. The conclusion is ˆ is statistically
significant.
 If t*< tc , accept H0 and reject H1. The conclusion is ˆ is statistically
insignificant.
94
 Conti..
Numerical Example:
Suppose that from a sample size n=20 we estimate the following consumption
function:
C  100  0.70  e
(75.5) (0.21)

The values in the brackets are standard errors. We want to test the null hypothesis:
H 0 :  i  0 against the alternative H 1 :  i  0 using the t-test at 5% level of

significance.
the t-value for the test statistic is
95
 Conti…
ˆ  0 ˆ 0.70
t*   =  3 .3
SE(ˆ ) SE(ˆ ) 0.21

a. Since the alternative hypothesis (H1) is stated by inequality sign (  ) ,it is a

two tail test, hence we divide  2  0.05 2  0.025 to obtain the critical value of
„t‟ at  2 =0.025 and 18 degree of freedom (df) i.e. (n-2=20-2). From the
t-table „tc‟ at 0.025 level of significance and 18 df is 2.10.
b. Since t*=3.3 and tc=2.1, t*>tc. It implies that ˆ is statistically significant.

96
 III) Confidence interval
Rejection of the null hypothesis doesn‟t mean that our estimate ˆ and ˆ is the
correct estimate of the true population parameter  and  . It simply means that
our estimate comes from a sample drawn from a population whose parameter  is
different from zero.
In order to define how close the estimate to the true parameter, we must construct
confidence interval for the true parameter, in other words we must establish limiting
values around the estimate with in which the true parameter is expected to lie within
a certain “degree of confidence”. In this respect we say that with a given probability
the population parameter will be with in the defined confidence interval (confidence
limits). 97
 Conti…
We choose a probability in advance and refer to it as confidence level (interval
coefficient). It is customarily in econometrics to choose the 95% confidence level.
This means that in repeated sampling the confidence limits, computed from the
sample, would include the true population parameter in 95% of the cases. In the
other 5% of the cases the population parameter will fall outside the confidence
interval.
In a two-tail test at  level of significance, the probability of obtaining the specific
t-value either –tc or tc is  2 at n-2 degree of freedom. The probability of obtaining
98
 Cont..
ˆ  
any value of t which is equal to at n-2 degree of freedom is
SE ( ˆ )

1 
2  
2  i.e. 1   .
i.e. Pr t c  t*  t c   1   …………………………………………(2.57)

ˆ  
but t*  …………………………………………………….(2.58)
SE( ˆ )

Substitute (2.58) in (2.57) we obtain the following expression.

 ˆ   
Pr t c   t c   1   ………………………………………..(2.59)
 SE( ˆ ) 

 
Pr  SE( ˆ )t c  ˆ    SE( ˆ )t c  1        by multiplying SE( ˆ )

 
Pr  ˆ  SE( ˆ )t c    ˆ  SE( ˆ )t c  1        by subtracting ˆ

Pr ˆ  SE( ˆ )    ˆ  SE( ˆ )t   1        by multiplying by  1

c

Prˆ  SE( ˆ )t    ˆ  SE( ˆ )t   1        int erchanging

c c
99
 Conti..
The limit within which the true  lies at (1   )% degree of confidence is:

[ ˆ  SE( ˆ )t c , ˆ  SE( ˆ )t c ] ; where t c is the critical value of t at 

2 confidence interval
and n-2 degree of freedom.
The test procedure is outlined as follows.
H0 :   0

H1 :   0

Decision rule: If the hypothesized value of  in the null hypothesis is within the

confidence interval, accept H0 and reject H1. The implication is that ˆ is

statistically insignificant; while if the hypothesized value of  in the null hypothesis

is outside the limit, reject H0 and accept H1. This indicates ˆ is statistically
significant. 100
 Cont.…
Numerical Example:
Suppose we have estimated the following regression line from a sample of 20
observations.
Y  128.5  2.88X  e
(38.2) (0.85)

The values in the bracket are standard errors.

a. Construct 95% confidence interval for the slope of parameter
b. Test the significance of the slope parameter using constructed confidence
interval.

101
 Cont.…
Solution:
a. The limit within which the true  lies at 95% confidence interval is:

ˆ  SE( ˆ )t c

ˆ  2.88

SE(ˆ )  0.85
t c at 0.025 level of significance and 18 degree of freedom is 2.10.

 ˆ  SE(ˆ )t c  2.88  2.10(0.85)  2.88  1.79.

The confidence interval is:

(1.09, 4.67)
The value of  in the null hypothesis is zero which implies it is out side the
confidence interval. Hence  is statistically significant.
102
 Cont..
2.2.3 Reporting the Results of Regression Analysis

The results of the regression analysis derived are reported in conventional formats.
It is not sufficient merely to report the estimates of  ‟s. In practice we report
regression coefficients together with their standard errors and the value of R2. It
has become customary to present the estimated equations with standard errors
placed in parenthesis below the estimated parameter values. Sometimes, the
103
 Conti…
placed in parenthesis below the estimated parameter values. Sometimes, the
estimated coefficients, the corresponding standard errors, the p-values, and some
other indicators are presented in tabular form.
These results are supplemented by R2 on ( to the right side of the regression
equation).

104
# END!!!!!!!

14