Name: Adewole Oreoluwa Adesina Matric. No.: RUN/ACC/19/8261 Course Code: Eco 307
Name: Adewole Oreoluwa Adesina Matric. No.: RUN/ACC/19/8261 Course Code: Eco 307
Name: Adewole Oreoluwa Adesina Matric. No.: RUN/ACC/19/8261 Course Code: Eco 307
explanatory variables and the error terms . Assumption 1 requires the specified model
to be linear in parameters, but it does not require the model to be linear in variables.
Equation 1 and 2 depict a model which is both, linear in parameter and variables. Note
(1)
(2)
In order for OLS to work the specified model has to be linear in parameters. Note that if the
true relationship between and is non linear it is not possible to estimate the coefficient
in any meaningful way. Equation 3 shows an empirical model in which is of quadratic
nature.
(3)
Assumption 1 requires the model to be linear in parameters. OLS is not able to estimate
Equation 3 in any meaningful way. However, assumption 1 does not require the model to be
linear in variables. OLS will produce a meaningful estimation of in Equation 4.
(4)
Using the method of ordinary least squares (OLS) allows us to estimate models which are
linear in parameters, even if the model is non linear in variables. On the contrary it is not
possible to estimate models which are non linear in parameters, even if they are linear in
variables.
plausible, the regressor may be stochastic in nature. The term stochastic regressor means that the
regressors, i.e. the explanatory variables are random with the change of time. The basic
assumption in case of Stochastic regressors are: i) X, Y, e random ii) (X,Y) obtained from iid
sampling iii) E(e|X)=0 iv) X takes atleast two values v) Var(e|X) =𝜎 ଶ vi) e is normal. The
Variables X, Y and e is already defined in section 1. It has been recognized, however, that in
numerous applications X might be stochastic and e might not be normal. This may give rise to
three problems (a) X is nonstochastic and e is nonnormal, (b) X is stochastic and e is normal, and
(c) X is stochastic and e is nonnormal. The basic problem of the stochastic regressor in General
Linear Model (GLM) is that the least square estimators may not be unbiased because when
taking the expectation of the random vector b (OLS estimator), E[𝑏෨] = 𝛽෨+ E [(𝑋′𝑋) −1𝑋′𝑒̃]
the second term E[(𝑋′𝑋) −1𝑋′𝑒̃] may not vanish. It may happen because E[(𝑋′𝑋) −1𝑋′𝑒̃] cannot
be expressed as (𝑋′𝑋) −1𝑋′𝐸[𝑒̃] as in the case of fixed regressor, nor, due to the stochastic nature
unbiased, efficient & consistent * biased but consistent * biased but inconsistent. Also, the
statistical inference become more difficult. Appropriateness of OLS depends on the stochastic
dependencies between the matrix X and the error vector e. Three dependency structures are
defined, in the first case error and regressor are stochastically independent, secondly, error term
and regressor are contemporary uncorrelated and thirdly error term and regressor are
contemporary correlated. In first two cases, OLS estimators are feasible since they are consistent.
In the third case, application of alternative estimation procedures is needed since OLS estimators
are biased and inconsistent. Instrumental Variable Regression can be applied in such cases.
3. Zero mean value of the mean value of the disturbance
error term
The error term accounts for the variation in the dependent variable that the independent
variables do not explain. Random chance should determine the values of the error term.
For your model to be unbiased, the average value of the error term must equal zero.
Suppose the average error is +7. This non-zero average error indicates that our model
like this as bias, and it signifies that our model is inadequate because it is not correct on
average.
Stated another way, we want the expected value of the error to equal zero. If the expected
value is +7 rather than zero, part of the error term is predictable, and we should add that
information to the regression model itself. We want only random error left for the error
term.
In other words, the distribution of error terms has zero mean and doesn’t depend on the
independent variables X′s. Thus, there must be no relationship between the X′s and the
error term.
population corresponding to various X values have the same variance i.e. it neither increases nor
decreases a X increases.
Heteroscedasticity is when the variance is unequal i.e. the variance of y population varies with X.
One of the most popular example quoted on heteroscedasticity is the savings vs income data. As
incomes grow, people have more discretionary income and thus a wider choice about the income
disposition. Thus, regressing savings on income is likely to find higher variance as income
increases cause people have more choices regarding their saving decision.
Other reasons for Heteroscedasticity include presence of outliers, omitted important variables,
sum of squares (RSS). It gives equal weight to all observations. Thus when the RSS is minimized
to compute the estimate values, the observation with higher variance will have a larger pull in the
equation. Thus the beta estimated using OLS for heteroscedastic data will no longer have
minimum variance.
The beta estimated with heteroscedastic data will therefore have a higher variance and thus a high
standard error.
One observation of the error term should not predict the next observation. For instance, if
the error for one observation is positive and that systematically increases the probability
that the following error is positive, that is a positive correlation. If the subsequent error is
more likely to have the opposite sign, that is a negative correlation. This problem is
known both as serial correlation and autocorrelation. Serial correlation is most likely to
For example, if sales are unexpectedly high on one day, then they are likely to be higher
than average on the next day. This type of correlation isn’t an unreasonable expectation
for some subject areas, such as inflation rates, GDP, unemployment, and so on.
Assess this assumption by graphing the residuals in the order that the data were collected.
You want to see randomness in the plot. In the graph for a sales model, there is a cyclical
This assumption is most likely to be violated in time series regression models and, hence,
intuition says that there is no need to investigate it. However, you can still check for
the model, you can try taking lags of independent variables to correct for the trend
component. If you do not correct for autocorrelation, then OLS estimates won’t be
But the "true" errors 𝜀 may well be correlated with them, and this is what counts
as endogeneity.
To keep things simple, consider the regression model (you might see this
described as the underlying "data generating process" or "DGP", the theoretical
model that we assume to generate the value of 𝑦
𝑦𝑖=𝛽1+𝛽2𝑥𝑖+𝜀𝑖
When you estimate your regression model on the available data, we get
𝑦𝑖=𝛽̂1+𝛽̂2𝑥𝑖+𝜀̂𝑖
Because of the way OLS works*, the residuals 𝜀̂ will be uncorrelated with 𝑥. But
that doesn't mean we have avoided endogeneity — it just means that we can't
detect it by analysing the correlation between 𝜀̂and 𝑥, which will be (up to
numerical error) zero. And because the OLS assumptions have been breached,
we are no longer guaranteed the nice properties, such as unbiasedness, we enjoy
so much about OLS. Our estimate 𝛽̂2 will be biased
This is seen immediately you look at the problem. If the number of parameters to
parameters to be estimated and the observations are equal, then Ordinary Least
unbiased estimates with the minimum variance. However, satisfying this assumption
intervals and prediction intervals.
Another important OLS (Ordinary Least Squares) assumptions is the fact that when you
want to run a regression, you need to make sure that the sample is drawn randomly from
the population. When this doesn’t occur, you are basically running the risk of introducing
an unknown factor into your analysis and the model won’t take it into account.
Notice that this assumption also makes it clear that your independent variable causes the
dependent variable (in theory). So, simply put, the OLS is a causal statistical method that
investigates the ability of the independent variable to predict the dependent variable. This means
The easiest way to determine whether the residuals follow a normal distribution is to
assess a normal probability plot. If the residuals follow the straight line on this type of
Y=β1+β2∗(1/X)
OLS estimation provides the Best Linear Unbiased Estimate (BLUE) of beta if all assumptions of
What does BLUE mean? It means that the OLS method gives the estimates that are :
Linear :
Unbiased :
Its expected or average value is equal to the true value
Efficient :
It has minimum variance. An unbiased estimator with the least variance is known as efficient
estimator.
If all assumptions of the Linear Regression are satisfied, OLS gives us the best linear unbiased
estimates.
For a classical linear regression model with multiple regressors (explanatory variables),
there should be no exact linear relationship between the explanatory variables. The
Ragnar Frisch introduced this term, originally it means the existence of a “perfect”
where λ1,λ2,⋯,λk are constant and all of them all are non-zero, simultaneously, and X1=1 for all
Now a day, multicollinearity term is not only being used for the case of perfect multicollinearity
but also in case of not perfect collinearity (the case where the X variables are intercorrelated but
λ1X1+λ2X2+⋯λkXk+υi,
In case of a perfect linear relationship (correlation coefficient will be one in this case) among
explanatory variables, the parameters become indeterminate (it is impossible to obtain values for
each parameter separately) and the method of least square breaks down. However, if regressors
are not intercorrelated at all, the variables are called orthogonal and there is no problem