Regression Analysis:

Estimating Relationships

Regression Analysis is a study of relationship between a

set of independent variables and the dependent variable.
Independent variables are characteristics that can be measured directly
(example the area of a house). These variables are also caled predictor
variables (used to predict the dependent variable) or explanatory variables
(used to explain the behavior of the dependent variable).
Dependent variable is a characteristic whose value depends on the values of
independent variables.

Constant term Coefficients

Y = B0 + B1*X1 + B2*X2 + …… +/- E

Dependent Variable Independent Variable Random Error
 Purpose of Regression Analysis

Past / Experience / Known Future/Unknown

time
Now
Explanation:Use regression Prediction: If the regression
analysis to develop a mathematical model adequately explains the
model to explain the variance in the dependent variable, use the
dependent variable based on values model to predict values of the
of independent variables. dependent variable.
Explain Selling Price of a house (dependent) based on its characteristics
(independents). If the model is valid, use it for prediction.
Develop Regression Model using known data (sample)

Selling Price = 40,000 + 100(Sq.ft) + 20,000(#Baths)

If the above model is reliable and valid, Use this model to predict the Selling Price of any house
based on its area (Sq.ft.) and the number of bathrooms (#Baths)
The constant term (40,000) is the fixed price of the house. This is not dependent on the values of
the variables considered. Can be interpreted as the price of the lot and transaction costs.
The coefficient of Sq.ft. (100) is the change in Selling Price for an additional Square Foot. Can
be interpreted as Price per Sq.Foot.
 Procedure for Building Regression Models

Define Objectives Define/Clarify Purpose. Identify and describe the

measurement of the dependent variable.

Select Variables Identify possible independent variables (predictors – should

make sense). Use scatter plots and correlations for selection.

Estimate Model Estimate Regression Coefficients (using least squares

method).

Test Model Test to see if all coefficients are significant (reliability). Establish
validity (are relationships as expected, do predictions match actuals).

Implement the model in Decision Support System. Incorporate error in

Implement and Use predictions. Outline limitations/constraints of the model.

Compare predictions with actual values. Modify/Refine/Expand model

Monitor Performance if necessary. IT is about continuous improvement.
 Selecting Independent Variables: Scatter Plots
Scatter Plots are used to visualize the relationship between any two variables. For
regression analysis, we are looking for strong linear relationship between the independent and
dependent variable.
Overhead Vs. Production Runs (11.2)
Sales Vs. Promotion (Ex. 11.1) y = 25.12 + 0.7623x 130000
R2 = 0.4529

Overhead (y)
110000
130
90000
110
Sales

70000
90
50000
70 0 20 40 60 80
y = 75606 + 655.07x
50 R2 = 0.271
Production Runs (x)

60 80 100 120 Overhead Vs. Machine Utilization

Promotion
130000

Overhead
110000
90000

Y-Intercept (Constant): Value of the dependent variable 70000

50000
irrespective of the value(s) of the independent variable(s). 1000
y = 48621 + 1200
34.702x 1400 1600 1800 2000
2 Machine Hours
R = 0.3993

X-Coefficient (Slope): Change in dependent variable per

unit change in independent variable.
Overhead = 3996 + 43 M_Hrs + 883 Runs
R-Squared: Proportion of variance in dependent variable
explained by independent variable(s).
 Selecting Independent Variables: Correlation Analysis

Correlation Coefficients are used to measure the linear relationship between any two
variables. For regression analysis, we are looking for strong linear relationship between the
independent and dependent variable, and low correlations among independent variables .

MachHrs ProdRuns Overhead

MachHrs 1
ProdRuns -0.22909 1
Overhead 0.631885 0.520544 1

Correlation of MachHrs with Correlation of MachHrs with Correlation of ProdRuns with

ProdRuns (should be low) Overhead (should be high) Overhead (should be high)

Multicollinearity exists when two independent variables are highly correlated (redundancy).
 Simple Linear Regression

• Linear regression function

• One dependent and one independent variables
• Mathematical form : Y = b0+ b1X + e
 b0 and b1 are parameters (unknown constants) and their values are
estimated from a known sample of X and corresponding Y.

Estimated Model: Y-Pred = b0 + b1X Y-actual

b0 and b1 are estimates (based on a e *
Y-pred
sample) of b0 and b1 which are B1 = slope
parameters (based on population)
B0 = y -intercept
Estimation of b0 and b1 (coefficients) is
done by the Least Squares Method.
This method selects the line that has the X
smallest squared error
 Example of Simple Linear Regression: Defining Objective(s)

Define Objectives

• Pharmex is a chain of drugstores that operates around the country.

• To see how effective their advertising and other promotional activities are, the
company has collected data from 50 randomly selected metropolitan regions.
• In each region it has compared its own promotional expenditures and sales to
those of the leading competitor in the region over the past year.
So, Pharmex’s objective is to model the relationship
between Promotion expenditures and Sales
Since Pharmex is interested in improving its sales, relative to its largest
competitor, the dependent (outcome) variable for this situation is
Sales: Pharmex’s sales as a percentage of those of the leading
competitor. This is the dependent (or predicted) variable.
 Example of SLR: Select Independent Variable

Variable Selection

The company expects that there is a positive relationship between the

Relative measures of Sales and Promotion Expenditures, so that regions
with relatively more expenditures have relatively more sales.
Promote: Pharmex’s promotional expenditures as a percentage of
those of the leading competitor. This is the independent variable (or
predictor variable), one which can be controlled by Pharmex.
Selection Criteria: Sales Vs. Promotion (Ex. 11.1) y = 25.12 + 0.7623x
R2 = 0.4529
130

•Based in Common Sense and Experience 110

Sales
90
70

•Scatter Plots and Correlations 50

60 80 100 120
Promotion

Description of Variables:
•List each variable, how measured, and expected relationship with dependent
variable.
•In this section report results of Correlation Analyses, Scatter Plots, etc.
 Example of SLR: Collect and Organize Data

Data Collection

Pharmex ($) Competitor ($) Indexes (regr. data)

Sales= Promote =
Region Sales(Sp) Prom (Pp) Sales(Sc) Prom (Pc) Sp/Sc Pp/Pc

Collect all relevant Data and Organize it in a Dataset –

one which can be analyzed by a solver (like Excel)
 Example of SLR: Estimate Coefficients

Estimate Model Regression Procedure in Excel

R-Square: 45% of the variance in Sales

is explained by Promote (model)

Estimated Coefficients: Y-
intercept (b0) = 25.12 Slope
(b1) = 0.762 Sales-
predicted = 25.12 + 0.762 Promote

P-Value: Indicates the probability of

making a Type I error (the possibility
that the coefficient is = 0, that is there is
no relationship). If this value is greater
than .05 do not use the variable as a
predictor.
 Example of SLR: Testing the Model

Reliability and Validity:

•Does the model make intuitive sense? Is the model easy to
understand and interpret?
•Are all coefficients statistically significant? (p-values less
than .05)
•Are the signs associated with the coefficients as expected?
•Does the model predict values that are reasonably close to
the actual values?
•Is the model sufficiently sound? (High R-square, low
standard error, etc.)
 Example of SLR: Implementing and Using the Model

Develop a Spreadsheet Model (Decision Support System)

Competitor's Promotion 125 Estimated

Pharmex's Promotion 150 Decision Variable

Promotion Index (Promote) 120

Predicted Sales Index (Sales) 116.602 Forecast (regression formula)

What-if Pharmex spent 160K on promotions? (Sensitivity analysis)

What will Pharmex have to do to achieve 20% sales more than its competitor?
(goal seeking)
What will happen to Pharmex’s sales if its Competitor’s promotion can be any
value between 130K and 140K? (Monte-Carlo Simulation)