Regression: Finding the equation of the line of best fit

Objectives: To find the equation of the least squares regression line of y on x.

Background and general principle

The aim of regression is to calculate the

equation of the line of best fit on a scatter 85
80
graph.

Weight (kg)
75
Consider the scatter graph on the right. One 70
possible line of best fit has been drawn on 65

the diagram. Some of the points lie above 60

55
the line and some lie below it.
50
The vertical distance each point is above or
below the line has been added to the 1·4 1·5 1·6 1·7 1·8
Height (m)
diagram. These distances are called
deviations or errors – they are symbolised as
d 1 , d 2 ,..., d n .
When drawing in a regression line, the aim is to make the line fit the points as closely as possible. We
do this by making the total of the squares of the deviations as small as possible, i.e. we minimise
∑ di2 .
If a line of best fit is found using this principle, it is called the least-squares regression line.

Example 1:
A patient is given a drip feed containing a particular chemical and its concentration in his blood is
measured, in suitable units, at one hour intervals. The doctors believe that a linear relationship will
exist between the variables.

Time, x (hours) 0 1 2 3 4 5 6
Concentration, y 2.4 4.3 5.0 6.9 9.1 11.4 13.5

We can plot these data on a scatter graph –

time would be plotted on the horizontal axis 12
(as it is the independent variable). Time is
Concentration

10
here referred to as a controlled variable,
since the experimenter fixed the value of this 8
variable in advance (measurements were 6
taken every hour). 4
Concentration is the dependent variable as the 2
concentration in the blood is likely to vary
according to time. 1 2 3 4 5 6
Time (hours)
The doctor may wish to estimate the
concentration of the chemical in the blood after 3.5 hours.
She could do this by finding the equation of the line of best fit.

There is a formula which gives the equation of the line of best fit.
The equation of the line is y = a + bx

S xy
where b = and a = y − bx .
S xx

Note: S xy = ∑ xy −
∑ x∑ y and
S xx . =∑x −
(∑ x )
2
2

n n
Note 2: x and y are the mean values of x and y respectively.

This line is called the (least-squares) regression line of y on x (because the equation has been given
with y the subject).
b is sometimes called the regression coefficient of y on x.

We can work out the equation for our example as follows:

21
∑ x = 0 + 1 + ... + 6 = 21 so x=
7
=3
52.6 These could all be
∑ y = 2.4 + 4.3 + ... + 13.5 = 52.6 so y=
7
= 7.514... found on a
calculator (if you
∑ xy = (0 × 2.4) + (1× 4.3) + ... + (6 ×13.5) = 209.4 enter the data into a
21 calculator).
∑ x = 0 + 1 + ... + 6 = 91 so x = 7 = 3
2 2 2 2

S xy = ∑ xy −
∑ x∑ y = 209.4 − 21× 52.6 = 51.6
n 7

S xx = ∑ x 2 −
(∑ x ) 2

= 91 −
(21)2 = 28
n 7

S xy51.6
So, b = = = 1.843 and a = y − b x = 7.514 − 1.843 × 3 = 1.985 .
S xx 28
So the equation of the regression line is y = 1.985 + 1.843x.

To work out the concentration after 3.5 hours: y = 1.985 + 1.843 × 3.5 = 8.44 (3sf)

If you want to find how long it would be before the concentration reaches 8 units, we substitute y = 8
into the regression equation:
8 = 1.985 + 1.843x
Solving this we get: x = 3.26 hours

Note: It would not be sensible to predict the concentration after 8 hours from this equation – we don’t
know whether the relationship will continue to be linear. The process of trying to predict a value from
outside the range of your data is called extrapolation.
Example 2:
The heights and weights of a sample of 11 students are:

Height 1.36 1.47 1.54 1.56 1.59 1.63 1.66 1.67 1.69 1.74 1.81
(m) h
Weight 52 50 67 62 69 74 59 87 77 73 67
(kg) w

[ n = 11 ∑ h = 17.72 ∑ h 2
= 28.705 ∑ w = 737 ∑ w 2
= 50571 ∑ hw = 1196.1 ]
a) Calculate the regression line of w on h.
b) Use the regression line to estimate the weight of someone whose height is 1.6m.

Note: Both height and weight are referred to as random variables – their values could not have been
predicted before the data were collected. If the sampling were repeated again, different values would
be obtained for the heights and weights.

Solution:
a) We begin by finding the mean of each variable:

h=
∑ h = 17.72 = 1.6109...
n 11

w=
∑ w = 737 = 67
n 11
Next we find the sums of squares:
(∑ h ) 2
17.72 2
S hh = ∑ h 2
− = 28.705 − = 0.1597
n 11
(∑ w) 737
2
2
S ww = ∑w − 2
= 50571 − = 1192
n 11

S hw = ∑ hw −
∑ h∑ w = 1196.1 − 17.72 × 737 = 8.86
n 11
The equation of the regression line is:
w = a + bh
where
S hw 8.86
b= = = 55.5
S hh 0.1597
and
a = w − bh = 67 − 55.5 × 1.6109 = −22.4
So the equation of the regression line of w on h is:
w = -22.4 + 55.5h

b) To find the weight for someone that is 1.6m high:

w = -22.4 + 55.5×1.6 = 66.4 kg
 Two possible regression lines
When x and y are both random variables, there are two possible regression lines that can be
calculated:
* the regression line of y on x;
* the regression line of x on y.

The regression line of y on x is the line that has already been met. Its equation is
y = a + bx
and it is used to find a value of y when we are given a value of x. This line minimises the vertical
distances of each point from the line.

The regression line of x on y minimises the horizontal distances of each point from the line. It is used
if you wish to work out a value of x when you are given a value of y. The equation of this regression
line is
x = a´ + b´y
where
S xy
b′ =
S yy
and
a ′ = x − b′ y .

The regression lines will not in general be the same (unless the points lie on a perfect straight line).
Both regression lines pass through the mean point ( x, y ) .

Note: If x is a controlled variable, you always use the regression line of y on x (since the regression
line of x on y doesn’t have any statistical meaning in this case).

Example:
A psychologist wants to investigate the relationship between the IQ of a child and the IQ of their
mother. She measures the IQ of a sample of 8 children and mothers:

Child’s IQ, x 87 91 94 98 103 108 111 123

Mother’s IQ, y 94 96 89 102 98 94 116 117

[ ∑ x = 815 ∑x 2
= 84013 ∑ y = 806 ∑ y 2
= 81962 ∑ xy = 82789 ]
a) Work out the product moment correlation coefficient between x and y.
b) Calculate the regression line of y on x and the regression line of x on y. Write down a point that
both lines pass through.
c) Use the appropriate regression line to estimate the IQ of a child born to a mother with an IQ of
100. Using your answer to part (a), explain how accurate you think this estimate is likely to be.

Solution:
a) Using the sums given in the question we find that:
815 × 806 815 2
S xy = 82789 − = 677.75 S xx = 84013 − = 984.875
8 8
806 2
S yy = 81962 − = 757.5
8
So,
677.75
r= = 0.785 (to 3 s.f.)
984.875 × 757.5

b) For the regression line of y on x:

677.75
b= = 0.688 (to 3 s.f.)
984.875
815 806
x= = 101.875 and y = = 100.75
8 8
a = y − b x = 100.75 − 0.688 × 101.875 = 30.66
So the equation of the regression line of y on x is:
y = 30.66 + 0.688x.

For the regression line of x on y:

677.75
b′ = = 0.895 (to 3 s.f.)
757.5
a ′ = x − b ′ y = 101.875 − 0.895 × 100.75 = 11.70
So the equation of the regression line of x on y is
x = 11.70 + 0.895y.

Both lines must pass through the mean point (101.875, 100.75).

c) Both variables are random variables.

To find x when y = 100, we use the regression line of x on y.
x = 11.70 + 0.895 ×100 = 101.2

This accurate should be reasonably accurate since the product moment correlation coefficient shows
fairly strong correlation.

Example 2:
The scores that 9 students obtained in their C1 and M1 mathematics examinations are as follows:

C1 mark, c% 82 51 68 45 30 55 64 77 28
M1 mark, m% 75 46 84 47 42 59 52 69 41

[ ∑ c = 500, ∑c 2
= 30708, ∑ m = 515, ∑ m 2
= 31397, ∑ cm = 30617
a) Calculate the equation of the appropriate regression line in order to estimate the mark that a student
scoring 53% in M1 might expect to obtain in C1.
b) Explain why it would not be sensible to use the same regression line to estimate the C1 mark that
might be expected if a student scored 20% in M1.

Solution:
a) Both c and m are random variables. So we need to find the regression line of c on m:

500 × 515 515 2

S cm = 30617 − = 2005.89 S mm = 31397 − = 1927.56
9 9
2005.89
So, b = = 1.04
1927.56
 500 515
c= = 55.56 and m = = 57.22
9 9
Therefore, a = 55.56 − 1.04 × 57.22 = −3.95

So, the regression line of c on m is:

c = -3.95 + 1.04m

When m = 53, c = -3.95 + 1.04×53 = 51.17%

b) The value m = 20 lies outside the range of M1 marks seen in the table. The regression line
calculated in part (a) cannot be assumed to still be valid outside the range of values given in the table.

Example 3:
A particular greenhouse plant is suspect able to a particular disease. An agricultural scientist wishes
to see how the temperature of the greenhouse affects the prevalence of the disease. She designs an
experiment in which she monitors the percentage of diseased leaves occurring at different
temperatures:

Temperature, t °F 70 72 74 76 78 80
Percentage of 12.3 9.5 7.7 6.1 4.3 2.3
diseased leaves, p

[ ∑ t = 450, ∑t 2
= 33820, ∑ p = 42.2, ∑ p 2
= 361.82, ∑ tp = 3097.8 ]
The scientist wishes to estimate the temperature that she should set the greenhouse if she is aims for
5% of leaves being diseased. Calculate an appropriate regression line and use it to find the required
temperature (giving your answer to the nearest whole number). Give a reason for your choice of
regression line.

Solution:
In this situation, temperature is a controlled variable. Only the regression line of p on t makes sense
here.

450 × 42.2 42.2 2

S tp = 3097.8 − = −67.2 S pp = 361.82 − = 65.013
6 6
− 67.2
So, b = = −1.0336
65.013
450 42.2
t= = 75 and p = = 7.033
6 6
Therefore, a = 7.033 − ( −1.0336) × 75 = 84.5

So, the regression line of t on p is:

p = 84.5 - 1.03t

Therefore to find t, we solve

5 = 84.5 – 1.03t
i.e. t = 77°C