Bivariate linear regression

Bivariate linear regression

• ASW, Chapter 12

• Economics 224 – Notes for November 12, 2008

Regression line

• For a bivariate or simple regression with an independent variable x and a dependent variable y, the regression equation is y = β0 + β1 x + ε.
• The values of the error term, ε, average to 0 so E(ε) = 0 and E(y) = β0 + β1 x.
• Using observed or sample data for values of x and y, estimates of the parameters β0 and β1 are obtained and the estimated regression line is
• where is the value of y that is predicted from the estimated regression line.

Bivariate regression line

• x

• y

• E(y) = β0 + β1x

• yi

• ε or error term

• xi

• y = β0 + β1x + ε

• E(yi)

Observed scatter diagram and estimated least squares line

• x

• y

• ŷ = b0 + b1x

• y (actual)

• ŷ (estimated)

• deviation

Example from SLID 2005

• According to human capital theory, increased education is associated with greater earnings.
• Random sample of 22 Saskatchewan males aged 35-39 with positive wages and salaries in 2004, from the Survey of Labour and Income Dynamics, 2005.
• Let x be total number of years of school completed (YRSCHL18) and y be wages and salaries in dollars (WGSAL42).

• Source: Statistics Canada, Survey of Labour and Income Dynamics, 2005 [Canada]: External Cross-sectional Economic Person File [machine readable data file]. From IDLS through UR Data Library.

ID#	YRSCHL18	WGSAL42
1	17	62500
2	12	15500
3	12	67500
4	11	9500
5	15	38000
6	15	36000
7	19	70000
8	15	47000
9	20	80000
10	16	28000
11	18	65000
12	11	48000
13	14	72500
14	12	33000
15	14.5	6000
16	13.5	62500
17	15	77500
18	13	42000
19	10	36000
20	12.5	21000
21	15	41000
22	12.3	52500

• YRSCHL18 is the variable “number of years of schooling”
• WGSAL42 is the variable “wages and salaries in dollars, 2004”

• x

• y

• x

• y

• Mean of x is 14.2 and sd is 2.64 years.

• Mean of y is $45,954 and sd is $21,960.

• n = 22 cases

• y

• x

Analysis and results

• H0: β1 = 0. Schooling has no effect on earnings.
• H1: β1 > 0. Schooling has a positive effect on earnings.
• From the least squares estimates, using the data for the 22 cases, the regression equation and associate statistics are:
• y = -13,493 + 4,181 x.
• R2 = 0.253, r = 0. 503.
• Standard error of the slope b0 is 1,606.
• t = 2.603 (20 df), significance = 0.017.
• At α = 0.05, reject H0, accept H1 and conclude that schooling has a positive effect on earnings.
• Each extra year of schooling adds $4,181 to annual wages and salaries for those in this sample.
• Expected wages and salaries for those with 20 years of schooling is -13,493 + (4,181 x 20) = $70,127.

Equation of a line

• y = β0 + β1 x. x is the independent variable (on horizontal) and y is the dependent variable (on vertical).
• β0 and β1 are the two parameters that determine the equation of the line.
• β0 is the y intercept – determines the height of the line.
• β1 is the slope of the line.
• Positive, negative, or zero.
• Size of β1 provides an estimate of the manner that x is related to y.

Positive Slope: β1 > 0

• x

• y

• β0

• Δx

• Δy

• Example – schooling (x) and earnings (y).

Negative Slope: β1 < 0

• x

• y

• β0

• Δx

• Δy

• Example – higher income (x) associated with fewer trips by bus (y).

Zero Slope: β1 = 0

• x

• y

• β0

• Δx

• Example – amount of rainfall (x) and student grades (y)

Infinite Slope: β1 = 

• x

• y

Infinite number of possible lines can be drawn. Find the straight line that best fits the points in the scatter diagram.

Least squares method (ASW, 469)

• Find estimates of β0 and β1 that produce a line that fits the points the best.
• The most commonly used criterion is least squares.
• The least squares line is the unique line for which the sum of the squares of the deviations of the y values from the line is as small as possible.
• Minimize the sum of the squares of the errors ε.
• Or, equivalent to this, minimize the sum of the squares of the differences of the y values from the values of E(y). That is, find b0 and b1 that minimize: