## Multiple Regression

## Forrest Young's Notes

## Copyright © 1997-9 by Forrest W. Young.

## Review of Simple Regression

Simple Regression, the previous topic, is very closely related to Multiple Regression.

Simple RegressionSimple Regression (just called "Regression", up to now) is a statistical technique that is used to measure and describe the function relating a predictor (independent) variable to a response (dependent) variable.

Multiple RegressionMultiple Regression is a statistical technique that is used to measure and describe the function relating two (or more) predictor (independent) variables to one response (dependent) variable.

Correlation Coefficient:- Correlation tells us about the
strength(and shape) of the relationship between two variables. The square of the correlation tells us the proportion of the variables' variance that is shared between them.Multiple Correlation Coefficient:- The Multiple Correlation Coefficient tells us about the
strengthof the relationship of a combination of the several predictor variables with the response variable. The square of the multiple correlation tells us the proportion of the response variable's variance that is shared with the predictor variables variance.Simple Regression:- Regression tells us about the nature of the
functionrelating the two variables. For linear regression, which is what we consider here, regression fits a straight line, called theregression line,to the data so that the line best describes their relationship.Multiple Regression- Multiple regression tells us about the nature of the function that relates a
set of predictor variablesto a single response variable. OLS (ordinary least squares) multiple regression assumes the function is alinear function(straight line, flat plane, etc), and fits the function so that the relationship is is a least squares relationship. The function is called theregression function.The Regression Line:- Regression Analysis fits a
linear functionto the relationship between the two variables. The line that is fit to the relationship has certain properties:

- The regression function is as close as possible, in a specific average
least squaressense, to all of the points.- The regression function identifies the "central tendency" of the relationship between the two variables, just as the mean identifies the "central tendency" of a single variable.
- The regression function provides a simplified description --- a
model--- of the relationship between the two variables.- The regression function gives us a way to
predictvalues for the response variable from values of the predictor variable.Regression Equation for a Linear Relationship- A linear relationship of n predictor variables, denoted
- X
1, X2, ... Xn- to a single response variable, denoted
- Y
- is described by the linear equation involving several variables. The general linear equation is:
- Y = a + b
1X1+ b2X2+ ... + bnXn- This equation shows that any linear relationship can be described by its:

Intercept: The linear combination of the X's is zero. Usually we don't interpret the intercept.Slopes: The slope for each predictor variable is denoted by the variousb's in the equation. The slope specifies how much the variable Y will change when the particular X changes by one unit.