Examples1

1.2 Setting the Regression Context 3

Should one be especially interested in a comparison of the means, one could

proceed descriptively with a conventional least squares regression analysis as

a special case. That is, for each observation i, one could let

ˆ

yi

= β0

+ β1

xi

, (1.1)

where the response variable yi

is each applicant’s SAT score, xi

is an indicator

variable coded “1” if the applicant is Asian and “0” if the applicant is Hispanic,

β0

is the mean SAT score for Hispanic applicants, β1

is how much larger (or

smaller) the mean SAT score for Asian applicants happens to be, and i is an

index running from 1 to the number of Hispanic and Asian applicants, N.

Fig. 1.2. Distribution of SAT scores for Asian applicants.

SAT Scores for Asian Applicants

SAT Score

Frequency

600 800 1000 1200 1400 1600

0 50 100 150

to equate regression analysis with causal modeling. This is too narrow and

even misleading. Causal modeling is actually an interpretive framework that

is imposed on the results of a regression analysis. An alternative knee-jerk

response may be to equate regression analysis with the general linear model.

At most, the general linear model can be seen as a special case of regression

analysis.

Statisticians commonly deﬁne regression so that the goal is to understand

“as far as possible with the available data how the conditional distribution

of some response y varies across subpopulations determined by the possible

values of the predictor or predictors” (Cook and Weisberg, 1999: 27). That

is, interest centers on the distribution of the response variable Y conditioning

on one or more predictors X.

This deﬁnition includes a wide variety of elementary procedures easily

implemented in R. (See, for example, Maindonald and Braun, 2007: Chapter

2.) For example, consider Figures 1.1 and 1.2. The ﬁrst shows the distribution

of SAT scores for recent applicants to a major university, who self-identify

as “Hispanic.” The second shows the distribution of SAT scores for recent

applicants to that same university, who self-identify as “Asian.”

1

Jake Hofman (Columbia University) Regression March 8, 2019 5 / 6