1. (a) Let . The normal equations are given by (3-12), , hence for each of the
columns of X, x
=
n
... [Show More] x
x
X
1
. .
1 1
X′e = 0
= i i i k, we know that xk’e=0. This implies that Σ = 0and x e . i i e Σ 0
(b) Use Σ = 0 to conclude from the first normal equation that i i e a = y − bx .
(c) Know that Σ = 0 and . It follows then that i i e Σ = 0 i i i x e Σ ( − ) = 0 i i i x x e . Further, the latter
implies ( )( ) = 0 i Σ − bx i − − a i i x x y or (x − x)(y − y − b(x − x))= 0 i i i i Σ from which the result
follows.
2. Suppose b is the least squares coefficient vector in the regression of y on X and c is any other Kx1 vector.
Prove that the difference in the two sums of squared residuals is
(y-Xc)′(y-Xc) - (y-Xb)′(y-Xb) = (c - b)′X′X(c - b).
Prove that this difference is positive.
Write c as b + (c - b). Then, the sum of squared residuals based on c is
(y - Xc)′(y - Xc) = [y - X(b + (c - b))] ′[y - X(b + (c - b))] = [(y - Xb) + X(c - b)] ′[(y - Xb) + X(c - b)]
= (y - Xb) ′(y - Xb) + (c - b) ′X′X(c - b) + 2(c - b) ′X′(y - Xb).
But, the third term is zero, as 2(c - b) ′X′(y - Xb) = 2(c - b)X′e = 0. Therefore,
(y - Xc) ′(y - Xc) = e′e + (c - b) ′X′X(c - b)
or (y - Xc) ′(y - Xc) - e′e = (c - b) ′X′X(c - b).
The right hand side can be written as d′d where d = X(c - b), so it is necessarily positive. This confirms what
we knew at the outset, least squares is least squares.
3. Consider the least squares regression of y on K variables (with a constant), X. Consider an alternative set of
regressors, Z = XP, where P is a nonsingular matrix. Thus, each column of Z is a mixture of some of the
columns of X. Prove that the residual vectors in the regressions of y on X and y on Z are identical. What
relevance does this have to the question of changing the fit of a regression by changing the units of
measurement of the independent variables?
The residual vector in the regression of y on X is MXy = [I - X(X′X)-1X′]y. The residual vector in
the regression of y on Z is
MZy = [I - Z(Z′Z)-1Z′]y
= [I - XP((XP)′(XP))-1(XP)′)y
= [I - XPP-1(X′X)-1(P′)-1P′X′)y
= MXy
Since the residual vectors are identical, the fits must be as well. Changing the units of measurement of the
regressors is equivalent to postmultiplying by a diagonal P matrix whose kth diagonal element is the scale
factor to be applied to the kth variable (1 if it is to be unchanged). It follows from the result above that this
will not change the fit of the regression.
4. In the least squares regression of y on a constant and X, in order to compute the regression coefficients on
X, we can first transform y to deviations from the mean, y , and, likewise, transform each column of X to
deviations from the respective column means; second, regress the transformed y on the transformed X without
a constant. Do we get the same result if we only transform y? What if we only transform X?
3 [Show Less]