next up previous print clean
Next: Differentiation by a complex Up: MULTIVARIATE LEAST SQUARES Previous: Inside an abstract vector

Normal equations

An important concept is that when energy is minimum, the residual is orthogonal to the fitting functions. The fitting functions are the column vectors $\bold f_1$, $\bold f_2$, and $\bold f_3$.Let us verify only that the dot product $ \bold r \cdot \bold f_2 $ vanishes; to do this, we'll show that those two vectors are orthogonal. Energy minimum is found by  
 \begin{displaymath}
0 \quad = \quad {\partial\over \partial x_2}\ \bold r \cdot ...
 ...r\over \partial x_2}
 \quad = \quad 2\; \bold r \cdot \bold f_2\end{displaymath} (31)
(To compute the derivative refer to equation (13).) Equation (31) shows that the residual is orthogonal to a fitting function. The fitting functions are the column vectors in the fitting matrix.

The basic least-squares equations are often called the ``normal" equations. The word ``normal" means perpendicular. We can rewrite equation (28) to emphasize the perpendicularity. Bring both terms to the left, and recall the definition of the residual $\bold r$from equation (13):
      \begin{eqnarray}
\bold F' ( \bold F \bold x - {\bf d}) &=& \bold 0 \\ \bold F' \bold r &=& \bold 0\end{eqnarray} (32)
(33)
Equation (33) says that the residual vector $\bold r$is perpendicular to each row in the $\bold F'$ matrix. These rows are the fitting functions. Therefore, the residual, after it has been minimized, is perpendicular to all the fitting functions.


next up previous print clean
Next: Differentiation by a complex Up: MULTIVARIATE LEAST SQUARES Previous: Inside an abstract vector
Stanford Exploration Project
4/27/2004