It often happens that some observations are considered more reliable
than others. One may desire to weight the more reliable data more heavily
in the calculation. In other words, we may multiply the *i*th equation by a
weight

(28) |

(29) |

(30) |

A case of common interest
is where some equations should be solved exactly.
Such equations are called constraint equations.
Constraint equations often
arise out of theoretical considerations so they may,
in principle,
not have any error.
The rest of the equations often involve some measurement.
Since the measurement can often be made many times,
it is easy to get a lot more
equations than unknowns.
Since measurement always involves error,
we then use the method of least squares
to minimize the average error.
In order to be certain that the constraint equations are solved exactly,
one could use the trick of applying very large weight factors
to the constraint equations.
A problem is that
``very large'' is not well defined.
A weight equal 10^{10}
might not be large enough
to guarantee the constraint equation is satisfied
with sufficient accuracy.
On the other hand, 10^{10} might lead to
disastrous round-off when solving
the simultaneous equations in a computer
with eight-digit accuracy.
The best approach is to analyze the situation
theoretically for .

An example of a constraint equation
is that the sum of the *x*_{i} equals *M*.
Another constraint would be *x _{1}* =

(31) |

(32) |

(33) |

(34) | ||

(35) |

(36) |

(37) | ||

(38) | ||

(39) |

Arranging (38) and (32)
together and dropping superscripts,
we get a square matrix in *m* + *k* unknowns.

(40) |

Equation (40) is now a simultaneous set for the unknowns and . It might also be thought of as the solution to the problem of minimizing the quadratic form

(41) |

According to the method of Lagrange multipliers, one may minimize a quadratic form subject to constraints by minimizing instead a sum of the quadratic form plus constraint terms where each constraint term is the product of a constraint equation multiplied by a Lagrange multiplier . This is precisely what we have in (41), and the solution is given by (40). Lagrange multipliers frequently arise in connection with integral equations. The concept is readily transformed to matrices merely by approximating integration by summation.

- In determining a density vs. depth profile
of the earth one might
minimize the squared difference between
some theoretical quantities (say, the frequencies of free oscillation)
and the observed quantities.
By astronomical means,
total mass and moment of inertia of the earth
are very well known.
If the earth is divided into
arbitrarily thin shells of equal thickness,
what are the two astronomical constraint equations
on the layer densities
*p*_{i}? If the least-squares problem is nonlinear (as it often is) it may be linearized by assuming that a given set of densities*p*_{i}is a good guess which satisfies the constraints and doing least squares for the perturbation*dp*_{i}. What are the constraint equations on*dp*_{i}?

10/30/1997