``Useful'' constraint equations

There is a theory for general constraints in quadratic form minimization. I haven't found the theory to be useful in any application I've run into so far, but it should be useful for writing erudite theoretical articles.

Constraint equations are an underdetermined set of equations, say $\bold d = \bold G\bold x$(the number of components in $\bold x$ exceeds that in $\bold d$), that must be solved exactly while some other set is solved in the least-squares sense, say $\bold y \approx \bold B\bold x$.This is formalized as  

$$\min_{\bold x} \ \{ Q_C(\bold x) \ =\ \lim_{\epsilon \righta... ...on} (\bold d - \bold G\bold x)' (\bold d - \bold G\bold x) ] \}$$ (1)

In my first book, ``Fundamentals of Geophysical Data Processing,'' I minimized Q_C by power series letting $\bold x = \bold x^{(0)}+\epsilon\,\bold x^{(1)}$ and hence $Q_C = Q^{(0)} + \epsilon Q^{(1)}+ \cdots$.I minimized both Q⁽⁰⁾ and Q⁽¹⁾ w.r.t. x⁽⁰⁾ and x⁽¹⁾. With a page of algebra this leads to the system of equations  

$$\left[ \begin{array} {cc} \bold B'\bold B & \bold G' \\ ... ...{array} {c} \bold B' \bold y \\ \bold d \end{array} \right]$$ (2)

where $\bold x^{(1)}$ has been superceded by the variable $\bold \lambda = \bold G \bold x^{(1)}$which has fewer components and where $\bold x^{(0)}$ has simply been replaced by $\bold x$.The second of the two equations by itself shows that the constraints are satisfied. But from (2) it is not clear that (1) is minimized.

The great mathematician Lagrange apparently looked at the result, equation (2), and realized that he could have it far more simply by extremalizing the following quadratic form  

$$Q_L(\bold x,\bold \lambda) {=} (\bold y-\bold B\bold x)' (\b... ...old x)' \bold \lambda + \bold \lambda' (\bold d-\bold G\bold x)$$ (3)

as you can quickly verify by setting to zero the derivatives w.r.t. $\bold x'$ and $\bold \lambda'$.Naturally everyone prefers to handle constraints by Lagrange's method. Unfortunately Lagrange failed to pass on to the teachers of this world an intuitive reason why extremalizing (3) gives the same result as extremalizing (1). Lagrange's quadratic form is not even positive definite (it cannot be written as something times its conjugate). In honor of Lagrange, the variables $\bold \lambda$ have come to be known as Lagrange multipliers.

When I sat down to write this book I promised myself to include no equations without a practical use, so now I'll explain how you can use these equations to prepare pedantic articles on geophysical inversion. You can derive marvelously complicated equations by formally solving equation (2) as you would any $2\times 2$ set. The solution balloons up in size, particularly since the elements of equation (2) are submatrices and they cannot be commuted. You justify these equations by stating that geophysical measurements are finite in number and thus all equations with measurements are constraints to an infinite-dimensional optimization problem for finding the earth model which is a continuous function. You can obfuscate further by introducing weighting functions that are inverse covariance matrices of the unknowns, and thus fully as unknown as the unknowns themselves.