OPPORTUNITIES FOR SMART DIRECTIONS

Recall the fitting goals (10)   Without preconditioning we have the search direction

(11)

and with preconditioning we have the search direction

(12)

The essential feature of preconditioning is not that we perform the iterative optimization in terms of the variable $\bold p$.The essential feature is that we use a search direction that is a gradient with respect to not .Using we have .This enables us to define a good search direction in model space.

(13)

Define the gradient by and notice that . 

The search direction (14) shows a positive-definite operator scaling the gradient. Each component of any gradient vector is independent of each other. All independently point a direction for descent. Obviously, each can be scaled by any positive number. Now we have found that we can also scale a gradient vector by a positive definite matrix and we can still expect the conjugate-gradient algorithm to descend, as always, to the ``exact'' answer in a finite number of steps. This is because modifying the search direction with is equivalent to solving a conjugate-gradient problem in $\bold p$.