REVIEW

Inverse problems obtain an estimate of a model $\bf m$, given some data $\bf d$ and an operator $\bf L$ relating the two. We can write our estimate of the model as minimizing the objective function in a least-squares sense,

$$f(\bf m) = \Vert\bf d- \bf L\bf m\Vert^2 .$$ (1)

We can think of this same minimization in terms of fitting goals as

$$\bf 0\approx \bf r_{} = \bf d- \bf L\bf m,$$ (2)

where $\bf r_{}$ is a residual vector.

Bayesian theory tells us Tarantola (1987) that convergence rate and the final quality of the model is improved the closer $\bf r_{}$ is to being Independent Identically Distributed (IID). If we include the inverse noise covariance $\bf N$ in our inversion our residual beomes IID,  

$$\bf 0\approx \bf r_{} = \bf N( \bf d- \bf L\bf m) .$$ (3)

A regularized inversion problem can be thought of as a more complicated version of (3) with an expanded data vector and an additional covariance operator,

$$\begin{eqnarray} \bf 0&\approx&\bf r_{d} = \bf N_{noise} ( \bf d- \bf L\bf m) \n... ...0&\approx&\bf r_{m} =\epsilon \bf N_{model} ( \bf 0- \bf I\bf m) .\end{eqnarray}$$ (4)

In this new formulation

$\bf r_{d}$

is the residual from the data fitting goal,

$\bf r_{m}$

is the residual from the model styling goal,

$\bf N_{noise}$

is the inverse noise covariance,

$\bf N_{model}$

is the inverse model covariance,

$\bf I$

is the identity matrix, and

$\epsilon$

is a scalar that balances the fitting goals against each other.

Normally we think of $\bf N_{model}$ as the regularization operator $\bf A$. Simple linear algebra leads to a more standard set of fitting goals:

$$\begin{eqnarray} \bf 0&\approx&\bf r_{d} = \bf N_{noise} ( \bf d- \bf L\bf m) \nonumber \\ \bf 0&\approx&\bf r_{m} = \epsilon \bf A\bf m.\end{eqnarray}$$ (5)

The problem with this approach is that we never know the true inverse noise or model covariance and therefore are only capable of applying approximate forms of these matrices.