Forward residuals

The forward residuals $\varepsilon_{n,T}$ can be expressed in terms of orthogonal projections. Effectively, defining P_1,n,T=A_1,n,T(A'_1,n,TA_1,n,T)^-1A'_1,n,T, we have:

$$\varepsilon_{n,T}=y_T-A_{1,n,T}f_{n,T}=(I-P_{1,n,T})y_T \;.$$

Because P²_1,n,T=P'_1,n,T=P_1,n,T, the operator P_1,n,T is an orthogonal projector. It projects the data vector y_T on the space Y_1,n,T spanned by the columns of A_1,n,T. These columns are simply the past observations (or shifted data), Zy_T to Zⁿy_T, where Z is the unit-delay shift operator.

The predictable part of y_T is P_1,n,Ty_T, the orthogonal projection of y_T on Y_1,n,T. The residuals $\varepsilon_{n,T}$ are then orthogonal to this space: they are called the orthogonal complement of y_T on Y_1,n,T. Their geometric interpretation is illustrated in Figure . Defining the operator $P^{\perp}_{1,n,T}=I-P_{1,n,T}$, the forward residuals $\varepsilon_{n,T}$ are $P^{\perp}_{1,n,T}y_T$; $P^{\perp}_{1,n,T}$ is also an orthogonal projector.

So, we need to compute P_1,n,Ty_T. But this projection is easy to compute, if we already know an orthogonal basis $u_1,\cdots,u_n$ of the space Y_1,n,T. Effectively, the projection of y_T is:

$$P_{1,n,T}y_T={y'u_1\over u'_1u_1}u_1 + \cdots + {y'u_n\over u'_nu_n}u_n \;.$$

The backward residuals will in fact form this orthogonal basis.