next up previous contents
Next: Spaces of Random Variables Up: Hilbert Spaces Previous: Hilbert Spaces

State Space Decomposition

Of great importance in all that follows are the fundamental concepts of orthogonality and orthogonal projection. We briefly characterize them here, in the familiar euclidean case, in order to more fully appreciate the brain twisting beauty of their natural extension into infinite dimensional probability spaces a little later.

Two vectors (in the same vector space) are orthogonal if their inner product is zero. This is written tex2html_wrap_inline721

In a Euclidean space, this simply means the (smallest) angle between them is a right angle. We will also want to talk about orthogonality relations between vectors and entire spaces. Following the above definition naturally,

Given a vector x and a closed subspace S, both in some larger Hilbert Space, we say x is orthogonal to S, written tex2html_wrap_inline731 , if



Thus, as a quick example, in tex2html_wrap_inline715 , the x-axis is orthogonal to the yz-plane, but not to the plane x=y

Given a Hilbert Space tex2html_wrap_inline737 , a closed subspace tex2html_wrap_inline739 , and a vector tex2html_wrap_inline741 , we may uniquely decompose x into orthogonal components in tex2html_wrap_inline745 and its orthogonal complement. That is,


where tex2html_wrap_inline747 and tex2html_wrap_inline749 are uniquely determined by tex2html_wrap_inline745 , and are orthogonal.

Equivalently, we might say


By tex2html_wrap_inline753 we will mean the Linear Span of tex2html_wrap_inline755 .

The ability to decompose any vector into orthogonal components leads naturally to the definition of a special linear transormation: the orthogonal projection onto a subspace. By the above theorem, we know that given any vector x and subspace S there exists a unique element, say y, in S such that x-y is orthogonal to S. We call it the orthogonal projection of x on S, written tex2html_wrap_inline773 .

This should seem familiar. We perform these decomositions under various rubrics and aliases all the time. In elementary mechanics, we decompose a particle's momentum and position into independant components along the appropriate axes. Taking the real part of a complex valued function would seem to be no more than the projection of the plane onto the line.

Thomas Pynchon supplies us with a further example of this in his book ``V.''. If we imagine that we are looking at the rotation of the Earth about the Sun from a point in the plane of the ecliptic, from far enough away that depth information is lost, we would observe the rotation as motion in a one dimensional space. We have projected the two-dimensional system of the Earth's rotation gif onto a one-dimensional subspace - the line crossing through the center of the sun perpendicular to the line of our observation. Note that when we talk about these projections, we're generally talking about the relationship with the observer - we're not changing the system at all, just how we're looking at it.

The following facts about projections will be used quite often:

  1. Linearity


  2. Minimizes Induced Norm of ``Error''.


    tex2html_wrap_inline775 , where the norm is induced by the inner product, ie, tex2html_wrap_inline777 .

  3. Orthogonal ``Error''.


    tex2html_wrap_inline775 .

  4. Inner Product Equality (x,s)=(y,s) for all s in S.

These results find direct application in so called Gramm-Schmidt Orthogonalization. Often, the space generated by a set of vectors will be of greater interest to us than the vectors themselves. This being the case, we may seek out another set of vectors generating the same space which is easier to work with. Generally, we will wish to reduce a given basis down to an orthogonal basis with identical span. We do this as follows, given tex2html_wrap_inline781 :


By simple application of the above properties of projections, we see that this definition will indeed yield the fact that


as desired, so that the set is pairwise orthogonal.

next up previous contents
Next: Spaces of Random Variables Up: Hilbert Spaces Previous: Hilbert Spaces

Scot Free Kennedy
Sat Sep 13 00:27:51 PDT 1997