Definitions

An set of vectors is an orthogonal set if each pair of vectors is orthogonal to eachother such that

(1)
\begin{align} \begin{Bmatrix} u_1,...,u_n \end{Bmatrix}\\ u_1 \cdot u_2 = 0, u_1 \cdot u_2 = 0, ... , u_{n-1\text{}} \cdot u_n = 0 \end{align}

In addition, if there is a subspace S formed by an orthogonal set, that set is linearly independent and is thus a basis for S. Therefore, if there exists a vector $\vec{y}$ is S, then $\vec{y}$ can be formed by the orthogonal set

(2)
\begin{align} \vec{y} = c_1\vec{u_1}+c_2\vec{u_2}+...+c_n\vec{u_n} \end{align}

We can therefore multiply each term by $\vec{u_1}$ for instance, and since all other u's are perpendicular to $u_1$, all other terms drop out except $\vec{y} \cdot \vec{u_1} = c_1 \vec{u_1}^2$. This can be simplified and $c_1$ can be found using

(3)
\begin{align} \frac{\vec{y} \cdot \vec{u_1}}{\vec{u_1}^2} = c_1 \end{align}

This is true for all c's so we now have the tools to find any vector in S by a linear combination of the orthogonal basis.


Orthogonal Projection

Now, if we're looking to express $\vec{y}$ as a sum of two vectors, one which is a multiple of $\vec{u}$ and one perpendicular to $\vec{u}$, we must apply some of what we just learned. We know the scalar $c_1$ to create a vector in S that is a component of $\vec{y}$ is $\frac{\vec{y} \cdot \vec{u_1}}{\vec{u_1}^2}$. Using this to scale $\vec{u_1}$ which we'll call $\hat{y}$, we find

(4)
\begin{align} \vec{y} = \hat{y} + \vec{v} \end{align}

where $\vec{v}$ is the perpendicular vector such that

(5)
\begin{align} \vec{y} - \hat{y} = \vec{v} \\ \vec{v} \cdot \hat{y} = 0 \end{align}

Orthonormal Sets

An Orthonormal set is a set of orthogonal vectors whose magnitudes are all 1.

To normalize a vector, just divide each component by the magnitude of the vector so…

(6)
\begin{align} \hat{x} = \frac{\vec{x}}{|\vec{x}|} \end{align}

Then the orthonormal set has the property $U^TU = I$. Thus, $U^T = U^{-1}$.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License