**Observation**: As we observed in Matrix Operations, two non-null vectors *X* = [*x _{i}*] and

*Y*= [y

_{i}] of the same shape are

**orthogonal**if their dot product is 0, i.e. 0 =

*X ∙ Y*=

*x*y

_{i}_{i}. Note that if

*X*and

*Y*are

*n*× 1 column vectors, then

*X ∙ Y*=

*X*, while if

^{T}Y = Y^{T}X*X*and

*Y*are 1 ×

*n*row vectors, then

*X ∙ Y = XY*. It is easy to see that (

^{T }= YX^{T}*cX*) ∙

*Y*=

*c*(

*X ∙ Y*), (

*X + Y*) ∙

*Z*=

*X ∙ Z*+

*Y ∙ Z*,

*X ∙ X =*> 0 and other similar properties of the dot product.

**Property 1**: If *A* is an *m × n* matrix, *X* is an *n* *×* 1 vector and *Y* is an *m* *×* 1 vector, then

(*AX*) ∙ *Y* = *X* ∙ (*A ^{T}Y*)

Proof: (*AX*) ∙ *Y *= (*AX*)* ^{T}Y* = (

*X*)

^{T}A^{T}*Y*=

*X*(

^{T}*A*) =

^{T}Y*X*∙ (

*A*)

^{T}Y**Property 2**: If *X*_{1}, …, *X _{m}* are mutually orthogonal vectors, then they are independent.

Proof: Suppose *X*_{1}, …, *X _{m}* are mutually orthogonal and let = 0. Then for any

*j*, 0 =

*X*∙ = =

_{j}*c*(

_{j}*X*) since

_{j}∙ X_{j}*X*= 0 when

_{j}∙ X_{i}*i ≠ j*. Thus,

*c*(

_{j}*X*) = 0. But since

_{j}∙ X_{j}*X*> 0, it follows that

_{j}∙ X_{j}*c*= 0. Since this is true for any

_{j}*j*,

*X*

_{1}, …,

*X*are independent.

_{m}**Property 3**: Any set of *n* mutually orthogonal *n* × 1 column vectors is a basis for the set of *n* × 1 column vectors. Similarly, any set of *n* mutually orthogonal 1 × *n* row vectors is a basis for the set of 1 × *n* row vectors.

Proof: This follows by Corollary 4 of Linear Independent Vectors and Property 2.

**Observation**: Let *C _{j}* be the

*j*th column of the identity matrix

*I*. As we mentioned in the proof of Corollary 4 of Linear Independent Vectors, it is easy to see that for any

_{n}*n*,

*C*

_{1}, …,

*C*forms a basis for the set of all

_{n}*n*× 1 column vectors. It is also easy to see that the

*C*

_{1}, …,

*C*are mutually orthogonal.

_{n}We next show that any set of vectors has a basis consisting of mutually orthogonal vectors.

**Theorem 1** (**Gram-Schmidt Process**): Suppose *X*_{1}, …, *X _{m}* are independent

*n*× 1 column vectors. Then we can find

*n*× 1 column vectors

*V*

_{1}, …,

*V*which are mutually orthogonal and have the same span.

_{m}Proof: We show how to construct the *V*_{1}, …, *V _{m}* from the

*X*

_{1}, …,

*X*as follows.

_{m}Define *V*_{1}, …, *V _{m} *as follows:

Proof: We first show that the *V _{k}* are mutually orthogonal by induction on

*k*. The case where

*k*= 1 is trivial. Assume that

*V*

_{1}, …,

*V*are mutually orthogonal. To show that

_{k}*V*

_{1}, …,

*V*

_{k+}_{1}are mutually orthogonal, it is sufficient to show that

*V*

_{k}_{+1}∙

*V*= 0 for all

_{i}*i*where 1 ≤

*i ≤ k.*Using the induction hypothesis that

*V*∙

_{j}*V*= 0 for 1 ≤

_{i}*j ≤ k*and

*j ≠ i*and

*V*∙

_{i}*V*≠ 0 (since

_{i}*V*≠ 0), we see that

_{i}This completes the proof that *V*_{1}, …, *V _{m}* are mutually orthogonal. By Property 2, it follows that

*V*

_{1}, …,

*V*are also independent.

_{m}We next show that the span of *V*_{1}, …, *V _{k}* is a subset of the span of

*X*

_{1}, …,

*X*for all

_{k}*k ≤ m*. The result for

*k*= 1 is trivial. We assume the result is true for

*k*and show that it is true for

*k*+ 1. Based on the induction hypothesis, it is sufficient to show that

*V*

_{k+1}can be expressed as a linear combination of

*X*

_{1}, …,

*X*

_{k}_{+1}. This is true since by definition

and by the induction hypothesis all the *V _{j}* can be expressed as a linear combination of the

*X*

_{1}, …,

*X*.

_{k}By induction, we can now conclude that the span of *V*_{1}, …, *V _{m}* is a subset of the span of

*X*

_{1}, …,

*X*, and so trivially

_{m}*V*

_{1}, …,

*V*are elements in the span of

_{m}*X*

_{1}, …,

*X*But since the

_{m}*V*

_{1}, …,

*V*are independent, by Property 3 of Linear Independent Vectors, we can conclude that the span of

_{m}*V*

_{1}, …,

*V*is equal to the span of

_{m}*X*

_{1}, …,

*X*.

_{m}**Corollary 1**: For any closed set of vectors we can construct an orthogonal basis

Proof: By Corollary 1 of Linear Independent Vectors, every closed set of vectors *V* has a basis. In fact, we can construct this basis. By Theorem 1, we can construct an orthogonal set of vectors which spans the same set. Since this orthogonal set of vectors is independent, it is a basis for *V*.

**Definition 1**: A set of vectors is **orthonormal** if the vectors are mutually orthogonal and each vector is a unit vector.

**Corollary 2**: For any closed set of vectors we can construct an orthonormal basis

Proof: If *V*_{1}, …, *V _{m}* is the orthogonal basis, then

*Q*

_{1}, …,

*Q*is an orthonormal normal basis where

_{m}**Observation**: The following is an alternative way of constructing *Q*_{1}, …, *Q _{m}* (which yields the same result).

Define *V*_{1}, …, *V _{m}* and

*Q*

_{1}, …,

*Q*from

_{m}*X*

_{1}, …,

*X*as follows:

_{m}**Definition 2**: A matrix *A* is **orthogonal** if *A*^{T}*A* = *I*.

**Observation**: The following property is an obvious consequence of this definition.

**Property 4**: A matrix is orthogonal if and only if all of its columns are orthonormal.

**Property 5**: If *A* is an *m × n* orthogonal matrix and *B* is an *n × p* orthogonal then *AB* is orthogonal.

Proof: If *A* and *B* are orthogonal, then

(*AB*)^{T}(*AB*) = (*B*^{T}*A*^{T})(*AB*) = *B*^{T}(*A*^{T}*A*)*B *= *B*^{T}*IB *= *B*^{T}*B* =* I *

**Example 1**: Find an orthonormal basis for the three column vectors which are shown in range A4:C7 of Figure 1.

**Figure 1 – Gram Schmidt Process**

The columns in matrix *Q* (range I4:K7) are simply the normalization of the columns in matrix *V*. E.g., the third column of matrix *Q* (range K4:K7) is calculated using the array formula G4:G7/SQRT(SUMSQ(G4:G7)). The columns of *V* are calculated as described in Figure 2.

**Figure 2 – Formulas for V in the Gram Schmidt Process**

The orthonormal basis is given by the columns of matrix *Q*. That these columns are orthonormal is confirmed by checking that *Q ^{T}Q = I* by using the array formula =MMULT(TRANSPOSE(I4:K7),I4:K7) and noticing that the result is the 3 × 3 identity matrix.

We explain how to calculate the matrix *R* in Example 1 of QR Factorization.

**Property 6**: If *A* is an orthogonal square matrix, then

*A*^{T}= A^{-1}*AA*^{T}= I*A*is orthogonal^{T}- det
*A*= ±1 (the converse is not necessarily true)

Proof:

a) Since *A ^{T}* is a left inverse of

*A*, by Property 5 of Rank of a Matrix,

*A*is the inverse of

^{T}*A*

b) This follows from (a)

c) This follows from (b) since (

*A*)

^{T}

^{T}A^{T}= AA^{T}= Id) By Property 1 of Determinants and Linear Equations, |

*A*|

^{2}= |

*A*| ∙ |

*A*| = |

*A*| ∙ |

^{T}*A*|= |

*A*| = |

^{T}A*I*| = 1. Thus |A|=±1.

**Property 7**: A square matrix is orthogonal if and only if all of its rows are orthonormal.

Proof: By Property 4 and 6b.

**Observation**: Multiplying a vector *X* by an orthogonal matrix *A* has the effect of rotating or reflecting the vector. Thus we can think of *X* as point in n-space which is transformed into the point *AX* in n-space. Note that the distance between the point *X* and the origin (i.e. the length of vector *X*) is the same as the distance between *AX* and the origin (i.e. the length of vector *AX*), which can be seen from

Also multiplication of two vectors by *A* also preserves the angle between the two vectors, which is characterized by the dot product of the vectors (since the dot product of two unit vectors is the cosine of this angle), as can be seen from

Note too that *A* represents a rotation if det *A* = +1 and a reflection if det *A* = -1.