Orthogonality and Gram Schmidt process

Orthogonal bases
Let $\{v_1,\cdots,v_n\}$ be an orthogonal basis of the Euclidean space $V$. We want to find coordinates of the vector $u$ in this basis, namely the numbers $a_1,\cdots,a_n$ such that $$u=a_1v_1+\cdots+a_nv_n.$$ We can take the inner product with $v_1$, so $$\langle u,v_1\rangle=a_1\langle v_1,v_1\rangle+\cdots+a_n\langle v_n,v_1\rangle.$$ Since $v_1,\cdots,v_n$ are pairwise orthogonal, we have $\langle v_i,v_j \rangle=0$ for $i\neq j$. Thus, $\langle u,v_1\rangle=a_1\langle v_1,v_1\rangle \Rightarrow a_1=\dfrac{\langle u,v_1\rangle}{\langle v_1,v_1\rangle}$. Similarly, multiplying by $v_2,\cdots,v_n$, we have other coefficients $$a_2=\dfrac{\langle u,v_2\rangle}{\langle v_2,v_2\rangle},\cdots,a_n=\dfrac{\langle u,v_n\rangle}{\langle v_n,v_n\rangle}.$$ These coefficients are called Fourier coefficients of the vector $u$ with respect to basis $\{v_1,\cdots,v_n\}$.

Theorem: Let $\{v_1,\cdots,v_n\}$ be an orthogonal basis of the Euclidean space $V$. Then for any vector $u$, $$u=\dfrac{\langle u,v_1\rangle}{\langle v_1,v_1\rangle}v_1+\cdots+\dfrac{\langle u,v_n\rangle}{\langle v_n,v_n\rangle}v_n.$$ This expression is called Fourier decomposition and can be obtained in any Euclidean space.

Projections

The projection of the vector $v$ along the vector $w$ is the vector $\text{proj}_w v=cw$ such that $u=v-cw$ is orthogonal to $w$. We have $\langle u,w \rangle=0$, which means $$\langle v-cw,w \rangle=0 \iff \langle v,w \rangle-c\langle w,w \rangle=0 \iff c=\dfrac{\langle v,w \rangle}{\langle w,w \rangle}.$$ Thus the orthogonal complement $u$ is $$u=v-\text{proj}_w v=v-\dfrac{\langle v,w \rangle}{\langle w,w \rangle}w.$$ With this formula, we can find the distance between a point and a plane. Now, let's generalise our constructions.

Theorem: Let $W$ be a subspace with orthogonal basis $\{w_1,\cdots,w_n\}$. The projection $\text{proj}_w v$ of any vector $v$ along $W$ is $$\text{proj}_w v=\dfrac{\langle v,w_1 \rangle}{\langle w_1,w_1 \rangle}w_1+\cdots+\dfrac{\langle v,w_n \rangle}{\langle w_n,w_n \rangle}w_n.$$ [This follows from the Fourier decomposition above.] In particular, $u=v-\text{proj}_w v$ is orthogonal to the subspace $W$.

Proof: Take the inner product of $u$ with $w_i$, $$\langle u,w_i \rangle=\langle v,w_i \rangle-\bigg(\dfrac{\langle v,w_1 \rangle}{\langle w_1,w_1 \rangle}\langle w_1,w_i \rangle+\cdots+\dfrac{\langle v,w_n \rangle}{\langle w_n,w_n \rangle}\langle w_n,w_i \rangle\bigg).$$ Since $\langle w_i,w_j \rangle =0$ for $i\neq j$, we have $$\begin{align}\langle u,w_i \rangle&=\langle v,w_i \rangle-\dfrac{\langle v,w_i \rangle}{\langle w_i,w_i \rangle} \langle w_i,w_i \rangle\\&=\langle v,w_i \rangle-\langle v,w_i \rangle\\&=0.\end{align}$$ Thus $u$ is orthogonal to every $w_i$, and so it is orthogonal to $W$. $\Box$

Gram Schmidt orthogonalisation process
Let $\{v_1,\cdots,v_n\}$ be a basis in the Euclidean space. We can construct orthogonal basis $\{w_1,\cdots,w_n\}$ of this space. $$\begin{align}w_1&=v_1\\w_2&=v_2-\dfrac{\langle v_2,w_1\rangle}{\langle w_1,w_1\rangle}w_1\\w_3&=v_3-\dfrac{\langle v_3,w_1\rangle}{\langle w_1,w_1\rangle}w_1-\dfrac{\langle v_3,w_2\rangle}{\langle w_2,w_2\rangle}w_2\\ \cdots\\ w_n&=v_n-\dfrac{\langle v_n,w_1\rangle}{\langle w_1,w_1 \rangle}w_1-\dfrac{\langle v_n,w_2\rangle}{\langle w_2,w_2\rangle}w_2-\cdots-\dfrac{\langle v_n,w_{n-1}\rangle}{\langle w_{n-1},w_{n-1}\rangle}w_{n-1}.\end{align}$$

[...]