Orthogonal functions

Legendre polynomials (important in spherical harmonics)
There are three general ways of forming these polynomials.
1. Schmidt orthogonalisation of $\{x^n\}$ on $[-1,1]$
Let $p_1(x)=\dfrac{1}{\lVert 1\rVert}$. Then $\lVert 1\rVert^2=\int_{-1}^1 1dx=2$ and thus $\boxed{p_1(x)=\dfrac{1}{\sqrt{2}}}$. Notice that $\int_{-1}^1 f(x)dx=0$ for any odd function $f(x)$. So $x$ is orthogonal to $1$. We then find $p_2(x)=\dfrac{x}{\lVert x \rVert}$. Since $\lVert x \rVert^2=\int_{-1}^1 x^2dx=\dfrac{2}{3}$, we have $\boxed{p_2(x)=\sqrt{\dfrac{3}{2}}x}$. Although $x^2$ is orthogonal to $x$ (and hence $p_2$), it is not orthogonal to $1$. Let $q_3(x)=x^2-\langle x^2,p_1 \rangle p_1$, and then normalise to get $p_3=\dfrac{q_3}{\lVert q_3 \rVert}$. Now $\langle x^2,p_1 \rangle=\dfrac{1}{\sqrt{2}}\int_{-1}^1 x^2 dx=\dfrac{\sqrt{2}}{3}$, so $q_3(x)=x^2-\dfrac{\sqrt{2}}{3}\dfrac{1}{\sqrt{2}}=x^2-\dfrac{1}{3}$. $\lVert q_3 \rVert^2=\int_{-1}^1 (x^2-\dfrac{1}{3})^2dx=\dfrac{8}{45}$. Thus $\boxed{p_3(x)=\sqrt{\dfrac{45}{8}}(x^2-\dfrac{1}{3})=\sqrt{\dfrac{5}{2}}(\dfrac{3}{2}x^2-\dfrac{1}{2})}$.
The whole orthonormal set of functions is $\{\sqrt{\dfrac{2n+1}{2}}P_n(x)\}$, where $P_n(x)$ signifies the Legendre polynomial of rank $n$.

2. Solution of Legendre's differential equation
$$\dfrac{d}{dx}[(1-x^2)\dfrac{df}{dx}]+l(l+1)f=0,$$ where $l$ is a positive integer or zero.
One can verify that the polynomials that are generated by orthogonalisation of $\{x^n\}$ on $[-1,1]$ are solutions of this differential equation. Now we attempt a power series solution $\sum_n a_nx^n$ for $f$. Then $$\dfrac{d}{dx}[(1-x^2)\sum_n na_nx^{n-1}]+[l(l+1)\sum_n a_nx^n]=0\\ \dfrac{d}{dx}[\sum_n na_nx^{n-1}-\sum_n na_nx^{n+1}]+\sum_n l(l+1)a_nx^n=0\\ \sum_n n(n-1)a_nx^{n-2}-\sum_n n(n+1)a_nx^n+\sum_n l(l+1)a_nx^n=0.$$ Collecting all similar powers of $x$, we obtain $$\sum_n x^n\underbrace{[(n+2)(n+1)a_{n+2}-n(n+1)a_n+l(l+1)a_n]}_{(*)}=0.$$ Since the functions $\{x_n\}$ are linearly independent, $(*)$ must be zero. This gives the recurrence relation for the expansion coefficients: $$a_{n+2}=\dfrac{[n(n+1)-l(l+1)]a_n}{(n+2)(n+1)}.$$ This equation allows us to find every other coeffcient when we are given the first. For example, if $a_0=1$, then $a_2=\dfrac{-l(l+1)}{2},a_4=\dfrac{6-l(l+1)}{12}\cdot \dfrac{-l(l+1)}{2}$, and so on.

If $l$ is even, eventually $n$ will equal $l$ for some term, then $n(n+1)$ will equal $l(l+1)$ and the following coefficients will also equal zero. So the even terms of the power series cut off at $n=l$ whereas the infinite series of odd powers diverges at $x=\pm 1$. The finite series of even powers gives $$\begin{align}l=0\quad &a_0=1\quad &f_0=1\\ l=2\quad &a_0=1\\ &a_2=-3\quad &f_2=-3x^2+1,\end{align}$$ which are proportional to the Legendre polynomials derived by orthogonalisation. We can choose $a_0$ such that the polynomial $f_n=1$ at $x=+1$. Then $$\begin{align}l=0\quad &a_0=1\quad &P_0=1\\l=2\quad &a_0=\dfrac{-1}{2}\\&a_2=\dfrac{3}{2}\quad &P_2=\dfrac{3}{2}x^2-\dfrac{1}{2}.\end{align}$$
If $l$ is odd, the series in odd powers terminates at $n=l$, and the series in even powers is an infinite series, divergent at $x=\pm 1$. Again, we choose $a_0$ such that $f_n(+1)=1$. Then $$\begin{align}l=1\quad &a_1=1\quad &P_1=x\\ l=3\quad &a_1=\dfrac{-3}{2}\\ &a_3=\dfrac{5}{2}\quad &P_3=\dfrac{5}{2}x^3-\dfrac{3}{2}x.\end{align}$$
3. Generating function
A generating function for a set of functions is a function of two variables such that when this function is expanded as power series in one of the variables, the coefficients are the set of functions in other variable. Often many useful relations may be derived from a generating function; nonetheless, there are no general methods for constructing a generating function.

The generating function for Legendre polynomials is $$F(x,t)=(1-2xt+t^2)^{-1/2}=\sum_n P_n(x)t^n.$$ It can be expanded as a binomial series $$F(x,t)=1+\dfrac{1\cdot 1}{2\cdot 1!}(2xt-t^2)+\dfrac{1\cdot 3\cdot 1}{2\cdot 2\cdot 2!}(2xt-t^2)^2+\frac{1\cdot 3\cdot 5\cdot 1}{2\cdot 2\cdot 2\cdot 3!}(2xt-t^2)^3+\cdots$$ and then rearranged in powers of $t$: $$\begin{align}F(x,t)&=1+(x)(t)+(\dfrac{-1}{2}+\dfrac{3}{2}x^2)(t^2)+(\dfrac{-3}{2}x+\dfrac{5}{2}x^3)(t^3)+\cdots\\&=\sum_n P_n(x)t^n.\end{align}$$
Hermite polynomials (important in Quantum Mechanics)
They are solutions of the differential equation
$$y''-2xy'+2py=0\quad y(0)=1,\quad y'(0)=1.$$
Chebyshev polynomials (important in numerical analysis)
They are solutions of the differential equation
$$(1-x^2)y''-xy'+n^2y=0\quad y(0)=1,\quad y'(0)=0.$$
They can also be represented as $$T_n(x)=\cos (n\cos^{-1} x).$$
Applications

$\min \int_{-1}^1 |x^3-a-bx-cx^2|dx$
Geometrically, the minimum, call it $Q$, is the square of the distance from $x^3$ to the subspace $S$ spanned by the functions $1,x,x^2$. We can compute the orthogonal projection $\phi(x)$ of $x^3$ into $S$ and use the Pythagoras' theorem: $Q^2=\lVert x^3 \rVert^2-\lVert \phi(x)\rVert^2$. We first need to transform the basis $\{1,x,x^2\}$ into an orthonormal one, namely $\{p_1,p_2,p_3\}$ as above. $\phi(x)=\langle x^3,p_1\rangle p_1+\langle x^3,p_2 \rangle p_2+\langle x^3,p_3 \rangle p_3$. $x^3$ is orthogonal to $p_1$ and $p_2$ by oddness. Since $\langle x^3,p_2 \rangle=\sqrt{\dfrac{3}{2}}\int_{-1}^1 x^3x dx=\sqrt{\dfrac{3}{2}}\dfrac{2}{5}=\dfrac{6}{5}$, we have $\lVert\phi(x)\rVert^2=\dfrac{6}{25}\lVert p_2 \Vert^2=\dfrac{6}{25}$. With $\lVert x^3 \Vert^2=\int_{-1}^1 x^6 dx=\dfrac{2}{7}$, we have $Q=\dfrac{2}{7}-\dfrac{6}{25}=\dfrac{8}{175}$.

physical applications [later]

The Legendre polynomials are important in some quantum-chemical problems. They are the basis for the wave functions for angular momentum, and thus occur in problems involving spherical motion, such as that of the electron in a hydrogen atom or the rotations of a molecule. They are also important in describing the angular dependence of one-electron atomic orbitals; this dependence, in turn, forms the basis of the geometry of chemical compounds.

Reference:
Mathematics for Quantum Chemistry by Jay Martin Anderson

Mathematics for quantum chemistry

Introduction
Quantum mechanics grew up from two different points of view, which represent two analogous mathematical formulations of eigenvalue problems. One is the wave mechanics of Schrodinger. In wave mechanics, operators are differential expressions, such as $\dfrac{d}{dx}$, and the eigenvalue equation takes the form of a differential equation, and relies on the calculus for its solution. The other formulation is the matrix mechanics of Heisenberg, in which operators are represented by algebraic entities called matrices; instead of a function in the eigenvalue equation, the matrix operator operates on a vector $\zeta$ to transform $\zeta$ into a vector parallel to $\zeta$, but $q$ times as long: $$Q\zeta=q\zeta.$$ This is the matrix-mechanical formulation of the eigenvalue problem. The solution of this form of the eigenvalue problem relies on algebra.

These two approaches to quantum mechanical problems are deeply interrelated; the work of Dirac shows the underlying equivalence of the two points of view, as well as of the corresponding mathematical techniques. [...]

Orthogonal functions
Definition: The expansion interval is the range of the independent variables assumed by the functions under consideration.

Definition: The inner product of the two complex-valued functions $f$ and $g$ of a continuous variable on their expansion interval $[a,b]$ is $$\langle f|g \rangle=\int_a^b f(x)^*g(x)dx.$$
The order is not important for real-valued functions but important for complex-valued functions: $$\begin{align}\langle g|f \rangle&=\int g(x)^*f(x)dx\\&=\big(\int f(x)^*g(x)dx\big)^*\\ &=\langle f|g \rangle^*.\end{align}$$

Definition: Two functions $f(x)$ and $g(x)$ are said to be orthogonal on the interval $[a,b]$ if their inner product on $[a,b]$ is zero: $$\langle f|g \rangle=\int_a^b f^*g=0=\int_a^b g^*f=\langle g|f \rangle.$$

Definition: The norm of a function on the interval $[a,b]$ is the inner product of the function with itself, and may be symbolised by $N$: $$N(f)=\langle f|f \rangle=\int_a^b f^*g=0=\int_a^b f^*f.$$

The norm of a function is a real, positive quantity: $$f^*f=(\text{Re}\:f-i\text{Im}\:f)(\text{Re}\:f+i\text{Im}\:f)=(\text{Re}\:f)^2+(\text{Im}\:f)^2.$$

Definition: A function is said to be normalised if its norm is one: $\langle f|f \rangle=1$.
Suppose $f$ has a norm $N$. Then the function $\dfrac{f}{\sqrt{N}}$ will have a norm of one, since $$\langle \dfrac{f}{\sqrt{N}}|\dfrac{f}{\sqrt{N}} \rangle=\dfrac{1}{N}\langle f|f \rangle=\dfrac{N}{N}=1.$$
Reminder: The expansion interval must be specified before a statement about orthogonality or normality can be made. On $[-1,1]$, $\langle x|x^2 \rangle=\int_{-1}^1 x^*x^2 dx=\int_{-1}^1 x^3 dx=\dfrac{x^4}{4}|_{-1}^1=0$, so we can say that on $[-1,1]$, $x$ and $x^2$ are orthogonal functions. However, on $[0,1]$, $\langle x|x^2 \rangle=\int_0^1 x^3 dx=\dfrac{x^4}{4}|_0^1=\dfrac{1}{4}\neq 0$, so the functions are not orthogonal in this interval.

Definition: $\{F_i\}$ is a complete set of functions such that any other function $f$ may be expressed as a linear combination of members of the set $\{F_i\}$ on a prescribed expansion interval. [Namely, if the set $\{F_i\}$ is complete, then we may expand $f$ in terms of the functions $F_i$: $f(x)=\sum_{n=1}^\infty a_n F_n(x)$.]

Definition: An orthogonal set of functions is a set of functions each of which is orthogonal on a prescribed interval to all other members of the set. [That is, the set $\{F_i\}$ is is an orthogonal set if every member is orthogonal to every other member: $\langle F_i|F_j \rangle=0$ for all $i,j$ such that $i\neq j$.]

Example: The set $\{\sin nx, \cos nx\}$, on $[-\pi,\pi]$, for $n$ zero or positive integers, is an orthogonal set of functions.

Definition: An orthonormal set of functions is an orthogonal set of functions, each of which is normalised. [That is, the set $\{F_i\}$ is orthonormal if $\langle F_i|F_j \rangle=0$ for all $i\neq j$ and $\langle F_i|F_i \rangle=1$ for all $j$.]

These two equations occurs so often in discussing orthonormal functions that a special symbol has been introduced to combine them. The Kronecker delta symbol $\delta_{ij}$ has the meaning $\delta_{ij}=0$ for $i\neq j$, $\delta_{ij}=1$ for $i=j$.

The Fourier series is an expansion of a function in the orthonormal functions which are proportional to $\{\sin mx,\cos mx\}$. Let us derive their norm on the interval $[-\pi,-\pi]$. $$N(\sin mx)=\langle \sin mx|\sin mx \rangle=\int_{-\pi}^\pi \sin^2 mx dx=\dfrac{1}{m}\int_{-m\pi}^{m\pi}\sin^2 y dy=\pi.$$ Similarly, $$N(\cos mx)=\langle \cos mx|\cos mx \rangle=\dfrac{1}{m}\int_{-m\pi}^{m\pi}\cos^2 y dy=\pi.$$ Also, $$N(\sin 0x)=\langle \sin 0x|\sin 0x \rangle=\int_{-\pi}^\pi 0 dx=0$$ and $$N(\cos 0x)=\langle \cos 0x|\cos 0x \rangle=\int_{-\pi}^{\pi}1dx=2\pi.$$ Summarising the results, $$N(\sin mx)=\begin{cases} \pi\quad m\neq 0\\ 0\quad m=0 \end{cases}\\ N(\cos mx)=\begin{cases} \pi\quad m\neq 0\\ 2\pi \quad m=0 \end{cases}$$ or using the Kronecker delta, we have $$N(\sin mx)=\pi-\pi\delta_{m0}\\N(\cos mx)=\pi+\pi\delta_{m0}.$$ To calculate the expansion coefficients for an expansion in orthonormal functions, we use the set of functions $\{\dfrac{1}{\sqrt{2\pi}},\dfrac{1}{\sqrt{\pi}}\sin mx,\dfrac{1}{\sqrt{\pi}}\cos mx\}$ for $m=1,2,\cdots$, for expansion on $[-\pi,\pi]$. For these functions, the expansion coefficients will be $$a_0=\langle \dfrac{1}{\sqrt{2\pi}}|f\rangle\\a_m=\langle \dfrac{1}{\sqrt{\pi}}\cos mx|f\rangle\\b_m=\langle \dfrac{1}{\sqrt{\pi}}\sin mx|f\rangle,$$ where the expansion is $$f(x)=a_0\dfrac{1}{\sqrt{2\pi}}+\sum_{m=1}^\infty a_m(\dfrac{1}{\sqrt{\pi}}\cos mx)+\sum_{m=1}^\infty b_m(\dfrac{1}{\sqrt{\pi}}\sin mx).$$ Usually the constants are removed from the terms by explicitly writing out the expansion coefficients $$f(x)=\dfrac{1}{2\pi}\langle 1|f\rangle+\dfrac{1}{\pi}\sum_{m=1}^\infty [\langle \cos mx|f\rangle \cos mx+\langle \sin mx|f\rangle \sin mx].$$ This gives the final result $$f(x)=\dfrac{c_0}{2}+\sum_{m=1}^\infty c_m \cos mx+\sum_{m=1}^\infty d_m \sin mx\\c_m=\dfrac{1}{\pi}\langle \cos mx|f \rangle\\ d_m=\dfrac{1}{\pi}\langle \sin mx|f \rangle$$ on $[-\pi,\pi]$. This is the usual form of Fourier series. Note the lead term is divided by two because the norm of $\cos 0x$ is $2\pi$, whereas for all other values of $m$, the norm of $\cos mx$ is $\pi$.

Using the property of evenness and oddness of functions, we can make two simple extensions of the Fourier series. The Fourier expansion on $[-\pi,\pi]$ of an odd function is made up of sine terms $$f(x)=\sum_{m=1}^\infty d_m \sin mx$$ since if $f$ is odd, all inner products $\langle \cos mx|f \rangle \equiv 0$. The Fourier expansion on $[-\pi,\pi]$ of an even function is made up only of cosine terms: $$f(x)=\dfrac{c_0}{2}+\sum_{m=1}^\infty c_m \cos mx$$ since if $f$ is even, all inner products $\langle \sin mx|f \rangle \equiv 0$.

[...]

Reference:
Mathematics for Quantum Chemistry by Jay Martin Anderson

Orthogonality and Gram Schmidt process

Orthogonal bases
Let $\{v_1,\cdots,v_n\}$ be an orthogonal basis of the Euclidean space $V$. We want to find coordinates of the vector $u$ in this basis, namely the numbers $a_1,\cdots,a_n$ such that $$u=a_1v_1+\cdots+a_nv_n.$$ We can take the inner product with $v_1$, so $$\langle u,v_1\rangle=a_1\langle v_1,v_1\rangle+\cdots+a_n\langle v_n,v_1\rangle.$$ Since $v_1,\cdots,v_n$ are pairwise orthogonal, we have $\langle v_i,v_j \rangle=0$ for $i\neq j$. Thus, $\langle u,v_1\rangle=a_1\langle v_1,v_1\rangle \Rightarrow a_1=\dfrac{\langle u,v_1\rangle}{\langle v_1,v_1\rangle}$. Similarly, multiplying by $v_2,\cdots,v_n$, we have other coefficients $$a_2=\dfrac{\langle u,v_2\rangle}{\langle v_2,v_2\rangle},\cdots,a_n=\dfrac{\langle u,v_n\rangle}{\langle v_n,v_n\rangle}.$$ These coefficients are called Fourier coefficients of the vector $u$ with respect to basis $\{v_1,\cdots,v_n\}$.

Theorem: Let $\{v_1,\cdots,v_n\}$ be an orthogonal basis of the Euclidean space $V$. Then for any vector $u$, $$u=\dfrac{\langle u,v_1\rangle}{\langle v_1,v_1\rangle}v_1+\cdots+\dfrac{\langle u,v_n\rangle}{\langle v_n,v_n\rangle}v_n.$$ This expression is called Fourier decomposition and can be obtained in any Euclidean space.

Projections

The projection of the vector $v$ along the vector $w$ is the vector $\text{proj}_w v=cw$ such that $u=v-cw$ is orthogonal to $w$. We have $\langle u,w \rangle=0$, which means $$\langle v-cw,w \rangle=0 \iff \langle v,w \rangle-c\langle w,w \rangle=0 \iff c=\dfrac{\langle v,w \rangle}{\langle w,w \rangle}.$$ Thus the orthogonal complement $u$ is $$u=v-\text{proj}_w v=v-\dfrac{\langle v,w \rangle}{\langle w,w \rangle}w.$$ With this formula, we can find the distance between a point and a plane. Now, let's generalise our constructions.

Theorem: Let $W$ be a subspace with orthogonal basis $\{w_1,\cdots,w_n\}$. The projection $\text{proj}_w v$ of any vector $v$ along $W$ is $$\text{proj}_w v=\dfrac{\langle v,w_1 \rangle}{\langle w_1,w_1 \rangle}w_1+\cdots+\dfrac{\langle v,w_n \rangle}{\langle w_n,w_n \rangle}w_n.$$ [This follows from the Fourier decomposition above.] In particular, $u=v-\text{proj}_w v$ is orthogonal to the subspace $W$.

Proof: Take the inner product of $u$ with $w_i$, $$\langle u,w_i \rangle=\langle v,w_i \rangle-\bigg(\dfrac{\langle v,w_1 \rangle}{\langle w_1,w_1 \rangle}\langle w_1,w_i \rangle+\cdots+\dfrac{\langle v,w_n \rangle}{\langle w_n,w_n \rangle}\langle w_n,w_i \rangle\bigg).$$ Since $\langle w_i,w_j \rangle =0$ for $i\neq j$, we have $$\begin{align}\langle u,w_i \rangle&=\langle v,w_i \rangle-\dfrac{\langle v,w_i \rangle}{\langle w_i,w_i \rangle} \langle w_i,w_i \rangle\\&=\langle v,w_i \rangle-\langle v,w_i \rangle\\&=0.\end{align}$$ Thus $u$ is orthogonal to every $w_i$, and so it is orthogonal to $W$. $\Box$

Gram Schmidt orthogonalisation process
Let $\{v_1,\cdots,v_n\}$ be a basis in the Euclidean space. We can construct orthogonal basis $\{w_1,\cdots,w_n\}$ of this space. $$\begin{align}w_1&=v_1\\w_2&=v_2-\dfrac{\langle v_2,w_1\rangle}{\langle w_1,w_1\rangle}w_1\\w_3&=v_3-\dfrac{\langle v_3,w_1\rangle}{\langle w_1,w_1\rangle}w_1-\dfrac{\langle v_3,w_2\rangle}{\langle w_2,w_2\rangle}w_2\\ \cdots\\ w_n&=v_n-\dfrac{\langle v_n,w_1\rangle}{\langle w_1,w_1 \rangle}w_1-\dfrac{\langle v_n,w_2\rangle}{\langle w_2,w_2\rangle}w_2-\cdots-\dfrac{\langle v_n,w_{n-1}\rangle}{\langle w_{n-1},w_{n-1}\rangle}w_{n-1}.\end{align}$$

[...]

Exact sequences

A motivating example:
In linear algebra, we know that the linear transformation $f:V\to W$ induces the linear transformations $V\xrightarrow{g}\text{im}(f)\xrightarrow{i}W$ where $i$ is the inclusion map and $g$ is the same as $f$ except that the range of $f$ has been restricted to $\text{im}(f)$. Then $f=i\circ g$ and so we can represent this by a commutative diagram. Intuitively, we can think of exact sequence as a factorisation of maps with certain condition.

Definition: Let $U\xrightarrow{f}V\xrightarrow{g}W$ be a sequence of linear transformations. (This means $f\in L(U,V)$ and $g\in L(V,W)$.) We say $U\xrightarrow{f}V\xrightarrow{g}W$ is an exact sequence at $V$ if $\text{im}(f)=\text{ker}(g)$. A sequence $V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}\cdots \xrightarrow{f_{n-2}}V_{n-1}\xrightarrow{f_{n-1}}V_n$ is an exact sequence if $\text{im}(f_{i-1})=\text{ker}(f_i), 2\leq i\leq n-1$, that is, if the sequence is exact at $V_2,\cdots,V_{n-1}$.

When writing sequences of linear transformations, it is customary to write $0$ for the vector space $\{0\}$.

Lemma:
1. The sequence $0\to V\to 0$ is exact iff $V=\{0\}$.
2. The sequence $0\to U\xrightarrow{f} V$ is exact iff $f$ is injective.
3. The sequence $V\xrightarrow{g}W\to 0$ is exact iff $g$ is surjective.
4. The sequence $0\to V\xrightarrow{f}W\to 0$ is exact iff $f$ is an isomorphism.
5. The sequence $0\to U\xrightarrow{f}V\xrightarrow{g}W\to 0$ is exact iff $f$ is injective and $g$ is surjective, and $\text{im}(f)=\text{ker}(g)$.
Proof: Just note that from the definition, $\text{im}(0\to V)=0$ and $\text{ker}(V\to 0)=V.\:\Box$

Definition: A short exact sequence is a sequence of five vector spaces, which is exact at all three internal nodes: $0\to U\xrightarrow{f}V\xrightarrow{g}W\to 0$. A long exact sequence is a doubly infinite sequence of vector spaces which is exact at every node.

A nice feature of exact sequences is that there are simple rules to create new sequences from old ones. The simplest way to do this is to split and glue exact sequences as follows.

Splitting and gluing exact sequences
Splitting: If $V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}V_3\xrightarrow{f_3}V_4$ is an exact sequence, the two sequences $V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}W\to 0$ and $0\to W\to V_3\xrightarrow{f_3}V_4$ are also exact, where $W=\text{im}(f_2)=\text{ker}(f_3)$ and $f_2$ is the inclusion of $\text{ker}(f_3)$ in $V_3$.

Gluing: Conversely, if $V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}W\to 0$ and $0\to W\to V_3\xrightarrow{f_3}V_4$ are exact sequences with $W \subset V_3$ and $f_2$ being the inclusion, the sequence $V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}V_3\xrightarrow{f_3}$ is also exact.

Proposition: Assume that $0\to V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}\cdots\xrightarrow{f_{n-1}}V_n\to 0$ is an exact sequence of finite dimensional vector spaces, $n\geq 1$. Then $\sum\limits_{i=1}^n (-1)^i \: \text{dim}(V_i)=0$.

Proof by MI: If $n=1$, then $V_1=\{0\}$, and so $\text{dim}(V_1)=0$.
If $n=2$, then $V_1 \cong V_2$ by the lemma, and so $\text{dim}(V_1)-\text{dim}(V_2)=0$.
If $n=3$, then $0\to V_1\xrightarrow{f_1}V_2\xrightarrow{f_2}V_3\to 0$ is an exact sequence.
By the dimension theorem, we have $$\begin{align}\text{dim}(V_2)&=\text{dim}\:\text{ker}(f_2)+\text{dim}\:\text{im}(f_2)\\
&=\text{dim}\:\text{im}(f_1)+\text{dim}(V_3)\\
&=\text{dim}(V_1)-\text{dim}\:\text{ker}(f_1)+\text{dim}(V_3)\\
&=\text{dim}(V_1)+\text{dim}(V_3),\end{align}$$ which means $-\text{dim}(V_1)+\text{dim}(V_2)-\text{dim}(V_3)=0$. So the result holds for $n=3$.
For the general case, suppose $n\geq 4$. To use the induction assumption, we can split the given exact sequence into $$0\to V_1\xrightarrow{f_1}\cdots\xrightarrow{f_{n-3}}V_{n-2}\xrightarrow{f_{n-2}}\text{im}(f_{n-2})\to 0,$$ $$0\to \text{im}(f_{n-2})\xrightarrow{i}V_{n-1}\xrightarrow{f_{n-1}}V_n\to 0$$ because $\text{im}(f_{n-2})=\text{ker}(f_{n-1})$. By induction, we have $\sum\limits_{i=1}^{n-2} (-1)^i \:\text{dim}(V_i)+(-1)^{n-1}\:\text{dim}(\text{im}(f_{n-2}))=0$ and $(-1)^{n-2}\:\text{dim}(\text{im}(f_{n-2}))+(-1)^{n-1}\:\text{dim}(V_{n-1})+(-1)^n\:\text{dim}(V_n)=0$.
Adding these two gives $\sum\limits_{i=1}^n (-1)^i \:\text{dim}(V_i)=0.\:\Box$

Direct proof: Applying the rank nullity theorem to all vector spaces $V_1,\cdots,V_n$, we have $$\begin{align}\sum\limits_{i=1}^{n-1}(-1)^i\:\text{dim}(V_i)&=\sum\limits_{i=1}^{n-1}(-1)^i \:(\text{dim}\:\text{ker}(f_i)+\text{dim}\:\text{im}(f_i))\\
&=-\text{dim}\:\text{ker}(f_1)+(-1)^{n-1}\text{dim}\:\text{im}(f_{n-1})\\&+\sum\limits_{i=2}^{n-1}(-1)^i \:(\text{dim}\:\text{ker}(f_i)-\text{dim}\:\text{im}(f_{i-1})).\end{align}$$
But $\text{dim}\:\text{ker}(f_1)=0$ since $f_1$ is injective, $\text{dim}\:\text{im}(f_{n-1})=\text{dim}(V_n)$ since $f_{n-1}$ is surjective, and $\text{dim}\:\text{ker}(f_i)=\text{dim}\:\text{im}(f_{n-1})$ for all $i=2,\cdots,n-1$ by definition.
Therefore $\sum\limits_{i=1}^n (-1)^i \: \text{dim}(V_i)=(-1)^{n-1}\:\text{dim}(V_n)+(-1)^n\:\text{dim}(V_n)=0.\:\Box$

Linear algebra
Dimension theorems
For any short exact sequence of finite-dimensional vector spaces $0\to V_1\xrightarrow{T_1}V_2\xrightarrow{T_2}V_3 \to 0$, $\text{dim}(V_2)=\text{dim}(V_1)+\text{dim}(V_3)$. [This is just the $n=3$ case of the proposition above. Here we provide a proof without the aid of rank nullity theorem.]

Proof: Suppose $\text{dim}(V_1)=n$. Then we can take $\{e_i\}_{i=1}^n$ to be a basis of $V_1$. Thus $\{T_1(e_i)\}_{i=1}^n$ span the image of $T_1$. Since $T_1$ is injective, $\{T_1(e_i)\}_{i=1}^n$ are linearly independent in $V_2$. We can find $\{v_j\}_{j=1}^r$ such that $\{T_1(e_i)\}_{i=1}^n \cup \{v_j\}_{j=1}^r$ form a basis of $V_2$. Thus $\text{dim}(V_2)=n+r$. Now $T_2$ is surjective, so $\{T_2(v_j)\}_{j=1}^r$ span $V_3$. It remains to show that $\{T_2(v_j)\}_{j=1}^r$ is linearly independent. Suppose $\sum_j a_jT_2(v_j)=0$. Then $\sum_j T_2(a_jv_j)=0\Rightarrow \sum a_jv_j \in \text{ker}(T_2)=\text{im}(T_1)$. Since $\{T_1(e_i)\}_{i=1}^n$ span the image of $T_1$, we have $\sum\limits_{j=1}^r a_jv_j=\sum\limits_{i=1}^n b_i T_1(e_i)$ for some constants $\{b_i\in \mathbb{R}\}_{i=1}^n$. In particular, $\sum\limits_{i=1}^n -b_i T_1(e_i)+\sum\limits_{j=1}^r a_jv_j=0$. Since $\{T_1(e_i)\}_{i=1}^n \cup \{v_j\}_{j=1}^r$ are linearly independent in $V_2$, all the coefficients $\{b_i\}$ and $\{a_j\}$ must be zero. Therefore, $\text{dim}(V_3)=r$. We have established that $\text{dim}(V_2)=n+r=\text{dim}(V_1)+\text{dim}(V_3).\:\Box$

Using this, we can derive many results in linear algebra.

Theorem: If $T:V\to W$, then $0 \to \text{ker}(T) \xrightarrow{i} V \xrightarrow{T} \text{im}(T) \to 0$ is a short exact sequence, where $i$ is the inclusion map.
Corollary (Rank-nullity): If $T:V\to W$ is a linear map of finite-dimensional vector spaces, then $\text{dim}(\text{ker}(T))+\text{dim}(\text{im}(T))=\text{dim}(V)$.

Theorem: If $A,B \subset V$ are subspaces such that $V=\text{span}(A\cup B)$, then $0\to A \cap B \to A \oplus B \to V \to 0$ is a short exact sequence, where $A \cap B\to A\oplus B$ is the diagonal map $v\mapsto (v,v)$, and the map $A\oplus B\to V$ is the difference map $(a,b)\mapsto a-b$.
Corollary: $\text{dim}(V)=\text{dim}(A)+\text{dim}(B)-\text{dim}(A \cap B)$.

Theorem: If $W\subset V$ is a subspace, then $0\to W \xrightarrow{i} V \xrightarrow{g} V/W \to 0$ is exact, where $i$ is the inclusion map and $g$ is the quotient map $g(v)=[v]$.
Corollary: $\text{dim}(V/W)=\text{dim}(V)-\text{dim}(W)$.

Dual space
Proposition: If $U\xrightarrow{f}V\xrightarrow{g}W$ is an exact sequence of linear transformations of vector spaces, then $W^*\xrightarrow{g^*}V^*\xrightarrow{f^*}U^*$ is also an exact sequence of linear transformations.

Corolloary: Let $f\in L(V,W)$ and let $f^*\in L(W^*,V^*)$ be the induced linear transformation on dual spaces.
1. If $f$ is injective, then $f^*$ is surjective.
2. If $f$ is surjective, then $f^*$ is injective.
Proof:
If $f$ is injective, then $0\to V\to W$ is exact at $V$ and so $W^*\to V^*\to 0$ is an exact sequence. Thus $f^*$ is surjective.
If $f$ is surjective, then $V\to W\to 0$ is exact at $W$ and so $0\to W^*\to V^*$ is an exact sequence. Thus $f^*$ is injective.

More intuition about exact sequences
In Euclidean space $\mathbb{R}^n$, if $V$ is a subspace of $\mathbb{R}^n$, then we think of it as filling out "some of" the dimensions in $\mathbb{R}^n$ and its orthogonal complement $V^\perp$ fills out the other directions. Together they span $\mathbb{R}^n$ with no redundancies: $\mathbb{R}^n=V\oplus V^\perp$. See here for more examples of direct sum.

More generally, we don't have an inner product and so we can't form orthogonal complements, but we can still talk about submodules and quotients.

If $A$ is a submodule of $B$, then $A$ fills up "some of the directions" in $A$, and the remaining directions are encoded in $B/A$. When we have a submodule $A\subset B$, a surjection $B \to C$, and if $A$ happens to be the kernel of the map $B \to C$, then we are in the previous situation: $A$ fills out some of the directions in $B$, and all the complementary directions are encoded in $C$. So we introduce the terminology $0\to A\to B\to C\to 0$ is a short exact sequence to capture this situation.

Graph theory: Euler's formula
Inclusion-exclusion

References
Intuitive meaning of exact sequence
Linear algebra [p38-40,75,76,84]
Exact sequences in linear algebra [p4-6]
Definition of short exact sequence [p1,2]
Splitting and gluing [p1,2,5]
Simple groups and short exact sequences [p2,4]

More to explore:
Homological algebra

Quotient spaces

Recall the definition of sum of subspaces: if $U,W$ are subspaces of $V$, then $U+W=\{u+w\:|\:u \in U\:\text{and}\: w\in W\}$. In case one of the sets contain only one element, we can say $\{u_0\}+W=u_0+W=\{u_0+w\:|\: w\in W\}$. We call $u_0+W$ a coset (translate) of $W$. [A coset $v+W$ is not a subspace unless $v \in W$. The simple reason is that after translation, the zero element is no longer in $v+W$. In fact, such cosets are examples of affine subspaces.] Geometrically, all of the sets $u_0+W$ are parallel pictures of $W$ that are translated in $V$ as we change $u_0$. We can also say that $U$ and $W$ are parallel and denote $W \parallel U$ if $U=u_0+W$ for some $u_0\in V$.

Note that several of these translates can be equal. Let's take a look at an example. Let $W=\{(x,0)\:|\: x\in \mathbb{R}\},v_1=(0,1),v_2=(1,1)$. Then $v_1+W=\{(x,1)\:|\:x\in \mathbb{R}\}=v_2+W$. Also notice that $v_2-v_1=(1,0)\in W$. Thus translating $W$ by $v_1$ or $v_2$ is the same. This motivates the following proposition.

Proposition (i): $v+W=v'+W \iff v-v'\in W$.
Proof: Suppose $v+W=v'+W$. Then $v=v'+w$ for some $w\in W$, implying $v-v'=w\in W$.
Conversely, suppose $v-v'\in W$. Note that if $z\in W$, then $z+W=W$ since $W$ is a subspace. Then $v+W=v'+(v-v')+W=v'+W.\:\Box$

According to the proposition, two elements $v$ and $v'$ of $V$ determine the same element of $V /W$ if and only if $v-v'\in W$. So we can think of $V/W$ as the set of all elements of $V$ under the extra condition that two elements of $V$ are declared to be the same if their difference is in $W$. Thus we can define $V/W$ to be the set of equivalence classes of the equivalence relation $∼$ on $V$ defined by $v \sim v'$ if $v-v'\in W$. Notice how similar this definition of relation is to congruence relation. So we can also say $v \equiv v' \:\text{mod}\:W$ if $v-v' \in W$. As an exercise, try to prove that the equivalence classes of $\equiv \text{mod}\:W$ are precisely the cosets of $W$.

Definition: A quotient set $V/W$, pronounced $V\:\text{mod}\:W$, is the set of all cosets of $W$, defined by $V/W=[v]=\{v+W\:|\:v\in V\}$.
[Informally, a quotient set is what we get when we "divide" a set $A$ by $B\subset A$, wherein we set all elements of $B$ to the identity in $A$. For example, if $A=\mathbb{Z}$ and $B=\{5n\:|\:n\in \mathbb{Z}\}$, then we're making all multiples of $5$ zero, so the quotient is $\{0,1,2,3,4\}$ (the set of all equivalence classes).]

We want to turn $V/W$ into a vector space (quotient space), so we have to define addition and scalar multiplication.

Definition: The sum of two elements $v+W$ and $v'+W$ of $V/W$ is defined by $(v+W)+(v'+W) := (v+v')+W$.
For $a\in \mathbb{F}$, the scalar multiplication of $a$ on $v+W$ is defined by $a\cdot (v+W) := av+W$.

It is necessary to check that these definitions are well defined, that is, these operations should not depend on which element of $v+W$ we choose to represent $v+W$.

Proposition (ii): Suppose $v+W=v'+W$. Then $(v+W)+(v''+W)=(v'+W)+(v''+W)$ for any $v''+W\in V/W$ and $a\cdot (v+W)=a\cdot (v'+W)$ for any $a\in \mathbb{F}$.
Proof:
WLOG it is enough to show that $(v+W)+(v''+W) \subset (v'+W)+(v''+W)$. Let $u\in(v+W)+(v''+W)=(v+v'')+W$. Then there exists $w\in W$ such that $u=(v+v'')+w$. Since $v+W=v'+W$, we have $v-v'\in W$. Denote this element by $w'$. Then $v=v'+w'$. Thus $u=(v+v'')+w=((v'+w')+v'')+w=(v'+v'')+(w'+w)$, showing that $u\in (v'+v'')+W$ since $w'+w\in W$.
We conclude that $(v+W)+(v''+W)=(v'+W)+(v''+W)$.
WLOG it is enough to show that $a\cdot (v+W)\subset a\cdot (v'+W)$. Let $u\in a\cdot (v+W)=av+W$. Then $u=av+w$ for some $w\in W$. Denote by $w'$ the element $v-v'\in W$. Then $u=av+w=a(v'+w')+w=av'+(aw'+w)\in av'+W$ since $aw'+w\in W$ as $W$ is closed under addition and scalar multiplication.
We have $a\cdot (v+W)=a\cdot (v'+W)$
Thus both addition and scalar multiplication on $V/W$ is well-defined. $\Box$

Note that the above depends crucially on the assumption that $W$ is a subspace of $V$; if $W$ were just an arbitrary subset of $V$, addition and scalar multiplication on $V/W$ would not be well-defined. With these operations then, $V/W$ becomes a vector space over $\mathbb{F}$.

It is important to keep in mind that the quotient space is in general not a subspace, nor naturally equivalent to one. But because of the presence of a natural inner product on $\mathbb{R}^n$, there is a natural way to choose a second subspace orthogonal to the given one, and that second subspace serves as a natural isomorphic model of the quotient space by the given subspace. Namely, if $V$ has an inner product, then $V/W$ is isomorphic to $W^\perp=\{v\:|\:\langle v,w \rangle =0\: \forall\: w\in W\}$.

Summary of concepts: sum of subspaces, coset (translate), subspaces, equivalence relation & equivalence class, congruence relation, quotient set

---

Examples:

The quotient space in $\mathbb{R}^3$ by a line is a two-dimensional vector space that is isomorphic to a plane. It does not necessarily equal a plane in $\mathbb{R}^3$. In particular, the quotient space doesn't need to be a subset of $\mathbb{R}^3$. The quotient space is something very abstract. It's an entirely new vector space.

The quotient space in $\mathbb{R}^3$ by a plane is the set of all planes parallel to the plane given by the equation $z=0$. So any plane with equation $z=k$ is an element of the quotient space, which is isomorphic to $\mathbb{R}$, but note that the quotient space does not necessarily equal a line in $\mathbb{R}^3$.

In the quotient $\mathbb{R}[x]/(x^2+1)$, the polynomial $x^2+1$ is equivalent to $0$. It follows from the fact that $0$ and $x^2+1$ are in the same equivalence class since $x^2+1=0+(x^2+1)$. So we have $x^2=-1$, which means $x=i$ or $x=-i$. Since an arbitrary element of $\mathbb{R}[x]$ looks like $a_0+a_1x+a_2x^2+\cdots+a^nx^n\; a_i\in \mathbb{R}$, in the quotient $\mathbb{R}[x]/(x^2+1)$ the polynomial becomes $f=a_0+a_1 i+a_2 i^2+\cdots+a^n i^n$. Since $i^2=-1$, we have $i^3=-i, i^4=1$. Collecting terms we can write $f$ as $a+bi$ for some $a,b\in \mathbb{R}$. Therefore, we can say $\mathbb{R}[x]/(x^2+1)\cong \mathbb{C}$.

Quotient space $\mathbb{P}_n/\mathbb{R}$
Let $V=\mathbb{P}_n=\{a_nx^n+a_{n-1}x^{n-1}+\cdots+a_1x+a_0\:|\:a_0,a_1,\cdots,a_n\in \mathbb{R}\}$ and $W=\{a_0\:|\:a_0\in \mathbb{R}\}=\mathbb{R}$.
Let $[f(x)], [g(x)]\in V/W$. Then $[f(x)]=[g(x)] \iff f(x)-g(x) \in W \iff f(x)-g(x)=c$ for $c\in \mathbb{R}$.
Let a differential operator be $D:V \to V$ where $D[f(x)]=\frac{df}{dx}$.
We have $D[c]=\frac{d}{dx}(c)=0$, which means $\text{ker}(D)=[c]=W$, the set of constant polynomials. This operator is not 1-1 because different functions after differentiation can give the same constant polynomial.
Now let's change the domain of the differential operator to $\overline{D}:V/W \to V$ where $\overline{D}[f(x)]=\frac{df}{dx}$. Now $\text{ker}(\overline{D})=[c]=W=[0]+W=0_{V/W}$. So the operator is 1-1 in the quotient space.

The key idea is this: if a mapping is not 1-1 in which different points map to the same point, we can take all those different points to be elements of a common coset. In other words, we are identifying points that are equivalent to one another in some equivalence relation. With the quotient space formed by squashing those points to the same coset, we have a 1-1 mapping. Similarly, we can modify the codomain and turn a mapping into a surjection, which leads to the following proposition.

Proposition (iii) Canonical epimorphism: The function $\pi:V\to V/W$ defined by $\pi(v)=[v]=v+W$ is a linear transformation with $\text{ker}(\pi)=W, \text{im}(\pi)=V/W$.
Proof:
$\pi(u+v)=(u+v)+W=(u+W)+(v+W)=\pi(u)+\pi(v)$
$\pi(av)=(av)+W=a(v+W)=a\pi(v)$
It follows that $\pi$ is a linear transformation.
The image of $\pi$ is $V/W$ by definition.
For the kernel of $\pi$, $v\in \text{ker}(\pi) \iff v+W=0+W \iff v\in W.\:\Box$

Corollory: Let $V$ be a vector space and $W$ be a subspace of $V$. Then $\text{dim}(W)+\text{dim}(V/W)=\text{dim}(V)$.
Proof: Let $\pi:V\to V/W$ be the canonical epimorphism. By the dimension theorem, we have $\text{dim}\:\text{ker}(\pi)+\text{dim}\:\text{im}(\pi)=\text{dim}(V)$. But $\text{ker}(\pi)=W$ and $\text{im}(\pi)=V/W$, so the claim follows. $\Box$

Theorem: Let $W$ be a subspace of $V$ with quotient mapping $T:V\to V/W$. A subspace $U$ is a complement to $W$ iff $T|_U:U\to V/W$ is an isomorphism.

Universal property of the quotient
Let $V$ be a vector space over $\mathbb{F}$ and $W$ be a subspace of $V$. Define $\pi:V\to V/W$ by $v \mapsto [v]$. If $W'$ is a vector space over $\mathbb{F}$ and $f:V\to W'$ is a linear map whose kernel contains $W$, then there exists a unique linear map $g:V/W\to W'$ such that $f=g\circ \pi$. This is summarized by the following commutative diagram:

$V \xrightarrow{\qquad f \qquad} W'\\ \; \pi \searrow \quad \qquad \nearrow g\\ \qquad\;\; V/W$

Proof:
Let $W'$ be a vector space over $\mathbb{F}$, $f:V\to W'$ be a linear map with $W \subset \text{ker}f$. Then define $g:V/W\to W'$ to be the map $[v]\mapsto f(v)$.
$g$ is well-defined because if $[v]\in V/W$ and $v_1,v_2\in V$ are both representatives of $[v]$, then there exists $w\in W$ such that $v_1=v_2+W$.
Hence $f(v_1)=f(v_2+w)=f(v_2)+f(w)=f(v_2)$ since f is linear.
The linearity of $g$ comes from linearity of $f$.
Also, by definition $f=g\circ \pi$.
It remains to prove $g$ is unique.
Suppose $h:V/W\to W'$ is such that $h\circ \pi=f=g\circ \pi$. Then $h([v])=g([v])$ for all $v\in V$. Since $\pi$ is surjective, $h=g.\:\Box$

First isomorphism theorem
Let $V,W$ be vector spaces over $\mathbb{F}$ and $T:V \to W$ a linear map. Then $V/\text{ker}(T) \cong \text{im}(T)$, namely $V/\text{ker}(T)$ is isomorphic to $\text{im}(T)$.

Proof: Consider the mapping $\overline{T}:V/\text{ker}(T) \to W \;\;\; \overline{T}:[v]=v+\text{ker}(T) \mapsto T(v)$.

A commutative diagram would express this as

$V \xrightarrow{\qquad T \qquad} W\\ \; \pi \searrow \quad \qquad \nearrow \overline{T}\\ \qquad V/ \text{ker}(T)$

We can use similar arguments as the previous example to prove that $\overline{f}$ is well-defined and linear.
Now suppose $\overline{T}([v])=0$. Then $T(v)=0$, which implies $v\in \text{ker}(T)$. This means $[v]=0_{V/\text{ker}(T)}$. Then $\overline{T}$ is injective. Hence if we confine the codomain of $\overline{T}$ to $\text{im}(T)$, then $\overline{T}$ becomes a bijection and hence $V/\text{ker}(T) \cong \text{im}(T).\:\Box$

Example:
$T:\mathbb{R}^{n\times n} \to \mathbb{R}^{n\times n}\quad T(A)=A-A^T$
$\text{ker}(T)=\{A\in \mathbb{R}^{n\times n}\:|\:A=A^T\}$ set of symmetric matrices
$\text{im}(T)=\{B=A-A^T\:|\:A\in \mathbb{R}^{n\times n}\}$
$B^T=A^T-A=-B$ set of antisymmetric matrices
$\mathbb{R}^{n\times n}/ \{n\times n\;\text{symmetric matrices}\} \cong \{n\times n\;\text{antisymmetric matrices}\}$

Strong form of the first isomorphism theorem
Let $T:V\to W'$ be a linear transformation of vector spaces over $\mathbb{F}$. Suppose $W \subset \text{ker}(T)$. Then there exists a unique linear transformation $\overline{T}:V/W\to \text{im}(T)$ such that $\overline{T}\circ \pi=T$, where $\pi:V\to V/W$, and $\overline{T}$ is surjective. Also, $\text{ker}(\overline{T})\cong \text{ker}(T)/W$ and $(V/W)/(\text{ker}(T)/W)\cong \text{im}(T)$.

Second isomorphism theorem
Let $U,W$ be subspaces $V$. Then $(U+W)/U \cong W/(U \cap W)$.

Third isomorphism theorem
Let $V$ be a vector space and $U\subset W\subset V$ be subspaces. Then $(V/U)/(W/U)\cong V/W$.

References:
An awesome video on quotient space
Definition of quotient set
Basis for $V/U$
Examples of quotient spaces
Intuition about quotient spaces
Constructing $\mathbb{C}$ from $\mathbb{R}$