Chapter 9 — Eigen Decomposition

Proof

\( T \in \Lin(V) \) is self-adjoint, i.e. \( T = T^* \). \(T\) has at least one eigenvalue (by existence of roots of the characteristic polynomial). Let \( \lambda \in \C \) be an eigenvalue with \( Tv = \lambda v \), \( v \in V \), \( v \neq 0 \). We show \( \lambda \) is real: \[\begin{aligned} \langle Tv, v \rangle = \langle v, T^*v \rangle &\implies \langle \lambda v, v \rangle = \langle v, Tv \rangle \\ &\implies \overline{\lambda}\langle v,v\rangle = \langle v, \lambda v\rangle \\ &\implies \overline{\lambda}\langle v,v\rangle = \lambda\langle v,v\rangle. \end{aligned}\] Since \( v \neq 0 \), \( \langle v,v\rangle = \|v\|^2 > 0 \), so \( \overline{\lambda} = \lambda \), i.e. \( \lambda \) is real.

Alternate solution. Let \( A = [T] \). By Schur, \( A = U\Lambda U\herm \) with \( \Lambda \) upper triangular. Then \( A\herm = (U\Lambda U\herm)\herm = U\Lambda\herm U\herm \), and \( A = A\herm \Rightarrow \Lambda = \Lambda\herm \), so \( \Lambda \) is diagonal with real diagonal entries (the eigenvalues). They may all be the same, but nonetheless we have a real eigenvalue.

Proof of Theorem 15

\( T \in \Lin(V) \) has at least one eigenvalue (existence of roots of the complex polynomial). Let \( \lambda \in \C \), \( Tv = \lambda v \), \( v \neq 0 \). By the argument of Lemma 3, \( \lambda \) is real. We prove the claim by induction on \( \dim V \).

Base case \( \dim V = 1 \): one real eigenvalue \( \lambda \) exists, \( Tv = \lambda v \), \( v \neq 0 \). Then \( \tfrac{v}{\|v\|} \) forms an ONB for \(V\), with \( \lambda \) real.

Inductive step. Assume the result for \( \dim V = n \). For \( \dim V = n+1 \): \( T \in \Lin(V) \), \( T^* = T \), with a real eigenvalue \( \lambda \) and \( Tv = \lambda v \). Let \( u_1 = \tfrac{v}{\|v\|} \) (so \( \|u_1\| = 1 \)) and \( U = \operatorname{span}(u_1)^{\perp} \). Since \(T\) is invariant on \( \operatorname{span}(u_1) \), \( T^* \) is invariant on \( U = \operatorname{span}(u_1)^{\perp} \) (Ch. 8, Proposition 6; Ex 93); and as \( T = T^* \), \(T\) is invariant on \(U\). Let \( T_U \) be the restriction of \(T\) on \(U\); then \( T_U \in \Lin(U) \) is self-adjoint (Ex 92). Now \( \dim U = n \), so by induction there is an ONB of eigenvectors of \( T_U \), say \( u_2, \dots, u_{n+1} \); these are also eigenvectors of \( T \in \Lin(V) \), and \( u_1 \perp U = \operatorname{span}(u_2, \dots, u_{n+1}) \). So \( u_1, \dots, u_{n+1} \) forms an ONB of eigenvectors of \(V\), completing the induction.

Proof (sketch)

Suppose \( TS = ST \). Let \( \lambda \) be an eigenvalue of \(S\) with eigenspace \( E_\lambda = \Null(S - \lambda I) \). For \( v \in E_\lambda \), applying \(T\): \( TSv = T(\lambda v) = \lambda(Tv) \), and since \( TS = ST \), \( S(Tv) = \lambda(Tv) \), so \( Tv \in E_\lambda \). Hence \(T\) is invariant on every eigenspace of \(S\); exchanging the roles of \(S\) and \(T\) gives the converse.

Since \(T\) and \(S\) are self-adjoint they are each diagonalizable. Decompose \( V = E_1 \oplus \dots \oplus E_k \) into eigenspaces \( E_i = \Null(T - \lambda_i I) \) of \(T\). By the above, \(S\) is invariant on each \( E_i \); let \( S_i \) be the restriction of \(S\) to \( E_i \). By Ex 92, \( S_i \in \Lin(E_i) \) is self-adjoint, hence diagonalizable, giving \( \dim E_i \) orthonormal eigenvectors that are eigenvectors of both \(T\) and \(S\). Doing this for each \( E_i \) yields \( \sum_i \dim E_i = n \) orthonormal vectors forming an ONB of \(V\) of common eigenvectors. Hence \(T\) and \(S\) share a common ONB in which they are diagonalized.

Proof

Idea (even–odd decomposition). For functions, even means \( f(-t) = f(t) \) and odd means \( f(-t) = -f(t) \); any \(g\) splits as \( g(t) = \tfrac12[g(t) + g(-t)] + \tfrac12[g(t) - g(-t)] \) (even + odd).

For a normal operator \(T\) (\( TT^* = T^*T \)), write \( T = \tfrac12(T + T^*) + \tfrac12(T - T^*) \) (self-adjoint + skew-adjoint). To use Theorem 15 we want self-adjoint components, so \[ T = \underbrace{\tfrac12(T + T^*)}_{T_e} + i\,\underbrace{\tfrac{1}{2i}(T - T^*)}_{T_o}, \qquad i = \sqrt{-1}, \] where \( T_e \) and \( T_o \) are both self-adjoint: \( T = T_e + iT_o \), \( T_e^* = T_e \), \( T_o^* = T_o \).

Claim: \( T_e T_o = T_o T_e \). \[ T_e T_o = \tfrac12(T + T^*)\cdot\tfrac{1}{2i}(T - T^*) = \tfrac{1}{4i}(TT - TT^* + T^*T - T^*T^*), \] \[ T_o T_e = \tfrac{1}{2i}(T - T^*)\cdot\tfrac12(T + T^*) = \tfrac{1}{4i}(TT + TT^* - T^*T - T^*T^*). \] Since \(T\) is normal (\( TT^* = T^*T \)), \( T_e T_o = T_o T_e \). By Proposition 7 they are diagonalized in a common basis: \[ [T_e] = Q\Lambda_e Q\herm, \quad [T_o] = Q\Lambda_o Q\herm \quad (Q \text{ unitary, columns an ONB of eigenvectors}). \] Therefore \( [T] = [T_e] + i[T_o] = Q\Lambda_e Q\herm + iQ\Lambda_o Q\herm = Q(\Lambda_e + i\Lambda_o)Q\herm \), so \(T\) has an ONB of eigenvectors.

Proof

Let \( \varphi(U) = \min_{x\in U,\ \|x\|=1}\langle x, Tx\rangle \); we show \( \lambda_k = \max_{\dim U = k}\varphi(U) \). Take \( U \subseteq V \) with \( \dim U = k \), \( 1 \le k \le n = \dim V \). As \(T\) is self-adjoint, an ONB of eigenvectors \( q_1, \dots, q_n \) exists with \( Tq_i = \lambda_i q_i \).

Let \( W = \operatorname{span}(q_k, \dots, q_n) \), so \( \dim W = n - k + 1 \). Since \( \dim W + \dim U > \dim V \), \( W \cap U \) is non-empty with \( \dim(W \cap U) \ge 1 \). Take \( x_0 \in W \cap U \), \( \|x_0\| = 1 \). Writing \( x_0 = a_1 q_1 + \dots + a_n q_n \) (with \( \sum_{i=1}^{n}|a_i|^2 = 1 \) as the \( q_i \) are an ONB), \[ Tx_0 = a_1\lambda_1 q_1 + \dots + a_n\lambda_n q_n, \qquad \langle x_0, Tx_0\rangle = \sum_{i=1}^{n}\sum_{j=1}^{n}\lambda_j a_j\overline{a_i}\underbrace{\langle q_i, q_j\rangle}_{\delta_{ij}} = \sum_{i=1}^{n}\lambda_i|a_i|^2. \] The largest eigenvalue appearing in \( W \cap U \) is \( \lambda_k \), so \( \langle x_0, Tx_0\rangle \le \lambda_k\sum_{i=1}^{n}|a_i|^2 = \lambda_k \). Hence for all \(U\) with \( \dim U = k \) there is \( x_0 \in U \), \( \|x_0\| = 1 \), with \( \langle x_0, Tx_0\rangle \le \lambda_k \), so \( \min_{x\in U,\|x\|=1}\langle x, Tx\rangle \le \lambda_k \), i.e. \( \varphi(U) \le \lambda_k \).

This bound is achieved: for \( U^* = \operatorname{span}(q_1, \dots, q_k) \) (\( \dim U^* = k \)), every unit \( x = a_1 q_1 + \dots + a_k q_k \in U^* \) has \( \langle x, Tx\rangle = \sum_{i=1}^{k}\lambda_i|a_i|^2 \ge \lambda_k\sum_{i=1}^{k}|a_i|^2 = \lambda_k \) (since \( \lambda_i \ge \lambda_k \) for \( i \le k \)), with equality at \( x = q_k \). So \( \varphi(U^*) = \lambda_k \); combined with \( \varphi(U) \le \lambda_k \) for every \(k\)-dimensional \(U\), \[ \lambda_k = \max_{\dim U = k}\ \min\big\{\langle x, Tx\rangle : x \in U,\ \|x\| = 1\big\}. \]