Linear Algebra and its Applications · IISc 2024 · typeset reproduction
The trace is a linear functional on the space of (square) matrices — an element of \( \Lin(\F^{n\times n}, \F) \).
Definition 34 — Trace
The trace of \( A \in \F^{n\times n} \) is \[ \tr(A) = A_{11} + \dots + A_{nn} = \text{(sum of the diagonal elements of } A\text{)}. \]
Ex 73
Show that
Solution — (1)
\( \tr(AB) = \sum_{i=1}^{n} (AB)_{ii} \), and \( (AB)_{ii} = \sum_{k=1}^{n} a_{ik} b_{ki} \) (the diagonal elements of \( AB \)). Hence \[\begin{aligned} \tr(AB) &= \sum_{i=1}^{n}\sum_{k=1}^{n} a_{ik} b_{ki} \\ &= \sum_{k=1}^{n}\sum_{i=1}^{n} b_{ki} a_{ik} = \sum_{k=1}^{n} (BA)_{kk} = \tr(BA). \end{aligned}\] \( \therefore\ \tr(AB) = \tr(BA) \quad \forall\, A, B \in \F^{n\times n}. \)
Solution — (2)
\( (A\herm)_{ij} = \overline{a_{ji}} = \overline{(A)_{ji}} \). With \( (AB)_{ij} = \sum_{k} a_{ik} b_{kj} \) (summing over the shared index), \[ (A\herm A)_{ij} = \sum_{k=1}^{m} \overline{a_{ki}}\, a_{kj}. \] So \[ \tr(A\herm A) = \sum_{i=1}^{n} (A\herm A)_{ii} = \sum_{i=1}^{n}\sum_{k=1}^{m} \overline{a_{ki}}\, a_{ki} = \sum_{i=1}^{n}\sum_{k=1}^{m} |a_{ki}|^2 \qquad \big( |z|^2 = \overline{z}\,z = z\,\overline{z} \big). \] Therefore \( \tr(A\herm A) = \|A\|_F^2 \). Since \( \tr(AB) = \tr(BA) \), we also have \( \tr(AA\herm) = \tr(A\herm A) \), and so \[ \|A\|_F^2 = \tr(A\herm A) = \tr(AA\herm). \]
Correction
The scan runs every sum to \(n\). Since claim (2) is for \( A \in \F^{m\times n} \), the product \( A\herm A \) is \( n\times n \) — so the trace index \( i = 1,\dots,n \) is right — but the contraction is over the \(m\) rows of \(A\), so those \(k\)-sums run \( k = 1,\dots,m \). (For square \(A\), \(m=n\) and the distinction vanishes.)
Solution — (3)
For \( u \in \R^n \), \( \|u\|^2 = u \cdot u = u_1^2 + \dots + u_n^2 \). Now \[ uu\T = \begin{bmatrix} u_1 \\ \vdots \\ u_n \end{bmatrix}_{n\times 1} \begin{bmatrix} u_1 & \cdots & u_n \end{bmatrix}_{1\times n} = (\text{an } n\times n \text{ matrix}), \qquad (uu\T)_{ij} = u_i u_j. \] Hence \( \tr(uu\T) = \sum_{i=1}^{n} u_i \cdot u_i = \sum_{i=1}^{n} u_i^2 = \|u\|^2. \)
Solution — (4)
For \( u \in \C^n \), \( \|u\|^2 = |u_1|^2 + \dots + |u_n|^2 \) (for \( z \in \C \), \( |z|^2 = z\overline{z} = \overline{z}z \)). Since \( (uu\herm)_{ij} = u_i \overline{u_j} \), \[ \tr(uu\herm) = \sum_{i=1}^{n} u_i \overline{u_i} = \sum_{i=1}^{n} |u_i|^2 = \|u\|^2. \]
Solution — (5)
From Ch. 3, \( M \sim N \) iff there is an invertible matrix \( P \) such that \( M = PNP^{-1} \). So \( A \sim B \implies \exists\, P : A = PBP^{-1} \), and \[\begin{aligned} \tr(A) &= \tr(PBP^{-1}) = \tr\big((PB)P^{-1}\big) \\ &= \tr\big(P^{-1}(PB)\big) = \tr(IB) = \tr(B) \qquad (\text{as } \tr(AB) = \tr(BA)). \end{aligned}\] \( \therefore\ \tr(A) = \tr(B). \)
Ex 74
If \( A \) and \( B \) are matrix representations of an operator \( T \), show that \( \tr A = \tr B \). Thus we can define the trace of \( T \) as \( \tr A \), where \( A = [T] \).
Solution
From Ch. 3 (Ex 37), if \( M \) and \( N \) are two matrix representations of a linear operator corresponding to two different bases, then \( M \sim N \).
Let \( T \in \Lin(V) \), and let \( \phi \) and \( \psi \) be the coordinate maps for the two bases of \( V \). Then \[ T = \phi^{-1} A\, \phi = \psi^{-1} B\, \psi \implies A = \phi\psi^{-1} B\, \psi\phi^{-1}. \] Put \( P = \phi\psi^{-1} \Rightarrow P^{-1} = \psi\phi^{-1} \). Then \( A = PBP^{-1} \implies A \sim B \). By Ex 73 (5), \( \tr A = \tr B \), so we are done.
\( [n] = \{1, 2, \dots, n\} \). A permutation on \( [n] \) is a reordering of the elements of \( [n] \).
Definition 35 — Permutation
A permutation on \( [n] \) is a bijection \( \sigma : [n] \to [n] \). We use \( \Pi_n \) to denote the set of permutations on \( [n] \).
Example 10
\( \Pi_1 = \{(1)\}, \qquad \Pi_2 = \{(1,2),\ (2,1)\}. \)
Ex 75
Compute \( |\Pi_n| \), i.e. the number of possible permutations on \( [n] \).
Solution
\( [n] = \{1, 2, \dots, n\} \). Filling the \( n \) positions:
\[ n \times (n-1) \times (n-2) \times \dots \times 1 = n!. \qquad \therefore\ |\Pi_n| = n!. \]
→ A permutation can be decomposed into elementary permutations.
Definition 36 — Transposition
A permutation \( \tau \in \Pi_n \) is called a transposition if there exist \( p, q \in [n] \), \( p \neq q \), such that \[ \sigma(i) = \begin{cases} q & i = p \\ p & i = q \\ i & i \notin \{p, q\} \end{cases} \] We use \( T_n \) to denote the set of transpositions on \( [n] \). In other words, if a permutation exchanges just two elements and keeps the rest the same, it is a transposition.
Example 11
\( (2,1) \), \( (3,2,1) \) — transpositions.
\( (2,3,1) \), \( (3,1,2) \) — not transpositions.
Ex 76
Compute \( |T_n| \) and compare with \( |\Pi_n| \).
Solution
For a transposition, choose any two positions to exchange. Choosing 2 out of \( n \): \( {}^{n}C_{2} = \dfrac{n(n-1)}{2} \). Hence \[ |T_n| = {}^{n}C_{2} = \frac{n(n-1)}{2}, \qquad \text{while}\quad |\Pi_n| = n! = n(n-1)(n-2)\cdots 2 \cdot 1. \]
Ex 77
Show that any permutation is a composition of transpositions, i.e. \[ \forall\, \sigma \in \Pi_n,\ \exists\, \tau_1, \dots, \tau_k \in T_n : \sigma = \tau_1 \circ \dots \circ \tau_k. \]
Solution
Define a cycle \( (a_1\, a_2 \cdots a_k) \) by \[ \sigma(a_i) = \begin{cases} a_{i+1} & 1 \le i < k \\ a_1 & i = k \\ a_i & \text{for all other } i \end{cases} \]
For \( \sigma \in \Pi_n \), define a relation \( \sim\ \subseteq [n] \times [n] \) by \[ \forall\, a, b \in [n], \quad a \sim b \iff \exists\, n \in \Z : \sigma^{n}(a) = b. \] This is an equivalence relation:
Thus \( \sim \) partitions \( [n] \) into orbits, and each orbit corresponds to a cycle (since \( \sigma \) is a bijection, \( \sigma^{k}(a) = a \) after some \( k \) steps). So any permutation \( \sigma \in \Pi_n \) decomposes into disjoint cycles.
Each cycle decomposes into transpositions: \[ (a_1\, a_2 \cdots a_k) = (a_1\, a_k)(a_1\, a_{k-1}) \cdots (a_1\, a_2) = \tau_1 \circ \dots \circ \tau_{k-1}. \] Doing this for each disjoint cycle, any permutation breaks down into a composition of transpositions.
Example 12
We can write \( (1\,2\,3) \) as a composition of transpositions: \( (1\,2\,3) = (1\,3)(1\,2) \).
→ There are different ways to express a permutation using transpositions; for example, applying the same transposition an even number of times makes no difference. What is invariant is the parity of the number of transpositions in the decomposition.
Ex 78
Let \( \sigma \in \Pi_n \). Suppose there exist \( \tau_1, \dots, \tau_k \in T_n \) and \( s_1, \dots, s_l \in T_n \) such that \[ \sigma = \tau_1 \circ \dots \circ \tau_k = s_1 \circ \dots \circ s_l. \] Show that \( k \) and \( l \) are both even or both odd. Thus every \( \sigma \in \Pi_n \) has even parity (an even permutation) or odd parity (an odd permutation).
Solution
First, the identity permutation \( \sigma_I(x) = x \) can be written as \( \sigma_I = (1\,2)(1\,2) \) — exchange the first two elements, then exchange again — so \( \sigma_I \) has even parity.
Now take \( \sigma = \tau_1 \circ \dots \circ \tau_k = s_1 \circ \dots \circ s_l \). Since a transposition exchanges just two elements, it is its own inverse: \( \tau_i^{-1} = \tau_i\ (1 \le i \le k) \), \( s_j^{-1} = s_j\ (1 \le j \le l) \). So \[ \sigma^{-1} = \tau_k \circ \dots \circ \tau_1 = s_l \circ \dots \circ s_1, \] and therefore \[ \sigma_I = \sigma\sigma^{-1} = \underbrace{\tau_1 \circ \dots \circ \tau_k \circ s_l \circ \dots \circ s_1}_{k + l \text{ transpositions}}. \] Since \( \sigma_I \) has even parity, \( k + l \) must be even \( \implies \) both \( k \) and \( l \) are odd, or both are even.
Completing the parity argument — Ex 78, added in review
The step "\( \sigma_I \) has even parity" still needs proof: a priori \( \sigma_I \) might also be writable as an odd number of transpositions. Use the inversion count \( \operatorname{inv}(\pi) = \#\{(i,j) : i \lt j,\ \pi(i) \gt \pi(j)\} \). Composing \( \pi \) with a transposition changes \( \operatorname{inv} \) by an odd number, so it flips the parity of \( \operatorname{inv}(\pi) \). Hence if \( \pi = t_1\circ\cdots\circ t_m \) is any product of \(m\) transpositions, \( (-1)^{\operatorname{inv}(\pi)} = (-1)^m \). For \( \pi = \sigma_I \), \( \operatorname{inv}(\sigma_I)=0 \), so \( (-1)^m = +1 \) forces \(m\) even — i.e. \( \sigma_I \) is genuinely even, completing the step above. (Equivalently and more directly, \( (-1)^k = (-1)^{\operatorname{inv}(\sigma)} = (-1)^l \), so \(k,l\) share parity.)
Definition 37 — Sign
The sign of a permutation \( \sigma \in \Pi_n \) is \[ \sign(\sigma) = \begin{cases} +1 & \sigma \text{ is even} \\ -1 & \sigma \text{ is odd} \end{cases} \]
Ex 79
Prove the following results.
Solution — (1)
Let \( \sigma = \tau_1 \circ \dots \circ \tau_k \) and \( s = s_1 \circ \dots \circ s_l \) (compositions of transpositions). Then \( \sigma s = \tau_1 \circ \dots \circ \tau_k \circ s_1 \circ \dots \circ s_l \), so \[ \sign(\sigma s) = (-1)^{k+l} = (-1)^{k}(-1)^{l} = \sign(\sigma)\,\sign(s). \]
Solution — (2)
Take any \( w \in \Pi_n \). From \( \sigma s = w \) we get \( s = \sigma^{-1} w \); and from \( s\sigma = \theta \) (with \( \theta \in \Pi_n \)) we get \( s = \theta\sigma^{-1} \). Since \( \sigma \) is bijective, \( \sigma^{-1} \in \Pi_n \). Hence any \( s \in \Pi_n \) can be written as \( s = \sigma^{-1} w \) or \( s = \theta\sigma^{-1} \).
So for any \( \tau \in \Pi_n \), \[ \tau = \sigma s = \sigma(\sigma^{-1} w) = w, \qquad \text{or} \qquad \tau = s\sigma = (\theta\sigma^{-1})\sigma = \theta. \] Thus \( \Pi_n = \{ \sigma s : s \in \Pi_n \} = \{ s\sigma : s \in \Pi_n \}\ \forall\, \sigma \in \Pi_n. \)
Solution — (3)
\( \sigma \) is bijective, so \( \sigma^{-1} \) exists and \( \sigma^{-1} \in \Pi_n \); hence \( \{ \sigma^{-1} : \sigma \in \Pi_n \} \subseteq \Pi_n \). We can also write \( \sigma = (\sigma^{-1})^{-1} \) for any \( \sigma \in \Pi_n \) — every permutation is the inverse of some permutation. So \( \Pi_n = \{ \sigma^{-1} : \sigma \in \Pi_n \} \).
A \( 3 \times 3 \) example. Let \[ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix}. \] Then \[ \det(A) = a_{11}(a_{22}a_{33} - a_{32}a_{23}) - a_{12}(a_{21}a_{33} - a_{31}a_{23}) + a_{13}(a_{21}a_{32} - a_{31}a_{22}) \] \[ = a_{11}a_{22}a_{33} - a_{11}a_{32}a_{23} - a_{12}a_{21}a_{33} + a_{12}a_{31}a_{23} + a_{13}a_{21}a_{32} - a_{13}a_{31}a_{22}. \]
Rearranging terms, \[ \det(A) = a_{11}a_{22}a_{33} - a_{11}a_{32}a_{23} - a_{21}a_{12}a_{33} + a_{21}a_{32}a_{13} + a_{31}a_{12}a_{23} - a_{31}a_{22}a_{13}, \] where each product is read as \( a_{\sigma(1),\,1}\, a_{\sigma(2),\,2}\, a_{\sigma(3),\,3} \): the column indices are \( 1, 2, 3 \) and the row indices are \( \sigma(1), \sigma(2), \sigma(3) \) for \( \sigma \in \Pi_n \).
Definition 38 — Determinant (Leibniz)
The determinant of \( A = (a_{ij}) \in \F^{n\times n} \) is defined as
Ex 80
Let \( A, B \in \F^{n\times n} \). Deduce the following properties from the Leibniz definition of determinants.
(The original notes stated these as exercises without proof; worked solutions are added in review below.)
Solution — Ex 80, added in review
Throughout, \( \det(A) = \sum_{\sigma\in\Pi_n}\sign(\sigma)\prod_{j=1}^n a_{\sigma(j),\,j} \).
(1) Row form. In \( \prod_j a_{\sigma(j),j} \) set \( i=\sigma(j) \), i.e. \( j=\sigma^{-1}(i) \): \( \prod_j a_{\sigma(j),j} = \prod_i a_{i,\,\sigma^{-1}(i)} \). As \(\sigma\) ranges over \(\Pi_n\) so does \(\rho=\sigma^{-1}\) (Ex 79.3), with \( \sign(\sigma)=\sign(\sigma^{-1}) \); hence \( \det(A)=\sum_\rho\sign(\rho)\prod_i a_{i,\rho(i)} \).
(2) \( \det(I_n)=1 \). For \( I_n \), \( a_{\sigma(j),j}=\delta_{\sigma(j),j} \), so \( \prod_j a_{\sigma(j),j}\neq0 \) only for \( \sigma=\mathrm{id} \); that term is \( \sign(\mathrm{id})\cdot1=1 \).
(3) \( \det(cA)=c^n\det(A) \). Each product takes one entry from each of the \(n\) columns, so \( \prod_j(cA)_{\sigma(j),j}=c^n\prod_j a_{\sigma(j),j} \).
(4) \( \det(A\T)=\det(A) \). \( (A\T)_{\sigma(j),j}=a_{j,\sigma(j)} \), so \( \det(A\T)=\sum_\sigma\sign(\sigma)\prod_j a_{j,\sigma(j)}=\det(A) \) by the row form (1).
(8) Two equal columns \( \Rightarrow \det=0 \). Swapping columns \(p,q\) sends each \(\sigma\) to \(\sigma\circ\tau_{pq}\), flipping \( \sign \); so the swap negates \( \det \). If columns \(p,q\) are equal the swap leaves \(A\) unchanged, giving \( \det(A)=-\det(A) \Rightarrow \det(A)=0 \). (Rows: apply (4).)
(5) \( \det(AB)=\det(A)\det(B) \). With \( (AB)_{i,\sigma(i)}=\sum_k a_{ik}b_{k,\sigma(i)} \), expand the row-form product over \(i\): \[ \det(AB)=\sum_{f:[n]\to[n]}\Big(\prod_i a_{i,f(i)}\Big)\sum_\sigma\sign(\sigma)\prod_i b_{f(i),\sigma(i)}. \] If \(f\) is not a bijection, two rows \(b_{f(i),\cdot}\) coincide and the inner sum is \(0\) by (8); if \(f=\pi\in\Pi_n\), the inner sum is \( \sign(\pi)\det(B) \). So \( \det(AB)=\sum_\pi\sign(\pi)\big(\prod_i a_{i,\pi(i)}\big)\det(B)=\det(A)\det(B) \).
(7) \( A\sim B\Rightarrow\det(A)=\det(B) \). \( A=PBP^{-1} \), so by (5),(2), \( \det(A)=\det(P)\det(B)\det(P^{-1})=\det(B)\det(PP^{-1})=\det(B) \).
(6) \( \det(A)=0\iff A \) is not invertible. If \(A\) is invertible, \( \det(A)\det(A^{-1})=\det(I)=1 \) by (5),(2), so \( \det(A)\neq0 \). If \(A\) is singular, its columns are dependent, so one column is a combination of the others; \( \det \) is linear in that column and vanishes whenever two columns coincide (8), so each resulting term is \(0\) and \( \det(A)=0 \).
(9) \(A\) unitary \( \Rightarrow|\det(A)|=1 \). \( A\herm A=I \), and \( \det(A\herm)=\overline{\det(A)} \) (as \( A\herm=\overline{A\T} \) and (4)). By (5),(2), \( 1=\det(I)=\det(A\herm A)=\overline{\det(A)}\det(A)=|\det(A)|^2 \), so \( |\det(A)|=1 \).