Chapter 7 — Trace and Determinant

Linear Algebra and its Applications · IISc 2024 · typeset reproduction

Trace

The trace is a linear functional on the space of (square) matrices — an element of \( \Lin(\F^{n\times n}, \F) \).

Definition 34 — Trace

The trace of \( A \in \F^{n\times n} \) is \[ \tr(A) = A_{11} + \dots + A_{nn} = \text{(sum of the diagonal elements of } A\text{)}. \]

Ex 73

Show that

  1. \( \tr(AB) = \tr(BA) \quad \forall\, A, B \in \F^{n\times n} \)
  2. \( \|A\|_F^2 = \tr(A\herm A) \quad \forall\, A \in \F^{m\times n} \)
  3. \( \|u\|^2 = \tr(uu\T) \quad \forall\, u \in \R^n \)
  4. \( \|u\|^2 = \tr(uu\herm) \quad \forall\, u \in \C^n \)
  5. \( A \sim B \implies \tr(A) = \tr(B) \)

Solution — (1)

\( \tr(AB) = \sum_{i=1}^{n} (AB)_{ii} \), and \( (AB)_{ii} = \sum_{k=1}^{n} a_{ik} b_{ki} \) (the diagonal elements of \( AB \)). Hence \[\begin{aligned} \tr(AB) &= \sum_{i=1}^{n}\sum_{k=1}^{n} a_{ik} b_{ki} \\ &= \sum_{k=1}^{n}\sum_{i=1}^{n} b_{ki} a_{ik} = \sum_{k=1}^{n} (BA)_{kk} = \tr(BA). \end{aligned}\] \( \therefore\ \tr(AB) = \tr(BA) \quad \forall\, A, B \in \F^{n\times n}. \)

Solution — (2)

\( (A\herm)_{ij} = \overline{a_{ji}} = \overline{(A)_{ji}} \). With \( (AB)_{ij} = \sum_{k} a_{ik} b_{kj} \) (summing over the shared index), \[ (A\herm A)_{ij} = \sum_{k=1}^{m} \overline{a_{ki}}\, a_{kj}. \] So \[ \tr(A\herm A) = \sum_{i=1}^{n} (A\herm A)_{ii} = \sum_{i=1}^{n}\sum_{k=1}^{m} \overline{a_{ki}}\, a_{ki} = \sum_{i=1}^{n}\sum_{k=1}^{m} |a_{ki}|^2 \qquad \big( |z|^2 = \overline{z}\,z = z\,\overline{z} \big). \] Therefore \( \tr(A\herm A) = \|A\|_F^2 \). Since \( \tr(AB) = \tr(BA) \), we also have \( \tr(AA\herm) = \tr(A\herm A) \), and so \[ \|A\|_F^2 = \tr(A\herm A) = \tr(AA\herm). \]

Correction

The scan runs every sum to \(n\). Since claim (2) is for \( A \in \F^{m\times n} \), the product \( A\herm A \) is \( n\times n \) — so the trace index \( i = 1,\dots,n \) is right — but the contraction is over the \(m\) rows of \(A\), so those \(k\)-sums run \( k = 1,\dots,m \). (For square \(A\), \(m=n\) and the distinction vanishes.)

Solution — (3)

For \( u \in \R^n \), \( \|u\|^2 = u \cdot u = u_1^2 + \dots + u_n^2 \). Now \[ uu\T = \begin{bmatrix} u_1 \\ \vdots \\ u_n \end{bmatrix}_{n\times 1} \begin{bmatrix} u_1 & \cdots & u_n \end{bmatrix}_{1\times n} = (\text{an } n\times n \text{ matrix}), \qquad (uu\T)_{ij} = u_i u_j. \] Hence \( \tr(uu\T) = \sum_{i=1}^{n} u_i \cdot u_i = \sum_{i=1}^{n} u_i^2 = \|u\|^2. \)

Solution — (4)

For \( u \in \C^n \), \( \|u\|^2 = |u_1|^2 + \dots + |u_n|^2 \) (for \( z \in \C \), \( |z|^2 = z\overline{z} = \overline{z}z \)). Since \( (uu\herm)_{ij} = u_i \overline{u_j} \), \[ \tr(uu\herm) = \sum_{i=1}^{n} u_i \overline{u_i} = \sum_{i=1}^{n} |u_i|^2 = \|u\|^2. \]

Solution — (5)

From Ch. 3, \( M \sim N \) iff there is an invertible matrix \( P \) such that \( M = PNP^{-1} \). So \( A \sim B \implies \exists\, P : A = PBP^{-1} \), and \[\begin{aligned} \tr(A) &= \tr(PBP^{-1}) = \tr\big((PB)P^{-1}\big) \\ &= \tr\big(P^{-1}(PB)\big) = \tr(IB) = \tr(B) \qquad (\text{as } \tr(AB) = \tr(BA)). \end{aligned}\] \( \therefore\ \tr(A) = \tr(B). \)

Ex 74

If \( A \) and \( B \) are matrix representations of an operator \( T \), show that \( \tr A = \tr B \). Thus we can define the trace of \( T \) as \( \tr A \), where \( A = [T] \).

Solution

From Ch. 3 (Ex 37), if \( M \) and \( N \) are two matrix representations of a linear operator corresponding to two different bases, then \( M \sim N \).

Let \( T \in \Lin(V) \), and let \( \phi \) and \( \psi \) be the coordinate maps for the two bases of \( V \). Then \[ T = \phi^{-1} A\, \phi = \psi^{-1} B\, \psi \implies A = \phi\psi^{-1} B\, \psi\phi^{-1}. \] Put \( P = \phi\psi^{-1} \Rightarrow P^{-1} = \psi\phi^{-1} \). Then \( A = PBP^{-1} \implies A \sim B \). By Ex 73 (5), \( \tr A = \tr B \), so we are done.

View original handwritten scans — Trace (pages 1–5)
Original handwritten page 1 Original handwritten page 2 Original handwritten page 3 Original handwritten page 4 Original handwritten page 5

Permutations

\( [n] = \{1, 2, \dots, n\} \). A permutation on \( [n] \) is a reordering of the elements of \( [n] \).

Definition 35 — Permutation

A permutation on \( [n] \) is a bijection \( \sigma : [n] \to [n] \). We use \( \Pi_n \) to denote the set of permutations on \( [n] \).

Example 10

\( \Pi_1 = \{(1)\}, \qquad \Pi_2 = \{(1,2),\ (2,1)\}. \)

Ex 75

Compute \( |\Pi_n| \), i.e. the number of possible permutations on \( [n] \).

Solution

\( [n] = \{1, 2, \dots, n\} \). Filling the \( n \) positions:

\[ n \times (n-1) \times (n-2) \times \dots \times 1 = n!. \qquad \therefore\ |\Pi_n| = n!. \]

→  A permutation can be decomposed into elementary permutations.

Definition 36 — Transposition

A permutation \( \tau \in \Pi_n \) is called a transposition if there exist \( p, q \in [n] \), \( p \neq q \), such that \[ \sigma(i) = \begin{cases} q & i = p \\ p & i = q \\ i & i \notin \{p, q\} \end{cases} \] We use \( T_n \) to denote the set of transpositions on \( [n] \). In other words, if a permutation exchanges just two elements and keeps the rest the same, it is a transposition.

Example 11

\( (2,1) \), \( (3,2,1) \) — transpositions.
\( (2,3,1) \), \( (3,1,2) \) — not transpositions.

Ex 76

Compute \( |T_n| \) and compare with \( |\Pi_n| \).

Solution

For a transposition, choose any two positions to exchange. Choosing 2 out of \( n \): \( {}^{n}C_{2} = \dfrac{n(n-1)}{2} \). Hence \[ |T_n| = {}^{n}C_{2} = \frac{n(n-1)}{2}, \qquad \text{while}\quad |\Pi_n| = n! = n(n-1)(n-2)\cdots 2 \cdot 1. \]

Ex 77

Show that any permutation is a composition of transpositions, i.e. \[ \forall\, \sigma \in \Pi_n,\ \exists\, \tau_1, \dots, \tau_k \in T_n : \sigma = \tau_1 \circ \dots \circ \tau_k. \]

Solution

Define a cycle \( (a_1\, a_2 \cdots a_k) \) by \[ \sigma(a_i) = \begin{cases} a_{i+1} & 1 \le i < k \\ a_1 & i = k \\ a_i & \text{for all other } i \end{cases} \]

For \( \sigma \in \Pi_n \), define a relation \( \sim\ \subseteq [n] \times [n] \) by \[ \forall\, a, b \in [n], \quad a \sim b \iff \exists\, n \in \Z : \sigma^{n}(a) = b. \] This is an equivalence relation:

Thus \( \sim \) partitions \( [n] \) into orbits, and each orbit corresponds to a cycle (since \( \sigma \) is a bijection, \( \sigma^{k}(a) = a \) after some \( k \) steps). So any permutation \( \sigma \in \Pi_n \) decomposes into disjoint cycles.

Each cycle decomposes into transpositions: \[ (a_1\, a_2 \cdots a_k) = (a_1\, a_k)(a_1\, a_{k-1}) \cdots (a_1\, a_2) = \tau_1 \circ \dots \circ \tau_{k-1}. \] Doing this for each disjoint cycle, any permutation breaks down into a composition of transpositions.

Example 12

We can write \( (1\,2\,3) \) as a composition of transpositions: \( (1\,2\,3) = (1\,3)(1\,2) \).

→  There are different ways to express a permutation using transpositions; for example, applying the same transposition an even number of times makes no difference. What is invariant is the parity of the number of transpositions in the decomposition.

Ex 78

Let \( \sigma \in \Pi_n \). Suppose there exist \( \tau_1, \dots, \tau_k \in T_n \) and \( s_1, \dots, s_l \in T_n \) such that \[ \sigma = \tau_1 \circ \dots \circ \tau_k = s_1 \circ \dots \circ s_l. \] Show that \( k \) and \( l \) are both even or both odd. Thus every \( \sigma \in \Pi_n \) has even parity (an even permutation) or odd parity (an odd permutation).

Solution

First, the identity permutation \( \sigma_I(x) = x \) can be written as \( \sigma_I = (1\,2)(1\,2) \) — exchange the first two elements, then exchange again — so \( \sigma_I \) has even parity.

Now take \( \sigma = \tau_1 \circ \dots \circ \tau_k = s_1 \circ \dots \circ s_l \). Since a transposition exchanges just two elements, it is its own inverse: \( \tau_i^{-1} = \tau_i\ (1 \le i \le k) \), \( s_j^{-1} = s_j\ (1 \le j \le l) \). So \[ \sigma^{-1} = \tau_k \circ \dots \circ \tau_1 = s_l \circ \dots \circ s_1, \] and therefore \[ \sigma_I = \sigma\sigma^{-1} = \underbrace{\tau_1 \circ \dots \circ \tau_k \circ s_l \circ \dots \circ s_1}_{k + l \text{ transpositions}}. \] Since \( \sigma_I \) has even parity, \( k + l \) must be even \( \implies \) both \( k \) and \( l \) are odd, or both are even.

Completing the parity argument — Ex 78, added in review

The step "\( \sigma_I \) has even parity" still needs proof: a priori \( \sigma_I \) might also be writable as an odd number of transpositions. Use the inversion count \( \operatorname{inv}(\pi) = \#\{(i,j) : i \lt j,\ \pi(i) \gt \pi(j)\} \). Composing \( \pi \) with a transposition changes \( \operatorname{inv} \) by an odd number, so it flips the parity of \( \operatorname{inv}(\pi) \). Hence if \( \pi = t_1\circ\cdots\circ t_m \) is any product of \(m\) transpositions, \( (-1)^{\operatorname{inv}(\pi)} = (-1)^m \). For \( \pi = \sigma_I \), \( \operatorname{inv}(\sigma_I)=0 \), so \( (-1)^m = +1 \) forces \(m\) even — i.e. \( \sigma_I \) is genuinely even, completing the step above. (Equivalently and more directly, \( (-1)^k = (-1)^{\operatorname{inv}(\sigma)} = (-1)^l \), so \(k,l\) share parity.)

Definition 37 — Sign

The sign of a permutation \( \sigma \in \Pi_n \) is \[ \sign(\sigma) = \begin{cases} +1 & \sigma \text{ is even} \\ -1 & \sigma \text{ is odd} \end{cases} \]

Ex 79

Prove the following results.

  1. \( \sign(\sigma s) = \sign(\sigma)\,\sign(s) \quad \forall\, \sigma, s \in \Pi_n \).
  2. \( \Pi_n = \{ \sigma s : s \in \Pi_n \} = \{ s\sigma : s \in \Pi_n \} \quad \forall\, \sigma \in \Pi_n \).
  3. \( \Pi_n = \{ \sigma^{-1} : \sigma \in \Pi_n \} \).

Solution — (1)

Let \( \sigma = \tau_1 \circ \dots \circ \tau_k \) and \( s = s_1 \circ \dots \circ s_l \) (compositions of transpositions). Then \( \sigma s = \tau_1 \circ \dots \circ \tau_k \circ s_1 \circ \dots \circ s_l \), so \[ \sign(\sigma s) = (-1)^{k+l} = (-1)^{k}(-1)^{l} = \sign(\sigma)\,\sign(s). \]

Solution — (2)

Take any \( w \in \Pi_n \). From \( \sigma s = w \) we get \( s = \sigma^{-1} w \); and from \( s\sigma = \theta \) (with \( \theta \in \Pi_n \)) we get \( s = \theta\sigma^{-1} \). Since \( \sigma \) is bijective, \( \sigma^{-1} \in \Pi_n \). Hence any \( s \in \Pi_n \) can be written as \( s = \sigma^{-1} w \) or \( s = \theta\sigma^{-1} \).

So for any \( \tau \in \Pi_n \), \[ \tau = \sigma s = \sigma(\sigma^{-1} w) = w, \qquad \text{or} \qquad \tau = s\sigma = (\theta\sigma^{-1})\sigma = \theta. \] Thus \( \Pi_n = \{ \sigma s : s \in \Pi_n \} = \{ s\sigma : s \in \Pi_n \}\ \forall\, \sigma \in \Pi_n. \)

Solution — (3)

\( \sigma \) is bijective, so \( \sigma^{-1} \) exists and \( \sigma^{-1} \in \Pi_n \); hence \( \{ \sigma^{-1} : \sigma \in \Pi_n \} \subseteq \Pi_n \). We can also write \( \sigma = (\sigma^{-1})^{-1} \) for any \( \sigma \in \Pi_n \) — every permutation is the inverse of some permutation. So \( \Pi_n = \{ \sigma^{-1} : \sigma \in \Pi_n \} \).

View original handwritten scans — Permutations (pages 6–13)
Original handwritten page 6 Original handwritten page 7 Original handwritten page 8 Original handwritten page 9 Original handwritten page 10 Original handwritten page 11 Original handwritten page 12 Original handwritten page 13

Leibniz Formula for Determinants

A \( 3 \times 3 \) example. Let \[ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix}. \] Then \[ \det(A) = a_{11}(a_{22}a_{33} - a_{32}a_{23}) - a_{12}(a_{21}a_{33} - a_{31}a_{23}) + a_{13}(a_{21}a_{32} - a_{31}a_{22}) \] \[ = a_{11}a_{22}a_{33} - a_{11}a_{32}a_{23} - a_{12}a_{21}a_{33} + a_{12}a_{31}a_{23} + a_{13}a_{21}a_{32} - a_{13}a_{31}a_{22}. \]

Rearranging terms, \[ \det(A) = a_{11}a_{22}a_{33} - a_{11}a_{32}a_{23} - a_{21}a_{12}a_{33} + a_{21}a_{32}a_{13} + a_{31}a_{12}a_{23} - a_{31}a_{22}a_{13}, \] where each product is read as \( a_{\sigma(1),\,1}\, a_{\sigma(2),\,2}\, a_{\sigma(3),\,3} \): the column indices are \( 1, 2, 3 \) and the row indices are \( \sigma(1), \sigma(2), \sigma(3) \) for \( \sigma \in \Pi_n \).

Definition 38 — Determinant (Leibniz)

The determinant of \( A = (a_{ij}) \in \F^{n\times n} \) is defined as

\[ \det(A) = \sum_{\sigma \in \Pi_n} \sign(\sigma) \prod_{j=1}^{n} a_{\sigma(j),\, j}. \]

Properties of Determinants

Ex 80

Let \( A, B \in \F^{n\times n} \). Deduce the following properties from the Leibniz definition of determinants.

  1. \( \det(A) = \displaystyle\sum_{\sigma \in \Pi_n} \sign(\sigma) \prod_{i=1}^{n} a_{i,\,\sigma(i)} \).
  2. \( \det(I_n) = 1 \).
  3. \( \forall\, c \in \F : \det(cA) = c^{n}\det(A) \).
  4. \( \det(A\T) = \det(A) \).
  5. \( \det(AB) = \det(A)\det(B) \).
  6. \( \det(A) = 0 \iff A \text{ is not invertible} \).
  7. \( A \sim B \implies \det(A) = \det(B) \).
  8. \( A \text{ has two identical columns (or rows)} \implies \det(A) = 0 \).
  9. \( A \text{ is unitary} \implies |\det(A)| = 1 \).

(The original notes stated these as exercises without proof; worked solutions are added in review below.)

Solution — Ex 80, added in review

Throughout, \( \det(A) = \sum_{\sigma\in\Pi_n}\sign(\sigma)\prod_{j=1}^n a_{\sigma(j),\,j} \).

(1) Row form. In \( \prod_j a_{\sigma(j),j} \) set \( i=\sigma(j) \), i.e. \( j=\sigma^{-1}(i) \): \( \prod_j a_{\sigma(j),j} = \prod_i a_{i,\,\sigma^{-1}(i)} \). As \(\sigma\) ranges over \(\Pi_n\) so does \(\rho=\sigma^{-1}\) (Ex 79.3), with \( \sign(\sigma)=\sign(\sigma^{-1}) \); hence \( \det(A)=\sum_\rho\sign(\rho)\prod_i a_{i,\rho(i)} \).

(2) \( \det(I_n)=1 \). For \( I_n \), \( a_{\sigma(j),j}=\delta_{\sigma(j),j} \), so \( \prod_j a_{\sigma(j),j}\neq0 \) only for \( \sigma=\mathrm{id} \); that term is \( \sign(\mathrm{id})\cdot1=1 \).

(3) \( \det(cA)=c^n\det(A) \). Each product takes one entry from each of the \(n\) columns, so \( \prod_j(cA)_{\sigma(j),j}=c^n\prod_j a_{\sigma(j),j} \).

(4) \( \det(A\T)=\det(A) \). \( (A\T)_{\sigma(j),j}=a_{j,\sigma(j)} \), so \( \det(A\T)=\sum_\sigma\sign(\sigma)\prod_j a_{j,\sigma(j)}=\det(A) \) by the row form (1).

(8) Two equal columns \( \Rightarrow \det=0 \). Swapping columns \(p,q\) sends each \(\sigma\) to \(\sigma\circ\tau_{pq}\), flipping \( \sign \); so the swap negates \( \det \). If columns \(p,q\) are equal the swap leaves \(A\) unchanged, giving \( \det(A)=-\det(A) \Rightarrow \det(A)=0 \). (Rows: apply (4).)

(5) \( \det(AB)=\det(A)\det(B) \). With \( (AB)_{i,\sigma(i)}=\sum_k a_{ik}b_{k,\sigma(i)} \), expand the row-form product over \(i\): \[ \det(AB)=\sum_{f:[n]\to[n]}\Big(\prod_i a_{i,f(i)}\Big)\sum_\sigma\sign(\sigma)\prod_i b_{f(i),\sigma(i)}. \] If \(f\) is not a bijection, two rows \(b_{f(i),\cdot}\) coincide and the inner sum is \(0\) by (8); if \(f=\pi\in\Pi_n\), the inner sum is \( \sign(\pi)\det(B) \). So \( \det(AB)=\sum_\pi\sign(\pi)\big(\prod_i a_{i,\pi(i)}\big)\det(B)=\det(A)\det(B) \).

(7) \( A\sim B\Rightarrow\det(A)=\det(B) \). \( A=PBP^{-1} \), so by (5),(2), \( \det(A)=\det(P)\det(B)\det(P^{-1})=\det(B)\det(PP^{-1})=\det(B) \).

(6) \( \det(A)=0\iff A \) is not invertible. If \(A\) is invertible, \( \det(A)\det(A^{-1})=\det(I)=1 \) by (5),(2), so \( \det(A)\neq0 \). If \(A\) is singular, its columns are dependent, so one column is a combination of the others; \( \det \) is linear in that column and vanishes whenever two columns coincide (8), so each resulting term is \(0\) and \( \det(A)=0 \).

(9) \(A\) unitary \( \Rightarrow|\det(A)|=1 \). \( A\herm A=I \), and \( \det(A\herm)=\overline{\det(A)} \) (as \( A\herm=\overline{A\T} \) and (4)). By (5),(2), \( 1=\det(I)=\det(A\herm A)=\overline{\det(A)}\det(A)=|\det(A)|^2 \), so \( |\det(A)|=1 \).

View original handwritten scans — Determinant (pages 13–15)
Original handwritten page 13 Original handwritten page 14 Original handwritten page 15