Eigenvectors from eigenvalues

Peter Denton, Stephen Parke, Xining Zhang, and I have just uploaded to the arXiv the short unpublished note “Eigenvectors from eigenvalues“. This note gives two proofs of a general eigenvector identity observed recently by Denton, Parke and Zhang in the course of some quantum mechanical calculations. The identity is as follows:

Theorem 1 Let ${A}$ be an ${n \times n}$ Hermitian matrix, with eigenvalues ${\lambda_1(A),\dots,\lambda_n(A)}$ . Let ${v_i}$ be a unit eigenvector corresponding to the eigenvalue ${\lambda_i(A)}$ , and let ${v_{i,j}}$ be the ${j^{th}}$ component of ${v_i}$ . Then

$\displaystyle |v_{i,j}|^2 \prod_{k=1; k \neq i}^n (\lambda_i(A) - \lambda_k(A)) = \prod_{k=1}^{n-1} (\lambda_i(A) - \lambda_k(M_j))$

where ${M_j}$ is the ${n-1 \times n-1}$ Hermitian matrix formed by deleting the ${j^{th}}$ row and column from ${A}$ .

For instance, if we have

$\displaystyle A = \begin{pmatrix} a & X^* \\ X & M \end{pmatrix}$

for some real number ${a}$ , ${n-1}$ -dimensional vector ${X}$ , and ${n-1 \times n-1}$ Hermitian matrix ${M}$ , then we have

$\displaystyle |v_{i,1}|^2 = \frac{\prod_{k=1}^{n-1} (\lambda_i(A) - \lambda_k(M))}{\prod_{k=1; k \neq i}^n (\lambda_i(A) - \lambda_k(A))} \ \ \ \ \ (1)$

assuming that the denominator is non-zero.

Once one is aware of the identity, it is not so difficult to prove it; we give two proofs, each about half a page long, one of which is based on a variant of the Cauchy-Binet formula, and the other based on properties of the adjugate matrix. But perhaps it is surprising that such a formula exists at all; one does not normally expect to learn much information about eigenvectors purely from knowledge of eigenvalues. In the random matrix theory literature, for instance in this paper of Erdos, Schlein, and Yau, or this later paper of Van Vu and myself, a related identity has been used, namely

$\displaystyle |v_{i,1}|^2 = \frac{1}{1 + \| (M-\lambda_i(A))^{-1} X \|^2}, \ \ \ \ \ (2)$

but it is not immediately obvious that one can derive the former identity from the latter. (I do so below the fold; we ended up not putting this proof in the note as it was longer than the two other proofs we found. I also give two other proofs below the fold, one from a more geometric perspective and one proceeding via Cramer’s rule.) It was certainly something of a surprise to me that there is no explicit appearance of the ${a,X}$ components of ${A}$ in the formula (1) (though they do indirectly appear through their effect on the eigenvalues ${\lambda_k(A)}$ ; for instance from taking traces one sees that ${a = \sum_{k=1}^n \lambda_k(A) - \sum_{k=1}^{n-1} \lambda_k(M)}$ ).

One can get some feeling of the identity (1) by considering some special cases. Suppose for instance that ${A}$ is a diagonal matrix with all distinct entries. The upper left entry ${a}$ of ${A}$ is one of the eigenvalues of ${A}$ . If it is equal to ${\lambda_i(A)}$ , then the eigenvalues of ${M}$ are the other ${n-1}$ eigenvalues of ${A}$ , and now the left and right-hand sides of (1) are equal to ${1}$ . At the other extreme, if ${a}$ is equal to a different eigenvalue of ${A}$ , then ${\lambda_i(A)}$ now appears as an eigenvalue of ${M}$ , and both sides of (1) now vanish. More generally, if we order the eigenvalues ${\lambda_1(A) \leq \dots \leq \lambda_n(A)}$ and ${\lambda_1(M) \leq \dots \leq \lambda_{n-1}(M)}$ , then the Cauchy interlacing inequalities tell us that

$\displaystyle 0 \leq \lambda_i(A) - \lambda_k(M) \leq \lambda_i(A) - \lambda_k(A)$

for ${1 \leq k < i}$ , and

$\displaystyle \lambda_i(A) - \lambda_{k+1}(A) \leq \lambda_i(A) - \lambda_k(M) < 0$

for ${i \leq k \leq n-1}$ , so that the right-hand side of (1) lies between ${0}$ and ${1}$ , which is of course consistent with (1) as ${v_i}$ is a unit vector. Thus the identity relates the coefficient sizes of an eigenvector with the extent to which the Cauchy interlacing inequalities are sharp.

— 1. Relating the two identities —

We now show how (1) can be deduced from (2). By a limiting argument, it suffices to prove (1) in the case when ${\lambda_i(A)}$ is not an eigenvalue of ${M}$ . Without loss of generality we may take ${i=n}$ . By subtracting the matrix ${\lambda_n(A) I_n}$ from ${A}$ (and ${\lambda_n(A) I_{n-1}}$ from ${M_j}$ , thus shifting all the eigenvalues down by ${\lambda_i(A)}$ , we may also assume without loss of generality that ${\lambda_i(A)=0}$ . So now we wish to show that

$\displaystyle |v_{n,1}|^2 \prod_{k=1}^{n-1} \lambda_k(A) = \prod_{k=1}^{n-1} \lambda_k(M).$

The right-hand side is just ${\mathrm{det}(M)}$ . If one differentiates the characteristic polynomial

$\displaystyle \mathrm{det}(A - \lambda I_n) = \prod_{k=1}^n (\lambda_k(A) - \lambda) = - \lambda \prod_{k=1}^{n-1} (\lambda_k(A) - \lambda)$

at ${\lambda=0}$ , one sees that

$\displaystyle \prod_{k=1}^{n-1} \lambda_k(A) = -\frac{d}{d\lambda}\mathrm{det}(A - \lambda I_n)|_{\lambda=0}.$

Finally, (2) can be rewritten as

$\displaystyle |v_{n,1}|^2 = \frac{1}{1 + X^* M^{-2} X}$

so our task is now to show that

$\displaystyle \frac{d}{d\lambda}\mathrm{det}(A - \lambda I_n)|_{\lambda=0} = - \mathrm{det}(M) ( 1 + X^* M^{-2} X ). \ \ \ \ \ (3)$

By Schur complement, we have

$\displaystyle \mathrm{det}(A - \lambda I_n) = \mathrm{det}(M - \lambda I_{n-1}) ( a - \lambda - X^* (M_1 - \lambda I_{n-1})^{-1} X ). \ \ \ \ \ (4)$

Since ${\lambda=0}$ is an eigenvalue of ${A}$ , but not of ${M_1}$ (by hypothesis), the factor ${a - \lambda - X^* (M_1 - \lambda I_{n-1})^{-1} X}$ vanishes when ${\lambda=0}$ . If we then differentiate (4) in ${\lambda}$ and set ${\lambda=0}$ we obtain (3) as desired.

— 2. A geometric proof —

Here is a more geometric way to think about the identity. One can view ${\lambda_i(A) - A}$ as a linear operator on ${{\bf C}^n}$ (mapping ${w}$ to ${(\lambda_i(A) w - Aw)}$ for any vector ${w}$ ); it then also acts on all the exterior powers ${\bigwedge^k {\bf C}^n}$ by mapping ${w_1 \wedge \dots \wedge w_k}$ to ${(\lambda_i(A) w_1 - Aw_1) \wedge \dots \wedge (\lambda_i(A) w_k - Aw_k)}$ for all vectors ${w_1,\dots,w_k}$ . In particular, if one evaluates ${A}$ on the basis ${v_1 \wedge \dots \wedge v_{j-1} \wedge v_{j+1} \wedge \dots \wedge n_n}$ of ${\bigwedge^{n-1} {\bf C}^n}$ induced by the orthogonal eigenbasis ${v_1,\dots,v_n}$ , we see that the action of ${\lambda_i(A) - A}$ on ${\bigwedge^{n-1} {\bf C}^n}$ is rank one, with

$\displaystyle \langle (\lambda_i(A) - A) \omega, \omega \rangle = \prod_{k=1; k \neq i}^n (\lambda_i(A) - \lambda_k(A)) |\omega \wedge v_i|^2$

for any ${\omega \in \bigwedge^{n-1} {\bf C}^n}$ , where ${\langle,\rangle}$ is the inner product on ${\bigwedge^{n-1} {\bf C}^n}$ induced by the standard inner product on ${{\bf C}^n}$ . If we now apply this to the ${n-1}$ -form ${\omega = e_1 \wedge \dots \wedge e_{j-1} \wedge e_{j+1} \wedge \dots \wedge e_n}$ , we have ${|\omega \wedge v_i| = |v_{i,j}|}$ , while ${(\lambda_i(A)-A) \omega}$ is equal to ${\mathrm{det}(\lambda_i(A) I_{n-1} - M) \omega}$ plus some terms orthogonal to ${\omega}$ . Since ${\mathrm{det}(\lambda_i(A) I_{n-1} - M ) = \prod_{k=1}^{n-1} (\lambda_i(A) - \lambda_k(M))}$ , Theorem 1 follows.

— 3. A proof using Cramer’s rule —

By a limiting argument we can assume that all the eigenvalues of ${A}$ are simple. From the spectral theorem we can compute the resolvent ${(\lambda I_n - A)^{-1}}$ for ${\lambda \neq \lambda_1(A),\dots,\lambda_n(A)}$ as

$\displaystyle (\lambda I_n - A)^{-1} = \sum_{k=1}^n \frac{1}{\lambda - \lambda_k(A)} v_k v_k^*.$

Extracting the ${(j,j)}$ component of both sides and using Cramer’s rule, we conclude that

$\displaystyle \frac{\mathrm{det}(\lambda I_{n-1} - M_j)}{\mathrm{det}(\lambda I_n - A)} = \sum_{k=1}^n \frac{1}{\lambda - \lambda_k(A)} |v_{k,j}|^2$

or in terms of eigenvalues

$\displaystyle \frac{\prod_{k=1}^{n-1} (\lambda - \lambda_k(M_j)) }{\prod_{k=1}^{n} (\lambda - \lambda_k(A)) } = \sum_{k=1}^n \frac{1}{\lambda - \lambda_k(A)} |v_{k,j}|^2.$

Both sides are rational functions with a simple pole at the eigenvalues ${\lambda_i(A)}$ . Extracting the residue at ${\lambda = \lambda_i(A)}$ we conclude that

$\displaystyle \frac{\prod_{k=1}^{n-1} (\lambda_i(A) - \lambda_k(M_j)) }{\prod_{k=1; k \neq i}^{n} (\lambda_i(A) - \lambda_k(A)) } = |v_{i,j}|^2$

and Theorem 1 follows. (Note that this approach also gives a formula for ${v_{i,j} \overline{v_{i,j'}}}$ for ${j,j'=1,\dots,n}$ , although the formula becomes messier when ${j \neq j'}$ because the relevant minor of ${\lambda I_n}$ is no longer a scalar multiple of the identity ${I_{n-1}}$ .)