Diagonalization by a Similarity Transformation

Definition.

A matrix $\mathit{A}\in M_n(C)$ is diagonalizable by a similarity transformation if there exists an invertible matrix $\mathit{P}\in M_n(C)$ such that

(1)$${\mathit{P}}^{-1}\mathit{A}\mathit{P}=\mathit{D},$$

where $\mathit{D}\in M_n(C)$ is a diagonal matrix.

We say that $\mathit{P}$ is a diagonalizing matrix or that it diagonalizes the matrix $\mathit{A}.$

Lemma.

Consider matrices $\mathit{A},\mathit{P}\in M_n(C).$ Columns ${\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n$ of the matrix $\mathit{P}$ are eigenvectors of the matrix $\mathit{A}:$

(2)$$\mathit{A}{\mathit{X}}_1={\lambda }_1{\mathit{X}}_1,\mathit{A}{\mathit{X}}_2={\lambda }_2{\mathit{X}}_2,\dots \mathit{A}{\mathit{X}}_n={\lambda }_n{\mathit{X}}_n$$

if and only if

(3)$$\mathit{A}\mathit{P}=\mathit{P}\mathit{D},$$

where $\mathit{D}=\text{diag}({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n).$

Indeed, according to the column rule of matrix multiplication:

$$\begin{array}{r}\begin{array}{rl}\mathit{A}\mathit{P}& =\mathit{A}[{\mathit{X}}_1|{\mathit{X}}_2|\dots |{\mathit{X}}_n]=\\ & =[\mathit{A}{\mathit{X}}_1|\mathit{A}{\mathit{X}}_2|\dots |\mathit{A}{\mathit{X}}_n]\\ \mathit{P}\mathit{D}& =[{\lambda }_1{\mathit{X}}_1|{\lambda }_2{\mathit{X}}_2|\dots |{\lambda }_n{\mathit{X}}_n],\end{array}\end{array}$$

and thus the conditions (2) and (3) equivalent.

Theorem 7.

A matrix $\mathit{A}$ is diagonalizable by a similarity transformation (1) if and only if the space $C^n$ has a basis $\mathcal{B}=({\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n)$ consisting of eigenvectors of the matrix $\mathit{A}:$

$$\mathit{A}{\mathit{X}}_1={\lambda }_1{\mathit{X}}_1,\mathit{A}{\mathit{X}}_2={\lambda }_2{\mathit{X}}_2,\dots ,\mathit{A}{\mathit{X}}_n={\lambda }_n{\mathit{X}}_n,{\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n\in C.$$

Then the matrix $\mathit{P}=[{\mathit{X}}_1|{\mathit{X}}_2|\dots |{\mathit{X}}_n],$ whose columns are vectors from the basis $\mathcal{B},$ diagonalizes the matrix $\mathit{A}.$

Proof.

A matrix $\mathit{A}$ is diagonalizable by a similarity transformation if there exists an invertible matrix $\mathit{P}\equiv [{\mathit{X}}_1|{\mathit{X}}_2|\dots |{\mathit{X}}_n]$ such that

$${\mathit{P}}^{-1}\mathit{A}\mathit{P}=\mathit{D},$$

where $\mathit{D}$ is a diagonal matrix: $\mathit{D}=\text{diag}({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n).$ This is equivalent to the conditions

$$\mathit{A}\mathit{P}=\mathit{P}\mathit{D}\text{and}det\mathit{P}\ne 0.$$

The condition $det\mathit{P}\ne 0$ means that $\mathcal{B}=({\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n)$ comprises a linearly independent set of vectors of the space $C^n.$ Moreover, by the above Lemma:

$$\mathit{A}{\mathit{X}}_1={\lambda }_1{\mathit{X}}_1,\mathit{A}{\mathit{X}}_2={\lambda }_2{\mathit{X}}_2,\dots \mathit{A}{\mathit{X}}_n={\lambda }_n{\mathit{X}}_n.$$

In an $n$-dimensional vector space every set of $n$ linearly independent vectors is a basis. Therefore, since $\dim C^n=n,$ so $\mathcal{B}$ is a basis of the space $C^n;$ it is the basis consisting of eigenvectors of the matrix $\mathit{A}.$

On the other hand, if eigenvectors ${\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n$ of the matrix $\mathit{A}\in M_n(C)$ span the space $C^n,$ then the matrix $\mathit{P}$ whose columns are the basis vectors diagonalizes the matrix $\mathit{A}.$

Comments and corollaries.

1.) Every matrix $\mathit{A}\in M_n(C)$ has at least one eigenvalue $\lambda$ and the associated eigenvector $\mathit{X}.$ Hence, because the equation (2) does not require that the eigenvalues ${\lambda }_i$ and $$ the associated eigenvectors ${\mathit{X}}_i$ are distinct, there always exists a matrix $\mathit{P}$ such that the equation (3) holds. In particular, one may take

$${\lambda }_1={\lambda }_2=\dots {\lambda }_n=\lambda ,{\mathit{X}}_1={\mathit{X}}_2=\dots {\mathit{X}}_n=\mathit{X}.$$

Then $\mathit{A}\mathit{P}=\mathit{P}\mathit{D}=\lambda \mathit{P},$ but the matrix $\mathit{P}$ is not invertible and thus the relation does not hold (1).

2.) The formula $\mathit{D}={\mathit{P}}^{-1}\mathit{A}\mathit{P}$ may be interpreted in terms of transformation of a matrix of a linear operator under a change of basis. Consider the space $C^n$ with the canonical basis $\mathcal{E}=({\mathit{e}}_1,{\mathit{e}}_2,\dots {\mathit{e}}_n).$ Let $\mathit{A}$ be the matrix of a linear operator $F\in \text{End}(C^n)$ defined by $F(\mathit{x}):=\mathit{A}\mathit{x},$ $\mathit{x}\in C^n.$ If eigenvectors ${\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_n$ of the operator $F$ are linearly independent, then the matrix $\mathit{P}=[{\mathit{X}}_1|{\mathit{X}}_2|\dots |{\mathit{X}}_n]$ is the transition matrix from the canonical basis $\mathcal{E}$ to the basis $\mathcal{B}=({\mathit{X}}_1,{\mathit{X}}_2,\dots {\mathit{X}}_n)$ consisting of the eigenvectors.

Hence, $\mathit{D}$ is a matrix of the operator $F$ in the basis $\mathcal{B}$ consisting of its eigenvectors. As one should expect, this is a diagonal matrix with the eigenvalues of $F$ on the diagonal.

3.) We know already that the eigenevectors of a linear operator which are associated to different eigenvalues are linearly independent.

Corollary. If a matrix $\mathit{A}\in M_n(C)$ has $n$ distinct eigenvalues, then there exists a similarity transformation which diagonalizes this matrix.

Indeed, if columns of the matrix $\mathit{P}$ are eigenvectors of the matrix $\mathit{A}$ which are associated with distinct eigenvalues, then the matrix $\mathit{P}$ is non-degenerate: $det\mathit{P}\ne 0,$ and thus invertible.

4.) Eigenvectors of a normal operator which are associated with distinct eigenvalues comprise an orthogonal system, and after normalization - an orthonormal system. A matrix whose columns comprise an orthonormal system is unitary.

Corollary. Let $\mathit{A}\in M_n(C)$ be a normal (e.g. Hermitian or unitary) matrix. If $\mathit{A}$ has $n$ distinct eigenvalues, then there exists a unitary similarity transformation which diagonalizes this matrix (a diagonalizing matrix $\mathit{P}$ is unitary: ${\mathit{P}}^{+}\mathit{P}={\mathit{I}}_n).$

Remark. A normal matrix does not have to have $n$ distinct eigenvectors to be diagonalizable. Namely, one can prove a more general

Theorem 8.

A matrix $\mathit{A}\in M_n(C)$ is diagonalizable by a unitary similarity transformation if and only if it is normal.

Application to real matrices.

For a real matrix $\mathit{A}:$ $\mathit{A}\in M_n(R),$ we have ${\mathit{A}}^{+}={\mathit{A}}^T.$ Therefore

$${\mathit{A}}^{+}=\mathit{A}\iff {\mathit{A}}^T=\mathit{A}$$

(a real Hermitian matrix is symmetric), and

$${\mathit{A}}^{+}\mathit{A}={\mathit{I}}_n\iff {\mathit{A}}^T\mathit{A}={\mathit{I}}_n$$

(a real unitary matrix is orthogonal).

Theorem 9.

Every real symmetric or orthogonal matrix is diagonalizable by a unitary similarity transformation.

Eigenvalues, and thus also eigenvectors, of a real symmetric matrix are real. Hence, a unitary diagonalizing matrix is a real orthogonal matrix.

Corollary. Every real symmetric matrix is diagonalizable by a real orthogonal similarity transformation.

In comparison to the previous case, eigenvalues of a real orthogonal matrix (and thus also its eigenvectors) may be complex and not real. Then the unitary diagonalizing matrix will be also complex and not real.

Theorem 10.

If a matrix $\mathit{A}$ is diagonalizable by a similarity transformation, then an algebraic multiplicity of every eigenvalue is equal to the geometric multiplicity.

Proof. $$ If a transformation $\mathit{A}\to {\mathit{P}}^{-1}\mathit{A}\mathit{P}\equiv \mathit{D}$ diagonalizes the matrix $\mathit{A},$ then $\mathit{D}=\text{diag}({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_k),$ where ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_k$ are eigenvalues of the matrix $\mathit{A}.$ The number with which ${\lambda }_i$ occurs on a diagonal of the matrix $\mathit{D}$ is equal to both an algebraic and geometric multiplicity of this eigenvalue.