Elementary matrices are obtained by performing a single elementary row operation on an identity matrix. Recall that an elementary row operation is one of the following:

multiplying a row by a non-zero constant.
swapping one row with another row.
adding a multiple of a row to another to another row.

Because there are three types of elementary row operations, there are also three types of elementary matrices.

Example.

Non-elementary matrices

Consider the following matrix:

$$\boldsymbol{A}_1= \begin{pmatrix} 4&0&0\\ 0&5&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{A}_2= \begin{pmatrix} 0&1&0\\ 0&0&1\\ 1&0&0\\ \end{pmatrix}$$

Show that these are not elementary matrices.

Solution. The $3\times3$ identity matrix is:

$$\boldsymbol{I}_3=\begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Matrix $\boldsymbol{A}_1$ can be obtained by performing two elementary row operations on the identity matrix:

multiply the first row of the identity matrix by $4$.
multiply the second row by $5$.

Since an elementary matrix is defined as a matrix that can be obtained from a single elementary operation, $\boldsymbol{A}_1$ is not an elementary matrix.

Similarly, $\boldsymbol{A}_2$ is not an elementary matrix because we must perform two swapping elementary operations on the identity matrix to obtain $\boldsymbol{A}_2$.

■

Example.

Elementary matrices

Consider the following matrices:

$$\boldsymbol{E}_1= \begin{pmatrix} 1&0&0\\ 0&4&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{E}_2= \begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\;\boldsymbol{E}_3= \begin{pmatrix} 1&0&4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Show that these are elementary matrices.

Solution. The $3\times3$ identity matrix is:

$$\boldsymbol{I}_3=\begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Note the following:

matrix $\boldsymbol{E}_1$ can be obtained by the elementary row operation of multiplying the second row of the identity matrix by $4$.
matrix $\boldsymbol{E}_2$ can be obtained by the elementary row operation of swapping the first row and the second row of the identity matrix.
finally, $\boldsymbol{E}_3$ can be obtained by the elementary row operation of multiplying the third row by $4$ and then adding it to the first row.

■

Theorem.

Effect of multiplying elementary matrix on a matrix

Let $\boldsymbol{E}$ be any elementary matrix and $\boldsymbol{A}$ be any matrix. Taking the matrix product $\boldsymbol{EA}$ is equivalent to performing a single elementary row operation on $\boldsymbol{A}$ where the type of the row operation corresponds to the type of the elementary matrix.

Sketch proof. Let's go through a sketch proof and assume that the shape of the elementary matrix $\boldsymbol{E}$ and matrix $\boldsymbol{A}$ are both $3\times3$. This sketch proof can easily be generalized to any square shape.

Consider the following matrices:

$$\boldsymbol{A}= \begin{pmatrix} a&b&c\\ d&e&f\\ g&h&i\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{E}_1= \begin{pmatrix} 1&0&0\\ 0&4&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{E}_2= \begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\;\boldsymbol{E}_3= \begin{pmatrix} 1&0&4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Here, $\boldsymbol{E}_1$, $\boldsymbol{E}_2$ and $\boldsymbol{E}_3$ are the elementary matrices from our previous examplelink. Recall that each of these matrices corresponds to a different type of elementary row operation. Our goal is to compute $\boldsymbol{E}_1\boldsymbol{A}$, $\boldsymbol{E}_2\boldsymbol{A}$ and $\boldsymbol{E}_3\boldsymbol{A}$ and show that these matrix products can be thought of as performing the corresponding elementary row operation on $\boldsymbol{A}$.

Let's start with $\boldsymbol{E}_1\boldsymbol{A}$. Recall that $\boldsymbol{E}_1$ can be obtained by multiplying the second row of the identity matrix by $4$. Let's see what $\boldsymbol{E}_1\boldsymbol{A}$ gives us:

$$\begin{align*} \boldsymbol{E}_1\boldsymbol{A}&= \begin{pmatrix} 1&0&0\\0&4&0\\0&0&1\\ \end{pmatrix}\begin{pmatrix} a&b&c\\d&e&f\\g&h&i\\\end{pmatrix}= \begin{pmatrix}a&b&c\\4d&4e&4f\\g&h&i\\ \end{pmatrix} \end{align*}$$

Multiplying $\boldsymbol{E}_1$ to $\boldsymbol{A}$ results in the second row of $\boldsymbol{A}$ getting multiplied by $4$ - just like the way $\boldsymbol{E}_1$ is obtained from the identity matrix! In other words, the product $\boldsymbol{E}_1\boldsymbol{A}$ represents an elementary row operation of multiplying a single row by a non-zero constant value.

Let's now move on to $\boldsymbol{E}_2\boldsymbol{A}$. Recall that $\boldsymbol{E}_2$ can be obtained by swapping the first row and second row of the identity matrix. Let's see what $\boldsymbol{E}_2\boldsymbol{A}$ gives us:

$$\begin{align*} \boldsymbol{E}_2\boldsymbol{A}&= \begin{pmatrix} 0&1&0\\1&0&0\\0&0&1\\ \end{pmatrix}\begin{pmatrix} a&b&c\\d&e&f\\g&h&i\\\end{pmatrix}= \begin{pmatrix}d&e&f\\a&b&c\\g&h&i\\ \end{pmatrix} \end{align*}$$

Notice how multiplying $\boldsymbol{E}_2$ to $\boldsymbol{A}$ also swaps the first row and the second row of $\boldsymbol{A}$! By multiplying $\boldsymbol{E}_2$ to $\boldsymbol{A}$, we're essentially performing an elementary row operation on $\boldsymbol{A}$ in which we swap the two rows.

Finally, let's compute $\boldsymbol{E}_3\boldsymbol{A}$. We know that $\boldsymbol{E}_3$ is obtained by multiplying the third row by $4$, and then adding it to the first row. Let's see what $\boldsymbol{E}_3\boldsymbol{A}$ gives us:

$$\begin{align*} \boldsymbol{E}_3\boldsymbol{A}&= \begin{pmatrix} 1&0&4\\0&1&0\\0&0&1\\ \end{pmatrix}\begin{pmatrix} a&b&c\\d&e&f\\g&h&i\\\end{pmatrix}= \begin{pmatrix}a+4g&b+4h&c+4i\\d&e&f\\g&h&i\\ \end{pmatrix} \end{align*}$$

Multiplying $\boldsymbol{E}_3$ to $\boldsymbol{A}$ results in multiplying the third row of $\boldsymbol{A}$ by $4$ and then adding it to the first row. Again, multiplying $\boldsymbol{E}_3$ to $\boldsymbol{A}$ can be treated as performing a single elementary row operation on $\boldsymbol{A}$ in which we add a multiple of one row to another.

We have just shown that for any of the three types of elementary matrix $\boldsymbol{E}$, multiplying $\boldsymbol{E}$ to some matrix $\boldsymbol{A}$ results in a single elementary row operation on $\boldsymbol{A}$ whose type corresponds to that of the elementary matrix.

■

Theorem.

Elementary matrices are invertible

Any elementary matrix is invertible, and its inverse is also an elementary matrix. Specifically, the inverses of the three types of elementary matrices are:

	Example of elementary matrix	Inverse of elementary matrix
Multiplying a row by a non-zero constant k	$$\boldsymbol{E}_1=\begin{pmatrix} 1&0&0\\ 0&k&0\\ 0&0&1\\ \end{pmatrix}$$	$$\boldsymbol{E}^{-1}_1 =\begin{pmatrix} 1&0&0\\ 0&1/k&0\\ 0&0&1\\ \end{pmatrix}$$
Swapping one row with another row	$$\boldsymbol{E}_2=\begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix}$$	$$\boldsymbol{E}^{-1}_2 =\begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix}$$
Adding a multiple of a row to another	$$\boldsymbol{E}_3= \begin{pmatrix} 1&0&4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$	$$\boldsymbol{E}_3^{-1} =\begin{pmatrix} 1&0&-4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Proof. Recall that elementary matrices are obtained by performing one of the following elementary row operations on an identity matrix:

multiplying a row by a non-zero constant.
swapping one row with another row.
adding a multiple of a row to another row.

Once again, we will use the elementary matrices below that represent each type of operation:

Let's now show that each type of elementary matrix has an inverse. For the first type, the elementary matrix will be of the following form:

$$\boldsymbol{E}_1=\begin{pmatrix} 1&0&0\\ 0&4&0\\ 0&0&1\\ \end{pmatrix}$$

Remember that the inverse of $\boldsymbol{E}_1$, denoted by $\boldsymbol{E}^{-1}_1$, is a matrix such that $\boldsymbol{E}^{-1}_1\boldsymbol{E}_1= \boldsymbol{E}_1\boldsymbol{E}^{-1}_1 =\boldsymbol{I}_n$. Our goal is to find $\boldsymbol{E}^{-1}$ below:

$$\boldsymbol{E}^{-1}_1\boldsymbol{E}_1=\boldsymbol{I}_n \;\;\;\;\;\Longleftrightarrow\;\;\;\;\; \boldsymbol{E}^{-1}_1 \begin{pmatrix} 1&0&0\\0&4&0\\0&0&1\\ \end{pmatrix}= \begin{pmatrix} 1&0&0\\0&1&0\\0&0&1\\ \end{pmatrix}$$

By definition, an elementary matrix is obtained by performing a single elementary row operation on the identity matrix. This means that we can perform a single elementary row operation that reverses this to give us back the identity matrix. Therefore, the inverse matrix $\boldsymbol{E}^{-1}$ must also be an elementary matrix! In this specific case, the inverse $\boldsymbol{E}^{-1}$ is:

$$\boldsymbol{E}^{-1}_1= \begin{pmatrix} 1&0&0\\0&1/4&0\\0&0&1\\ \end{pmatrix}$$

Multiplying this elementary matrix $\boldsymbol{E}^{-1}_1$ to $\boldsymbol{E}_1$ will result in multiplying the second row of $\boldsymbol{E}_1$ by $1/4$, thereby giving us back the identity matrix!

Next, let's find the inverse of the elementary matrix $\boldsymbol{E}_2$ below:

$$\boldsymbol{E}^{-1}_2\boldsymbol{E}_2=\boldsymbol{I}_n \;\;\;\;\;\Longleftrightarrow\;\;\;\;\; \boldsymbol{E}^{-1}_2 \begin{pmatrix} 0&1&0\\1&0&0\\0&0&1\\ \end{pmatrix}= \begin{pmatrix} 1&0&0\\0&1&0\\0&0&1\\ \end{pmatrix}$$

We know that $\boldsymbol{E}_2$ swaps the first row and the second row. Therefore, its inverse matrix should be another elementary matrix that swaps back the first row and the second row:

$$\boldsymbol{E}^{-1}_2=\begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix}$$

Finally, let's look at the inverse of $\boldsymbol{E}_3$ below:

$$\boldsymbol{E}^{-1}_3\boldsymbol{E}_3=\boldsymbol{I}_n \;\;\;\;\;\Longleftrightarrow\;\;\;\;\; \boldsymbol{E}^{-1}_3 \begin{pmatrix}1&0&4\\0&1&0\\0&0&1\\\end{pmatrix} = \begin{pmatrix} 1&0&0\\0&1&0\\0&0&1\\ \end{pmatrix}$$

The inverse matrix $\boldsymbol{E}_3^{-1}$ is an elementary matrix that corresponds to the elementary row operation of multiplying the third row by $-4$ and then adding it to the first row:

$$\boldsymbol{E}^{-1}_3=\begin{pmatrix} 1&0&-4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

We have managed to show that:

every elementary matrix has an inverse and is thus invertible.
the inverse of an elementary matrix is also an elementary matrix.

This completes the proof.

■

Theorem.

Transpose of an elementary matrix is also an elementary matrix

The transpose of an elementary matrix is also an elementary matrix.

Proof. Let's consider the three types of elementary matrices:

matrix $\boldsymbol{E}_1$ corresponds to multiplying a single row by $k$.
matrix $\boldsymbol{E}_2$ corresponds to interchanging two rows.
matrix $\boldsymbol{E}_3$ corresponds to adding a multiple of one row to another row.

We shall consider $3\times3$ elementary matrices but this proof can be easily extended to a general $n\times{n}$ shape. Here are some examples of each type of elementary matrix:

$$\boldsymbol{E}_1= \begin{pmatrix} 4&0&0\\ 0&1&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{E}_2= \begin{pmatrix} 0&1&0\\ 1&0&0\\ 0&0&1\\ \end{pmatrix},\;\;\;\;\;\boldsymbol{E}_3= \begin{pmatrix} 1&0&4\\ 0&1&0\\ 0&0&1\\ \end{pmatrix}$$

Elementary matrices of type $\boldsymbol{E}_1$ is a diagonal matrix, which means that their transpose is itself.

Next, elementary matrices of type $\boldsymbol{E}_2$ are symmetric because swapping two rows, say the $i$-th and $j$-th row, is equivalent to swapping the $i$-th and $j$-th column. You can confirm this using $\boldsymbol{E}_2$ above. The transpose of a symmetric matrix is itself.

Finally, elementary matrices of type $\boldsymbol{E}_3$ involves adding a multiple $k$ of the $i$-th row to some other $j$-th row. The transpose of $\boldsymbol{E}_3$ is:

$$\boldsymbol{E}_3^T= \begin{pmatrix} 1&0&0\\ 0&1&0\\ 4&0&1\\ \end{pmatrix}$$

This is yet another elementary matrix corresponding to adding a multiple $k$ of the $j$-th row to the $i$-th row.

This completes the proof.

■

Theorem.

Equivalent statements of matrix invertibility

If $\boldsymbol{A}$ is an $n\times{n}$ matrix, then the following statements are equivalent:

$\boldsymbol{A}$ is invertible.
$\boldsymbol{Ax}=\boldsymbol{0}$ has only the trivial solution of $\boldsymbol{x}=\boldsymbol{0}$.
the reduced row echelon form of $\boldsymbol{A}$ is the identity matrix $\boldsymbol{I}_n$. Equivalently, the reduced row echelon form of $\boldsymbol{A}$ does not contain a row with all zeros.
$\boldsymbol{A}$ can be expressed as a product of elementary matrices.

Note the following:

if one of these statements is true, then all statements will be true.
if one of these statements is false, then all statements will be false.

Proof. Our proof will follow the following flow:

$$1\implies2\implies3\implies4\implies1$$

$\color{blue}1\implies2$ - Let's begin by assuming that statement $1$ is true, that is, matrix $\boldsymbol{A}$ is invertible. In other words, the inverse $\boldsymbol{A}^{-1}$ exists. To prove statement $2$, suppose $\boldsymbol{x}_0$ is any solution to $\boldsymbol{Ax}=0$. Multiplying both sides by the inverse matrix $\boldsymbol{A}^{-1}$ yields:

$$\begin{align*} \boldsymbol{A}^{-1}\boldsymbol{A}\boldsymbol{x}_0&=\boldsymbol{A}^{-1}\boldsymbol{0}\\ \boldsymbol{I}_n\boldsymbol{x}_0&=\boldsymbol{0}\\ \boldsymbol{x}_0&=\boldsymbol{0}\\ \end{align*}$$

We have shown that any solution $\boldsymbol{x}_0$ must be the zero vector, which is the trivial solution.

$\color{blue}2\implies3$ - We assume that the only solution to $\boldsymbol{Ax}=\boldsymbol{0}$ is the zero vector $\boldsymbol{x}=\boldsymbol{0}$. The augmented matrix of $\boldsymbol{Ax}=\boldsymbol{0}$ is:

$$\begin{pmatrix} a_{1,1}&a_{1,2}&\cdots&a_{1,n}&\color{purple}0\\ a_{2,1}&a_{2,2}&\cdots&a_{2,n}&\color{purple}0\\ \vdots&\vdots&\smash\ddots&\vdots&\color{purple}\vdots\\ a_{n,1}&a_{n,2}&\cdots&a_{n,n}&\color{purple}0\\ \end{pmatrix}$$

Performing elementary row operations on this augmented matrix does not affect columns with all zeros. After Gaussian elimination, we are guaranteed to end up with an identity matrix:

$$\begin{equation}\label{eq:YsTnZmrYsImILLMMSka} \begin{pmatrix} 1&0&\cdots&0&\color{purple}0\\ 0&1&\cdots&0&\color{purple}0\\ \vdots&\vdots&\smash\ddots&\vdots&\color{purple}0\\ 0&0&\cdots&1&\color{purple}0\\ \end{pmatrix} \end{equation}$$

To understand why, consider the following augmented matrix:

$$\begin{pmatrix} 1&0&0&\color{purple}0\\ 0&1&0&\color{purple}0\\ 0&0&0&\color{purple}0\\ \end{pmatrix}$$

Here, the first two rows imply that $x_1=0$ and $x_2=0$. However, the last row implies that $0x_3=0$, which means $x_3$ can take on infinitely many values - not just $0$. This contradicts our assumption that the only solution to $\boldsymbol{A}\boldsymbol{x}=\boldsymbol{0}$ is the zero vector $\boldsymbol{x}=\boldsymbol{0}$. This means that the reduced row echelon form cannot have a row with all zeros. In other words, the reduced row echelon form must look like \eqref{eq:YsTnZmrYsImILLMMSka}.

$\color{blue}3\implies4$ - We assume that the reduced row echelon form of $\boldsymbol{A}$ is the identity matrix:

$$\begin{pmatrix} 1&0&\cdots&0&\color{purple}0\\ 0&1&\cdots&0&\color{purple}0\\ \vdots&\vdots&\smash\ddots&\vdots&\color{purple}0\\ 0&0&\cdots&1&\color{purple}0\\ \end{pmatrix}$$

This means that we managed to perform elementary row operations on $\boldsymbol{A}$ to obtain the identity matrix. From theoremlink, we know that performing elementary row operations on $\boldsymbol{A}$ is equivalent to multiplying corresponding elementary matrices $\boldsymbol{E}_1$, $\boldsymbol{E}_2$, $\cdots$, $\boldsymbol{E}_m$ to $\boldsymbol{A}$, that is:

$$\begin{equation}\label{eq:Gtg9cIjbX0SgWBZFJC4} \boldsymbol{E}_1\boldsymbol{E}_2\cdots\boldsymbol{E_m} \boldsymbol{A}=\boldsymbol{I}_n \end{equation}$$

We know from theoremlink that all elementary matrices are invertible, which means that each elementary matrix has an inverse. Therefore, \eqref{eq:Gtg9cIjbX0SgWBZFJC4} can be written as:

$$\begin{align*} \boldsymbol{E}_1\boldsymbol{E}_2\cdots\boldsymbol{E_m} \boldsymbol{A}&=\boldsymbol{I}_n\\ \boldsymbol{E}_2\cdots\boldsymbol{E_m} \boldsymbol{A}&=\boldsymbol{E}_1^{-1}\boldsymbol{I}_n\\ \boldsymbol{A}&=\boldsymbol{E}^{-1}_m\cdots\boldsymbol{E}^{-1}_2\boldsymbol{E}^{-1}_1\boldsymbol{I}_n\\ \boldsymbol{A}&=\boldsymbol{E}^{-1}_m\cdots\boldsymbol{E}^{-1}_2\boldsymbol{E}^{-1}_1\\ \end{align*}$$

We have also shown in theoremlink that these inverse matrices are also elementary matrices. Therefore, we have managed to write $\boldsymbol{A}$ as a product of elementary matrices!

$\color{blue}4\implies1$ - We assume that $\boldsymbol{A}$ can be written as a product of elementary matrices, say:

$$\boldsymbol{A} =\boldsymbol{E}^{-1}_m\cdots\boldsymbol{E}^{-1}_2\boldsymbol{E}^{-1}_1$$

All elementary matrices are invertible, and by theorem, we know that the product of invertible matrices is also invertible.

■

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!