Matrix Operations and Invertibility

9 min read

Before A product of matrices extends the product Ax, and an inverse is computed by row reducing [A | I]. Builds on: The Matrix Equation Ax = b · Systems of Linear Equations and Row Reduction

Next Invertibility leads to the Invertible Matrix Theorem, determinants, eigenvalues, and diagonalization. Leads to: The Invertible Matrix Theorem · Determinants · Eigenvalues and Eigenvectors · Diagonalization

Reference: Lay, Linear Algebra and Its Applications, Section 2.1: Matrix Operations; Section 2.2: The Inverse of a Matrix

Video: Dr Lynch’s MATH 231 lectures (YouTube playlists)

Before You Start: Prerequisite Check

Can you do these? (Click to reveal self-test)

Test yourself on these prerequisite skills:

Matrix-vector product: What is $\begin{pmatrix}2&-1\\0&3\end{pmatrix}\begin{pmatrix}4\\2\end{pmatrix}$?

Check

\[ \begin{pmatrix}2(4)+(-1)(2)\\0(4)+3(2)\end{pmatrix} = \begin{pmatrix}6\\6\end{pmatrix} \]
Identity matrix: What is $I_2$ and what does $I_2\mathbf{x}$ equal?

Check

$I_2 = \begin{pmatrix}1&0\\0&1\end{pmatrix}$. For any $\mathbf{x} \in \mathbb{R}^2$: $I_2\mathbf{x} = \mathbf{x}$.
Row reduction: What is the RREF of the augmented matrix $\left[\begin{array}{rr|r}2&0&4\\1&1&3\end{array}\right]$?

Check

$R_1 \leftrightarrow R_2$, then $R_2 - 2R_1$, gives RREF $\left[\begin{array}{rr|r}1&0&2\\0&1&1\end{array}\right]$. Solution: $x_1=2, x_2=1$.

If you struggled: Review The Matrix Equation Ax = b.

Try This First

Compute both $AB$ and $BA$ for:

\[ A = \begin{pmatrix}1&2\\3&4\end{pmatrix}, \quad B = \begin{pmatrix}0&1\\1&0\end{pmatrix} \]

Do you get the same matrix both times?

Show Solution

\[ AB = \begin{pmatrix}1(0)+2(1)&1(1)+2(0)\\3(0)+4(1)&3(1)+4(0)\end{pmatrix} = \begin{pmatrix}2&1\\4&3\end{pmatrix} \]

\[ BA = \begin{pmatrix}0(1)+1(3)&0(2)+1(4)\\1(1)+0(3)&1(2)+0(4)\end{pmatrix} = \begin{pmatrix}3&4\\1&2\end{pmatrix} \]

$AB \neq BA$. Matrix multiplication is not commutative. The order matters: reversing the factors generally changes the result.

This is the most important difference between matrix arithmetic and number arithmetic. Nearly every other property from numbers (associativity, distributivity) does carry over to matrices, but commutativity does not.

The Algebra of Matrices

Matrices form an arithmetic: they can be added, scaled, multiplied, and transposed. The resulting algebra is essential for everything that follows in linear algebra.

The key new feature: matrix multiplication encodes composition of linear transformations. The product $AB$ represents “first apply $B$, then apply $A$.” Reversing the order applies the transformations in the opposite sequence, so $BA$ is generally different from $AB$.

Prerequisite Map

Quick Reference

Property	Value
Course	MATH 231
Difficulty	Intermediate
Time	~30 minutes
Source	Lay, Linear Algebra and Its Applications, Sections 2.1 and 2.2

Matrix Operations Summary

Operation	Rule	Size of Result
Addition $A + B$	Add entry by entry	Same as $A$ and $B$ (must match)
Scalar multiply $cA$	Multiply every entry by $c$	Same as $A$
Product $AB$	$(AB)_{ij} = $ row $i$ of $A$ dotted with col $j$ of $B$	$m \times p$ if $A$ is $m \times n$ and $B$ is $n \times p$
Transpose $A^T$	$(A^T)_{ij} = A_{ji}$	$n \times m$ if $A$ is $m \times n$

Key Concepts

Matrix Multiplication

If $A$ is $m \times n$ and $B$ is $n \times p$, then $AB$ is $m \times p$. The $(i,j)$ entry of $AB$ is:

\[ (AB)_{ij} = \text{row } i \text{ of } A \cdot \text{col } j \text{ of } B = \sum_{k=1}^{n} a_{ik} b_{kj} \]

The inner dimensions must match: the number of columns of $A$ must equal the number of rows of $B$.

Column interpretation: The columns of $AB$ are $A$ times the corresponding columns of $B$: \[ AB = A[\mathbf{b}_1 \; \mathbf{b}_2 \; \cdots \; \mathbf{b}_p] = [A\mathbf{b}_1 \; A\mathbf{b}_2 \; \cdots \; A\mathbf{b}_p] \]

Properties of matrix multiplication:

Associative: $(AB)C = A(BC)$
Distributive: $A(B+C) = AB + AC$
Not commutative: $AB \neq BA$ in general
No cancellation: $AB = AC$ does not imply $B = C$
Identity: $AI = IA = A$ for the appropriate identity matrix

Source: Lay, Linear Algebra and Its Applications, Section 2.1: Matrix Operations.

Transpose

The transpose of $A$, written $A^T$, is obtained by reflecting across the main diagonal: rows become columns.

\[ \text{If } A = \begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}, \text{ then } A^T = \begin{pmatrix}1&4\\2&5\\3&6\end{pmatrix} \]

Key transpose property: $(AB)^T = B^T A^T$ (the order reverses).

Invertible Matrices

An $n \times n$ matrix $A$ is invertible (or nonsingular) if there exists an $n \times n$ matrix $C$ such that: \[ CA = I_n \quad \text{and} \quad AC = I_n \] The matrix $C$ is unique and is written $A^{-1}$.

Consequences:

If $A$ is invertible, the system $A\mathbf{x} = \mathbf{b}$ has the unique solution $\mathbf{x} = A^{-1}\mathbf{b}$.
Not every matrix is invertible: a matrix with a free variable in its RREF is singular (not invertible).
Only square matrices can be invertible.

Properties of inverses:

$(A^{-1})^{-1} = A$
$(AB)^{-1} = B^{-1}A^{-1}$ (order reverses)
$(A^T)^{-1} = (A^{-1})^T$

Source: Lay, Linear Algebra and Its Applications, Section 2.2: The Inverse of a Matrix.

Finding the Inverse by Row Reduction

Algorithm: To find $A^{-1}$, form the augmented matrix $[A \mid I_n]$ and row-reduce until the left block is $I_n$. The right block is then $A^{-1}$.

\[ [A \mid I_n] \xrightarrow{\text{row reduce}} [I_n \mid A^{-1}] \]

If the left block cannot be reduced to $I_n$, then $A$ is not invertible.

Worked example:

Find $A^{-1}$ for $A = \begin{pmatrix}2&1\\5&3\end{pmatrix}$.

\[ [A \mid I] = \left[\begin{array}{rr|rr} 2 & 1 & 1 & 0 \\ 5 & 3 & 0 & 1 \end{array}\right] \]

$R_1 \to \frac{1}{2}R_1$: \[ \left[\begin{array}{rr|rr} 1 & \frac{1}{2} & \frac{1}{2} & 0 \\ 5 & 3 & 0 & 1 \end{array}\right] \]

$R_2 \to R_2 - 5R_1$: \[ \left[\begin{array}{rr|rr} 1 & \frac{1}{2} & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & -\frac{5}{2} & 1 \end{array}\right] \]

$R_2 \to 2R_2$: \[ \left[\begin{array}{rr|rr} 1 & \frac{1}{2} & \frac{1}{2} & 0 \\ 0 & 1 & -5 & 2 \end{array}\right] \]

$R_1 \to R_1 - \frac{1}{2}R_2$: \[ \left[\begin{array}{rr|rr} 1 & 0 & 3 & -1 \\ 0 & 1 & -5 & 2 \end{array}\right] \]

$A^{-1} = \begin{pmatrix}3&-1\\-5&2\end{pmatrix}$.

Verify: $A \cdot A^{-1} = \begin{pmatrix}2&1\\5&3\end{pmatrix}\begin{pmatrix}3&-1\\-5&2\end{pmatrix} = \begin{pmatrix}6-5&-2+2\\15-15&-5+6\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}$ (correct).

Common Pitfalls

Mistake	Why It Is Wrong	Correct Approach
Assuming $AB = BA$	Commutativity fails for matrices; order determines which transformation is applied first	Always write factors in the order they were given; verify with a counterexample if needed
Thinking every nonzero matrix has an inverse	A matrix is invertible iff it has a pivot in every row and column; many nonzero matrices do not	Row-reduce $A$; if any row becomes all zeros, $A$ is not invertible
Computing the inverse by “dividing”: $A^{-1} = \frac{1}{A}$	Matrix division is not defined	Use row reduction on $[A \mid I]$; the $2 \times 2$ formula also works for $2 \times 2$ only
Reversing the order in $(AB)^{-1} = A^{-1}B^{-1}$	The correct formula is $(AB)^{-1} = B^{-1}A^{-1}$ (reversed)	Think: “undo $B$ first, then undo $A$” because $AB$ applies $B$ first

When $A$ Is Not Invertible

Consider $A = \begin{pmatrix}1&2\\2&4\end{pmatrix}$.

$[A \mid I] = \left[\begin{array}{rr|rr}1&2&1&0\\2&4&0&1\end{array}\right]$

$R_2 - 2R_1$: $\left[\begin{array}{rr|rr}1&2&1&0\\0&0&-2&1\end{array}\right]$

The left block cannot be reduced to $I_2$ (row 2 of the left block is all zeros). $A$ is singular.

Geometrically: both columns of $A$ are multiples of $\begin{pmatrix}1\\2\end{pmatrix}$, so the column space is only a line, not all of $\mathbb{R}^2$. No matrix can “undo” a transformation that collapses two dimensions to one.

Practice Problems

Level 1 Matrix Multiplication

Compute $AB$ where $A = \begin{pmatrix}3&-1\\2&0\end{pmatrix}$ and $B = \begin{pmatrix}1&4\\-2&1\end{pmatrix}$.

Then compute $(AB)^T$ and verify that it equals $B^TA^T$.

Thought Process

For $AB$: each entry $(AB)_{ij}$ is the dot product of row $i$ of $A$ with column $j$ of $B$.

For the transpose: $(AB)^T$ flips $AB$. Then check $B^TA^T$ by computing $B^T$ and $A^T$ and multiplying.

Show Answer

\[ AB = \begin{pmatrix}3(1)+(-1)(-2)&3(4)+(-1)(1)\\2(1)+0(-2)&2(4)+0(1)\end{pmatrix} = \begin{pmatrix}5&11\\2&8\end{pmatrix} \]

\[ (AB)^T = \begin{pmatrix}5&2\\11&8\end{pmatrix} \]

\[ B^T = \begin{pmatrix}1&-2\\4&1\end{pmatrix}, \quad A^T = \begin{pmatrix}3&2\\-1&0\end{pmatrix} \]

\[ B^TA^T = \begin{pmatrix}1(3)+(-2)(-1)&1(2)+(-2)(0)\\4(3)+1(-1)&4(2)+1(0)\end{pmatrix} = \begin{pmatrix}5&2\\11&8\end{pmatrix} \]

$(AB)^T = B^TA^T$ confirmed.

Level 2 Find a 2x2 Inverse by Row Reduction

Find $A^{-1}$ for $A = \begin{pmatrix}3&1\\7&2\end{pmatrix}$ by row-reducing $[A \mid I]$. Verify your answer.

Thought Process

Set up $[A \mid I_2]$ and perform row operations until the left side is $I_2$. The right side will then be $A^{-1}$.

Show Answer

\[ \left[\begin{array}{rr|rr}3&1&1&0\\7&2&0&1\end{array}\right] \]

$R_2 \to 3R_2 - 7R_1$: $[0, 6-7, 0-7, 3-0] = [0, -1, -7, 3]$

\[ \left[\begin{array}{rr|rr}3&1&1&0\\0&-1&-7&3\end{array}\right] \]

$R_2 \to -R_2$: \[ \left[\begin{array}{rr|rr}3&1&1&0\\0&1&7&-3\end{array}\right] \]

$R_1 \to R_1 - R_2$: \[ \left[\begin{array}{rr|rr}3&0&-6&3\\0&1&7&-3\end{array}\right] \]

$R_1 \to \frac{1}{3}R_1$: \[ \left[\begin{array}{rr|rr}1&0&-2&1\\0&1&7&-3\end{array}\right] \]

$A^{-1} = \begin{pmatrix}-2&1\\7&-3\end{pmatrix}$

Verify: $\begin{pmatrix}3&1\\7&2\end{pmatrix}\begin{pmatrix}-2&1\\7&-3\end{pmatrix} = \begin{pmatrix}-6+7&3-3\\-14+14&7-6\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}$ (correct).

Level 3 Find a 3x3 Inverse

Find $A^{-1}$ for $A = \begin{pmatrix}1&2&0\\0&1&3\\0&0&1\end{pmatrix}$.

Thought Process

Set up $[A \mid I_3]$ and row-reduce. Since $A$ is upper triangular with 1s on the diagonal, the forward phase is trivial (no zeros to create below diagonal). Only the backward phase is needed.

Show Answer

\[ [A \mid I_3] = \left[\begin{array}{rrr|rrr} 1&2&0&1&0&0\\0&1&3&0&1&0\\0&0&1&0&0&1 \end{array}\right] \]

$A$ is already in REF (upper triangular, no zero rows). Apply backward phase:

$R_2 \to R_2 - 3R_3$: \[ \left[\begin{array}{rrr|rrr} 1&2&0&1&0&0\\0&1&0&0&1&-3\\0&0&1&0&0&1 \end{array}\right] \]

$R_1 \to R_1 - 2R_2$: \[ \left[\begin{array}{rrr|rrr} 1&0&0&1&-2&6\\0&1&0&0&1&-3\\0&0&1&0&0&1 \end{array}\right] \]

$A^{-1} = \begin{pmatrix}1&-2&6\\0&1&-3\\0&0&1\end{pmatrix}$

Verify row 1: $[1,2,0] \cdot$ cols of $A^{-1}$: $(1, -2+2, 6-6) = (1,0,0)$ (correct).

Level 4 Solve a System Using the Inverse

Given $A = \begin{pmatrix}2&1\\5&3\end{pmatrix}$ with $A^{-1} = \begin{pmatrix}3&-1\\-5&2\end{pmatrix}$ (from the worked example), solve:

(a) $A\mathbf{x} = \begin{pmatrix}7\\18\end{pmatrix}$ using $\mathbf{x} = A^{-1}\mathbf{b}$.

(b) $A\mathbf{x} = \begin{pmatrix}0\\-1\end{pmatrix}$ using $\mathbf{x} = A^{-1}\mathbf{b}$.

Thought Process

If $A$ is invertible and $A\mathbf{x} = \mathbf{b}$, then $\mathbf{x} = A^{-1}\mathbf{b}$. Compute $A^{-1}\mathbf{b}$ directly.

For part (c): think about when using an inverse is efficient versus when row reduction is better.

Show Answer

(a) $\mathbf{x} = A^{-1}\begin{pmatrix}7\\18\end{pmatrix} = \begin{pmatrix}3&-1\\-5&2\end{pmatrix}\begin{pmatrix}7\\18\end{pmatrix} = \begin{pmatrix}21-18\\-35+36\end{pmatrix} = \begin{pmatrix}3\\1\end{pmatrix}$

Verify: $A\begin{pmatrix}3\\1\end{pmatrix} = \begin{pmatrix}7\\18\end{pmatrix}$ (correct).

(b) $\mathbf{x} = A^{-1}\begin{pmatrix}0\\-1\end{pmatrix} = \begin{pmatrix}0+1\\0-2\end{pmatrix} = \begin{pmatrix}1\\-2\end{pmatrix}$

Verify: $A\begin{pmatrix}1\\-2\end{pmatrix} = \begin{pmatrix}2-2\\5-6\end{pmatrix} = \begin{pmatrix}0\\-1\end{pmatrix}$ (correct).

(c) $A^{-1}$ can be used because $A$ is invertible ($\det(A) = 2(3)-1(5) = 1 \neq 0$). Using $A^{-1}$ is especially efficient when you need to solve $A\mathbf{x} = \mathbf{b}$ for many different right-hand sides $\mathbf{b}$: compute $A^{-1}$ once, then multiply. If there is only one right-hand side, row reduction on $[A \mid \mathbf{b}]$ is equally efficient without computing the full inverse.

Level 5 Properties of Inverses and Non-Commutativity

Let $A = \begin{pmatrix}2&1\\5&3\end{pmatrix}$ and $B = \begin{pmatrix}1&0\\-1&1\end{pmatrix}$.

(a) Compute $B^{-1}$ by row reduction.

(b) Compute $(AB)^{-1}$ using the formula $(AB)^{-1} = B^{-1}A^{-1}$.

(d) Does $(AB)^{-1} = A^{-1}B^{-1}$? Check with numbers.

Thought Process

For (a): set up $[B \mid I]$ and row-reduce.

For (b): apply the formula $(AB)^{-1} = B^{-1}A^{-1}$ using $A^{-1} = \begin{pmatrix}3&-1\\-5&2\end{pmatrix}$ from the worked example.

For (c): compute $AB$ first, then find its inverse.

For (d): compute $A^{-1}B^{-1}$ and compare to $(AB)^{-1}$.

Show Answer

(a) $[B \mid I] = \left[\begin{array}{rr|rr}1&0&1&0\\-1&1&0&1\end{array}\right]$. $R_2 + R_1$: $[0,1,1,1]$.

$B^{-1} = \begin{pmatrix}1&0\\1&1\end{pmatrix}$

Verify: $BB^{-1} = \begin{pmatrix}1&0\\-1&1\end{pmatrix}\begin{pmatrix}1&0\\1&1\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}$ (correct).

(b) $(AB)^{-1} = B^{-1}A^{-1} = \begin{pmatrix}1&0\\1&1\end{pmatrix}\begin{pmatrix}3&-1\\-5&2\end{pmatrix} = \begin{pmatrix}3&-1\\3-5&-1+2\end{pmatrix} = \begin{pmatrix}3&-1\\-2&1\end{pmatrix}$

(c) $AB = \begin{pmatrix}2&1\\5&3\end{pmatrix}\begin{pmatrix}1&0\\-1&1\end{pmatrix} = \begin{pmatrix}2-1&1\\5-3&3\end{pmatrix} = \begin{pmatrix}1&1\\2&3\end{pmatrix}$

$[AB \mid I] = \left[\begin{array}{rr|rr}1&1&1&0\\2&3&0&1\end{array}\right]$, $R_2 - 2R_1$: $[0,1,-2,1]$.

RREF: $(AB)^{-1} = \begin{pmatrix}3&-1\\-2&1\end{pmatrix}$. Matches (b).

(d) $A^{-1}B^{-1} = \begin{pmatrix}3&-1\\-5&2\end{pmatrix}\begin{pmatrix}1&0\\1&1\end{pmatrix} = \begin{pmatrix}3-1&-1\\-5+2&2\end{pmatrix} = \begin{pmatrix}2&-1\\-3&2\end{pmatrix}$

$(AB)^{-1} = \begin{pmatrix}3&-1\\-2&1\end{pmatrix} \neq \begin{pmatrix}2&-1\\-3&2\end{pmatrix} = A^{-1}B^{-1}$.

The formula $(AB)^{-1} = A^{-1}B^{-1}$ is wrong. The correct formula reverses the order: $(AB)^{-1} = B^{-1}A^{-1}$.

Common Misconceptions

Common misconception

matrix multiplication is commutative. For real numbers, $ab = ba$ always. For matrices, $AB = BA$ is the exception, not the rule. For any two matrices where both products are defined, compute both and compare: they are typically different. Specific pairs that commute (such as $A$ and $I$, or $A$ and $A^{-1}$) are special cases, not the default.

Common misconception

every nonzero matrix has an inverse. Only square matrices that row-reduce to the identity are invertible. A matrix with a zero row, or any $m \times n$ matrix with $m \neq n$, is not invertible. Even among square matrices, many fail: $\begin{pmatrix}1&2\\2&4\end{pmatrix}$ is nonzero but not invertible because its columns are linearly dependent.

Common misconception

$(AB)^{-1} = A^{-1}B^{-1}$. The correct formula reverses the order: $(AB)^{-1} = B^{-1}A^{-1}$. This follows from the fact that $AB$ applies $B$ first, then $A$; to undo that, you first undo $A$ (apply $A^{-1}$) and then undo $B$ (apply $B^{-1}$). The reversal is the same reason $(AB)^T = B^TA^T$.

Mastery Checklist

Novice (Level 1-2):

I can compute $AB$ using the row-times-column dot product.
I can show that $AB \neq BA$ for a specific pair of matrices.
I can find the inverse of a $2 \times 2$ matrix using $[A \mid I]$.

Competent (Level 3-4):

I can find the inverse of a $3 \times 3$ matrix by row-reducing $[A \mid I]$.
I can use $A^{-1}$ to solve $A\mathbf{x} = \mathbf{b}$ quickly when multiple right-hand sides are given.
I can determine when $[A \mid I]$ shows that $A$ is not invertible.

Proficient (Level 5):

I can apply the formula $(AB)^{-1} = B^{-1}A^{-1}$ and verify it numerically.
I can explain why the reversed-order formula for inverses parallels the transpose rule.
I can decide when to use $A^{-1}\mathbf{b}$ versus row reducing $[A \mid \mathbf{b}]$ for efficiency.

Mental Model

The Composition and Undo

Matrix multiplication models composition: $AB$ means “apply $B$ first, then $A$.” To undo this composite operation, you must undo the steps in reverse: first undo $A$, then undo $B$. That is $(AB)^{-1} = B^{-1}A^{-1}$.

The same logic explains why commutativity fails: “put on socks, then shoes” and “put on shoes, then socks” are different operations with very different results.

An invertible matrix is a transformation that can be completely undone. A singular matrix collapses some dimension: once two distinct inputs are mapped to the same output, there is no way to recover which input produced that output, so no inverse exists.

Connections

Looking ahead in MATH 231:

The Invertible Matrix Theorem: invertibility is equivalent to roughly a dozen other conditions; the computation covered here is the foundation
Determinants: the determinant provides a single-number test for invertibility
Eigenvalues and Eigenvectors: eigenvalue equations involve $(A - \lambda I)$, a matrix operation built on matrix addition and scalar multiplication

Real-world connections:

Cryptography uses matrix multiplication to encode messages; the inverse decodes them.
Solving multiple linear systems with the same coefficient matrix (but different right-hand sides) is most efficient with a precomputed inverse.

Source: Lay, Linear Algebra and Its Applications, Section 2.1: Matrix Operations; Section 2.2: The Inverse of a Matrix.

Previous	Up	Next
Linear Transformations	MATH231 Skills Index	The Invertible Matrix Theorem

Last updated: 2026-06-16

Watch and Read

Lynch, MATH 231 lecture playlists (YouTube)Recorded MATH 231 lectures at EWU. Use them to see each topic worked through in a full class session.