What Is Linear Algebra?

12 min read

2 numbers add to 11 and differ by 3. Which numbers are they? They are 7 and 4, found with a little algebra. Linear algebra studies problems like this one with 3, 300, or 3 million unknowns at once, and the vectors and matrices that hold them.

Learning Objectives

• Solve a system of 2 linear equations in 2 unknowns by elimination, 1 operation per step.
• Sketch the 2 lines of such a system and read the solution as the point where they cross.
• Write a system as a matrix equation $A\mathbf{x} = \mathbf{b}$ and check a solution by computing $A\mathbf{x}$.
• Decide from the picture whether a system of 2 equations in 2 unknowns has 0, 1, or infinitely many solutions.
• Name the 4 course questions and match a problem to the question it asks.

New Vocabulary

New Notation

1. The Objects of Study

1.1 Linear Equations and Systems

A linear equation is an equation in which each unknown is only multiplied by a number and the results are added. The number that multiplies an unknown is its coefficient. So $3x_1 - 2x_2 = 5$ is linear, with coefficients 3 and $-2$. The equation $x_1^2 + x_2 = 1$ is not linear, because $x_1$ is squared, and $x_1 x_2 = 6$ is not linear, because 2 unknowns are multiplied together.

A system of linear equations is 2 or more linear equations in the same unknowns. A solution of the system is a list of numbers, 1 for each unknown, that makes every equation true at the same time. A common mistake is to check only the first equation. A point that satisfies 1 equation and fails another is not a solution of the system.

Your turn 1.1. Is $(x_1, x_2) = (2, 5)$ a solution of the system $x_1 + x_2 = 7$ and $2x_1 - x_2 = 1$?

1.2 Vectors

A vector is an ordered list of numbers written as a column. Each number in it is an entry. The vector $\begin{bmatrix} 7 \\ 4 \end{bmatrix}$ has first entry 7 and second entry 4. We picture it as the point $(7, 4)$ in the plane, or as an arrow from the origin to that point. The set of all vectors with 2 entries is called $\mathbb{R}^2$.

Order matters. The vector $\begin{bmatrix} 7 \\ 4 \end{bmatrix}$ is a different vector from $\begin{bmatrix} 4 \\ 7 \end{bmatrix}$, just as the point $(7, 4)$ is a different point from $(4, 7)$.

1.3 Matrices and the Product $A\mathbf{x}$

A matrix is a rectangular table of numbers. A matrix with 2 rows and 2 columns is called a 2 by 2 matrix. Each number in it is also called an entry.

A matrix can multiply a vector when the matrix has as many columns as the vector has entries. To compute $A\mathbf{x}$, take each row of $A$, multiply its entries by the matching entries of $\mathbf{x}$, and add. Each row gives 1 entry of the answer.

Example. Compute $A\mathbf{x}$ for $A = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}$ and $\mathbf{x} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$.

Row 1 of $A$ is $2, 1$. Multiply by the entries 1, 2 of $\mathbf{x}$ and add: $2(1) + 1(2) = 4$.

Row 2 of $A$ is $1, 3$. Multiply and add: $1(1) + 3(2) = 7$.

So $A\mathbf{x} = \begin{bmatrix} 4 \\ 7 \end{bmatrix}$.

Your turn 1.2. Compute $\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} 3 \\ -1 \end{bmatrix}$.

1.4 Transformations

A transformation is a rule that turns each input vector into an output vector. Every matrix $A$ gives a transformation: the input $\mathbf{x}$ goes to the output $A\mathbf{x}$.

For example, the matrix $R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ turns every vector a quarter turn counterclockwise. The vector $\begin{bmatrix} 1 \\ 0 \end{bmatrix}$, which points right, goes to $R\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0(1) + (-1)(0) \\ 1(1) + 0(0) \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$, which points up.

2. The Same Problem 3 Ways

2.1 The Problem as Equations

“2 numbers add to 11 and differ by 3.” Call the larger number $x_1$ and the smaller number $x_2$. The 2 sentences become 2 linear equations:

$$x_1 + x_2 = 11$$

$$x_1 - x_2 = 3$$

2.2 Solving by Elimination

We remove 1 unknown, solve for the other, and then go back. Each step is 1 operation.

Step 1. Replace equation 2 by (equation 2) minus (equation 1).

$$(x_1 - x_2) - (x_1 + x_2) = 3 - 11, \quad \text{so} \quad -2x_2 = -8.$$

Why this step: $x_1$ has coefficient 1 in both equations, so subtracting removes $x_1$.

Step 2. Divide both sides by $-2$.

$$x_2 = 4.$$

Why this step: it leaves $x_2$ alone on the left.

Step 3. Put $x_2 = 4$ into equation 1.

$$x_1 + 4 = 11.$$

Why this step: equation 1 now has only 1 unknown.

Step 4. Subtract 4 from both sides.

$$x_1 = 7.$$

Answer. The numbers are 7 and 4.

Check. $7 + 4 = 11$ and $7 - 4 = 3$. Both equations hold.

A common mistake in Step 1 is to subtract only the left sides. Whatever we do to the left side of an equation, we do to the right side too, so $3 - 11$ appears on the right.

2.3 The Picture

Each linear equation in 2 unknowns is a line in the plane. Every point on the line $x_1 + x_2 = 11$ has coordinates that add to 11. Every point on the line $x_1 - x_2 = 3$ has coordinates that differ by 3. A solution of the system lies on both lines at once, so it is the point where the lines cross.

2.4 A Matrix Equation

Put the coefficients in a matrix $A$, the unknowns in a vector $\mathbf{x}$, and the right sides in a vector $\mathbf{b}$:

$$A = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}, \quad \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}, \quad \mathbf{b} = \begin{bmatrix} 11 \\ 3 \end{bmatrix}.$$

Row 1 of $A\mathbf{x}$ is $1x_1 + 1x_2$, the left side of equation 1. Row 2 is $1x_1 - 1x_2$, the left side of equation 2. So the whole system is the single equation $A\mathbf{x} = \mathbf{b}$.

Check. With $\mathbf{x} = \begin{bmatrix} 7 \\ 4 \end{bmatrix}$:

$$A\mathbf{x} = \begin{bmatrix} 1(7) + 1(4) \\ 1(7) + (-1)(4) \end{bmatrix} = \begin{bmatrix} 11 \\ 3 \end{bmatrix} = \mathbf{b}.$$

A common mistake is to put the right sides 11 and 3 into the matrix. The matrix holds only coefficients; the right sides go in $\mathbf{b}$.

Your turn 2.1. 2 numbers add to 10 and differ by 4. Write the 2 equations, solve them by elimination, name the point where the 2 lines cross, and check the answer with the matrix equation.

2.5 Counting Solutions

2 lines in the plane can meet in 3 ways, so a system of 2 linear equations in 2 unknowns has 3 possible kinds of answer.

Elimination shows the same 3 cases. When an unknown remains, there is 1 solution. When every unknown cancels and a false statement such as $0 = 2$ is left, there is no solution. When every unknown cancels and $0 = 0$ is left, every point of the line is a solution. With more unknowns there are still only these 3 cases, and row reduction tells them apart.

Your turn 2.2. How many solutions does the system $x_1 + x_2 = 4$, $x_1 + x_2 = 6$ have?

3. The 4 Course Questions

3.1 Solutions

Does this system have a solution, and how many? Section 2.5 answered this for 2 lines. For larger systems the tool is row reduction, which runs elimination in a fixed order. These lessons answer the question: Systems of Linear Equations and Row Reduction, Solution Sets of Linear Systems, The Invertible Matrix Theorem, and Determinants.

3.2 Reach

What can these vectors reach? Given some vectors, which other vectors can we build by stretching them and adding the results? The answer decides whether $A\mathbf{x} = \mathbf{b}$ has a solution for a given $\mathbf{b}$. These lessons answer it: Vector Equations and the Span, The Matrix Equation Ax = b, Linear Independence, Vector Spaces and Subspaces, Null Space, Column Space, and Basis, and Dimension, Rank, and Coordinate Systems. When a vector is out of reach, the closest reachable vector is the next best answer: Inner Products, Length, and Orthogonality and Orthogonal Projections and the Gram-Schmidt Process.

3.3 Action on Space

What does this matrix do to space? Section 1.4 showed a matrix that turns the plane a quarter turn. Other matrices stretch, flip, or flatten it. These lessons answer the question: Linear Transformations and Their Matrices, Matrix Operations and Invertibility, and Determinants, which measures how much a matrix scales area.

3.4 Directions a Matrix Leaves Alone

Which directions does it leave alone? Most vectors change direction when a matrix acts on them. A few special directions only get stretched or flipped. Those directions explain what the matrix does when it is applied again and again. These lessons answer the question: Eigenvalues and Eigenvectors and Diagonalization.

4. The Course Map

4.1 Every Lesson in Order

Each line gives a lesson, the question it answers, and the lessons to finish first. The order meets every prerequisite.

1. What Is Linear Algebra? (this lesson)What does linear algebra study, and what is it for?First: high school algebra
2. Systems of Linear Equations and Row ReductionDoes this system have a solution, and how do we find every solution?First: What Is Linear Algebra?
3. Vector Equations and the SpanWhich vectors can be built from a given set of vectors?First: Systems of Linear Equations and Row Reduction
4. The Matrix Equation Ax = bFor which right sides b does Ax = b have a solution?First: Vector Equations and the Span · Systems of Linear Equations and Row Reduction
5. Solution Sets of Linear Systems and Free VariablesWhen a system has many solutions, how do we write all of them?First: The Matrix Equation Ax = b · Systems of Linear Equations and Row Reduction
6. Linear IndependenceDoes any vector in this set add nothing new?First: The Matrix Equation Ax = b · Solution Sets of Linear Systems and Free Variables
7. Linear Transformations and Their MatricesWhat does a matrix do to the vectors of the plane or of space?First: The Matrix Equation Ax = b
8. Matrix Operations and InvertibilityWhen can the action of a matrix be undone, and how do we find the matrix that undoes it?First: The Matrix Equation Ax = b · Systems of Linear Equations and Row Reduction
9. The Invertible Matrix TheoremIn how many ways can we tell that a square matrix is invertible?First: Matrix Operations and Invertibility · Linear Independence
10. DeterminantsWhich single number tells whether a square matrix is invertible?First: Matrix Operations and Invertibility · The Invertible Matrix Theorem
11. Vector Spaces and SubspacesWhich sets of vectors stay closed under adding and scaling?First: Linear Independence · The Matrix Equation Ax = b
12. Null Space, Column Space, and BasisWhat is the smallest set of vectors that builds a whole subspace?First: Vector Spaces and Subspaces · Solution Sets of Linear Systems and Free Variables · Linear Independence
13. Dimension, Rank, and Coordinate SystemsHow big is a subspace, and how do we name its vectors in a new basis?First: Null Space, Column Space, and Basis · Linear Independence
14. Eigenvalues and EigenvectorsWhich directions does a matrix leave alone?First: Determinants · Matrix Operations and Invertibility · The Invertible Matrix Theorem · Linear Transformations and Their Matrices
15. DiagonalizationWhen can a matrix be written in its simplest form, as a diagonal matrix?First: Eigenvalues and Eigenvectors · Linear Independence · Matrix Operations and Invertibility
16. Inner Products, Length, and OrthogonalityHow do we measure length and angle, and when are 2 vectors perpendicular?First: Vector Equations and the Span · Linear Independence
17. Orthogonal Projections and the Gram-Schmidt ProcessWhat is the closest vector in a subspace when b is out of reach?First: Inner Products, Length, and Orthogonality · Null Space, Column Space, and Basis · Linear Independence

4.2 Reading the Map

The lessons are in the order of the course weeks, so reading from top to bottom always meets a lesson’s prerequisites before the lesson itself. A lesson with several prerequisites joins 2 threads of the course, such as solving systems and describing what vectors reach.

Where This Is Useful

• Many unknowns at once (1.1, 3.1). Traffic through a network of streets obeys 1 linear equation at each intersection: the cars that enter equal the cars that leave. A city map gives dozens of such equations at once, and solving the system finds the flow on every street.
• Computer graphics (1.4, 3.3). Every point of an image is a vector. Multiplying each point by 1 matrix rotates, scales, or flips the whole image at once, and a graphics card does millions of these products every second.
• Ranking web pages (3.4). A search engine can rank pages with a vector that a large matrix of links leaves unchanged. Finding that vector is a question about directions a matrix leaves alone.
• Fitting a line to data (3.2). Measured points almost never lie on 1 line, so the system that asks for a line through all of them has no solution. The closest reachable answer gives the line of best fit.

Check Yourself

Problem 1. 2 numbers add to 15 and differ by 5. Find them by elimination.

Problem 2. Write the system $x_1 + 2x_2 = 8$, $3x_1 - x_2 = 3$ as $A\mathbf{x} = \mathbf{b}$, then show that $\mathbf{x} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$ is the solution.

Problem 3. How many solutions does the system $x_1 + x_2 = 4$, $2x_1 + 2x_2 = 8$ have?

Problem 4. Compute $R\mathbf{v}$ for the quarter-turn matrix $R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ and $\mathbf{v} = \begin{bmatrix} 3 \\ 1 \end{bmatrix}$.

Problem 5. Which of these equations are linear? (a) $4x_1 - x_2 = 0$ (b) $x_1 x_2 = 4$ (c) $\frac{1}{2}x_1 + 7 = x_2$ (d) $x_1^2 = 9$.

Problem 6. Match each problem to 1 of the 4 course questions. (a) A game turns every point of a picture by 30 degrees. (b) A store has 3 prices and 3 total bills and wants each price. (c) A model of a population is applied year after year, and we want the mix of ages that only grows in size.

Watch and Read

• Lynch, MATH 231 lecture playlists (YouTube)Recorded MATH 231 lectures at EWU. Use them to see each topic worked through in a full class session.