Inner Products, Length, and Orthogonality
Before The dot product works on vectors as in span, and a set of nonzero orthogonal vectors is independent. Builds on: Vector Equations and the Span · Linear Independence
Next Orthogonality leads to projections and to the Gram-Schmidt process. Leads to: Orthogonal Projections and the Gram-Schmidt Process
Before You Start: Prerequisite Check
Can you do these? (Click to reveal self-test)
Test yourself on these prerequisite skills:
Vector arithmetic: Compute $3\begin{pmatrix}1\\-2\\0\end{pmatrix} + \begin{pmatrix}0\\1\\4\end{pmatrix}$.
Check
$\begin{pmatrix}3\\-6\\0\end{pmatrix} + \begin{pmatrix}0\\1\\4\end{pmatrix} = \begin{pmatrix}3\\-5\\4\end{pmatrix}$.
Linear independence: Is $\left\{\begin{pmatrix}1\\0\end{pmatrix}, \begin{pmatrix}0\\1\end{pmatrix}\right\}$ linearly independent?
Check
Yes: neither is a scalar multiple of the other, and the matrix $\begin{pmatrix}1&0\\0&1\end{pmatrix}$ has 2 pivots.
Pythagorean intuition: In $\mathbb{R}^2$, if $\mathbf{u}$ and $\mathbf{v}$ are perpendicular, what relationship holds between $\|\mathbf{u}\|$, $\|\mathbf{v}\|$, and $\|\mathbf{u}+\mathbf{v}\|$?
Check
The Pythagorean theorem: $\|\mathbf{u}+\mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2$.
If you struggled: Review Vector Equations and Span.
Try This First
Compute $\mathbf{u} \cdot \mathbf{v}$ for $\mathbf{u} = \begin{pmatrix}1\\2\\3\end{pmatrix}$ and $\mathbf{v} = \begin{pmatrix}4\\-2\\0\end{pmatrix}$.
Based on the result, are these vectors orthogonal?
Show Solution
\[ \mathbf{u} \cdot \mathbf{v} = 1(4) + 2(-2) + 3(0) = 4 - 4 + 0 = 0 \]
The dot product is zero. By definition, $\mathbf{u}$ and $\mathbf{v}$ are orthogonal.
Geometrically: these two vectors are perpendicular in $\mathbb{R}^3$.
The central test: $\mathbf{u} \perp \mathbf{v}$ iff $\mathbf{u} \cdot \mathbf{v} = 0$. This algebraic condition replaces any need to measure angles in high-dimensional spaces where angles are not directly visible.
Length and Geometry via the Dot Product
In $\mathbb{R}^2$ and $\mathbb{R}^3$, the familiar notions of length and angle have algebraic counterparts that extend to any $\mathbb{R}^n$:
- Length is the square root of the dot product of a vector with itself.
- Orthogonality (perpendicularity) is zero dot product.
- Angle between two vectors: $\cos\theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}$ (Cauchy-Schwarz inequality shows $-1 \leq \cos\theta \leq 1$).
These definitions give geometry an algebraic foundation that works in any dimension.
Prerequisite Map
Quick Reference
| Property | Value |
|---|---|
| Course | MATH 231 |
| Difficulty | Intermediate |
| Time | ~30 minutes |
| Source | Lay, Linear Algebra and Its Applications, Sections 6.1 and 6.2 |
Key Formulas
| Concept | Formula |
|---|---|
| Dot product | $\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n$ |
| Length (norm) | $\|\mathbf{u}\| = \sqrt{\mathbf{u} \cdot \mathbf{u}}$ |
| Unit vector | $\hat{\mathbf{u}} = \frac{\mathbf{u}}{\|\mathbf{u}\|}$ |
| Orthogonal? | $\mathbf{u} \perp \mathbf{v}$ iff $\mathbf{u} \cdot \mathbf{v} = 0$ |
| Distance | $\text{dist}(\mathbf{u}, \mathbf{v}) = \|\mathbf{u} - \mathbf{v}\|$ |
Key Concepts
The Dot Product
The inner product (or dot product) of $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$ is: \[ \mathbf{u} \cdot \mathbf{v} = \mathbf{u}^T\mathbf{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n \]
Properties (all entries $u_i, v_i, w_i \in \mathbb{R}$, $c \in \mathbb{R}$):
- $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$
- $(\mathbf{u}+\mathbf{v}) \cdot \mathbf{w} = \mathbf{u}\cdot\mathbf{w} + \mathbf{v}\cdot\mathbf{w}$
- $(c\mathbf{u}) \cdot \mathbf{v} = c(\mathbf{u}\cdot\mathbf{v})$
- $\mathbf{u} \cdot \mathbf{u} \geq 0$, with equality iff $\mathbf{u} = \mathbf{0}$
Source: Lay, Linear Algebra and Its Applications, Section 6.1: Inner Product, Length, and Orthogonality.
Length and Unit Vectors
The length (or norm) of $\mathbf{v} \in \mathbb{R}^n$ is: \[ \|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}} = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} \]
The length is always non-negative; $\|\mathbf{v}\| = 0$ only if $\mathbf{v} = \mathbf{0}$.
A unit vector has length 1. To normalize $\mathbf{v}$ (make it a unit vector), divide by its length: \[ \hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|} \]
Scaling: $\|c\mathbf{v}\| = |c| \|\mathbf{v}\|$.
Orthogonality
Two vectors $\mathbf{u}$ and $\mathbf{v}$ in $\mathbb{R}^n$ are orthogonal if $\mathbf{u} \cdot \mathbf{v} = 0$.
Pythagorean Theorem: if $\mathbf{u}$ and $\mathbf{v}$ are orthogonal, then: \[ \|\mathbf{u} + \mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2 \]
Orthogonal complement: the set of all vectors orthogonal to every vector in a subspace $W$ is the orthogonal complement $W^\perp$: \[ W^\perp = \{\mathbf{v} : \mathbf{v} \cdot \mathbf{w} = 0 \text{ for all } \mathbf{w} \in W\} \]
$W^\perp$ is always a subspace. For an $m \times n$ matrix $A$: $\text{Nul}(A) = (\text{Row}(A))^\perp$ and $\text{Nul}(A^T) = (\text{Col}(A))^\perp$.
Orthogonal Sets and Orthonormal Bases
A set of vectors $\{\mathbf{u}_1, \ldots, \mathbf{u}_p\}$ is an orthogonal set if every pair is orthogonal: $\mathbf{u}_i \cdot \mathbf{u}_j = 0$ whenever $i \neq j$.
An orthogonal set of nonzero vectors is automatically linearly independent.
An orthonormal set is an orthogonal set where every vector has length 1. For an orthonormal set: $\mathbf{u}_i \cdot \mathbf{u}_j = \delta_{ij}$ (1 if $i=j$, 0 otherwise).
Expansion in an orthonormal basis: if $\{\mathbf{u}_1, \ldots, \mathbf{u}_n\}$ is an orthonormal basis for $\mathbb{R}^n$, then for any $\mathbf{x}$: \[ \mathbf{x} = (\mathbf{x} \cdot \mathbf{u}_1)\mathbf{u}_1 + (\mathbf{x} \cdot \mathbf{u}_2)\mathbf{u}_2 + \cdots + (\mathbf{x} \cdot \mathbf{u}_n)\mathbf{u}_n \]
There is no system to solve: the coordinates are found by dot products.
Source: Lay, Linear Algebra and Its Applications, Section 6.2: Orthogonal Sets.
Common Pitfalls
| Mistake | Why It Is Wrong | Correct Approach |
|---|---|---|
| Thinking $\mathbf{u} \perp \mathbf{v}$ means $\mathbf{u} \cdot \mathbf{v} = 1$ | Perpendicular vectors have zero dot product, not one | $\mathbf{u} \perp \mathbf{v} \iff \mathbf{u} \cdot \mathbf{v} = 0$ |
| Thinking length can be negative | Length is a square root of a sum of squares: always non-negative | $\|\mathbf{v}\| \geq 0$; equal to zero only for the zero vector |
| Concluding an orthogonal set is automatically a basis | An orthogonal set is independent but might not span the space | An orthogonal set of $n$ nonzero vectors in $\mathbb{R}^n$ is both independent and a basis |
Practice Problems
For $\mathbf{u} = \begin{pmatrix}3\\-1\\2\end{pmatrix}$ and $\mathbf{v} = \begin{pmatrix}-1\\4\\1\end{pmatrix}$:
(a) Compute $\mathbf{u} \cdot \mathbf{v}$.
(b) Compute $\|\mathbf{u}\|$ and $\|\mathbf{v}\|$.
(c) Are $\mathbf{u}$ and $\mathbf{v}$ orthogonal?
(a) Find the unit vector in the direction of $\mathbf{w} = \begin{pmatrix}4\\3\end{pmatrix}$.
(b) Show that $\mathbf{e}_1 = \begin{pmatrix}3/5\\4/5\end{pmatrix}$ and $\mathbf{e}_2 = \begin{pmatrix}-4/5\\3/5\end{pmatrix}$ form an orthonormal set.
Show that the set $S = \left\{\begin{pmatrix}1\\0\\-1\end{pmatrix}, \begin{pmatrix}1\\2\\1\end{pmatrix}, \begin{pmatrix}1\\-1\\1\end{pmatrix}\right\}$ is orthogonal.
Then conclude it is linearly independent without row reduction.
The vectors $\mathbf{u}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\0\\-1\end{pmatrix}$ and $\mathbf{u}_2 = \frac{1}{\sqrt{6}}\begin{pmatrix}1\\2\\1\end{pmatrix}$ form part of an orthonormal set.
Find the coordinates of $\mathbf{y} = \begin{pmatrix}3\\1\\-1\end{pmatrix}$ in the directions $\mathbf{u}_1$ and $\mathbf{u}_2$ using dot products, without solving a linear system.
Also compute $\|\mathbf{y}\|^2$ and compare it with $(\mathbf{y}\cdot\mathbf{u}_1)^2 + (\mathbf{y}\cdot\mathbf{u}_2)^2$.
Let $W = \text{Span}\left\{\begin{pmatrix}1\\-2\\1\end{pmatrix}\right\}$ be a subspace of $\mathbb{R}^3$.
(a) Describe $W^\perp$ as the solution set of a homogeneous equation. What is its dimension?
(b) Find two vectors that form a basis for $W^\perp$.
(c) Verify that each of your basis vectors is orthogonal to the spanning vector of $W$.
Common Misconceptions
orthogonal vectors have dot product equal to 1. The identity element for multiplication in $\mathbb{R}$ is 1, but dot product is not multiplication; it is a sum of products. Orthogonal vectors have dot product equal to 0 (no alignment between them). A dot product of 1 occurs between, for example, $\begin{pmatrix}1\\0\end{pmatrix}$ and $\begin{pmatrix}1\\0\end{pmatrix}$ (same direction and unit length), which is the opposite of orthogonal.
the length of a vector can be negative. Length is the square root of a sum of squares. Since squares are non-negative and the square root is non-negative, $\|\mathbf{v}\| \geq 0$ always. The only vector with length 0 is the zero vector. The component $v_i$ of a vector can be negative, but the length is always non-negative.
checking pairwise orthogonality is the same as verifying the orthogonality of the full set. For a set with more than two vectors, pairwise orthogonality means every pair is orthogonal. This IS the definition of orthogonal set: “every pair.” The misconception is more subtle: verifying one-to-all orthogonality (checking each vector against only one reference vector) is not the same as checking all pairs.
Mastery Checklist
Novice (Level 1-2):
Competent (Level 3-4):
Proficient (Level 5):
Mental Model
The Right-Angle Test
The dot product measures the “alignment” of two vectors. When the dot product is zero, the vectors are pointing in completely independent directions: neither has any component in the other’s direction. In $\mathbb{R}^2$, zero dot product means a 90-degree angle. In higher dimensions, 90 degrees still makes sense via the formula $\cos\theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}$.
An orthogonal set is a collection of arrows, all at right angles to each other. Like the three coordinate axes in $\mathbb{R}^3$ (all mutually perpendicular), an orthogonal set has maximum “independence” structure. This is why an orthogonal set of nonzero vectors is always linearly independent.
Connections
Looking ahead in MATH 231:
- Orthogonal Projection and Gram-Schmidt: orthogonality enables projection formulas and the construction of orthonormal bases from any basis
Real-world connections:
- In signal processing, orthogonal waveforms (like Fourier basis functions) carry independent information; their signals can be separated without interference.
- In statistics, orthogonal regressors do not confound each other, making coefficient estimates independent.
- In quantum mechanics, orthogonal state vectors represent mutually exclusive measurement outcomes.
Source: Lay, Linear Algebra and Its Applications, Section 6.1: Inner Product, Length, and Orthogonality; Section 6.2: Orthogonal Sets.
| Previous | Up | Next |
|---|---|---|
| Diagonalization | MATH231 Skills Index | Orthogonal Projection and Gram-Schmidt |
Last updated: 2026-06-16
Watch and Read
- Lynch, MATH 231 lecture playlists (YouTube)Recorded MATH 231 lectures at EWU. Use them to see each topic worked through in a full class session.