Essence of Linear Algebra - 3b1b

CAUTION

Below are all my not-so-developed ideas/understandings during my course of watching 3Blue1Brown’s Essence of Linear Algebra series

IMPORTANT

Abstraction is the price of generality

On Capturing Information

Many concepts in linear algebra focus on capturing information — on distinguishing what is characteristic of some entity from what is not.

A basis is the representation of a space’s essence. It does double duty. Its vectors span the space, every vector is reachable as a linear combination of them, which is what makes it this space; and they are linearly independent, so those coefficients (the coordinates) are unique. Spanning determines the space; independence makes each vector’s description unambiguous. A basis is thus the minimal information needed both to build the space and to uniquely address every point in it.
A full-rank square matrix is invertible. Such a matrix describes a lossless transformation, a process that erases no information from the resulting vector compared to the entering one. In contrast, a singular matrix removes information permanently and unrecoverably (e.g., collapsing a plane onto a line). This is the geometric heart of rank and nullity: the rank is what survives, the nullity is what gets crushed to zero.
Often there are multiple ways to describe the same entity. For example, a plane in 3D can be described by a point plus two non-collinear direction vectors, or simply by a point and a normal. But note that the cross product $\mathbf{n} = \mathbf{u} \times \mathbf{v}$ does not preserve the two direction vectors, you cannot recover $\mathbf{u}$ and $\mathbf{v}$ from $\mathbf{n}$ , since infinitely many pairs spanning the same plane (with the same signed parallelogram area) yield the same normal. What survives is exactly the plane’s orientation (the normal direction) and the parallelogram’s area (the magnitude). The point is, the cross product is a deliberately lossy map that discards everything except the information characterizing the plane. It is purposeful compression, it keeps precisely the plane-defining part and throws the rest away.

Duality

Duality describes a situation where two notions have a certain inherent, natural correspondence. There are multiple instances of duality in linear algebra.

One of the foremost concepts when learning linear algebra is the dot product, intuitively, a linear-algebraic adaptation of multiplying two objects. One major question I had was how and why the dot product is, loosely speaking, defined/computed in two different ways: $\mathbf{a} \cdot \mathbf{b} = a_1 b_1 + a_2 b_2 + \cdots = \|\mathbf{a}\|\|\mathbf{b}\| \cos(\theta)$ The abstract sum of component products on one side, the geometric projection on the other. There are two separate things worth pulling apart here: why these two views correspond at all (the duality), and a concrete proof that the two formulas are numerically equal (the law of cosines).

The duality: a vector is secretly a transformation

Consider the following equivalency:

\begin{pmatrix} a & b \end{pmatrix} \begin{pmatrix} c \\ d \end{pmatrix} = \begin{pmatrix} a \\ b \end{pmatrix} \cdot \begin{pmatrix} c \\ d \end{pmatrix}

The LHS is a matrix–vector multiplication that transforms a 2D vector in the plane onto a 1D number line; geometrically, this is projecting that vector onto a line (and scaling). The RHS is the dot product. Their equality is the duality itself: every $1 \times n$ “row” is a linear map into 1D (a linear functional), and “tipping it on its side” turns it into a vector. So each vector $\mathbf{v}$ has an associated transformation — project onto $\mathbf{v}$ ‘s line, then scale by $\|\mathbf{v}\|$ — and that is why dotting with $\mathbf{v}$ equals projecting onto $\mathbf{v}$ . The duality explains where projection comes from in the first place.

The proof: why the two formulas coincide

Separately, the two formulas can be shown equal by a purely Euclidean argument. This establishes that they agree; it doesn’t itself invoke the duality above.

Sum of component products

|a - b|^2 = \sum(a_i - b_i)^2 = \sum(a_i^2 - 2a_i b_i + b_i^2) = \sum a_i^2 + \sum b_i^2 - 2\sum a_i b_i

Since $\sum a_i^2 = |a|^2$ and $\sum b_i^2 = |b|^2$ ,

|a-b|^2 = |a|^2 + |b|^2 - 2(a_1 b_1 + \cdots + a_n b_n)

Projection. Using the Law of Cosines,

|a - b|^2 = |a|^2 + |b|^2 - 2|a||b|\cos\theta

Setting the two expressions equal:

\begin{align*} |a|^2 + |b|^2 - 2|a||b|\cos\theta &= |a|^2 + |b|^2 - 2(a_1 b_1 + \dots + a_n b_n) \\ a_1 b_1 + a_2 b_2 + \dots + a_n b_n &= |a|\,|b|\cos\theta \end{align*}

This equivalency holds in any number of dimensions.

Interface

What is a vector?

Mathematicians say it describes a point on a coordinate system relative to the origin.
Physicists say it represents a measurement that has both magnitude and direction.
Computer scientists say it defines a data structure that stores a list of values.

In essence, a vector is abstract — it is inherently meaningless. It defines a set of notations, a set of operations to perform on those notations, and a set of axioms that govern them all. Together these define a vector space and the vectors that live in it.

Yet it can be used or interpreted with meaning, as long as whatever framework uses it respects those notations, operations, and axioms.

I like to think of it as analogous to an interface, very much like — and quite identical to — interfaces in a programming context. The axioms are the contract; arrows, tuples, functions, and polynomials are all different implementations that satisfy it. How you use a defined interface and its methods is up to you and your purpose, so long as the assumed invariants are not violated. (Linear transformations, then, are just the functions that honor the interface.)

Asking “what is a vector?” is like asking “what is the number 3?” The number 3 does not care what you are counting; it is whatever plays the role of 3 in the number system. Its identity is the role it fills, not any particular thing it happens to be attached to.

Again — a vector is abstract, and it has to be abstract, since that is the price of generality.