Math Vectors vs ML Vectors

Vectors in mathematical applications are generally talked about differently than vectors in ML applications. Why is that? Are math vectors and ML vectors the same thing?

The short answer is yes, the underlying vector concepts are the same between math and ML vectors. Vectors for mathematical and ML applications will also support the same operations like dot products and matrix multiplication.

But there are differences in how vectors are used in math vs ML applications which we discuss below.

Vectors in Mathematics

A vector is a quantity (often a physical property) that can't be measured with a single number. A vector has two components, magnitude and direction.

Since mathematical vectors often represent physical properties, we generally see them with 4 or fewer dimensions. Movement in space is often represented by 1, 2 or 3 dimensions, while time can be represented by a 4th dimension.

For example the velocity of a drone flying in space could be represented in 3 dimensions. * Movement along the X axis (i.e. east/west) * Movement along the Y axis (i.e. north/south) * Movement along the Z axis (i.e. up/down)

In mathematics vectors are most commonly represented vertically as column vectors. So our drone velocity vector will look something like this.

v = $\left[\begin{array}{}{V}_{\mathrm{x}}\\ V_{\mathrm{y}}\\ V_{\mathrm{z}}\end{array}\right]$

Vectors in ML

We mentioned earlier that vectors have both magnitude and direction components. But how does direction apply to ML?

ML vectors still have both magnitude and direction components but the meaning of these is more abstract. ML vectors often don't represent a physical space, but represent data points of object features, model parameters, embeddings, etc. For example, a restaraunt menu item could be represented with the following vector.

v = $\left[\begin{array}{c}price,\; weight,\; number\_of\_ingredients,\; is\_entree,\; is\_vegetarian\end{array}\right]$

ML vectors are often very highly dimensional due to the number of data points they can represent.

In ML as well as general programming vectors are most commonly represented horizontally as row vectors (shown above). This is more convenient to read and write when programming or when using data tables. But these row vectors must often be converted to column vectors for linear algebra operations with methods like transpose() or reshape().