search
Search
Login
Unlock 100+ guides
menu
menu
web
search toc
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
What does this mean?
Why is this true?
Give me some examples!
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
check_circle
Mark as learned
thumb_up
10
thumb_down
0
chat_bubble_outline
2
Comment
auto_stories Bi-column layout
settings

Introduction to vectors in linear algebra

schedule Jan 27, 2024
Last updated
local_offer
Linear Algebra
Tags
mode_heat
Master the mathematics behind data science with 100+ top-tier guides
Start your free 7-days trial now!
Definition.

Vectors

A vector is an object with both a magnitude (length) and a direction. An object with only the magnitude is called a scalar (e.g. $5$, $\pi$).

Vectors can be represented using an arrow that connects two points. For instance, consider the following two points:

The vector $\overrightarrow{AB}$ that points from $A$ to $B$ is as follows:

In two-dimensional space, the vector $\overrightarrow{AB}$ contains two numbers:

  • the first (top) component of the vector represents the horizontal distance between the points. This is also called the horizontal or $x$ component.

  • the second (bottom) component of the vector represents the vertical distance between the two points. This is also called the vertical or $y$ component.

Note the following two terms:

  • the start point of the vector is called the tail - $A$ is the tail in this case.

  • the end point of the vector is called the head - $B$ is the head in this case.

NOTE

Vectors are typically represented using a lowercase bold letter (e.g. $\boldsymbol{v}$ and $\boldsymbol{u}$).

We have defined a vector as an object with both direction and magnitude (length). Let's now consider a pair of vectors with the same direction but different magnitude:

When two vectors are parallel and pointing to the same side like in this case, we say that they have the same direction. The magnitude (length) of the two vectors is different - the bottom vector is twice as long as the top vector.

Next, let's consider the case when two vectors have the same magnitude but different directions:

We can see that they have the same length, but they are pointing in two different directions.

Definition.

Equality of vectors

Two vectors are considered equal if they have the same direction and magnitude.

Consider the vectors $\overrightarrow{AB}$ and $\overrightarrow{CD}$ below:

Even though the starting and ending points of the vectors are different, they are considered to be equal because they have the same magnitude (length) and direction. Because we can draw a vector given its magnitude and direction from any point, it is considered to be free. We will later see a non-free vector whose starting point is pinned to the origin (position vector).

Definition.

Taking scalar multiples of vectors

Scalars are quantities that have a magnitude without any notion of direction. We can multiply a vector by a scalar by multiplying every component of the vector by the scalar. For instance, consider the scalar $c$ and vector $\boldsymbol{v}$ below:

$$c=3,\;\;\;\;\; \boldsymbol{v}=\begin{pmatrix} 1\\2 \end{pmatrix}$$

Taking their product gives:

$$3\begin{pmatrix}1\\2 \end{pmatrix}= \begin{pmatrix}3\times1\\3\times2 \end{pmatrix}= \begin{pmatrix}3\\6 \end{pmatrix}$$
Theorem.

Geometric interpretation of scalar multiples of vectors

Taking a scalar multiple of a vector involves stretching or shrinking the vector and does not change the direction of the vector. For instance, suppose we double the vector $\boldsymbol{a}$ like so:

Notice how the length of the new vector $2\boldsymbol{a}$ has doubled, but it is still pointing in the same direction as $\boldsymbol{a}$.

What if we multiply a vector by a negative number? In this case, the new vector will point in the opposite direction, as illustrated below:

Here, the new vector $-2\boldsymbol{a}$ has doubled in length but is now pointing in the opposite direction because we have multiplied by a negative constant. Note that if we multiply a vector by $-1$, then we only reverse its direction but preserve its length:

Definition.

Adding two vectors

Suppose we have the following two vectors:

$$\boldsymbol{v}_1=\begin{pmatrix} 3\\1 \end{pmatrix},\;\;\;\;\;\; \boldsymbol{v}_2=\begin{pmatrix} 2\\4 \end{pmatrix}$$

To add two vectors, we add each pair component-wise:

$$\boldsymbol{v}_1+\boldsymbol{v}_2=\begin{pmatrix} 3\\1 \end{pmatrix}+ \begin{pmatrix} 2\\4 \end{pmatrix}= \begin{pmatrix} 3+2\\1+4 \end{pmatrix} = \begin{pmatrix} 5\\5 \end{pmatrix}$$

Vector addition is said to be commutative, which means that the order of the addition does not matter:

$$\boldsymbol{v}_1+\boldsymbol{v}_2= \boldsymbol{v}_2+\boldsymbol{v}_1$$
Theorem.

Geometric interpretation of vector-vector addition (head-to-tail method)

Consider the same set of vectors $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ as above:

$$\boldsymbol{v}_1=\begin{pmatrix} 3\\1 \end{pmatrix},\;\;\;\;\;\; \boldsymbol{v}_2=\begin{pmatrix} 2\\4 \end{pmatrix}$$

We know that their sum is:

$$\boldsymbol{v}_1+\boldsymbol{v}_2= \begin{pmatrix} 3+2\\1+4 \end{pmatrix} = \begin{pmatrix} 5\\5 \end{pmatrix}$$

Geometrically, $\boldsymbol{v}_1+\boldsymbol{v}_2$ represents the green vector below:

To geometrically add two vectors, we use the so-called head-to-tail method:

  1. we place the tail (non-pointy part) of the second vector ($\color{blue}{\boldsymbol{v}_2}$) at the head of the first vector ($\color{red}{\boldsymbol{v}_1}$).

  2. the vector ${\color{red}{\boldsymbol{v}_1}}+ \color{blue}{\boldsymbol{v}_2}$ is the vector whose tail is at the tail of $\color{red}{\boldsymbol{v}_1}$ and whose head is at the head of the $\color{blue}{\boldsymbol{v}_2}$.

Intuition. The reason why the head-to-tail method works can be explained using the following diagram:

Here, vectors $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are denoted as $\boldsymbol{a}$ and $\boldsymbol{b}$ for brevity. We can see that:

  • the horizontal component for $\boldsymbol{a}+\boldsymbol{b}$ is $a_1+b_1$.

  • the vertical component for $\boldsymbol{a}+\boldsymbol{b}$ is $a_2+b_2$.

We can also geometrically show that vector addition is commutative:

We can see that reversing the order of the addition does not matter - we end up with the green vector either way!

Theorem.

Subtracting two vectors

Consider the following vectors:

$$\boldsymbol{v}_1=\begin{pmatrix} 3\\1 \end{pmatrix},\;\;\;\;\;\; \boldsymbol{v}_2=\begin{pmatrix} 2\\4 \end{pmatrix}$$

Let's now derive vector subtraction using vector addition:

$$\begin{align*} \boldsymbol{v}_1-\boldsymbol{v}_2 &=\begin{pmatrix}3\\1\end{pmatrix}- \begin{pmatrix}2\\4\end{pmatrix}\\ &=\begin{pmatrix}3\\1\end{pmatrix}+ (-1)\begin{pmatrix}2\\4\end{pmatrix}\\ &=\begin{pmatrix}3\\1\end{pmatrix}+ \begin{pmatrix}-2\\-4\end{pmatrix}\\ &=\begin{pmatrix}1\\-3\end{pmatrix} \end{align*}$$

The key trick in deriving vector subtraction is to convert the operation into addition by multiplying the second vector by $-1$.

We already know how to geometrically add two vectors using the head-to-tail method. The same logic applies to subtraction except that we reverse the direction of the second vector since we are multiplying it by $-1$. Here's the diagram:

Notice how the direction of $\boldsymbol{v}_2$ is reversed here.

Definition.

Magnitude of vector

The magnitude of a vector is simply its length.

Example.

Computing the magnitude of a vector

For vectors in two-dimensional space, we can use the Pythagoras theorem to determine the magnitude. For instance, consider the following vector:

From Pythagoras theorem, we know that the length of $\boldsymbol{v}$, denoted as $\Vert{\boldsymbol{v}}\Vert$ is:

$$\begin{align*} \Vert{\boldsymbol{v}}\Vert &= \sqrt{4^2+3^2}\\ &=\sqrt{25}\\&=5 \end{align*}$$

Therefore, the magnitude of $\boldsymbol{v}$ is 5.

Theorem.

Useful property of the magnitude of a vector

If $c$ is a scalar and $\boldsymbol{v}$ is a vector, then:

$$\Vert{\boldsymbol{cv}}\Vert= \vert{c}\vert\times\Vert{\boldsymbol{v}}\Vert$$

Where $\vert{c}\vert$ is the absolute value of $c$.

Proof. We will prove the case for vectors in two-dimensional space but the proof can be easily generalized to higher dimensions. Let vector $\boldsymbol{v}$ be denoted as follows:

$$\boldsymbol{v}=\begin{pmatrix} v_1\\v_2 \end{pmatrix}$$

The scalar-vector product $c\boldsymbol{v}$ is:

$$c\boldsymbol{v}=\begin{pmatrix} cv_1\\cv_2 \end{pmatrix}$$

Taking the magnitude:

$$\begin{align*} \Vert{c\boldsymbol{v}}\Vert&= \sqrt{(cv_1)^2+(cv_2)^2}\\&= \sqrt{c^2v_1^2+c^2v_2^2}\\ &=\sqrt{c^2(v_1^2+v_2^2)}\\ &=\sqrt{c^2}\times\sqrt{v_1^2+v_2^2}\\ &=\vert{c}\vert\times\Vert{\boldsymbol{v}}\Vert\\ \end{align*}$$

This completes the proof.

Definition.

Zero vector

A zero vector, which is denoted as $\boldsymbol{0}$, has all zeros as it entries. For instance, the zero vector in two-dimensional space is:

$$\boldsymbol{0}= \begin{pmatrix} 0\\0 \end{pmatrix}$$

The zero vector has a magnitude of $0$.

Definition.

Unit vectors

A unit vector is a vector with magnitude one. Typically, unit vectors are denoted by $\boldsymbol{u}$.

Theorem.

Converting vectors into unit vectors

We can convert any vector $\boldsymbol{v}$ (except the zero vector) into a unit vector $\boldsymbol{u}$ while preserving its direction by using the following formula:

$$\begin{equation}\label{eq:G6SKsOpSuQVO82XzN2F} \begin{aligned}[b] \boldsymbol{u}= \frac{1}{\Vert{\boldsymbol{v}}\Vert}\boldsymbol{v} \end{aligned} \end{equation}$$

Where $\Vert{\boldsymbol{v}}\Vert$ is the magnitude of $\boldsymbol{v}$.

Proof. Let's first prove that $\boldsymbol{u}$ as computed by \eqref{eq:G6SKsOpSuQVO82XzN2F} has a magnitude of one. We have previously proven the following property of the magnitude of a vector:

$$\begin{equation}\label{eq:HOTfsE5kD7uSQLKQSeN} \Vert{\boldsymbol{cv}}\Vert= \vert{c}\vert\times\Vert{\boldsymbol{v}}\Vert \end{equation}$$

Where $c$ is any constant and $\vert{c}\vert$ is the absolute value of $c$. We let $c$ be defined as follows:

$$c=\frac{1}{\Vert{\boldsymbol{v}}\Vert}$$

Substituting this into \eqref{eq:HOTfsE5kD7uSQLKQSeN} gives:

$$\begin{equation}\label{eq:L145f21PtXgsMwlPBir} \begin{aligned}[b] \left\Vert{\boldsymbol{\frac{1}{\Vert{\boldsymbol{v}}\Vert}\boldsymbol{v}}}\right\Vert &=\left\vert\frac{1}{\Vert{\boldsymbol{v}}\Vert}\right\vert \times\Vert{\boldsymbol{v}}\Vert\\ &=\frac{1}{\Vert{\boldsymbol{v}}\Vert} \times\Vert{\boldsymbol{v}}\Vert\\ &=1 \end{aligned} \end{equation}$$

For the second step, we discarded the absolute value because $\Vert{\boldsymbol{v}}\Vert$ is a measure of length and is therefore always positive. The left-hand side of \eqref{eq:L145f21PtXgsMwlPBir} is simply $\Vert{\boldsymbol{u}}\Vert$. Therefore, we conclude that:

$$\Vert{\boldsymbol{u}}\Vert=1$$

Great, we've managed to prove that the vector computed by \eqref{eq:G6SKsOpSuQVO82XzN2F} is a unit vector!

Next, let's prove that $\boldsymbol{u}$ has the same direction as $\boldsymbol{v}$. The relationship between $\boldsymbol{u}$ and $\boldsymbol{v}$ is as follows:

$$\boldsymbol{u}= \frac{1}{\Vert{\boldsymbol{v}}\Vert}\boldsymbol{v}$$

Since the fraction is a scalar value, $\boldsymbol{u}$ is a scalar multiple of $\boldsymbol{v}$. We know from earlierlink that scalar multiples of vector preserve the direction. This completes the proof.

Example.

Converting a vector into a unit vector

Consider the following vector:

Convert this vector into a unit vector.

Solution. Firstly, let's compute the magnitude of the vector:

$$\begin{align*} \Vert{\boldsymbol{v}}\Vert &= \sqrt{4^2+3^2}\\ &=\sqrt{25}\\&=5 \end{align*}$$

Clearly $\boldsymbol{v}$ is not a unit vector. We can convert $\boldsymbol{v}$ into a unit vector $\boldsymbol{u}$ like so:

$$\begin{align*} \boldsymbol{u}&= \frac{1}{\Vert{\boldsymbol{v}}\Vert}\boldsymbol{v}\\ &= \frac{1}{5} \begin{pmatrix}4\\3\end{pmatrix}\\ &= \begin{pmatrix}4/5\\3/5\end{pmatrix}\\ \end{align*}$$

Let's confirm that $\boldsymbol{u}$ has a length of one:

$$\begin{align*} \Vert\boldsymbol{u}\Vert&= \sqrt{\Big(\frac{4}{5}\Big)^2+\Big(\frac{3}{5}\Big)^2}\\ &=\sqrt{\frac{16}{25}+\frac{9}{25}}\\ &=1 \end{align*}$$

Visually, $\boldsymbol{u}$ looks like the following:

Notice how $\boldsymbol{u}$ preserves the direction of $\boldsymbol{v}$.

Vectors in higher dimensions

Up to now, we have only dealt with vectors in two dimensions. Vectors can exist in higher dimensions as well - for instance, here is a vector in three-dimensional space:

Regardless of what dimension a vector is in, we can always think of it as an arrow that connects one point to another.

Using set notation to describe the dimension of vectors

To denote a vector $\boldsymbol{v}$ in $n$-dimensional space, we use the following set notation:

$$\boldsymbol{v}\in\mathbb{R}^n$$

For instance, a three-dimensional vector is denoted as:

$$\boldsymbol{v}\in\mathbb{R}^3$$

The $\in$ symbol is read as "belongs to".

Practice problems

Compute the following:

$$2 \begin{pmatrix} 3\\1 \end{pmatrix}$$
Your answer
Submit answer
Show solution

Multiply each component by the scalar constant like so:

$$2\begin{pmatrix} 3\\1 \end{pmatrix}= \begin{pmatrix} 2\times3\\2\times1 \end{pmatrix}= \begin{pmatrix} 6\\2 \end{pmatrix}$$

Compute the following:

$$\begin{pmatrix} 2\\3 \end{pmatrix}+ \begin{pmatrix} 1\\-1 \end{pmatrix}$$
Your answer
Submit answer
Show solution

To add two vectors, add each pair of components like so:

$$\begin{pmatrix} 2\\3 \end{pmatrix}+ \begin{pmatrix} 1\\-1 \end{pmatrix}= \begin{pmatrix} 3\\2 \end{pmatrix}$$

Compute the following:

$$\begin{pmatrix} 1\\4 \end{pmatrix}- \begin{pmatrix} 1\\-1 \end{pmatrix}$$
Your answer
Submit answer
Show solution

To subtract two vectors, perform the subtraction for each pair of components like so:

$$\begin{pmatrix} 1\\4 \end{pmatrix}- \begin{pmatrix} 1\\-1 \end{pmatrix}= \begin{pmatrix} 0\\5 \end{pmatrix}$$

Consider the following vector:

$$\boldsymbol{v}= \begin{pmatrix} 4\\3\\1 \end{pmatrix}$$

Compute the magnitude of $\boldsymbol{v}$.

Your answer
Submit answer
Show solution

The magnitude represents the length of $\boldsymbol{v}$. We can therefore use the Pythagoras theorem to get:

$$\begin{align*} \Vert{\boldsymbol{v}}\Vert &=\sqrt{4^2+3^2+1^2}\\ &=\sqrt{26} \end{align*}$$

Consider the following vector:

$$\boldsymbol{v}=\begin{pmatrix} 3\\6 \end{pmatrix}$$

Convert $\boldsymbol{v}$ into a unit vector.

$$\frac{1}{45}\begin{pmatrix} 3\\6 \end{pmatrix}$$
$$\frac{1}{\sqrt{45}}\begin{pmatrix} 3\\6 \end{pmatrix}$$
$$\frac{1}{9}\begin{pmatrix} 1\\2 \end{pmatrix}$$
$$\frac{1}{\sqrt{9}}\begin{pmatrix} 3\\6 \end{pmatrix}$$

We first compute the magnitude of the vector:

$$\begin{align*} \Vert\boldsymbol{v}\Vert&= \sqrt{3^2+6^2}\\ &=\sqrt{45}\\ \end{align*}$$

We then use formula for unit vector:

$$\begin{align*} \boldsymbol{u}&= \frac{1}{\Vert{\boldsymbol{v}}\Vert}\boldsymbol{v}\\ &=\frac{1}{\sqrt{45}} \begin{pmatrix} 3\\6 \end{pmatrix} \end{align*}$$
robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...