Solution. The linear combination of $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ that generates $\boldsymbol{v}_3$ is:

We say that $\boldsymbol{v}_3$ can be expressed as a linear combination of $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$.

Linear dependence and independence of two vectors

Linearly dependent vectors

Consider the following two vectors:

Can one vector be expressed as a constant multiple of the other? To answer this question, consider the following equation:

Clearly, if we let $c=2$, then the equality holds. This means that if we double the shorter vector, we get the longer vector - this should be clear from the diagram as well.

In fact, recall from theoremlink that as long as the two vectors are pointing in the same direction, there will always exist a constant $c$ that satisfies \eqref{eq:m3vR3xzP1TnBBuSrILn} because multiplying a vector by a constant involves stretching/shrinking the vector but preserving the direction.

Whenever we can express two vectors as a multiple of one another, we say that the two vectors are linearly dependent.

Linearly independent vectors

Consider the following two vectors:

We ask ourselves the same question - can we express one vector as a multiple of the other? Again, we are interested in finding a constant that makes the following equality hold:

Clearly, there exists no constant value $c$ that satisfies the equality. Setting $c=3$ will make the first elements match, but the second elements will not match. Similarly, setting $c=2$ will only match the second elements.

The fact that $c$ does not exist should make sense from the diagram because $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are pointing in different directions. No matter how much we stretch $\boldsymbol{v}_1$, we will never be able to obtain $\boldsymbol{v}_2$.

Whenever we cannot express two vectors as a constant multiple of one another, we say that the two vectors are linearly independent.

Deriving the formal definition

Linearly dependent case

Let's now modify equation \eqref{eq:m3vR3xzP1TnBBuSrILn} slightly and include a constant multiple on the right-hand side as well:

Notice that \eqref{eq:BRDlEdUuNi3ZgHYtFz2} is true when $c_1=c_2=0$. However, this is trivial because shrinking any two vectors to a zero vector will always make them identical. If we can find a pair of non-zero constants $c_1$ and $c_2$ that satisfies the equality, then this implies that we can express one vector using a multiple of the other. To understand why this is true, divide both sides by $c_2$ to get:

Since $c_1$ and $c_2$ are just constants, the fraction can also be treated as a constant. In this case, $c_1=2$ and $c_2=1$ will satisfy the equation. Because we managed to express one vector as a multiple of the other, the two vectors must be linearly dependent.

The key takeaway here is that two vectors are linearly dependent only if there exist non-zero constants for the equality \eqref{eq:BRDlEdUuNi3ZgHYtFz2} to hold.

Linearly independent case

Similarly, let's now modify \eqref{eq:oDqV0a3tTqzI5YprYaC} from earlier using two constants:

There exists no pair of non-zero constants that satisfy the above equality. In other words, the only solution to \eqref{eq:VQbMJFwXncwU5HjWo1t} is $c_1=c_2=0$, which is the trivial solution. Because neither of the vectors can be expressed as a linear combination of the other, the vectors must be linearly independent.

This means that two vectors are linearly independent only if the constants have to be all zero for the equality \eqref{eq:VQbMJFwXncwU5HjWo1t} to hold.

Generalization

We have demonstrated that we can determine whether two vectors $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are linearly dependent by checking the existence of non-zero constants that satisfy:

Let's move the vector on the right-hand side to the left-hand side to get:

We now convert the sign of $c_2$ to positive:

We are allowed to alter the sign of the constants because if there exist constant terms $c_1$ and $c_2$ that satisfy \eqref{eq:mSkwupqaA4DqVy0fcn9}, then there will always exist constant terms $c_1$ and $c_2$ satisfying \eqref{eq:PxgyGjfIqOXoUf1mtmG}, and vice versa. For instance, consider the following equation:

One solution is $c_1=2$ and $c_2=1$. If we swap the sign of $c_2$, then the solution becomes $c_1=2$ and $c_2=-1$. The constant terms have changed but the actual values they take on do not matter - all we care about when determining the linear dependence of two vectors is whether or not there exist constant terms $c_1$ and $c_2$ such that the equality in \eqref{eq:mSkwupqaA4DqVy0fcn9} holds.

Linear dependence and independence of more than two vectors

When we have more than two vectors, the question changes from "are the two vectors dependent?" to "are the set of vectors dependent?". If a vector can be expressed as a linear combination of the other vectors in the set, then the set is considered to be dependent. Otherwise, the set is independent.

As an example, consider the following set of three vectors:

Here, $\boldsymbol{v}_3$ can be expressed as a linear combination of $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ like so:

Visually, this means that:

Therefore, our set of vectors is dependent.

Now, let's do what we did earlier and generalize the criterion for linear dependence. Recall that the criterion in the case of two vectors is as follows:

For the case when we have three vectors, we can guess that the criterion would be:

It turns out that this is precisely the criterion to check for linear dependence of three vectors! Let's now understand why. Suppose there exist $c_1$, $c_2$ and $c_3$ that are not all zeros and satisfy \eqref{eq:KPNLoqNjC0SvplKn6mK}. Let's rewrite \eqref{eq:KPNLoqNjC0SvplKn6mK} such that $\boldsymbol{v}_3$ is the subject:

Here, we can regard the fraction coefficients as constants. We have managed to express $\boldsymbol{v}_3$ using some linear combinations of vectors $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$, which means that the set of vectors is linearly dependent.

Now, what if the only coefficients that satisfy equation \eqref{eq:KPNLoqNjC0SvplKn6mK} are all zeros? In such cases, the vectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$ and $\boldsymbol{v}_3$ disappear and we are left with $\boldsymbol{0}=\boldsymbol{0}$. Because we cannot make $\boldsymbol{v}_1$, $\boldsymbol{v}_2$ or $\boldsymbol{v}_3$ the subject, none of the vectors in our set can be constructed using a linear combination of the other vectors in the set.

To generalize further, instead of just three vectors, what if we had $n$ vectors? The criterion to test for linear dependence would be:

What we have derived is a criterion to test for linear dependence given multiple vectors. If there exist coefficients that are not all zeros satisfying \eqref{eq:lpuN20Wpqn1PEOzGrWE}, then the set of vectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_n$ are linearly dependent. On the other hand, if the only way to get the zero vector is by having all the constant terms be zero, then we have a linearly independent set.

Remark. Instead of the above definition, we could have defined linear dependence and independence as follows:

• a set of vectors is linearly dependent if one vector can be constructed as a linear combination of the other vectors.

• a set of vectors is linearly independent if none of the vectors can be constructed as a linear combination of the other vectors.

The reason why the formal definition is preferred is that equation \eqref{eq:LB5A64eXwsweyudsytL} can be expressed in a more practical way using matrices. We shall get back to this laterlink in this guide.

Solution. At first glance, $\boldsymbol{v}_2$ is simply the double of $\boldsymbol{v}_1$, so we immediately know that they are linearly dependent. Let's still go through the steps to come to this conclusion mathematically. The formula to test for linear dependence is:

We can reformulate this as a system of linear equations:

From the first equation, we know that:

Substituting this into the second equation gives:

This means that there are infinitely many solutions - any pair of $c_1$ and $c_2$ that satisfies the equality \eqref{eq:T1uZG54EHUKQ84EhWv5} is a valid solution. For instance, $c_1=4$ and $c_2=-2$ is one possible solution. Since there exist coefficients $c_1$ and $c_2$ that are not both zeros and satisfy \eqref{eq:R2jcsXyswMYpOp4zuNL}, we conclude that our vectors are linearly dependent.

Because $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are linearly dependent, we can express one vector as a scalar multiple of the other. Let's pick the coefficients $c_1=4$ and $c_2=-2$ because they satisfy \eqref{eq:T1uZG54EHUKQ84EhWv5}. Substituting them into \eqref{eq:R2jcsXyswMYpOp4zuNL} gives:

Indeed, $\boldsymbol{v}_2$ can be expressed as $2$ times $\boldsymbol{v}_1$.

Solution. We can easily tell that $\boldsymbol{v}_2$ cannot be expressed as a scalar multiple of $\boldsymbol{v}_1$ and vice versa. Therefore, the two vectors must be linearly independent. Let's still confirm this mathematically using our criterion:

We can reformulate this as a system of linear equations:

From the first row, we have that:

Substituting this into the second row gives:

From \eqref{eq:vlpR9E6sNDEy66izBb9}, we have that $c_1=0$ as well. Since the only way \eqref{eq:ClVxXzPeL3eG5xOgjBO} can be satisfied is by having the coefficients be $0$, we conclude that the vectors are linearly independent.

Solution. Let's solve for $c_1$, $c_2$ and $c_3$ below:

Rewriting this as a system of linear equations:

Multiplying the top equation by $2$ gives us the middle equation, which means that they are identical. Since we now only have $2$ equations with $3$ unknowns, one constant term is allowed to vary. This means that there are infinitely many solutions of $c_1$, $c_2$ and $c_3$ that satisfy \eqref{eq:jobxCfha2MUEOEJEA5i}. Because there exists a set of non-zero coefficients that satisfies \eqref{eq:jobxCfha2MUEOEJEA5i}, the three vectors must be linearly dependent.

Solution. By inspection, these vectors are linearly independent because we cannot construct any of these vectors using the other vectors. Let's still solve for $c_1$, $c_2$ and $c_3$ below:

This corresponds to the following system of linear equations:

Because the only coefficients that satisfy \eqref{eq:yFQuisYj2aZCWXuWXmA} are all zeros, our set of vectors must be linearly independent.

Proof. This theorem is equivalent to theoremlink.

Solution. We can directly use theoremlink to express equation \eqref{eq:NN895vYDrnrfWzljlhZ} as a matrix-vector product:

Remark. The matrix-vector form has two advantages:

• we can easily use a computer program to solve for the coefficients.

• we can use Gaussian elimination to solve for the coefficients. We will see an example of this later.

Solution. Using matrix-vector notation, the criterion for linear dependence is:

By theoremlink, we can ignore the zero vector on the left during Gaussian elimination because performing any elementary row operationlink (e.g. summation, multiplication) will not have an impact on zeros. Therefore, we can focus on row-reducing the coefficient matrix only:

This means that \eqref{eq:r436dJsdMikBCUppLTT} can be reformulated as below since they share the same solution set:

The only way the above can be true is if $c_1$ and $c_2$ are both equal to zero. This means that the original vectors $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are linearly independent.

Solution. The criterion for linear dependence is:

Expressing this in matrix-vector form:

Now, we perform row-reduction on the coefficient matrix like so:

Because the last row is all zeros, $c_3$ can take on any value. By theoremlink, we already know that this system has infinitely many solutions, which means that the set of vectors is linearly dependent. Therefore, we can express a vector as a linear combination of the other vectors.

For simplicity, let's say $c_3=1$. Using the second row of the reduced row echelon form, we have that $c_2=-3$. Using the first row, we have that $c_1=-2$. Substituting these coefficients into \eqref{eq:Yp1nezlKJNYTxna9JZi} gives us:

Making $\boldsymbol{v}_3$ the subject:

We have managed to express $\boldsymbol{v}_3$ as a linear combination of $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$.

Practice problems