Comprehensive Guide on Covariance of Two Random Variables
Start your free 7-days trial now!
Formal definition of covariance of two random variables
The covariance of random variables $X$ and $Y$ is defined as follows:
Where $\mu_X$ and $\mu_Y$ are the mean of $X$ and $Y$, respectively.
Intuition behind covariance
For the intuition behind covariance, please refer to this section in our guide on sample covariance. In essence, the covariance captures the linear relationship between two random variables:
Negative covariance | Zero covariance | Positive covariance |
---|---|---|
Here, note the following:
negative covariance - as $X$ increases, $Y$ tends to decrease linearly.
zero covariance - as $X$ increases, $Y$ tends does not vary linearly.
positive covariance - as $X$ increases, $Y$ tends to increase linearly.
Mathematical properties of covariance
Another formula to compute the covariance
The covariance of random variables $X$ and $Y$ is:
This form is also called the computation form of covariance.
Proof. For both discrete and continuous random variables:
This completes the proof.
Covariance of X and Y is equal to the covariance of Y and X
If $X$ and $Y$ are random variables, then covariance of $X$ and $Y$ is equivalent to the covariance of $Y$ and $X$, that is:
Proof. From the definition of covariance:
This completes the proof.
Covariance of aX and bY where a and b are constants
The covariance of random variables $aX$ and $bY$ where $a$ and $b$ are constants is:
Proof. Using the definition of covariance:
This completes the proof.
Covariance of a+X and b+Y where a and b are constants
The covariance of random variables $a+X$ and $b+Y$ where $a$ and $b$ are constants is:
Proof. Using the TODO definition of covariance:
This completes the proof.
Covariance of a random variable and a constant
The covariance of a random variable $X$ and a constant $c$ is:
This should make sense intuitively because this means that $c$, which is a constant, doesn't vary at all with $X$.
Proof. From theorem, the covariance can also be computed as:
This completes the proof.
Covariance of two independent variables
The covariance of two independent variables $X$ and $Y$ is:
Proof. Recall from theorem that the expected value of independent random variables $X$ and $Y$ is:
Now, theorem tells us that the covariance can be expressed like so:
Substituting \eqref{eq:Xeo8ybtlEPTFBYChCEp} into \eqref{eq:i72kxQzT471RtXqPJ90} gives:
This completes the proof.
Covariance of random variable with itself
The covariance of random variable $X$ with itself is:
Proof. We start with the covariance formula given by theorem:
Here, the last equality holds because of the definition of variance.
Covariance of random variable X+Y and Z
If $X$, $Y$ and $Z$ random random variables, then the covariance of $X+Y$ and $Z$ is:
Proof. By propertylink of covariance, we have that:
This completes the proof.
Covariance of series of random variables
If $X_1,X_2,\cdots,X_m$ and $Y_1,Y_2,\cdots,Y_n$ are random variables, while $a_1,a_2,\cdots,a_m$ and $b_1,b_2,\cdots,b_n$ are constants, then:
Sketch proof. We will prove this for the case when $m=n=2$, but the proof is easy to generalize. To be able to fit our equations on the page, let's define $z$ as follows:
We use the properties of covariance from earlier:
This completes the proof.
Computing covariance of two random variables
Suppose random variables $X$ and $Y$ have the following joint probability mass function:
$f_{X,Y}(x,y)$ | $x$ | $f_Y(y)$ | |||
---|---|---|---|---|---|
$1$ | $2$ | $3$ | |||
$y$ | $1$ | $0.2$ | $0.1$ | $0.1$ | $0.4$ |
$2$ | $0.1$ | $0.3$ | $0.2$ | $0.6$ | |
$f_X(x)$ | $0.3$ | $0.4$ | $0.3$ |
Compute the covariance of $X$ and $Y$.
Solution. Let's find the covariance of $X$ and $Y$ by using the following property of covariance:
Firstly, $\mathbb{E}(XY)$ is:
Next, $\mathbb{E}(X)$ and $\mathbb{E}(Y)$ are:
Therefore, the covariance is:
Since the covariance is small, $X$ and $Y$ have a weak positive linear association.