Correlation – Advanced

Property A: For constant a and random variables x, y and z, the following are true both for the sample and population definitions of covariance

  1. cov(x, y) = cov(y, x)
  2. cov(x, x) = var(x)
  3. cov(a, y) = 0
  4. cov(ax, y) = a · cov(x, y)
  5. cov(x+z, y) = cov(x, y)+ cov(z, y)

Proof: We give the proofs for the population version of covariance. The proofs for the sample versions are similar.

a)      Clear from the definition of covariance

b)      Clear from the definitions of covariance and variance

c)      Follows from


d)      Follows from

image3415 image3416

e)      Follows from

image3417 image3418 image3419

Property B: If x and y are random variables and z = ax + b where a and b are constants then the correlation coefficient between x and y is the same as the correlation coefficient between z and y.

Proof: By Property A


By Property 3 of Expectation


And so stdev(z) = a · stdev(x). Thus


Property 1


Proof: Let




Collecting together identical terms, together with a change in summation indices, we find:



and so



from which the result follows.

Property 2:


Proof: The proof is similar to the proof of Property 1.

Property 3:


Proof: The proof is similar to that of Property 2 of Expectation.

Property 4: The following is true for both for the sample and population definitions of covariance:

If x and y are independent then cov(x, y) = 0

Proof: The proof follows from Property 1d of Expectation and Property 3.

Property 5: The following are true both for samples and populations:

image1521 image1522

Proof: We give the proof of the first property for the population. The proof of the sample case is similar, as is the proof of the second property. By Property 2 of Expectation and Property 3

image1523 image5046 image5047 image1525

4 Responses to Correlation – Advanced

  1. Natalie says:

    I am confused by the term ” Average variance extracted (AVE)”. Dr. Charles, can you please explain this to me? Thank you.

    • Charles says:

      I don’t see this term on the webpage that you are referencing. Are you referring to a term that I use on the website or is a term you found somewhere else? Generally this is a term used with Factor Analysis.

  2. Colin says:


    What is the meaning of property 1? How do you define t and e?

Leave a Reply

Your email address will not be published. Required fields are marked *