Expectation value or first moment of a random variable is the probability weighted sum of the possible values (weighted mean).
Expectation value of a 6-dice is 1+2+3+4+5+6 / 6 = 3.5

Covariance of 2 random variables is:
COV(X,Y)=E[(X-E(X))(Y-E(Y))]=E(XY) - E(X)E(Y)
i.e. the difference between the expected value of their product and the product of their expected values.
So if the variables change together, they will have a high covariance, if they are independent, their covariance is zero.

Variance is the covariance on the same variable, :
COV(X,X)=VAR(X)=E(X2) - E(X)2

Standard deviation is the square root of Variance

Correlation is:
COR(X,Y)=COV(X,Y)/STDEV(X)STDEV(Y)


http://mathworld.wolfram.com/Covariance.html