4.1 The R program to look at the convergence rate for LLN.
The following program makes a PDF file called LLNFigure.pdf.
n <- 100000; m <- 30
x <- matrix(rbinom(n*m, size=1, prob=0.5), ncol = m)
y <- apply(x, 2, function(z) cumsum(z)/seq_along(z))
pdf("LLNFigure.pdf")
matplot(y, type = "l", col= 1, ylim = c(0.3,.7))
dev.off()
4.3 The strong law of large numbers
The law of large numbers explained in the text is called the weak law of large numbers. Therefore, there must be a strong version. The strong law guarantees that almost all sample sequences {xn} almost surely give the correct expectation value. In the context of computing thermodynamic quantities for an equilibrium state, one macroscopic specimen corresponds to a single sample sequence, so the strong law guarantees that the instantaneous observation of a single specimen gives correct thermodynamic observables reproducibly. Exceptional cases may happen, but exceptional event becomes rarer and rarer in the thermodynamic limit. As already asserted, from a single macroscopic sample with an instantaneous observation, we may observe correct thermodynamic observables. To guarantee this the weak law is not enough. We need the strong law.
〈〈 Difference between the weak and the strong laws〉〉
The reader might think that there is no difference from the weak law. The weak law is the law for the set of sample sequences. That is, let N1 < N2 < ··· → ∞ and consider the set of sample sequences consisting of Ni samples. Then, the law only tells us that its subset Ai of sequences with their expectation values close to the true expectation value can be chosen so that P(Ai^c) = O[1/Ni], where ^c implies complement. As i increases Ai becomes closer and closer to the whole set of sequences, but still ∩i>nAi may be empty. In fact, the weak law does not guarantee the convergence of the expectation value of individual sample sequence to the true expectation value. See Section 1.8 of R. Durrett, Probability: theory and examples (Wadsworth & Brooks/Cole, 1991). A rough illustration of the difference between the weak and strong law of large numbers is given below. Each line illustrates the sample sequence {X_i}. Red crosses indicate that for such n the empirical expectations deviate too much. In the upper panel, the weak law holds but not the strong law. If we look at all the `runs’ in the ensemble, the runs with red crosses become progressively rarer, so the weak law holds, BUT the red crosses continue to jump around, so no run may be without very large errors, however large n may be. If the strong law holds (the lower panel), then increasing number of runs become free from large errors .
If the system size is fixed, then the weak law guarantees the reproducibility of thermodynamic observables. However, it does not exclude the possibility that most (extremely large) systems have subsystems that behave thermodynamically not reproducibly. Thus, the weak law is not quite sufficient.
For more details, see Section 1.7 of R. Durrett, Probability: theory and examples (Wadsworth & Brooks/Cole, 1991).
Strong law of large numbers --- Demonstration
Theorem [Kolmogorov 1933]
Kolmogorov’s original proof used the following strategy:
In more detail:
Finally, we use Borel-Cantelli to conclude the proof.
4.4
Chebyshev’s inequality and law of large numbers, illustrated
Mathematica illustration is available:
Chebyshev's Inequality and the Weak Law of Large Numbers for iid Two-Vectors
Notable related theorems: 01 law and Regellosigkeit
Kolmogorov’s 01 law implies that macroscopic events occur with probability 0 or 1. Thus, ignoring unobservable measure zero events, macroscopic events are deterministic: thermodynamics is deterministic in the large system limit.
Regellosigikeit implies however cleverly you design sampling schemes, the expectation values would never be affected if the number of samples is large enough.
