I found this link to compute standard deviation when we add samples one by one. If we add samples batch by batch ($N_B$), how could we incrementally calculate the variance?
Asked
Active
Viewed 524 times
2
-
Why not just use the "one by one" method on each value of the batch? If the batch is given as a set of values you need to factor in your standard deviation, I don't think there is an asymptotically faster way to do this than looping over the whole set of values, in which case you may as well use the technique described in that other question. – N.Bach Aug 10 '19 at 17:50
-
@N.Bach Thanks for the answer. So, isn't there another method to combine the results? let's say we have two batches, $X_1 = {x_1, x_2, x_3}$ with variance $\sigma_1^2$ and $X_2 = {x_4, x_5, x_6}$ with variance $\sigma_2^2$, can not we combine those to have estimation of true variance? – Panda Aug 10 '19 at 18:36
-
You probably could, but what's the point of computing the intermediate variance values ? The only reasons I could think of right now would be if those values are directly given to you, or if you are trying to mitigate numerical issues (e.g. floating point precision). – N.Bach Aug 10 '19 at 18:41