Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

72
LINES

< > BotCompany Repo | #1033575 // Welford [running average & standard deviation]

JavaX fragment (include) [tags: use-pretranspiled]

Libraryless. Click here for Pure Java version (10210L/57K).

1  
// from https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm
2  
srecord Welford(long count, dbl mean, dbl m2) {
3  
  // For a new value newValue, compute the new count, new mean, the new M2.
4  
  // mean accumulates the mean of the entire dataset
5  
  // M2 aggregates the squared distance from the mean
6  
  // count aggregates the number of samples seen so far
7  
  void add(double newValue) {
8  
    count++;
9  
    dbl delta = newValue - mean;
10  
    mean += delta / count;
11  
    dbl delta2 = newValue - mean;
12  
    m2 += delta * delta2;
13  
  }
14  
  
15  
  void remove(double valueToRemove) {
16  
    guarantee(count > 0);
17  
    
18  
    /* 
19  
      Let's reverse the formulas from add():
20  
      
21  
        delta = newValue - mean1
22  
        mean2 = mean1 + delta / count2
23  
        
24  
        => mean2 = mean1+(newValue-mean1)/count2
25  
                 = mean1 + newValue/count2 - mean1/count2
26  
        => mean2 = mean1*(1-1/count2) + newValue/count2
27  
                 
28  
      Solve for mean1. (| -newValue/count2)
29  
        
30  
        => mean2-newValue/count2 = mean1*(1-1/count2)  | :(1-1/count2)
31  
        
32  
        => (mean2-newValue/count2)/(1-1/count2) = mean1.
33  
        
34  
      "Count" is easy to reverse: count1 = count2-1, obviously.
35  
        
36  
      Now, about restoring m2.
37  
        
38  
        delta2 = newValue - mean2
39  
        m2_2 = m2_1 + delta * delta2
40  
41  
        => m2_2 - delta * delta2 = m2_1
42  
        
43  
      Well, that was easy!
44  
45  
    */
46  
    
47  
    double newValue = valueToRemove;
48  
    double delta2 = newValue-mean;
49  
    mean = (mean-newValue/count)/(1-1.0/count);
50  
    double delta = newValue-mean;
51  
    m2 -= delta*delta2;
52  
    --count;
53  
  }
54  
 
55  
  bool isEmpty() { ret count == 0; }
56  
57  
  dbl average aka avg aka mean() { ret count == 0 ? Double.NaN : mean; }
58  
  dbl variance() { ret count == 0 ? Double.NaN : m2/count; }
59  
  dbl sampleVariance() { ret count < 2 ? Double.NaN : m2/(count-1); }
60  
  dbl standardDeviation() { ret sqrt(variance()); }
61  
  
62  
  toString {
63  
    S s = nValues(count);
64  
    if (!isEmpty())
65  
      s += ", avg: " + avg() + ", std dev: " + standardDeviation();
66  
    ret s;
67  
  }
68  
  
69  
  AverageAndStandardDeviation get() {
70  
    ret new AverageAndStandardDeviation(avg(), standardDeviation());
71  
  }
72  
}

download  show line numbers  debug dex  old transpilations   

Travelled to 3 computer(s): bhatertpkbcr, mowyntqkapby, mqqgnosmbjvj

No comments. add comment

Snippet ID: #1033575
Snippet name: Welford [running average & standard deviation]
Eternal ID of this version: #1033575/9
Text MD5: a025a00f9c6bf3d5a5b07e090a5abe7b
Transpilation MD5: ab321ab3958147f54e6dc44c3445a673
Author: stefan
Category: javax / maths
Type: JavaX fragment (include)
Public (visible to everyone): Yes
Archived (hidden from active list): No
Created/modified: 2022-12-12 02:22:42
Source code size: 2244 bytes / 72 lines
Pitched / IR pitched: No / No
Views / Downloads: 143 / 280
Version history: 8 change(s)
Referenced in: [show references]