Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

42
LINES

< > BotCompany Repo | #1011722 // websMadeByProgram_parallel - use them cores!!

JavaX fragment (include)

1  
static L<Web> websMadeByProgram_parallel(S progID) {
2  
  ret websMadeByProgram_parallel(progID, null);
3  
}
4  
5  
static L<Web> websMadeByProgram_parallel(S progID, final Map<Web, SoftwareMadeWeb> furtherInfoMap) ctex {
6  
  // TODO: clean up file handle in case of error
7  
  IterableIterator<S> l = scanLog_iterator(progID, "webs-made.txt");
8  
  fS src = progID;
9  
  final new L<Pair<Web, Int>> webs;
10  
  
11  
  int poolSize = numberOfCores(), queueSize = 20;
12  
  NotifyingBlockingThreadPoolExecutor e = new(poolSize, queueSize, 15, TimeUnit.SECONDS);
13  
  
14  
  try {
15  
    int i = 0;
16  
    for (fS s : l) {
17  
      ++i;
18  
      final int _i = i;
19  
      e.execute(r { pcall {
20  
        SoftwareMadeWeb o = cast unstructure(s);
21  
        Web web = o.web;
22  
        if (web == null)
23  
          web = (Web) unstructure(o.structure);
24  
        if (web != null) synchronized(webs) {
25  
          webs.add(pair(web_intern(web_setSourceIfEmpty(web, src)), _i));
26  
          mapPut(furtherInfoMap, web, o);
27  
        }
28  
      }});
29  
    }
30  
    //e.await(); // ConcurrentModificationException sometimes??
31  
    e.shutdown();
32  
    e.awaitTermination(1, TimeUnit.DAYS);
33  
  } finally {
34  
    e.shutdown();
35  
  }
36  
  
37  
  time "Sort" {
38  
    sortBySecondOfPairs_inPlace(webs);
39  
  }
40  
  
41  
  ret concatLists(ai_triplesMadeByProgram(progID), firstOfPairs(webs));
42  
}

Author comment

Began life as a copy of #1011177

download  show line numbers  debug dex  old transpilations   

Travelled to 13 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt

No comments. add comment

Snippet ID: #1011722
Snippet name: websMadeByProgram_parallel - use them cores!!
Eternal ID of this version: #1011722/15
Text MD5: ce81be32ae31b0ba9398a101c7b192c0
Author: stefan
Category: javax / a.i.
Type: JavaX fragment (include)
Public (visible to everyone): Yes
Archived (hidden from active list): No
Created/modified: 2017-11-05 03:42:56
Source code size: 1311 bytes / 42 lines
Pitched / IR pitched: No / No
Views / Downloads: 437 / 458
Version history: 14 change(s)
Referenced in: [show references]