Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

22
LINES

< > BotCompany Repo | #1011919 // ai_indexWebs_parallel - note: indexes in different order - this makes no sense, should make multiple indexes, then merge

JavaX fragment (include)

1  
static MultiMap<S, WebNode> ai_indexWebs_parallel(Collection<Web> webs) ctex {
2  
  print("Making index in parallel.");
3  
  final MultiMap<S, WebNode> index = caseInsensitiveMultiMap();
4  
  
5  
  int queueSize = numberOfCores();
6  
  NotifyingBlockingThreadPoolExecutor e = new(numberOfCores(), queueSize, 15, TimeUnit.SECONDS);
7  
    
8  
  try {
9  
    for (final L<Web> l : chunksOf1000(webs)) {
10  
      e.execute(r { pcall {
11  
        for (Web web : l)
12  
          ai_addWebToIndex(index, web);
13  
      }});
14  
    }
15  
    e.shutdown();
16  
    e.awaitTermination(1, TimeUnit.DAYS);
17  
  } finally {
18  
    e.shutdown();
19  
  }
20  
  
21  
  ret index;
22  
}

Author comment

Began life as a copy of #1011347

download  show line numbers  debug dex  old transpilations   

Travelled to 13 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt

No comments. add comment

Snippet ID: #1011919
Snippet name: ai_indexWebs_parallel - note: indexes in different order - this makes no sense, should make multiple indexes, then merge
Eternal ID of this version: #1011919/9
Text MD5: 235531fb20d539813096168dc4d983fb
Author: stefan
Category: javax / a.i.
Type: JavaX fragment (include)
Public (visible to everyone): Yes
Archived (hidden from active list): No
Created/modified: 2017-11-10 03:29:30
Source code size: 622 bytes / 22 lines
Pitched / IR pitched: No / No
Views / Downloads: 481 / 499
Version history: 8 change(s)
Referenced in: [show references]