static L<Web> webs_readTripleFile_parallel(File f) ctex { if (!f.exists()) ret ll(); print("Reading " + f2s(f) + " in " + nCore(numberOfCores())); time "webs_readTripleFile_parallel" { time "Reading names" { Iterator<S> it = linesFromFile(f); final new L<S> names; while (it.hasNext()) { S s = trim(it.next()); if (empty(s)) break; names.add(unquote(s)); } } final new L<Pair<Web, Int>> webs; int poolSize = numberOfCores(), queueSize = 1000; NotifyingBlockingThreadPoolExecutor e = new(poolSize, queueSize, 15, TimeUnit.SECONDS); try { int i = 0; for (fS s : iterable(it)) { ++i; final int _i = i; e.execute(r { pcall { Web web = webs_readTripleFile_line(s, names); if (web != null) synchronized(webs) { webs.add(pair(web, _i)); } }}); } e.shutdown(); e.awaitTermination(1, TimeUnit.DAYS); } finally { e.shutdown(); } L<Web> result = firstOfPairs(sortBySecondOfPairs_inPlace(webs)); } ret result; }
Began life as a copy of #1011762
download show line numbers debug dex old transpilations
Travelled to 13 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt
No comments. add comment
Snippet ID: | #1011783 |
Snippet name: | webs_readTripleFile_parallel |
Eternal ID of this version: | #1011783/11 |
Text MD5: | e80f9c965bf7503fb11285fa7ce781a2 |
Author: | stefan |
Category: | javax / a.i. |
Type: | JavaX fragment (include) |
Public (visible to everyone): | Yes |
Archived (hidden from active list): | No |
Created/modified: | 2017-11-05 04:38:45 |
Source code size: | 1169 bytes / 42 lines |
Pitched / IR pitched: | No / No |
Views / Downloads: | 430 / 461 |
Version history: | 10 change(s) |
Referenced in: | #1006654 - Standard functions list 2 (LIVE, continuation of #761) |