please include function webs_readTripleFileAsTriples. static L<TripleWeb> webs_readTripleFileAsTriples_parallel2(File f) ctex { if (!f.exists()) ret ll(); time "Getting names" { Iterator<S> it = linesFromFile(f); final new L<Symbol> names; while (it.hasNext()) { S s = trim(it.next()); if (empty(s)) break; names.add(symbol(unquote(s))); } } if (webs_readTripleFileAsTriples_debug) print("Have " + l(names) + " names"); final L<Pair<L<TripleWeb>, Int>> triples = new L; int poolSize = numberOfCores(), queueSize = 10000; int blockSize = 1000; NotifyingBlockingThreadPoolExecutor e = new(poolSize, queueSize, 15, TimeUnit.SECONDS); print("Using " + poolSize + " cores, block size: " + blockSize + "."); try { int i = 0; L<S> block = emptyListWithCapacity(blockSize); while licensed { ++i; final Int _i = i; bool hasNext = it.hasNext(); if ((!hasNext || l(block) >= blockSize) && nempty(block)) { final L<S> _block = cloneList(block); e.execute(r { pcall { L<TripleWeb> l = emptyListWithCapacity(_block); for (S s : _block) { TripleWeb w = webs_readTripleFileAsTriples_line(s, names); if (w != null) l.add(w); } synchronized(triples) { triples.add(pair(l, _i)); } }}); block.clear(); } if (hasNext) block.add(it.next()); else break; } e.shutdown(); e.awaitTermination(1, TimeUnit.DAYS); } finally { e.shutdown(); } sortBySecondOfPairs_inPlace(triples); int n = 0; for (Pair<L<TripleWeb>, Int> p : triples) n += l(p.a); L<TripleWeb> l = emptyListWithCapacity(n); for (Pair<L<TripleWeb>, Int> p : triples) l.addAll(p.a); ret l; }
Began life as a copy of #1012453
download show line numbers debug dex old transpilations
Travelled to 13 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt
No comments. add comment
Snippet ID: | #1012454 |
Snippet name: | webs_readTripleFileAsTriples_parallel2 |
Eternal ID of this version: | #1012454/10 |
Text MD5: | b6d35ec8cdf426977c05a0a62697eae9 |
Author: | stefan |
Category: | javax / a.i. |
Type: | JavaX fragment (include) |
Public (visible to everyone): | Yes |
Archived (hidden from active list): | No |
Created/modified: | 2017-12-31 21:00:22 |
Source code size: | 1906 bytes / 66 lines |
Pitched / IR pitched: | No / No |
Views / Downloads: | 406 / 426 |
Version history: | 9 change(s) |
Referenced in: | #1006654 - Standard functions list 2 (LIVE, continuation of #761) |