loadTextFilePart_iterator - load in multiple chunks [1014618]

< > BotCompany Repo | #1014618 // loadTextFilePart_iterator - load in multiple chunks

JavaX fragment (include)

static int loadTextFilePart_iterator_chunkSize = 65536;

// TODO: close on failure
static Iterable<S> loadTextFilePart_iterator(File file, long start, long end) ctex {
  if (end-start <= loadTextFilePart_iterator_chunkSize)
    ret ll(loadTextFilePart(file, start, end));
  final FileInputStream in = newFileInputStream(file);
  in.skip(start);
  final byte[] buf = new byte[loadTextFilePart_iterator_chunkSize];
  final new UTF8Processor p;
  ret iteratorFF(func -> O {
    S s = readChunkThroughUTF8Processor(p, in, buf);
    //print("Chunk read: " + l(s));
    if (nempty(s)) ret s;
    in.close();
    ret endMarker();
  });
}

Author comment

Began life as a copy of #1006741

Travelled to 14 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, ekrmjmnbrukm, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt

Snippet ID:

#1014618

Snippet name:

loadTextFilePart_iterator - load in multiple chunks

Eternal ID of this version:

#1014618/8

Text MD5:

3302ecd39142bdc6c46dd37898eb01bd

Author:

stefan

Category:

javax

Type:

JavaX fragment (include)

Public (visible to everyone):

Yes

Archived (hidden from active list):

Created/modified:

2018-05-11 01:26:01

Source code size:

647 bytes / 18 lines

Pitched / IR pitched:

No / No

Views / Downloads:

440 / 461

Version history:

7 change(s)

Referenced in:

[show references]

1	static int loadTextFilePart_iterator_chunkSize = 65536;
2
3	// TODO: close on failure
4	static Iterable<S> loadTextFilePart_iterator(File file, long start, long end) ctex {
5	if (end-start <= loadTextFilePart_iterator_chunkSize)
6	ret ll(loadTextFilePart(file, start, end));
7	final FileInputStream in = newFileInputStream(file);
8	in.skip(start);
9	final byte[] buf = new byte[loadTextFilePart_iterator_chunkSize];
10	final new UTF8Processor p;
11	ret iteratorFF(func -> O {
12	S s = readChunkThroughUTF8Processor(p, in, buf);
13	//print("Chunk read: " + l(s));
14	if (nempty(s)) ret s;
15	in.close();
16	ret endMarker();
17	});
18	}