splitIntoSentences_v2 (with questions) [1008435]

< > BotCompany Repo | #1008435 // splitIntoSentences_v2 (with questions)

JavaX fragment (include) [tags: use-pretranspiled]

static L<S> splitIntoSentences_v2(S s, O... _) {
  ret splitIntoSentences_v2_tok(nlTok3(s), _);
}

static Cl<S> splitIntoSentences_v2_tok_endMarkers = litset(".", "?", "!");

static L<S> splitIntoSentences_v2_tok(L<S> s, O... _) {
  optPar Cl<S> endMarkers = splitIntoSentences_v2_tok_endMarkers;
  int i = 0;
  new L<S> l;
  while (i < l(s)) {
    int j;
    if (emptyAfterTrim(s.get(i)) && isRoundBracketed(get(s, i+1)))
      j = i+2;
    else
      j = smartIndexOfAny(s, i, endMarkers)+1;
    addIfNempty(l, trimJoinSubList(s, i, j));
    i = j;
  }
  ret l;
}

Author comment

Began life as a copy of #1008429

Travelled to 15 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, irmadwmeruwu, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt, xrpafgyirdlv

Snippet ID:

#1008435

Snippet name:

splitIntoSentences_v2 (with questions)

Eternal ID of this version:

#1008435/9

Text MD5:

0b713c04a32da2ee3583da5bafe7bee5

Transpilation MD5:

9aafebc0f1dd93d12a1e8b13882d3739

Author:

stefan

Category:

javax / parsing

Type:

JavaX fragment (include)

Public (visible to everyone):

Yes

Archived (hidden from active list):

Created/modified:

2020-02-23 11:03:21

Source code size:

585 bytes / 21 lines

Pitched / IR pitched:

No / No

Views / Downloads:

538 / 653

Version history:

8 change(s)

Referenced in:

[show references]

1	static L<S> splitIntoSentences_v2(S s, O... _) {
2	ret splitIntoSentences_v2_tok(nlTok3(s), _);
3	}
4
5	static Cl<S> splitIntoSentences_v2_tok_endMarkers = litset(".", "?", "!");
6
7	static L<S> splitIntoSentences_v2_tok(L<S> s, O... _) {
8	optPar Cl<S> endMarkers = splitIntoSentences_v2_tok_endMarkers;
9	int i = 0;
10	new L<S> l;
11	while (i < l(s)) {
12	int j;
13	if (emptyAfterTrim(s.get(i)) && isRoundBracketed(get(s, i+1)))
14	j = i+2;
15	else
16	j = smartIndexOfAny(s, i, endMarkers)+1;
17	addIfNempty(l, trimJoinSubList(s, i, j));
18	i = j;
19	}
20	ret l;
21	}