ai_parseToTreeWithGuessing_fromTok - uses pretokenized input [dev.] [1017679]

< > BotCompany Repo | #1017679 // ai_parseToTreeWithGuessing_fromTok - uses pretokenized input [dev.]

JavaX fragment (include)

static PTElement ai_parseToTreeWithGuessing_fromTok(L<S> tok) {
  tok = dropPunctuation(tok);
  L<S> words = ai_uncategorizedWordsInSentence(tok);
  //print("Have " + n2(words, "unknown word"));
  
  new AI_BottomUpParser1 parser;
  parser.wordsToCategories = DerivedMap(ai_wordToCategories());
  Map<S, Map<S, Double>> guessMap = new HashMap;
  temp tempSetThreadLocal(ai_getWordToCategoryWeight_map, guessMap);
  for (S word : words) {
    Map<S, Double> map = ai_guessCategoriesForUnknownWord_weighted(word);
    guessMap.put(word, map);
    parser.wordsToCategories.put(word, keys(map));
  }
  ret ai_parseBottomUpToTree(parser, tok);
}

Author comment

Began life as a copy of #1017469

Travelled to 14 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, irmadwmeruwu, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt

Snippet ID:

#1017679

Snippet name:

ai_parseToTreeWithGuessing_fromTok - uses pretokenized input [dev.]

Eternal ID of this version:

#1017679/1

Text MD5:

fdae03257d9e0600ddfb7643a2fe27c1

Author:

stefan

Category:

javax / a.i.

Type:

JavaX fragment (include)

Public (visible to everyone):

Yes

Archived (hidden from active list):

Created/modified:

2018-08-02 16:25:25

Source code size:

655 bytes / 16 lines

Pitched / IR pitched:

No / No

Views / Downloads:

322 / 354

Referenced in:

[show references]

1	static PTElement ai_parseToTreeWithGuessing_fromTok(L<S> tok) {
2	tok = dropPunctuation(tok);
3	L<S> words = ai_uncategorizedWordsInSentence(tok);
4	//print("Have " + n2(words, "unknown word"));
5
6	new AI_BottomUpParser1 parser;
7	parser.wordsToCategories = DerivedMap(ai_wordToCategories());
8	Map<S, Map<S, Double>> guessMap = new HashMap;
9	temp tempSetThreadLocal(ai_getWordToCategoryWeight_map, guessMap);
10	for (S word : words) {
11	Map<S, Double> map = ai_guessCategoriesForUnknownWord_weighted(word);
12	guessMap.put(word, map);
13	parser.wordsToCategories.put(word, keys(map));
14	}
15	ret ai_parseBottomUpToTree(parser, tok);
16	}