Libraryless. Click here for Pure Java version (1129L/7K/24K).
!752 p { //makeBot("Tokenizer Bot"); } answer { if "javatok *" ret structure(javaTok(m.unq(0))); if "pythontojavatok *" ret structure(pythonToJavaTok(m.unq(0))); s = s.trim(); if (swic(s, "snltok ")) { s = dropPrefixIgnoreCase("snltok ", s); ret structure(codeTokens(snlTok(s))); } }
download show line numbers debug dex old transpilations
Travelled to 15 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, onxytkatvevr, pyentgdyhuwx, pzhvpgtvlbxg, teubizvjbppd, tslmcundralx, tvejysmllsmz, vouqrxazstgt
No comments. add comment
Snippet ID: | #1002301 |
Snippet name: | Tokenizer Bot |
Eternal ID of this version: | #1002301/1 |
Text MD5: | 69869e11876bbad49f361ec04540297a |
Transpilation MD5: | bc02ffb8963abd3d9f86c9b2dd7a240b |
Author: | stefan |
Category: | javax |
Type: | JavaX source code |
Public (visible to everyone): | Yes |
Archived (hidden from active list): | No |
Created/modified: | 2016-03-12 17:51:40 |
Source code size: | 337 bytes / 19 lines |
Pitched / IR pitched: | No / No |
Views / Downloads: | 666 / 1563 |
Referenced in: | [show references] |