Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

56
LINES

< > BotCompany Repo | #728 // JsonTok - JSON Tokenizer (old, bad)

JavaX fragment (include)

1  
class JsonTok {
2  
  public static List<String> split(String s) {
3  
    List<String> tok = new ArrayList<String>();
4  
    int l = s.length();
5  
    
6  
    int i = 0;
7  
    while (i < l) {
8  
      int j = i;
9  
      char c; String cc;
10  
      
11  
      // scan for whitespace
12  
      while (j < l) {
13  
        c = s.charAt(j);
14  
        cc = s.substring(j, Math.min(j+2, l));
15  
        if (c == ' ' || c == '\t' || c == '\r' || c == '\n')
16  
          ++j;
17  
        else if (cc.equals("/*")) {
18  
          do ++j; while (j < l && !s.substring(j, Math.min(j+2, l)).equals("*/"));
19  
          j = Math.min(j+2, l);
20  
        } else if (cc.equals("//")) {
21  
          do ++j; while (j < l && "\r\n".indexOf(s.charAt(j)) < 0);
22  
        } else
23  
          break;
24  
      }
25  
      
26  
      tok.add(s.substring(i, j));
27  
      i = j;
28  
      if (i >= l) break;
29  
      c = s.charAt(i); // cc is not needed in rest of loop body
30  
31  
      // scan for non-whitespace (json strings and the "null" identifier. everything else automatically becomes a one character token.)
32  
      if (c == '\'' || c == '"') {
33  
        char opener = c;
34  
        ++j;
35  
        while (j < l) {
36  
          if (s.charAt(j) == opener) {
37  
            ++j;
38  
            break;
39  
          } else if (s.charAt(j) == '\\' && j+1 < l)
40  
            j += 2;
41  
          else
42  
            ++j;
43  
        }
44  
      } else if (Character.isLetter(c))
45  
        do ++j; while (j < l && Character.isLetter(s.charAt(j)));
46  
      else
47  
        ++j;
48  
49  
      tok.add(s.substring(i, j));
50  
      i = j;
51  
    }
52  
    
53  
    if ((tok.size() % 2) == 0) tok.add("");
54  
    return tok;
55  
  }
56  
}

Author comment

Began life as a copy of #727

download  show line numbers  debug dex  old transpilations   

Travelled to 14 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, onxytkatvevr, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt

Comments [hide]

ID Author/Program Comment Date
1072 stefan Don't use. Use function "jsonTok" 2015-08-30 17:56:36

add comment

Snippet ID: #728
Snippet name: JsonTok - JSON Tokenizer (old, bad)
Eternal ID of this version: #728/1
Text MD5: 02334b9da8660693066a16267dcca41e
Author: stefan
Category:
Type: JavaX fragment (include)
Public (visible to everyone): Yes
Archived (hidden from active list): No
Created/modified: 2015-08-30 17:56:55
Source code size: 1596 bytes / 56 lines
Pitched / IR pitched: No / Yes
Views / Downloads: 901 / 1613
Referenced in: [show references]