Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

15
LINES

< > BotCompany Repo | #1021204 // addRangesToTokenization

JavaX fragment (include)

1  
static L<TokenAndRange> addRangesToTokenization(S text, LS tokens) {

2  
  if (tokens == null) null;

3  
  L<TokenAndRange> out = emptyList(tokens);

4  
  int i = 0;

5  
  for (S tok : tokens) {

6  
    int j = i+l(tok);

7  
    out.add(TokenAndRange(tok, intRange(i, j), text));

8  
    i = j;

9  
  }

10  
  ret out;

11  
}

12  


13  
static L<TokenAndRange> addRangesToTokenization(LS tokens) {



14  
  ret addRangesToTokenization(join(tokens), tokens);



15  
}



      
download  show line numbers  debug dex  old transpilations   
Travelled to 7 computer(s): bhatertpkbcr, cfunsshuasjs, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tvejysmllsmz, vouqrxazstgt


No comments. add comment
      
      
  

  

    Snippet ID:
    #1021204
  

  

    Snippet name:
    addRangesToTokenization
  

    

    Eternal ID of this version:
    #1021204/2
  

  

    Text MD5:
    e9fa4a249932fe45cfc905e129f11ff5
  

    

    Author:
    stefan
  

  

    Category:
    javax / parsing
  

  

    Type:
    JavaX fragment (include)
  

        

    Public (visible to everyone):
    Yes
  

  

    Archived (hidden from active list):
    No
  

  

    Created/modified:
      2019-02-03 00:54:20
  

  

    Source code size:
    414 bytes / 15 lines
  

  

    Pitched / IR pitched:
    No / No
  

    

    Views / Downloads:
    395 / 422
  

      

    Version history:
    1 change(s)
  

    

    Referenced in:
[show references]