Not logged in.  Login/Logout/Register | List snippets | | Create snippet | Upload image | Upload data

78
LINES

< > BotCompany Repo | #1029090 // Instant Full-Text Snippet Search v7 [using deep chunked bit-set word index, dev.]

JavaX source code (Dynamic Module) [tags: use-pretranspiled] - run with: Stefan's OS

Uses 911K of libraries. Click here for Pure Java version (8432L/42K).

!7

sclass SearchResult {
  S snippetID;
  S title;
  int occurrences;
  
  sS _fieldOrder = "snippetID title";
}

cmodule FullTextSnippetSearch > DynTableWithInput2<SearchResult> {
  switchable int maxResults = 1000;
  switchable int maxTextLength = 100000;
  transient S status;
  
  JComponent mainComponent() {
    ret northAndCenterWithMargin(dm_rightAlignedLabel status(), super.mainComponent());
  }
  
  start {
    dontPersist();
    //thread { warmUp(); }
  }
  
  void calc {
    setList(search(input, true));
  }
  
  void warmUp enter {
    benchFor5Seconds(r { hijackPrint(r { search("this is cool", false) }) });
  }
  
  L<SearchResult> search(S input, bool setStatus) {
    S mod = dm_require("#1028945/CacheAllSnippets");
    SS data = cast dm_rcall getData(mod);
    SS titles = cast dm_rcall getTitles(mod);
    
    S index = dm_require("#1029089/SnippetsDeepBitSetWordIndex");
    long nanos = nanoTime();
    
    S query = upper(input);
    Iterable<S> preSnippetIDs = cast dm_call(index, 'snippetPreSearch, query);
    long nanos1 = nanoTime()-nanos;

    new L<SearchResult> out;
    if (preSnippetIDs == null) preSnippetIDs = keys(data);
    int scanned = 0;
    if (nempty(query)) {
      fOr ping (S snippetID : preSnippetIDs) {
        S text = data.get(snippetID);
        continue if l(text) > maxTextLength;
        S title = titles.get(snippetID);
        /*int n = 
          countOccurrencesIC(title, query)
          + countOccurrences_optimizedForZero(text, query);
        bool found = n > 0;*/
        scanned += l(text);
        bool found = contains(text, query); 
        if (found) {
          new SearchResult r;
          r.snippetID = snippetID;
          r.title = title;
          //r.occurrences = n;
          out.add(r);
          if (l(out) >= maxResults) break;
        }
      }
    }
    nanos = max(nanoTime()-nanos, 0);
    if (setStatus) {
      setField(status := empty(query) ? " " : "Found " + addPlusToCount(maxResults, l(out), nSnippets(out) + " in "
        + formatDouble(nanoSecondsToMilliseconds(nanos1), 3)
        + " / " + formatDouble(nanoSecondsToMilliseconds(nanos), 3)
        + " ms"));
    }
    ret out;
  }
}

Author comment

Began life as a copy of #1029044

download  show line numbers  debug dex  old transpilations   

Travelled to 7 computer(s): bhatertpkbcr, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tvejysmllsmz, vouqrxazstgt, xrpafgyirdlv

No comments. add comment

Snippet ID: #1029090
Snippet name: Instant Full-Text Snippet Search v7 [using deep chunked bit-set word index, dev.]
Eternal ID of this version: #1029090/3
Text MD5: a20fca2c5cabbdb2aa785dc50789b219
Transpilation MD5: 1fcf60c38f187009fbcbb729265539ca
Author: stefan
Category: javax
Type: JavaX source code (Dynamic Module)
Public (visible to everyone): Yes
Archived (hidden from active list): No
Created/modified: 2020-07-19 03:06:45
Source code size: 2266 bytes / 78 lines
Pitched / IR pitched: No / No
Views / Downloads: 187 / 742
Version history: 2 change(s)
Referenced in: [show references]