Tokenizing
----------

Functions: javaTok, wordTok, javaTokWithBrackets, javaTokNoQuotes, ...

All functions take a string return a string list in "CNC" format.
CNC, when counting from 1, has whitespace in odd positions and "real" tokens in even
positions.

Add a "C" to any function name to drop the white space elements, e.g. "javaTokC".

Nouns
-----

Mech Lists: "Nouns"
Functions: isNoun

Adjectives
----------

Mech Lists: "Adjectives", "Adjectives and comparatives"
Functions: isAdjective, adjectiveToAdverb
Snippets: #1011077 (List of adjectives)

Adverbs
-------

Mech Lists: "Adverbs", "Adverbial phrases"
Functions: ai_dropLeadingAdverbs, adjectiveToAdverb, isAdverb

Determiners
-----------

Mech Lists: "Determiners"
Functions: isDeterminer, ai_groupSimplestNounPhrases

Fillers
-------

("like", "uh", "maybe", ...)

Mech Lists: "Fillers", "Not fillers" (exceptions to first list)
Functions: ai_dropFillers, ai_dropFillers_*