Included Tokens

Discovery comes with the following token definitions included:

Symbol name Explanation Examples Default Cleaning Function
"NUM" Used for capturing number expressions. "one", "thousand", "million" text2int
"LETTER" Use for picking out an individual letter. "a", "b", "alpha", "bravo" alphaToA
"NUM_ORD" Used for ordinal numbers "first", "second" text2int
"NUM_PLURAL" Plural numbers "ones", "thousands" text2int
"TIME_MONTH" Calendar months "January", "February" none
"TIME_DAY" Days of the week "Monday", "Tuesday" none
"TIME_WEEK" Week words "week", "weeks" none

Included Cleaning Functions

Discovery comes with the following cleaning functions included:

Symbol name Explanation Default Spacing Example Input Example Output
text2int Convert text numbers to numerals No Spaces "one three five" 135
alphaToA Convert International Radiotelephony expressions to the alphabet. Single space "alpha bravo zulu" "a b z"