Text Mining Basics

Term Frequency

Count Tokens

table counts repeated values. Text mining uses this idea to turn word tokens into term frequencies.

Program

Play the script to choose a term and watch its frequency come out of the token table.

term_frequency.R 
tokens <- c("r", "data", "r", "model", "data", "data")
terms <- c("r", "data", "model")
target_index <- 
target <- terms[target_index]
counts <- table(tokens)
frequency <- as.integer(counts[[target]])
label <- paste(target, frequency, sep = "=")
cat(label, "\n", sep = "")

tokens <- c("r", "data", "r", "model", "data", "data")
terms <- c("r", "data", "model")
target_index <- 
target <- terms[target_index]
counts <- table(tokens)
frequency <- as.integer(counts[[target]])
label <- paste(target, frequency, sep = "=")
cat(label, "\n", sep = "")

tokens <- c("r", "data", "r", "model", "data", "data")
terms <- c("r", "data", "model")
target_index <- 
target <- terms[target_index]
counts <- table(tokens)
frequency <- as.integer(counts[[target]])
label <- paste(target, frequency, sep = "=")
cat(label, "\n", sep = "")
table `table(tokens)` counts how often each token appears.
term `target <- terms[target_index]` chooses which count to inspect.
frequency `as.integer(counts[[target]])` extracts the chosen count as a number.