KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the ConvertTextCase Operator to Convert Case.

The Convert Case node can be used to perform case conversions on tokenized text. It will convert the case on all tokens in a TextToken input column.

The case on all tokens will either be converted to lowercase or uppercase depending on the option selected. If the output column is unspecified then the original input column will be overwritten. This will produce a new tokenized text object that can then be used for further text processing.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the DictionaryFilter Operator to Filter Based on Dictionaries.

The Dictionary Filter node can be used to filter all words included in the dictionary input from a tokenized text column.

The dictionary field must be a string column provided through the dictionary input port. All tokens that match any of the strings in the dictionary field will be filtered from the TextToken. The filter can also be inverted, which will filter all the tokens that would have previously been kept. If the output column is unspecified, the original column in the input will be overwritten instead.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the FilterText Operator to Filter Tokenized Text.

The Length Filter node can be used to filter the tokens in a tokenized text column based on length. All tokens with a character length less than or equal to the specified length will be filtered from the TextToken. The filter can also be inverted, which will filter all the tokens that would have previously been kept. If the output column is unspecified, the original column in the input will be overwritten instead.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the FilterText Operator to Filter Tokenized Text.

The Punctuation Filter node can be used to filter the punctuation tokens in a tokenized text column. All tokens consisting of punctuation symbols will be filtered from the TextToken.

The filter can also be inverted which will filter all the tokens that would have previously been kept. If the output column is unspecified the original column in the input will be overwritten instead.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the FilterText Operator to Filter Tokenized Text.

The Regex Filter node filters the tokens in a tokenized text column based on a regular expression. All tokens that match the provided regular expression will be filtered from the TextToken.

The filter can also be inverted, which will filter all the tokens that would have previously been kept. If the output column is unspecified, the original column in the input will be overwritten instead.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the TextStemmer Operator to Stem Text.

The Text Stemmer node can be used to stem tokenized text. Stemming is the process of removing the more common morphological and inflexional endings from words. The node stems all tokens in a TextToken input column.

This node uses the Snowball stemmer algorithms to stem the tokens using the specified language algorithm.

If the output column is not set, then the original input column is overwritten. This nodes produces a new tokenized text object that can be used for further text processing.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the TextTokenizer Operator to Tokenize Text Strings.

The Text Tokenizer node can tokenize a String field as an object that can then be used for a variety of text processing tasks.

The Text Tokenizer node has two options, which specify the source column and target column. The source must be of type String while the target will be a tokenized text object. The contents of the string field are parsed for each row and tokenized to create a TextToken cell that is saved in the target column. This object can then be used for other text processing tasks.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the FilterText Operator to Filter Tokenized Text.

The Word List Filter node can be used to filter the tokens in a tokenized text column based on a list of words. All tokens that match any of the strings in the provided list are filtered from the TextToken.

The filter can also be inverted to filter all tokens that do not match the word list and keep the ones that do.

If the output column is not set, the original column in the input is overwritten.

This feature can be useful when filtering a few named entities or specific words from the original text.

When filtering a larger list of words, we recommend that the Dictionary Filter be used instead.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the CalculateNGramFrequency Operator to Calculate N-gram Frequencies.

The Calculate N-grams node can be used to create a list of n-grams using the specified n from a TextToken input column. It also determines the absolute frequency of each n-gram.

The node creates a new output column containing the calculated n-grams and their frequencies.

The source column must be a TextToken cell, while the output column is encoded as an NgramFrequency cell. The n-gram frequency object can then be used by various other nodes that operate on frequency cells.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the CalculateWordFrequency Operator to Calculate Word Frequencies.

The Calculate Word Frequencies node calculates the frequency of each unique word token in a tokenized text input column. The node creates a new output column containing the calculated word frequencies. The source column must be a TextToken cell, while the output column is encoded as a WordFrequency cell. The word frequency object can then be used by other nodes that operate on frequency cells.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the CountTokens Operator to Count Tokens.

The Count Tokens node counts the number of a particular text element token type. The types of tokens that can be counted are words, sentences, paragraphs, and documents. By default this node counts the number of word tokens in each TextToken. The total counts are put in the specified count column.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the ExpandTextFrequency Operator to Expand Text Frequencies.

The Expand Frequency node can be used to expand a word frequency or n-gram frequency field. It outputs a field containing the expanded text and a field containing the associated frequencies. If either of the output fields are unspecified, then the field is not included in the output. This dropping of fields causes an expansion of the original data, since any field not actively expanded is simply copied. The output for the frequencies can also be customized by selecting whether absolute or relative frequencies are written to the frequency output field.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the ExpandTextTokens Operator to Expand Text Tokens.

The Expand Text Tokens node can be used to expand a TextToken input column by token type. It creates a new string field specified by the target column name and populates it with elements from the original TextToken field, with a single element per copied row. This action causes an expansion of the original data, since any field not actively expanded is simply copied.

 KNIME  This topic describes a KNIME node. For the DataFlow operator it is based on, see Using the TextFrequencyFilter Operator to Filter Frequencies.

The Frequency Filter node can be used to filter the frequencies in a frequency cell. It can filter n-gram frequencies or word frequencies and outputs the filtered frequencies in the specified field. The frequencies can be filtered based on a minimum and maximum threshold value or filtered so that only the top frequencies are kept.

If the output field is not set, then the original frequency column is overwritten in the output. The new filtered frequency object can be used by other nodes operate on frequency cells.