Overview
The fundamental unit of data in DataFlow is the token. Operators consume and create tokens, sending and receiving them over ports. Tokens have two attributes:
• A data type. The data type indicates the kinds of values represented by the token.
• A value. A token may also have no value, in which case it is called null valued. This is analogous to NULL in SQL.
The DataFlow Software Development Kit (SDK) supports many token types, arranged in a hierarchy of more general and more specific types. For each supported token type, a corresponding Java type exists that is used when interacting with the typed object. Specifics of the mappings are provided in
Token Types and Related Java Types.