User Guide : Designing and Executing Processes : Process Steps and Associated Components : Iterator Step
 
Share this page             
Iterator Step
The Iterator step allows you to choose an Iterator for a process. The Iterator component is used to extract individual parts from a larger data object. The iterator places the individual parts into message objects returned by the GetMessage action. Each subsequent call to GetMessage returns the next part. Iterators do not change the source data object. A typical application of an Iterator component includes breaking apart EDI documents to allow processing of the document contents.
Iterator Step Properties
After adding an /download/attachments/24976125/Iterator_Step.png?version=1&modificationDate=1488346578954&api=v2 Iterator step from the Palette to the canvas, you can double-click the step on the canvas and specify the following step properties in the Properties tab displayed at the bottom.
Name
Description
Name
Unique name for the step. You can use alphabets, numbers, and special characters such as -, _. . , and #.
Description
Description of the step (optional).
Enabled
Select this option to enable the step for execution. If it is not enabled, then the step on the canvas is greyed-out.
This option is useful when troubleshooting a process, especially when there are numerous steps in the process.
You can also enable or disable the step using the right-click menu options. If the step is enabled, then to disable the step, right-click on the step and select Disable Step. If the step is disabled, then to enable the step, right-click on the step and select Enable Step.
Error Handling
Select either of the following:
Abort Process - All the steps after the current step are skipped or the entire process is stopped (depending on how you have set the error logging options) if this step stops due to an error.
Ignore Error - The remaining steps in the current process are executed even if this step stops due to an error. This is selected by default.
Select Iterator
Select an Iterator instance. These instances are displayed when you add message components in the Configuration tab. For more information, see Adding Message Components in Configuring Processes.
Action
Select an action for the iterator instance.
Parameters
Parameters for the selected action. Specify values for the parameters.
Properties
Properties for the selected action. Specify values for the properties.
Note:  The Actions, Parameters, and Properties are specific to the component type. For information about this, based on the component instance selected in the Select Iterator drop-down list, click the component type in the Iterator Step Components section below.
Iterator Step Components
EDI X12 Batch Transaction Iterator
EDI X12 Functional Group Iterator
EDI X12 Transaction Set Iterator
File Transfer Iterator
JDBC WebRowSet Iterator
JSONPath Iterator
XPath Iterator
EDI X12 Batch Transaction Iterator
EDI X12 Batch Transaction Iterator splits an EDI (X12) batch file based on its Interchange Control Header/Trailer (ISA/IEA).
EDI X12 Batch Transaction Iterator Properties
 
Property Name
Description
SourceName
Path and file name of the source.
Unicode
Indicates whether source file is Unicode. Default value is False.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the component source.
Disconnect
Break the connection with the component source.
GetMessage
Returns data from the Iterator in a djmessage object. The maximum size limit for DJMessage is 512 MB.
Supported Action Parameters
 
Action
Parameter
Description
Connect
Database
Name of the database.
GetMessage
Message
Type the message.
Supported Action Properties
There are no supported action properties.
EDI X12 Functional Group Iterator
EDI X12 Functional Group Iterator splits an EDI (X12) file based on its Functional Group Header/Trailer (GS/GE).
EDI X12 Functional Group Iterator Properties
 
Property Name
Description
SourceName
Path and file name of the source.
Unicode
Indicates whether source file is Unicode. Default value is False.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the component source.
Disconnect
Break the connection with the component source.
GetMessage
Returns data from the Iterator in a djmessage object. The maximum size limit for DJMessage is 512 MB.
Supported Action Parameters
 
Action
Parameter
Description
Connect
Database
Name of the database.
GetMessage
Message
Message string.
Supported Action Properties
There are no supported action properties.
EDI X12 Transaction Set Iterator
EDI X12 Transaction Set Iterator splits a EDI (X12) input and output files based on the Transaction Set Header/Trailer (ISA/IEA). It also has one functional group header (GS/GE). The input file has several transaction control set headers (ST/SE), while the output file has one transaction control set header (ST/SE) to preserve the message properties in a header file.
EDI X12 Transaction Set Iterator Properties
 
Property Name
Description
SourceName
Path and file name of the source.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the component source.
Disconnect
Break the connection with the component source.
GetMessage
Returns data from the Iterator in a djmessage object. The maximum size limit for DJMessage is 512 MB.
Supported Action Parameters
 
Action
Parameter
Description
Connect
Database
Name of the database.
GetMessage
Message
Message string.
Supported Action Properties
There are no supported action properties for this iterator.
File Transfer Iterator
The File Transfer Iterator splits a file into pieces that are stored in message objects that can be transferred to another location and reassembled. The iterator uses properties on the message object to store information about how the pieces are reassembled into a complete file. For an example of using the properties, see File Transfer Aggregator. Whatever transfer mechanism is used must preserve the message properties. The File Folder Queue components use the Serialize and Deserialize options to preserve the message properties.
File Transfer Iterator Properties
 
Property Name
Default Value
Description
TransferUnitType
Bytes
Type of unit to use when splitting the file into pieces. This option is used along with the TransferUnitCount option to determine the size of the pieces to split the file into. The options available are:
Bytes
Records
TransferUnitCount
0
Number of units each piece of the file contains. This option is used in conjunction with the TransferUnitType option to determine the size of the pieces to split the file into. For example, if the TranferUnitType is Records and the TransferUnitCount is 10, then each message object contains 10 records from the source file.
SourceName
 
Path and file name of file that is split into pieces.
Encoding
ENC_OEM or 0
Character set of the source data. This indicates using the default character set for the system. The ENC_ names can be used in expressions for setting the encoding value.
Allowable values for supported character sets are listed under Encoding Tips.
RecordSeparator
 
The record separator to use when dividing the source into groups of records. This should be Unicode characters. Default is none.
Some allowable values are:
\r\n (CR-LF)
\n (LF)
\r (CR)
\n\r (LF-CR)
\x2028 (Unicode line separator)
\x2029 (Unicode paragraph separator)
\f (Form feed)
\r\n\r\n (Windows empty line)
\n\n (Unix empty line)
Note:  The actual characters (or escape sequences) must be used, that is, using 'CR-LF' will not work, you must use '\r\n'.
RecordLength
 
If records are separated by a record separator, this property must be set to 0. If the records are fixed-length (no separator), then this should be the actual record length. Any time this property has a value other than 0, the RecordSeparator property will be ignored.
TargetName
 
Name that is given to the file when it is reassembled. The TargetName option includes the path and file name.
TargetHost
 
Name of the host where the target file is reassembled. This option is used by the File Transfer Aggregator to verify that the file is being reassembled on the correct host.
BinaryIndicator
False
Indicates whether the file being read should be opened in binary mode.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the component source.
Disconnect
Break the connection with the component source.
GetMessage
Returns data from the Iterator in a djmessage object. The maximum size limit for DJMessage is 512 MB.
Supported Action Parameters
 
Action
Parameter
Description
Connect
Database
Name of the database.
GetMessage
Message
When the GetMessage action is called for an Iterator, the Iterator advances to the next element in the Iterator source and returns it in the message body of the specified DJMessage object. The maximum size limit for DJMessage is 512 MB.
Supported Action Properties
There are no supported action properties.
Supported DJFT Properties and Parameters
The following properties must be set manually, if the previous process step does not generate it before calling the component.
Property
Description
SourceName
The name of the source file.
TranferUnitType
Indicates how to divide the source file into pieces. Allowable values are Bytes and Records.
TransferUnitCount
The number of bytes of number or records to include in each piece.
RecordLength
The number of characters to include in a record.
SourceRecordSeparator
The record separator to use when splitting the source. Default None.
SourceEncoding
The character set of the source data. The default is ENC_OEM (0).
TargetHostSystem
The name of the target host system.
TargetName
The name of the target file.
TargetExpiration
The time period for the transfer. The time expires when the current time equals the start transfer time plus the target expiration value.
TargetExpirationType
The type of the target expiration value. Allowable values Years, Months, Days, Hours, Minutes, Seconds.
TargetMode
Indicates whether to replace or append to the target. Allowable values are Replace and Append.
TargetRecordSeparator
The record separator to use when assembling the target. If not set, the SourceRecordSeparator value is used.
TargetEncoding
The character set of the target data. If not set, the SourceEncoding value is used.
The following are the parameters.
Parameter
Description
TransferId
Uniquely identifies each transfer. It is a hash of the source name, target host, target name and transfer start time.
PieceNumber
Indicates the sequential order of the piece.
PieceSize
Indicates the size of the message body in bytes.
Priority
The priority of the piece. Default 0. The last message has a priority of 1.
TotalPieceCount
The total number of pieces.
StartTime
The date/time the transfer started.
JDBC WebRowSet Iterator
The JDBC WebRowSet Iterator connects to the specified database, runs the provided SQL SELECT statement, recurses through the row sets, and returns the WebRowSet XML in a DJMessage object. The maximum size limit for DJMessage is 512 MB.
Note:  Use JDBC WebRowSet Iterator only for reading from a table and not for running INSERT/UPDATE/DELETE statements on a database. Use JDBC WebRowSet Aggregator to perform update/insert/delete operations on a database. For more information, see JDBC WebRowSet Aggregator.
To connect to a database using JDBC WebRowSet Iterator, you must provide information about the required type 4 JDBC drivers, database connection values, and the number of records per rowset.
The component performs the following actions:
1. Reads the connection information, loads the drivers, and connects to the target database using the Host URL, User ID, and password.
2. Connects to the target database and runs the provided SQL SELECT query.
3. Returns the specified number of records per rowset as specified in the transfer batch size.
4. Disconnects from the target database.
The main features of JDBC WebRowSet Iterator are as follows:
Supports the following databases (type 4 drivers only) when used with the appropriate local libraries:
Oracle
IBM DB2
PSQL
Microsoft SQL Server
Queries the database based on the SELECT statement provided.
Recurses through the result set and outputs it in the WebRowSet XML format.
You must have the following skill set to use JDBC WebRowSet Iterator:
Database administration
Java Database Connectivity (JDBC) knowledge
Scripting
JDBC WebRowSet Iterator Properties
 
Property
Default Value
Description
Transfer Batch Size
1000
Number of records returned per rowset.
Decimal values are rounded off to the nearest integer.
Note:  If you specify the value as 0, then XPath returns all the records in the rowset.
For example, if you set 500 as the Transfer Batch Size for a 1000 record rowset, then the number of WebRowSet XML responses returned is 2.
Null Support
None
Handles null and empty string values.
If you set Null Support to None, then empty strings and null values are written as <columnValue></columnValue>.
If you set Null Support to "null tag", then the following actions apply:
Null values are written as <columnValue><null/></columnValue>
Empty strings are written as <columnValue><emptyString/></columnValue>
If you set Null Support to "nil attribute", then the following actions apply:
Null values are written as <columnValue xsi:nil='true'></columnValue>
Empty strings are written as <columnValue></columnValue>
Driver
 
Name of JDBC driver to load
For example: com.ibm.db2.jcc.DB2Driver
Host URL
 
URL connection to the database
For example: jdbc:db2://localhost/mydbname
User ID
 
User name for the database session
For example: db2user
Note:  Specify the user name and password for the database session only if it is configured for your database.
Password
 
Password for the database session
For example: mydb2pwd
Jar Location
 
Location of the folder that contains the JAR files that are required to load the JDBC driver.
Note:  You cannot append multiple paths to the value. All the driver JAR files must be located in the same folder.
For example: For DB2, you must load the db2cc.jar (type 4 driver).
Caution!  Always use the latest .jar files for a particular database version.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the database.
Tip...  Call Connect explicitly in your process flow.
Disconnect
Closes an existing database connection and performs any clean up actions if required.
Tip...  Call Disconnect explicitly in your process flow.
GetMessage
Iterates through the records in the result set, and returns the specified number of records (Transfer Batch Size) in the WebRowSet XML format.
The GetMessage requires the DJMessage object that holds the WebRowSet XML output. Call GetMessage immediately after PutMessage to iterate over the record set.
The maximum size limit for DJMessage is 512 MB.
PutMessage
Runs the SQL Select statements, processes the values, and stores the result set.
The Put Message requires the DJMessage object that holds the SQL SELECT statement. The maximum size limit for DJMessage is 512 MB.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
PutMessage
Message
Message string.
Supported Action Properties
There are no supported properties.
Errors
 
Error Code
Error Name
Description
Reason
4
ERR_READERR
Error while reading messages from a DJMessage body or iterating through the result set
ERR_READERR is returned in the following cases:
Problem reading messages from a DJMessage body
Problem iterating through the result set
8
ERR_WRITERR
Error while writing messages to a DJMessage body or when running the SELECT query
ERR_WRITERR is returned in the following cases:
Problem writing messages to a DJMessage body
Problem running the SELECT query
19
ERR_OPENERR
Error while connecting to the database
Exception is encountered when connecting to the specified database.
27
ERR_CLOSERR
Error while disconnecting from the database
Exception is encountered when disconnecting from the specified database
86
ERR_MSG_END
Error indicating the end of a given set of messages
Returned when there are no more messages to be returned by the "getMessage" method
Supported Data Types
 
Database
Data Types
DB2
bigint
character
clob
date
decimal
double
integer
real
smallint
time
timestamp
varchar
PSQL v9
bfloat4
bfloat8
bigint
bit
char
currency
date
decimal
double
float
identity
integer
longvarchar
money
numeric
numericsa
numericsts
real
smallidentity
smallint
time
timestamp
tinyint
ubigint
uinteger
usmallint
utinyint
varchar
SQL Server 2000
bigint
bit
char
datetime
decimal
float
int
money
nchar
ntext
numeric
nvarchar
real
smalldatetime
smallint
smallmoney
sql_variant
text
tinyint
uniqueidentifier
varchar
Oracle 9i
Varchar2
Char
Number
Integer
Date
Long
NVarchar2
RowId
NChar
Clob (Use the latest drivers)
Nclob (Use the latest drivers)
Float
Char Varying
Character
Character varying
Decimal
Double Precision
Int
National char
National char varying
National character
National character varying
Nchar varying
Numeric
Real
SmallInt
Varchar
Unsupported Data Types
 
Database
Data Type
Oracle
Long Raw (Binary)
Raw (Binary)
MLSLabel
Blob (Binary)
BFile
URowId
URIType
DB2
BLOB (Binary)
SQLServer
Binary
Image (Binary)
VarBinary
PSQL
Binary
LongVarBinary
JSONPath Iterator
The World Wide Web Consortium (W3C) formalized an official specification for an XML path language (XPath) in 1999. XPath has served as a central mechanism for directly addressing parts of XML documents since that time. It is integral to XSLT, XPointer and countless software which consumes or uses XML.
Unfortunately, there is no "formal" JSON path language specification backed by an organization like the W3C. However, there is a convention, which was defined by Stefan Goessner in 2007. The convention defines a path language for JSON that is similar to XPath. He coined the name JSONPath for the notation. Since that time, multiple JSONPath implementations in various programming languages have been developed. JSONPath is the de facto "XPath for JSON".
Given the proliferation JSON-formatted content, the need exists to be able to extract specific properties or objects from JSON documents without having to write a map. The JSONPath Iterator provides a means to accomplish this task.
In a nutshell, the JSONPath Iterator supports the following features:
Specification of a source JSON document
Evaluation of a JSONPath expression against the source document
Iteratively retrieve matching objects or properties from the document
JSONPath Iterator 1.0.0 version is supported.
JSONPath Iterator Properties
When you create an instance of JSONPath Iterator component in the Configuration tab > Message Components section, properties will not be defined in the Edit Message Component window.
Supported Actions
 
Action
Description
PutMessage
Consumes the provided JSON document and JSONPath expression. Loads the document and evaluates the expression against it.
GetMessage
Can be called repeatedly to retrieve the values which were matched by the expression that was evaluated by PutMessage. Returns EOF if no additional values remain.
Supported Action Parameters
 
Action
Parameter
Description
PutMessage
Message
The source DJMessage for PutMessage. It can optionally contain the following information:
Source JSON document (if both the JSON File action property and JSONPath.sourcefile message property are not set)
JSONPath.expression message property. See DJMessage Option Override Properties
GetMessage
Message
When GetMessage is called, the body of the DJMessage referenced by the Message property will be populated with the next value from the set of results, which were produced by the evaluation of the JSONPath expression against the source document.
 
Supported Action Properties
 
Action
Property
Description
PutMessage Action
JSONPath Expression
The expression that will be evaluated against the loaded JSON document. This value can be overridden by the DJMessage property, JSONPath.expression. It is an error condition if both are missing.
JSON File
The name of a local file from which (if provided) the component will read the source JSON document. The value of this option (whether it is blank or populated) will be overridden by the DJMessage property, JSONPath.sourcefile. If neither is provided, then the source JSON document is expected to be in the body of the DJMessage argument to PutMessage. It is an error condition if the source JSON document cannot be loaded using any of the options.
GetMessage Action
Not applicable
Not applicable
DJMessage Option Override Properties
The values provided in the PutMessage action properties (JSONPath Expression and JSON File) can be overridden with DJMessage properties. This section describes the override properties:
JSONPath.expression: Can be set in the DJMessage referenced by the argument to the PutMessage action. If provided, it overrides the value set in the JSONPath Expression option.
Note:  It is considered an error if both JSONPath Expression action property and JSONPath.expression DJMessage property are missing.
JSONPath.sourcefile: Can be set in the DJMessage referenced by the argument to the PutMessage action. If provided, it will override any value that is set in the JSON File option.
Note:  If both are missing, then the source JSON document is read from the DJMessage body.
Errors
The following table describes error codes generated for the JSONPath Iterator component.
 
Action
Code
Description
PUTMESSAGE
ERR_BADOPTIONVALUE
Generated if the JSONPath expression is not set.
 
ERR_INVALID
Multiple errors can result in ERR_INVALID. These include the following:
Unable to read the source JSON document
Unable to parse the source JSON document
JSONPath expression is badly formatted
GETMESSAGE
ERR_READERR
Generated if GetMessage is called without initializing the component instance with PutMessage. In other words, the component is not "connected."
ERR_EOF
Generated if GetMessage is called after the last result from the expression evaluation has been retrieved.
XPath Iterator
The XPath Iterator is a non-XML language used to iterate through and identify specific sections in an XML document. Use XPath Iterator to extract content from an XML document and iterate over the extracted result set.
XPath Iterator 1.1.0, 1.1.1, and 1.2.1 versions are supported.
The XPath Iterator does the following:
Enables extracting data from complex XML documents
Facilitates iteration over the extracted result set
Accepts sources, and queries at design time or run time
The following skill set is recommended:
Scripting
XML
XML Path Language (XPath)
XPath Iterator Properties
The following properties are displayed only for XPath Iterator 1.1.0 and XPath Iterator 1.1.1.
Property
Description
Dynamic XPath or XML Source Document
Dynamic XPath or Dynamic XML source.
Source XML Document
Path to XML document.
In the step property, you can provide the following:
Step option value (design time) - XML file or XPath expression
Source DJMessage object (run time) - XML file or XPath expression
If the input is a source XML file, then the file content can be passed in the source message.
Source Message is the value supplied within the source DJMessage.
Supported Actions
 
Action
Description
PutMessage
Extracts data from an XML source document (based on the XPath expression provided) and prepares the result set.
PutMessage lists an optional source message parameter containing either an XPath expression or an XML source (URL).
The PutMessage action contains two step properties: XML source and XPath expression. You can pass the XML source or XPath expression at design time or through Source Message at run time.
GetMessage
Iterates through the result set. This action should be called after the PutMessage action. The GetMessage action requires a target DJMessage object. The maximum size limit for DJMessage is 512 MB.
Note:  The process returns error code 4 every time you call GetMessage before PutMessage.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
PutMessage
Message
Message string.
Supported Action Properties
There are no action properties for XPath Iterator version 1.1.0 and 1.1.1.
The following action properties are applicable only for XPath Iterator 1.2.1.
Action
Property
Description
PutMessage
XML Source
Source message
PutMessage
XPath Expression
Source message
This table outlines the various possible step property combinations supported by XPath.
XML Source
XPath Expression
Description
Example
Combination of Design and Run Time
Source Message (Content)
Expression
Provide XML content at run time
srcMsg.body=fileread("C:\people.xml")
Provide XPath expression at design time
//Person[@title="doctor"]
Source Message (URL)
Expression
Provide URL to the XML file at run time
srcMsg.Body = "C:\people.xml"
Provide XPath expression at design time
//Person[@title="doctor"]
URL
Source Message (Expression)
Provide XML content at design time
C:\people.xml
Provide XPath expression at run time
srcMsg.Body="//Person[@title="doctor"]"
Design Time
URL
Expression
Provide URL to the XML file at design time
C:\people.xml
 
 
Provide XPath expression at design time
//Person[@title="doctor"]
*Content: msg.Body = fileread("C:\myspace\people.xml")
**URL: msg.Body = "C:\myspace\people.xml"
Errors
 
Code
Name
Description
Reason
1
ERR_EOF
End of file reached
No more items to return
4
ERR_READERR
Component not initialized
XPath is not initialized. ERR_READERR occurs when GetMessage is called before PutMessage.
34
ERR_INVALID
Invalid XPath expression or XML source file
Error occurred while processing XPath expression or loading XML source
Possible reasons:
Invalid XPath expression or XML source
Empty field value
50
ERR_UNSPECIFIED
Unknown error
Unknown error identified by the component