Was this helpful?
File Transfer Iterator
The File Transfer Iterator splits a file into pieces then stores each piece as a message. Each piece/message can be processed separately if desired, and then reassembled using the File Transfer Aggregator.
File Transfer Iterator Properties
Property Name
Description
Source File Path
Path and name of the input file that is being split into pieces.
Encoding
Encoding of the source file:
ISO-8859-1 to ISO-8859-9
UTF-8 (default)
UTF-16
Shift_JIS
Cp1252
EUC-JP
For value descriptions, see Encoding Tips.
Transfer Unit Type
The unit to use when splitting the file into pieces:
Number of Bytes
Number of Records (default)
Number of Parts
The Encoding, Binary content, and Record Separator properties are not displayed when Transfer unit type is selected as Number of Bytes.
Number of Bytes
Determines the number of bites to include in a message. The default value is 1024 bytes.
This property is displayed when Transfer unit type is selected as Number of Bytes.
Number of Records
Determines the number of records to include in a message. The default value is 1 record.
This property is displayed when Transfer unit type is selected as Number of Records.
Number of Parts
This property allows the user to break the source file (referenced in the Source File Path property) into multiple smaller messages. The Number of Parts property dictates how many messages will be created from the source file.
The default value is 2 parts.
This property is displayed when Transfer unit type is selected as Number of Parts.
Binary Content
Indicates whether the file being read has binary content.
Valid values: TRUE, FALSE (default).
Record Separator
The record separator to use when dividing the source into groups of records.
Type of select from the following list of allowed values:
CR-LF
LF-CR
CR
LF
System-Default (default)
Preserve Header Record
The File Transfer Iterator splits a file into specified number of pieces, then stores each piece as a message. Each piece/message can be processed separately, if desired. If you are using the File Transfer Iterator to process each file piece separately (could be in parallel threads or otherwise), the Preserve Header Record property ensures that the file piece that contains the header doesn’t have to be processed first. For this set this property to TRUE.
TRUE: Set this property to TRUE to include the header in all file pieces. Then you do not have to worry about processing the file pieces in any specific order because they will all contain the header. The header will only be written once.
FALSE (default): Only one of the file pieces will include the header. To ensure that the target database is loaded correctly and without error, the piece containing the header should be processed first.
Supported Actions
Action
Description
Connect
Connects with the component source.
Disconnect
Breaks the connection with the component source.
GetMessage
Returns data from the Iterator in a DJMessage object. The maximum size limit for DJMessage is 2 GB.
Supported Action Parameters
Action
Parameter
Description
GetMessage
Message
When the GetMessage action is called, the Iterator advances to the next element in the Iterator source and returns it in the message body of the specified DJMessage object. The maximum size limit for DJMessage is 2 GB.
Supported Action Properties
There are no supported action properties.
Supported DJMessage Properties
The following properties must be set manually or by using a preceding step:
Property
Description
TransferId
Uniquely identifies each transfer. It is a hash of the source name, target host, target name and transfer start time.
SourceName
The name of the source file.
PieceNumber
Indicates the sequential order of the piece.
TotalPieces
The total number of pieces.
StartTime
A numeric representation of the date/time the transfer started.
This is the current time as UTC milliseconds from the epoch.
SourceEncoding
Encoding of the source file:
ISO-8859-1 to ISO-8859-9
UTF-8 (default)
UTF-16
Shift_JIS
Cp1252
EUC-JP
For value descriptions, see Encoding Tips.
TransferUnitCount
Tells you how many records are being sent for the current iteration. It is for analytical purpose only.
This is a read only value, generated by the Iterator, which may be used by other components.
DJMessage Example
The following example demonstrates how to set the message properties in the Script step:
msg.Properties(“PieceNumber”) = “1”
msg.Properties(“SourceName”) = “testdatanew.txt”
msg.Properties(“StartTime”) = “1657607994344”
msg.Properties(“TotalPieces”) = “1”
msg.Properties(“TransferId”) = “testdatanew1657607994344”
msg.Properties(“encoding”) = “UTF-8”
msg.Properties(“transferUnitCount”) = “20”
Last modified date: 10/22/2024