User Guide : Designing and Executing Processes : Process Steps and Associated Components : Queue Step
 
Share this page             
Queue Step
Queues provide access to applications or middleware that produce and consume data in the form of messages. Queues encapsulate the information necessary to establish a connection with the messaging system. Queue components can access data and services provided by message queuing systems. This includes the ability to read and write messages to and from a queue as well as transactional support. Queue components can provide access to common message queuing systems such as FTP Queue, AWS S3 Queue, File Folder Queue, Email Queue, JMS Queue, Oracle AQ, IBM MQ, and MSMQ. They can also be used to provide a queuing interface to a nonmessage-based system, like a file system. Typical queuing systems perform a destructive read. In other words, once a message is read, it is removed from the queue.
For each type of queue component, you create a corresponding Queue step. The Queue step allows you to select a specific instance of a component and to specify the Action for the component to execute.
Queue Step Properties
After adding a /download/attachments/24976145/Queue_step.png?version=1&modificationDate=1490033775548&api=v2 Queue step from the Palette to the canvas, you can double-click the step on the canvas and specify the following step properties in the Properties tab displayed at the bottom.
Name
Description
Name
Unique name for the step.
Description
Description of the step (optional).
Enabled
Select this option to enable the step for execution. If it is not enabled, then the step on the canvas is greyed-out.
This option is useful when troubleshooting a process, especially when there are numerous steps in the process.
Error Handling
Select either of the following:
Abort Process - All the steps after the current step are skipped or the entire process is stopped (depending on how you have set the error logging options) if this step stops due to an error.
Ignore Error - The remaining steps in the current process are executed even if this step stops due to an error. This is selected by default.
Select Queue
Select a Queue component instance. These instances are displayed when you add message components in the Configuration tab. For more information, see Adding Message Components in Configuring Processes.
Action
Select the action for the queue instance.
Parameter
Parameters for the selected action. Specify values for the parameters.
Property
Properties for the selected action. Specify values for the properties.
Note:  The Actions, Parameters, and Properties are specific to the component type. For information about this, based on the component instance selected in the Select Queue drop-down list, click the component type in the Queue Step Components section below.
Queue Step Components
AWS S3 Queue
Azure Storage Queue
POP3 Email Queue
IMAP Email Queue
File Folder Queue
FTP Queue
Google Cloud Storage Queue
Google Drive Queue
JMS Queue
Kafka Micro-batch Queue
Kafka Writer Queue
Websphere MQ Queue
IBM MQ Queue
MSMQ Queue
AWS S3 Queue
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. You can use it to store and protect any amount of data for websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. For more information on Amazon's Simple Storage Service, see https://aws.amazon.com/s3/.
DataConnect provides AWS S3 Queue component that can be used to connect to AWS S3. This component supports the following features:
Specifying the AWS S3 bucket name that stores the objects being written and read.
Specifying the Region that houses the bucket.
Selecting common AWS authentication methods
Uploading files or DJMessage content to S3 buckets
Downloading objects from S3 buckets to DJMessages or files
Removing objects from the S3 bucket when they are no longer needed in the process
Creating a temporary bucket to be used in the current process. When the process calls the disconnect method of the component, the temporary bucket is deleted.
You can use the S3 to Avalanche template for loading data to Avalanche. It leverages the S3 component. For more information about this template, see Integration Manager documentation.
AWS S3 Queue Properties
 
Property
Default Value
Description
AWS Region
 
Region that hosts the specific S3 bucket. The component will be configured to use this bucket. For more information about AWS regions, availability zones, and other related topics, see
S3 Bucket Type
Use Existing Bucket
Select one of the following:
Use Existing Bucket: Select this option to use an existing S3 bucket. The following additional options are displayed:
S3 Bucket Name: Name of the AWS S3 bucket. The value must match the name of an existing AWS S3 bucket for which you have the appropriate privileges. This bucket will be used to upload and retrieve stored objects. For more information about AWS S3 buckets, see Working with Amazon S3 Buckets.
Delete Objects When Finished: If the value is set to True, then the component retains a list of all files that it uploaded during its life cycle in the process. When the component's disconnect method is called by the process, the uploaded files are deleted. The default value is False.
Create Temp Bucket: Select this option to create a temporary S3 bucket. It is used while the component instance remains active in the process. The following additional options are displayed:
Temp S3 Bucket Prefix: Specify a custom prefix for the temporary bucket name that will be created by the component. The prefix must be consistent with AWS S3's bucket naming guidelines as defined in the Rules for Bucket Naming section in Bucket Restrictions and Limitations.
If you do not provide this name, the component uses the default prefix for all unique temporary bucket names that it creates. The default prefix is actian-dc-tempbucket-.
Authentication Type
Default Credential Provider Chain
Means by which identity is provided to AWS
Default Credential Provider Chain: Uses the "default" credentials provider chain that is provided in the AWS SDK. For information about the standard mechanism that AWS client-side automation tools use to obtain user credentials, see New and Standardized Way to Manage Credentials in the AWS SDKs.
Access Keys: Access keys are long-term credentials for an IAM user or the AWS account root user. Access keys are created in pairs that consist of the following (displayed as additional options):
Access Key ID: Provide the access key ID.
Secret Access Key: Provide the secret access key.
Note:  If the Access Keys authentication type is selected, the component will not search for additional means by which the credentials are provided.
Supported Actions
 
Action
Description
Connect
Creates an authenticated S3 client that will be used to perform all subsequent operations.
PutMessage
Provide a DJMessage or file name. The component will write this file or message to the S3 bucket specified in the S3 Queue properties. The component uses the authenticated S3 client to upload data to an S3 bucket and saves it using a provided "key". Since the data is represented in S3 as an object, the key is the "object name" or "file name".
GetMessage
Specify a DJMessage or a referenced file through which content can be downloaded. Uses the authenticated S3 client to download data from an S3 bucket. The data in the bucket is referenced by a key.
Disconnect
Removes the authenticated S3 client.
If the Create Temp Bucket option is enabled, then the temporary bucket (along with all uploaded content) is removed.
If Delete Objects When Finished option is selected, all uploaded content will be deleted.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Specify a DJMessage variable or file through which content can be downloaded.
PutMessage
Message
Specify a DJMessage variable or file through which content can be uploaded.
Supported Action Properties
The following action properties are applicable.
 
Action
Property
Description
GetMessage
Object Key
Object key that must be downloaded from the S3 bucket. The key is unique within the S3 bucket and may include separator characters, such as slash (/) to emulate folders. This option is mandatory.
Local File Name
Local file name, where the component writes the data retrieved from the S3 bucket. If you do not specify this file name, then the data read from the bucket is inserted into the DJMessage parameter.
Bytes Limit
Limit the number of bytes that are downloaded from the object in the S3 bucket. For example, the value 10000 means that only the first 10000 bytes are downloaded from the source object.
Delete Object After Retrieval
If this value is set to True, then the object is deleted from the S3 bucket after it is successfully retrieved. The object will not be deleted if an error occurs during download. The default value is False.
Note:  This option is displayed only if you have selected Use Existing Bucket for the component properties and have set Delete Object When Finished to False.
PutMessage
ObjectKey
Key that is assigned to the object after it is uploaded to the S3 bucket. The key is unique within the S3 bucket and may include separator characters, such as slash (/) to emulate folders. If this value is not provided, then the component expects to find an override property called ObjectKey within the DJMessage.
Local File Name
Local file name from which the component reads the data that it uploads to the S3 bucket. The value of this option (whether it is blank or populated) is overridden by the LocalFileName DJMessage property. If neither value is set, then the data that will be uploaded is read from the DJMessage body.
Override Properties
You can override the values provided for the object key and local file name in the PutMessage action with DJMessage properties:
ObjectKey DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Object Key option.
LocalFileName DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Local File Name option. If both are missing, then the uploaded data is read from the DJMessage body.
Errors
The following table describes error codes generated for AWS S3 Queue component.
 
Action
Code
Description
Connect
ERR_OK
Connect completed successfully with no errors
ERR_OPENERR
Connect failed. Typically associated with LT_ERROR log level. Associated error message describes the specific issue.
Disconnect
ERR_OK
Disconnect completed successfully
ERR_CLOSERR
Associated with warning-level log message. Disconnect is called again after it is already disconnected.
ERR_DELETERR
Attempt to delete non-temporary bucket. Associated with warning-level log message.
Attempt to delete one or more objects resulted in error. Associated with warning-level log message.
GetMessage
ERR_OK
Object retrieved and saved (to file or DJMessage) with no errors
ERR_BADOPTIONVALUE
Object key not set. Associated with error-level log message.
Local file referenced file already exists. Associated with error-level log message.
ERR_READERR
Error copying response content to the provided DJMessage. Associated with error-level log message.
Wide variety of errors produced by Amazon AWS S3 API will produce this error. The error code will include a message appropriate to the error. Typically associated with error-level log entry.
ERR_DELETERR
Delete after download option is set, but the component failed to delete the object from the S3 bucket. Associated with warning-level log entry
PutMessage
ERR_OK
Object uploaded to S3 bucket with no errors
ERR_BADOPTIONVALUE
Target object key property must be set. Associated with error-level log entry.
ERR_WRITERR
Error copying data from DJMessage. Associated with error-level log entry.
AWS-generated error occurred while sending or writing data to the remote S3 bucket. A number of messages could be produced. All associated with error-level log entry.
Azure Storage Queue
‎Azure Data Lake Storage (ADLS) Gen2 is a set of capabilities built on Azure Blob storage. Data Lake Storage Gen2 is the result of converging the capabilities of the two existing storage services—Azure Blob storage and Azure Data Lake Storage Gen1. Features from Azure Data Lake Storage Gen1, such as file system semantics, directory, and file level security and scale are combined with low-cost, tiered storage, high availability or disaster recovery capabilities from Azure Blob storage. For more information on Azure Data Lake Storage, see Introduction to Azure Data Lake Storage Gen2.
DataConnect provides the Azure Storage Queue component that can be used to connect to ADLS Gen2. This component supports the following features:
Storage account’s name and shared access key (for authentication)
Container’s specification within the storage account that hosts the data
Selecting Data Lake Storage Service (Blob Storage Service or File Storage Service)
Uploading files or DJMessage content
Downloading objects to DJMessages or files
You can use the ABFS to Avalanche Azure template for loading data to Avalanche. It leverages the Azure Storage Queue component. For more information about this template, see Integration Manager documentation.
Azure Storage Queue Properties
 
Property
Default Value
Description
Azure Storage Service Type
 
Select any of the following options for interacting with the selected container:
Blob Storage Service: Component uses Microsoft Azure's Blob-specific APIs to interact with the selected storage account and container. For more information, see Introduction to Azure Blob Storage.
ADLS File System Service: Component uses Microsoft Azure's Data Lake Storage Gen2 hierarchical namespace-specific APIs to interact with the selected storage account and container. For more information, see Azure Data Lake Storage Gen2 hierarchical namespace.
Storage Account Name
 
Name of a pre-existing storage account that is created within a Microsoft Azure account. For more information, see Storage account overview.
Container Name
 
Name of a pre-existing container within the selected storage account.
Delete Objects When Finished
False
If set to True, all files that are uploaded by the component during the process execution are deleted when the disconnect action is called.
Note:  The process engine automatically calls the disconnect action when the process is completed.
Azure Storage Authentication Type
 
Select the authentication scheme that the component will use:
Storage Shared Access Keys: Azure generates two 512-bit storage account access keys when you create a storage account. These keys can be used to authorize access to data in your storage account through Shared Key authorization. The following additional option is displayed:
Access Key: Provide one of the two 512-bit storage account access keys that was automatically generated by Microsoft Azure when the storage account was created.
Azure Active Directory Service Principal With Client Secret: Customer applications, which embed the Azure Storage Component, can be integrated with the Microsoft identity platform by registering it with an Azure Active Directory tenant. After this, a client secret can be generated from the registered application and it can be used for authentication. For more information, see Microsoft’s How to page. Actian Avalanche COPY VWLOAD directly supports this type of registration and authentication. The following additional options are displayed:
Application (Client) ID: Identity assigned to the application after registration. It is the service principal assigned to the application.
Directory (tenant) ID: Provide the tenant ID. A tenant is a representation of an organization in Azure Active Directory. Applications are registered relative to a tenant within Azure Active Directory.
Client Secret: Specify the client secret value. After an application or service principal is created, you can upload a certificate or create a client secret to be used for authentication.
Supported Actions
 
Action
Description
Connect
Creates an authenticated Azure storage client that is used to perform all subsequent operations.
PutMessage
Uses the authenticated Azure Storage client for uploading data to an Azure Storage container and saves it using the provided blob (or file) name. The content that must be uploaded can be provided to the PutMessage action through DJMessage or a referenced file.
GetMessage
Uses the authenticated Azure storage client to download data from the specified container. The downloaded content can be written to a file or injected into a DJMessage object that is provided to the GetMessage action.
Disconnect
Removes the authenticated Azure storage client. If you have chosen the option to delete uploaded objects, then all the uploaded files are deleted.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Specify a DJMessage variable or file through which content can be downloaded.
PutMessage
Message
Specify a DJMessage variable or file through which content can be uploaded.
Supported Action Properties
The following action properties are applicable.
 
Action
Property
Description
GetMessage
Object Name
Object name that you intend to download from the configured Azure Storage container.
If the component is interacting with the container using the blob service, then the name represents the blob name.
If the component is interacting with the container through the file system service, then the name represents the full path to the file including directory elements separated by the "/" character.
Note:  This option is required within the GetMessage action.
Local File Name
Local file name, where the component writes the data retrieved from the Azure Storage container. If you do not specify this file name, then the data read from the container is inserted into the DJMessage parameter.
Bytes Limit
Limits the number of bytes downloaded from the selected blob or file to the specified value. If you want to download the entire blob or file, do not specify this value.
Delete Object After Retrieval
If set to True, the specified blob or file is deleted from the Azure Storage container after the download is successful. The default value is False.
Note:  This option is displayed only if the Delete Objects When Finished component option's value is False.
PutMessage
Object Name
Name of the blob or file that is assigned to the object after it is uploaded to the Azure Storage container.
If the component is configured to use the File System Service for interacting with the selected Azure Storage container, then the name can include the full path (including directories) to the file. If the path includes missing directories in the container, then the component creates the missing directories.
If the object name is not specified, then the component tries to find an override property called ObjectName within the DJMessage. It is an error condition if the option is left blank and the override property is missing from the DJMessage.
Local File Name
If a value is provided, then the component reads the data that it uploads from the referenced file. An error will be generated if the file cannot be resolved or is inaccessible.
When this option is left blank or populated, it is overridden by the LocalFileName DJMessage property. If neither value is set, then the data that is uploaded is read from the DJMessage body.
Override Properties
You can override the values provided for the object name and local file name in the PutMessage action with DJMessage properties:
ObjectName DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Object Name option.
LocalFileName DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Local File Name option. If both are missing, then the uploaded data is read from the DJMessage body.
Errors
The following table describes error codes generated for the Azure Storage Queue component.
Action
Code
Description
Connect
ERR_OK
Connect completed successfully without any errors.
ERR_OPENERR
Connect failed. Typically associated with LT_ERROR log level.
Disconnect
ERR_OK
Disconnect completed successfully.
ERR_CLOSERR
Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
ERR_DELETERR
If the component is configured to delete uploaded objects during connect, this error code is generated when one or more objects that are deleted resulted in error. This is associated with warning-level log message.
GetMessage
ERR_OK
Object retrieved and saved (to file or DJMessage) without any errors.
ERR_BADOPTIONVALUE
Object name is not set. This is associated with error-level log message.
ERR_READERR
Local file or referenced file already exists. This is associated with error-level log message.
ERR_DELETERR
Error in copying response content to the provided DJMessage. This is associated with error-level log message.
Too many errors generated by the Azure Storage SDK produces this error. The error code will include a message appropriate to the error. This is associated with error-level log message.
PutMessage
ERR_OK
Object uploaded to Azure Storage container without any errors.
ERR_BADOPTIONVALUE
Target object name property must be set. This is associated with error-level log entry.
ERR_WRITERR
Error copying data from DJMessage. This is associated with error-level log entry
Azure-generated error occurred when sending or writing data to the remote Azure Storage container. A number of messages can be generated and all these messages are associated with error-level log entry.
POP3 Email Queue
POP3 Email Queue supports receiving text and binary attachments. DataConnect supports POP3 Email Queue versions 1.0.2 and 1.3.0.
The following table provides the supported content types.
Content Type
Details
text/*
text
HTML
image/*
.gif
.jpg
application/*
All applicable applications, such as .xls, .zip, and .doc
The email components support encoding in all character sets when an email is sent and received. The message header and body are encoded in the character set specified by the MIME. In addition, the email components support only Base64 and other MIME encoding. Enriched encoding is not supported.
Note:  The POP3 Email Queue component implicitly trust the certificates presented by the email servers that require SSL or TLS connections, regardless of the issuer.
POP3 Email Queue Properties
 
Property
Default Value
Description
User Name
 
User name to connect to the POP3 email server.
Password
 
Password to connect to the POP3 email server.
Server Name
 
Name of the POP3 email server
Port
110
Port to be used to connect to the server.
Note:  If the default port is not used, then the component assumes a secure (SSL or TLS) connection is required.
ConnectionTimeout
90
Connection time out value in seconds.
Retry Count
3
Number of attempts the server tries to establish connection when the connection times out or fails.
From Contains
 
From email address
To Contains
 
To email address
Subject Contains
 
Subject value
Browse Mode
false
Select true or false. If true is selected, the email message remains on the server even after downloading it.
Download
UNREAD
Download the read and/or unread messages from the server:
ALL - Download both read and unread messages from the server.
READ - Download only read messages from the server.
UNREAD - Download only unread messages from the server.
Note:  This property is displayed only for POP3 Email Queue version 1.0.2 only.
Encoding String
ISO-8859-1
Encoding string required for the email from the list.
Mark Failed Messages as Seen
False
Select True or False:
True - Marks unprocessed messages as seen. This value provides a method to bypass messages that the component cannot handle by marking them as read.
False - Leaves unprocessed messages as unseen.
This property is displayed only for POP3 Email Queue version 1.3.0.
Always Create Attachment Messages
False
Select True or False:
True - Component encodes any primary content that may be an email attachment, and then includes it into a new DJMessage attachment. The original content normally has a MIME type indicating that it is binary data.
False - Component places any primary content that may be an email attachment in the main DJMessage. The original content normally has a MIME type indicating that it is text data.
Note:  If the email is not multi-part MIME- formatted, then the content is included in the main DJMessage regardless of the MIME type.This property is displayed only for POP3 Email Queue version 1.3.0.
Tip...  You can set filters or flags using the From Contains, To Contains, and Subject Contains properties. However, the filters do not work if the server does not support the same.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the database.
GetMessage
Retrieves message into a message object.
Disconnect
Closes an existing database connection and performs any clean up actions if required.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Name for the incoming message
Supported Action Properties
There are no supported action properties.
Errors
Errors are generated when email messages cannot be sent or received due to various reasons. Typical errors include the following:
Connection timeout
Unable to disconnect from the email server
Errors while sending email messages
Errors while Receiving email messages
Miscellaneous or unknown errors
All errors are written to an error log, which you can read to diagnose and troubleshoot the problem. The error messages specify whether a problem occurred while receiving or sending.
The error log also contains error codes that help you to identify and understand the cause of the error.
The following table describes error codes generated while sending or receiving email.
Code
Description
19
Unable to connect to the mail server
27
Unable to disconnect from the mail server
8
Unable to send email
4
Unable to receive email
50
Unknown error
1
End of file
12
No more messages to download
0
Messages sent or received successfully
IMAP Email Queue
The IMAP Email Queue supports receiving text and binary attachments.
The supported content types are outlined here.
Content Type
Details
text/*
text
HTML
image/*
.gif
.jpg
application/*
All applicable applications, such as .xls, .zip, and .doc
The email components support encoding in all character sets when an email is sent and received. The message header and body are encoded in the character set specified by the MIME. In addition, the email components support only Base64 and other MIME encoding. Enriched encoding is not supported.
Note:  The IMAP Email Queue component implicitly trust the certificates presented by the email servers that require SSL or TLS connections, regardless of the issuer.
Instance Properties
IMAP Email Queue has the following instance properties.
Property
Default Value
Description
User Name
 
User name to connect to the IMAP email server.
Password
 
Password to connect to the IMAP email server
Server Name
 
Name of the IMAP email server
Port
143
Port to be used to connect to the server.
Note:  If the default port is not used, then the component assumes a secure (SSL or TLS) connection is required.
ConnectionTimeout
90
Connection time out value in seconds.
Retry Count
3
Number of attempts the server tries to establish connection when the connection times out or fails.
From Contains
 
From email address
To Contains
 
To email address
Subject Contains
 
Subject value
Folder Name
 
Folder that resides in the IMAP server. If you specify a folder name, all the emails are delivered to the specified folder.
Typically, an email is downloaded to the INBOX folder unless you specify another location. Examples of specific folders are INBOX.myemail, INBOX.general, and so on.
Browse Mode
TRUE
If TRUE is selected, the email message remains on the server even after downloading it.
Download
UNREAD
Download the read and/or unread messages from the server:
ALL - Download both read and unread messages from the server.
READ - Download only read messages from the server.
UNREAD - Download only unread messages from the server.
Encoding String
ISO-8859-1
Encoding string required for the email from the list.
Mark Failed Messages as Seen
False
Select True or False:
True - Marks unprocessed messages as seen. This value provides a method to bypass messages that the component cannot handle by marking them as read.
False - Leaves unprocessed messages as unseen.
Always Create Attachment Messages
False
Select True or False:
True - Component encodes any primary content that may be an email attachment, and then includes it into a new DJMessage attachment. The original content normally has a MIME type indicating that it is binary data.
False - Component places any primary content that may be an email attachment in the main DJMessage. The original content normally has a MIME type indicating that it is text data.
Note:  If the email is not multi-part MIME- formatted, then the content is included in the main DJMessage regardless of the MIME type.
Tip...  You can set filters or flags using the From Contains, To Contains, Subject Contains, or Download properties. It is recommended to download only unread messages. However, filters or flags do not work if the server does not support the same.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the database.
Disconnect
Closes an existing database connection and performs any clean up actions if required.
GetMessage
Retrieves message into a message object.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Name for the incoming message
Supported Action Properties
There are no supported action properties.
Errors
Errors are generated when email messages cannot be sent or received due to various reasons. Typical errors include the following:
Connection timeout
Unable to disconnect from the email server
Errors while sending email messages
Errors while Receiving email messages
Miscellaneous or unknown errors
All errors are written to an error log, which you can read to diagnose and troubleshoot the problem. The error messages specify whether a problem occurred while receiving or sending. The error log also contains error codes that help you to identify and understand the cause of the error.
This table describes error codes generated while sending or receiving email.
Code
Description
19
Unable to connect to the mail server
27
Unable to disconnect from the mail server
8
Unable to send email
4
Unable to receive email
50
Unknown error
1
End of File
12
No more messages to download
0
Messages sent or received successfully
File Folder Queue
File Folder Queue allows a file folder to be treated like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a Queue step. Message objects can also be written to the folder as files using the PutMessage action. The properties of the session control the files that are read and the names of the files that are written, as well as whether to preserve the message properties.
File Folder Queue Properties
 
Property
Default Value
Description
User name
-
User name to log in to the system where the file folder is available.
Password
-
Password to log in to the system.
Server Name
-
Name of the system where the file folder resides.
Directory
 
Folder for reading and writing files.
Working Directory
-
Directory name used to store temporary files that are created when retrieving files with the GetMessage action. If this property is not specified, the connector creates a directory called DJFTFile Folder Queue in the directory specified for the Directory property. This new directory is used to store temporary files.
Deserialize
False
If a message object was written to a folder with the Serialize property set, the message properties are stored in a separate file with an .hdr extension. When reading a file from a folder using the GetMessage action, the Deserialize property indicates that the message properties must be read from the .hdr file.
Serialize
False
When a message object is written to a file with the PutMessage action, only the message body is written. The Serialize property indicates whether to write the message properties to a separate file with an .hdr extension.
Pattern
*
Pattern used to match files to be read using the GetMessage action.
GetMsgClass
dat
The file extension that is appended to the file name when reading a file with the GetMessage Action.
PutMsgClass
dat
File extension that is appended to the file name generated by the FileFormat property when writing a message object to a file with the PutMessage action.
MessageTimeOut
5
Indicates the time to wait (in seconds) to retrieve the first file from the folder when retrieving files with the GetMessage action. The files to be retrieved are determined by the Pattern property.
The MessageTimeOut property works with the SleepInterval property . The File Folder Queue tries to read the first file then sleep for the specified sleep interval. This continues based on the value in the MessageTimeOut option.
PollingTimeOut
5
Indicates the time to wait to retrieve all of the files from the folder when retrieving files with the GetMessage action. The files to be retrieved are determined by the Pattern property. The PollingTimeOut property works with the PollingTimeOutType property.
PollingTimeOutType
Minutes
Type of the polling timeout value. The options available are:
Years
Months
Days
Hours
Minutes
Seconds
SleepInterval
1000
Indicates the time to sleep when no messages are available to process. The sleep interval is in milliseconds.
BrowseMode
False
If this property is set to true, the messages read from the folder are not removed. Else, the files are deleted after processing.
FilenameFormat
MessageFile%n
When writing a message object to a file using the PutMessage action, this format is used to create the file name. The following format specifications can be used to build the format. Each specification begins with a percent sign (%). The specifiers represent properties on the message object that are created by the File Folder Iterator. Any or all of the specifications can be included in any order.
Note:  The format must generate unique names for the files.
The supported formats are:
MessageFile%n
%i TransferId
%p PieceNumber
%s SourceName
%h TargetHostName
%t TargetName
%d Transport Start Date/Time
%n Number starting with 0 (zero)
%c Current date/time stamp displaying number of seconds elapsed since midnight GMT January 1, 1970 (value always contains 10 digits)
Examples
TargetName: TargetFFQFile
PieceNumber: 5
FileFormat: %t%p
MessageClass: dat
Filename: TargetFFQFile5.dat
 
 
Best Practice — The %n and %c format specifications enable you to create unique file names for generic messages. These should be used together to ensure greatest efficiency.
If only %n is used in the file name specification, for example, MessageFile%n, the connector tries to assign the file name "MessageFile0" when a message is created. If this is unique it is used, but if it already exists in the specified directory, %n is incremented by 1 and this is checked against existing files. This continues until a unique file name is found. This means that the more messages that are created, the longer it takes for a unique name to be found.
If only %c is used, the process is limited to writing one file per second as the file name it tries to assign for the new message is not unique until the timestamp has advanced. Therefore, use a combination of %n and %c. For example, MessageFile%c%n creates as many messages each second as required and %n is reset to 0 each time the timestamp changes.
Source Encoding
ENC_OEM or 0
Character set of the source data. This indicates using the default character set for the system.
The ENC_ names can be used in expressions for setting the encoding value. On the integration engine command line, the number must be used.
Binary Indicator
False
Indicates whether the files being read must be opened in binary mode.
Supported Actions
 
Action
Description
Connect
Opens a persistent connection to the database.
Disconnect
Closes an existing database connection and performs any clean up actions if required.
GetMessage
Retrieves message into a message object.
PutMessage
Writes a message object into the queue.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
PutMessage
Message
Name for the incoming message
Supported Action Properties
There are no supported properties for the actions.
FTP Queue
The FTP Queue enables you to send and receive files using a secure or non-secure FTP connection. It allows a folder on an FTP site to be considered like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a Queue step.
Message objects can also be written to the folder as files using the PutMessage action. The properties control the files that are read and the names of the files that are written.
The primary functions of FTP Queue are:
Connect to and disconnect from a secure or unsecure FTP server
Retrieve a file from an FTP server and insert it in a message object
Put a message object into the selected file on an FTP server
This component has the following versions:
FTP Queue 1.0.0 - Provides an non-secure connection. While version 4.1.0 is recommended, 1.0.0 is supported for backward compatibility with existing processes.
FTP Queue 4.1.0 - Provides either a secure or non-secure connection using one of the three FTP types:
FTP (non-secure)
SFTP (secure FTP over SSH)
FTPS (secure FTP over SSL (certificate-based authentication))
Active and Passive Modes
FTP Queue 1.0.0 tries passive mode first. If it fails to connect, then it tries active mode.
FTP Queue 4.1.0 uses active mode for processing.
Encoding
In secure connections, FTP Queue 4.1.0 uses ISO-8859-1 to encode binary data.
FTP Queue 1.0.0 Properties
 
Property
Default Value
Description
User Name
 
User name to connect to the server.
Password
 
Password to connect to the server.
FTPSite
 
Name of the server to connect to.
Type the FTP server name or IP address, port number, and directory in the following format:
FTP_server:port/directory_name
The port number and directory name are optional. If you specify the port number, the component uses the default port number for the selected FTP server type.
Serialize
False
When a message object is written to a file with the PutMessage action, only the message body is written. The Serialize property indicates whether to write the message properties to a separate file with an .hdr extension.
Deserialize
False
If a message object was written to a folder with the Serialize property set, the message properties are stored in a separate file with an .hdr extension. When reading a file from a folder using the GetMessage action, the Deserialize property indicates that the message properties must be read from the .hdr file.
RetryCount
0
Number of times to retry if there is a failure to connect to the FTP site.
FilenameFormat
MessageFile%n
When writing a message object to a file using the PutMessage action, this format is used to create the file name. The following format specifications can be used to build the format. Each specification begins with a percent sign (%). The specifiers represent properties on the message object that are created by the File Folder Iterator. Any or all of the specifications can be included in any order.
Note:  The format must generate unique names for the files.
The supported formats are:
MessageFile%n
%i TransferId
%p PieceNumber
%s SourceName
%h TargetHostName
%t TargetName
%d Transport Start Date/Time
%n Number starting with 0 (zero)
%c Current date/time stamp displaying number of seconds elapsed since midnight GMT January 1, 1970 (value always contains 10 digits)
Examples
TargetName: TargetFFQFile
PieceNumber: 5
FileFormat: %t%p
MessageClass: dat
Filename: TargetFFQFile5.dat
 
 
Best Practice — The %n and %c format specifications enable you to create unique file names for generic messages. These should be used together to ensure greatest efficiency.
If only %n is used in the file name specification, for example, MessageFile%n, the connector tries to assign the file name "MessageFile0" when a message is created. If this is unique it is used, but if it already exists in the specified directory, %n is incremented by 1 and this is checked against existing files. This continues until a unique file name is found. This means that the more messages that are created, the longer it takes for a unique name to be found.
If only %c is used, the process is limited to writing one file per second as the file name it tries to assign for the new message is not unique until the timestamp has advanced. Therefore, use a combination of %n and %c. For example, MessageFile%c%n creates as many messages each second as required and %n is reset to 0 each time the timestamp changes.
GetMsgClass
dat
The file extension that is appended to the file name when reading a file with the GetMessage Action.
PutMsgClass
dat
File extension that is appended to the file name generated by the FileFormat property when writing a message object to a file with the PutMessage action.
Working Directory
 
Directory name used to store temporary files that are created when retrieving files with the GetMessage action.
Pattern
*
Pattern used to detect files to be read, such as File* or *95. To retrieve a specific file, type the exact file name without the extension. The extension is read from the File Extension option.
MessageTimeOut
5
Indicates the number of times to query the FTP folder to get the desired message. Default is 5. This option works with the SleepInterval option.
PollingTimeOut
5
Indicates the maximum total time to take to retrieve all files from the folder. Use this option with the Polling TimeOut Type option.
PollingTimeOutType
Minutes
Type of the PollingTimeOut value. The available options are:
Years
Months
Days
Hours
Minutes
Seconds
SleepInterval
1000
Time interval, in milliseconds, to wait between queries if there are no messages in the queue to process. This option works with the Message TimeOut option.
BrowseMode
False
Indicates whether the file on the FTP server from which the message is read is deleted. If Browse Mode is true, the file is not deleted.
SourceEncoding
0
Character set of the source data. This indicates using the default character set for the system.
The ENC_ names can be used in expressions for setting the encoding value. On the integration engine command line, the number must be used.
Binary Indicator
False
Indicates whether the files being read should be opened in binary mode.
Supported Actions for FTP Queue 1.0.0
 
Action
Description
Connect
Establishes a connection to an FTP server, which is useful when you are testing your configuration.
Disconnect
Breaks the connection with an FTP server, which is useful when you need to disconnect before a process is complete.
GetMessage
Connects to an FTP server, retrieves the contents of a file into a message object, then disconnects. The GetMessage action sets the SourceName message property to the name of the file from which it retrieved the message.
PutMessage
Connects to an FTP server, writes a message object to the specified file, then disconnects.
Supported Action Parameters for FTP Queue 1.0.0
 
Action
Parameter
Description
GetMessage, PutMessage
Message
Name of the message.
FTP Queue 4.1 Properties
 
Property
Default Value
Description
User Name
 
User name to connect to the server.
Password
 
Password to connect to the server.
Server Name
localhost
Name of the server to connect to.
Type the FTP server name or IP address, port number, and directory in the following format:
FTP_server:port/directory_name
The port number and directory name are optional. If you do not specify the port number, the component uses the default port number for the selected FTP server type.
FTP Type
FTP
Specify the type of FTP:
FTP
SFTP
FTPS
FTP Connection Mode
 
Connection mode is either Active (the default) or
Passive.
Note:  This is not applicable for SFTP.
Message Payload Type
 
Select File Contents (the default) or File Location:
File Contents: If the actions are:
If GetMessage is the , contents of the retrieved file are stored in the DJMessage.
If PutMessage, contents of the DJMessage are sent to the remote FTP site and stored as a file.
File Location: If actions are:
GetMessage - File retrieved from the remote FTP site is stored directly as a file.
PutMessage - Local file contents are sent to the remote FTP location.
Encoding Type
ISO8859-1
Select ISO8859-1 or UTF-8.
Note:  This is not applicable for SFTP.
Binary Indicator
 
False
Indicates whether the files being read must be opened in binary mode.
Supported Actions for FTP Queue 4.1.0
 
Action
Description
Connect
Establishes a connection to an FTP server, which is useful when you are testing your configuration.
Disconnect
Breaks the connection with an FTP server, which is useful when you need to disconnect before a process is complete.
GetMessage
Connects to an FTP server, retrieves the contents of a file into a message object, then disconnects. The GetMessage action sets the SourceName message property to the name of the file from which it retrieved the message.
PutMessage
Connects to an FTP server, writes a message object to the specified file, then disconnects.
Supported Action Parameters for FTP Queue 4.1.0
 
Action
Parameter
Description
GetMessage, PutMessage
Message
Name of the message.
Supported Action Properties for FTP Queue 4.1.0
 
Action
Property
Default Value
Description
GetMessage
Target Directory
 
Target directory for storing the retrieved file on the local machine.
 
Pattern
*
Pattern used to detect files to be read. To retrieve a specific file, type the exact file name with or without the extension. The extension is read from the File Extension option. The pattern is case sensitive. The Pattern properties supports the use of regular expressions. Default is *.
For example, File*, *95, File?, ?95('*' matches any character sequence and '?' matches any character).
 
File Extension
 
File extension to append to the file name when retrieving a file and inserting into a message object. Use values such as txt and asc. Do not enter the dot (.), the extension string.
 
Browse Mode
False
Indicates whether the file on the FTP server from which the message is read is deleted. If Browse Mode is true, the file is not deleted.
Message Timeout
5
Indicates the number of times to query the FTP folder to get the desired message. Default is 5. This option works with the SleepInterval option.
Sleep Interval
1000
Time interval, in milliseconds, to wait between queries if there are no messages in the queue to process. This option works with the Message TimeOut option.
Polling TimeOut
5
Indicates the maximum total time to take to retrieve all files from the folder. Use this option with the Polling TimeOut Type option.
Polling TimeOut Type
Minutes
Type of the PollingTimeOut value. The available options are:
Years
Months
Days
Hours
Minutes
Seconds
PutMessage
Remote Directory
 
Remote location for storing the file on the FTP Server.
 
Filename Format
 
Format for the file name written to the message object.
Various format specifications are supported. Each specification begins with a %. Any or all specifications can be placed consecutive to each other. However, you may not place text after a format specification. For example, MessageFile%n%s is valid, but MessageFile%nTest is invalid.
Make sure that the format generates a unique name for each file. The default format is MessageFile%n, which produces the file name MessageFile0.
The supported format specifications are:
%n - Inserts a number, starting with 0 (zero).
%c - Inserts current date/time stamp, displaying number of seconds elapsed since midnight GMT January 1, 1970. The value has 10 digits.
%s - Inserts the source file name, which can be specified as a message property.
%t - Inserts the target file name, which can be specified as a message property.
 
File Extension
 
File extension to append to the file name when writing a message object to a file. Use values such as txt and asc. Do not enter the period, only the extension string.
 
Use Temporary Files
 
 
 
Local File
 
Local path to the location where the file exists.
Errors
The following table provides possible FTP Queue error codes.
Error Code
Description
4
Error occurs while trying to get a message from the FTP server.
8
Error occurs while trying to add a message on the FTP server.
12
Error occurs because the GetMessage or PutMessage actions cannot find the specified file.
19
Error occurs while trying to connect to the FTP server.
27
Error occurs while trying to disconnect from the FTP server.
The process log may also contain messages related to FTP Queue.
Error Code
Description
28047
An error occurred executing a queue step.
This run-time error can result from a change in the server certificate after you deploy your integration application. View the process log for the following message:
"Error connecting to FTP Server: Could not validate server certificate. Please make sure you run TestConnect and save the process."
To resolve this error, do one of the following.
Obtain a new certificate and add it manually to the client trust store file.
In the process file, use the Test Connect option to connect to the FTP server so that it prompts you to accept the new certificate.
Limitations
The file transfer is bound by the amount of physical memory and heap size.
If you are transferring binary files, you must encode the data in the message object in Base64. When you receive files with GetMessage in binary mode, the files will be binary, and will not be encoded in Base64.
Google Cloud Storage Queue
The Google Cloud Storage Queue component facilitates integration with Google Cloud Platform (GCP) Storage services. The key component features are:
Specify the name of the GCP Storage bucket which will store the objects being written and read. Optionally, create a bucket if missing.
Note:  A missing bucket will be created only if the component uploads content to the bucket. A new bucket will not be created if the component is downloading content.
Specify the storage location (region) of the bucket.
Select authentication methods:
Supports Service account keys contained within a file, which is provided by the GCP Storage account owner
Supports the following schemes for finding the key file:
Explicitly specify the file path
Implicitly discover the file through the GOOGLE_APPLICATION_CREDENTIALS environment variable
Upload files or DJMessage content to GCP Storage buckets
Download "objects" from GCP Storage buckets to DJMessages or files
For GCP overview, see https://cloud.google.com/. For information about GCP Storage services, see https://cloud.google.com/storage.
Google Cloud Storage Queue Property Descriptions
 
Property
Default Value
Description
Bucket Location
 
Location for storing your object data when you create a bucket. The location types are:
Region: Specific geographic place, such as London.
Dual-region: Specific pair of regions, such as Finland and the Netherlands.
Multi-region is a large geographic area, such as the United States, that contains two or more geographic places.
Objects stored in a multi-region or dual-region are geo-redundant.
Cloud Storage stores object data in the selected location in accordance with the Service Specific Terms.
For more information about locations, see https://cloud.google.com/storage/docs/locations.
Bucket Name
 
GCP Storage bucket that will be used to upload and retrieve stored objects.
The value must match the name of an existing GCP Storage bucket for which the service account has appropriate privileges.
If the name does not match an existing bucket, then an error occurs unless the Create missing bucket option is set to true. For more information about GCP Storage buckets, see https://cloud.google.com/storage/docs/key-terms.
Create Missing Bucket
False
If set to True, the component creates the specified bucket (if it is missing) when attempting to upload (PutMessage action) content to the specified bucket. This setting has no effect if the component is only downloading content. The default value is False.
Bucket Storage Class
Standard Storage
When creating a missing bucket, the component must know the GCP Storage class to be used. The options are:
Standard Storage
Nearline Storage
Coldline Storage
Archive Storage
For more information about storage classes, see https://cloud.google.com/storage/docs/storage-classes.
Authentication Type
Use path in environment variable (GOOGLE_APPLICATION_CREDENTIALS)
Select any of the following authentication type:
Use path in environment variable (GOOGLE_APPLICATION_CREDENTIALS): Instructs the component to configure the GCP client SDK in a way, which instructs it to implicitly find the service account key file using the GOOGLE_APPLICATION_CREDENTIALS environment variable. If the environment variable is set, then its value is expected to be the path to the file. If the environment variable is not set, then an error occurs.
Provide path to service account key file: The component will display the Service Account Key File option, which allows you to specify the complete path of the service account key file.
Supported Actions
 
Action
Description
Connect
Creates an authenticated GCP Storage client, which will be used to perform all subsequent operations. The client is stateless, so it is not actually "connected."
Disconnect
Disconnects from the authenticated client.
GetMessage
Uses the authenticated client to download data from a GCP Storage bucket. The data in the bucket is referenced using an object name. The downloaded content can be written to a file or injected into a DJMessage object which is provided to the GetMessage action.
PutMessage
Uses the authenticated client to upload data to a GCP Storage bucket, saving it using the provided object name. This object name is equivalent to an S3 "key" or local "file name". The content to be uploaded can be provided to the PutMessage action via DJMessage or a referenced file.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Name of a DJMessage to which the downloaded file will be written if a local file was not specified.
PutMessage
Message
Name of a DJMessage from which data will be uploaded if a local file was not specified.
Supported Action Properties
 
Action
Property
Description
GetMessage
Object Name
Name of the object to retrieve from GCP Storage bucket.
The name is unique within the bucket and may include separator characters, such as slash ("/") to emulate folders. This option is required.
Local File Name
Click Browse and specify a local file, to which the component will write the data retrieved from the bucket.
Bytes Limit
Number of bytes to download from the object in the bucket. This is useful for sampling. Leave this blank if you want to download the entire object.
PutMessage
Object Name
Name of the GCP Storage object to be written.
The name is unique within the bucket and may include separator characters, such as slash ("/" ) for emulating folders.
If left blank, then the component expects to find an override property called "ObjectName" within the DJMessage. It is an error condition if the option is left blank and the override property is missing from the DJMessage.
Local File Name
Click Browse and specify the local file from where the component will read the data that it uploads to the bucket.
The value of this option (whether it is blank or populated) is overridden by the LocalFile DJMessage property. If neither value is set, then the data that will be uploaded is read from the DJMessage body.
MIME Type
MIME type that will be assigned to the object after it is uploaded to the bucket.
This is a useful option if the content is being uploaded from the body of a DJMessage. It can be overridden through the DJMessage property MimeType.
Note:  MIME Type will be automatically determined when uploading from a local file.
A limited list of MIME types is available for selection. However, you can specify any MIME type supported by GCP Storage. The available MIME Types are:
text/xml
text/html
text/plain
application/json
application/xml
application/octet-stream
Replace Existing Object
If set to True, the existing object is replaced on GCP Storage. The default value is False.
Override Queue Properties
You can override the values provided for the object name and local file name in the PutMessage action using the following DJMessage properties:
ObjectName: This header property can be set in the DJMessage argument for the PutMessage action. If provided, it overrides any value that is set in the Object Name option.
LocalFile: If the DJMessage argument for the PutMessage action has a header property with this name, then its value will override any value that is set in the Local File Name option. If both are missing, then the upload data will be read from the DJMessage body.
MimeType: This header property can be set in the DJMessage argument for the PutMessage action. If provided, it overrides the value set in the MIME Type option.
ReplaceFile: If the DJMessage argument for the PutMessage action has a header property with this name, then its value will override any value that is set in the Replace Existing Object option. If neither value is set, then the default value is False.
Errors
The following table describes the error codes generated for the Google Cloud Storage Queue component.
 
Action
Code
Description
Connect
ERR_OK
Connect completed successfully without any errors
ERR_OPENERR
Connect failed. Typically associated with LT_ERROR log level.
Disconnect
ERR_OK
Disconnect completed successfully.
 
ERR_CLOSERR
Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
GetMessage
ERR_OK
File retrieved and saved (to file or DJMessage) without any errors.
 
ERR_BADOPTIONVALUE
GCP storage file name is not set. This is associated with error-level log message.
 
ERR_READERR
Local file or referenced file already exists. This is associated with error-level log message.
PutMessage
ERR_OK
File uploaded to the GCP Storage bucket without any errors.
ERR_BADOPTIONVALUE
Object name property must be set. This is associated with error-level log entry
ERR_WRITERR
Error copying data from DJMessage.
Google-generated error occurred when sending or writing data to the remote GCP Storage account. A number of messages can be generated and all these messages are associated with error-level log entry.
Google Drive Queue
Google Drive is a personal cloud storage service from Google that allows you to store and synchronize digital content across computers, laptops and mobile devices, including Android-powered tablets and smart phone devices. Once permissions are granted by the owner of a Google Drive account, the component facilitates uploading or downloading files to the account.
Google Drive Queue Property Descriptions
 
Property
Default Value
Description
Token Folder
 
Local directory that will be used to store an OAuth token after the owner of the Google Drive account has granted the requested permissions to the component.
Click Browse and select a folder. You can also right-click and use the Paste Macro option to specify the token folder.
After specifying this folder, you must test the connection and obtain a valid token for connecting to Google Drive. For the steps, see Obtaining Valid Token for Connecting to Google Drive.
The connection is successful if a valid token file is created in the Token Folder.
Delete Uploaded Files When Finished
False
Automatically deletes all the uploaded files after the process completes. The options are:
True
False
Obtaining Valid Token for Connecting to Google Drive
The Test Connection option helps you to obtain a valid token for connecting to Google drive.
To test the connection and obtain a valid token, perform the following:
1. In the Google Drive queue component properties, in the Token Folder, specify a path. This is the folder where the granted OAuth token file will be created. For example, C:\Temp.
2. Click Test Connection.
The Google Sign in page opens with the following message:
Choose an account to continue to Actian DataConnect
3. Select your Google or Gmail account that you want to connect with Actian DataConnect.
The Grant Actian DataConnect permission dialog is displayed asking for permission to view, edit, create, and delete all of your Google Drive files.
4. Click Allow.
The following message is displayed:
View and manage metadata of files in your Google Drive files
5. Click Allow.
The following message is displayed:
Confirm your choices
6. Click Allow.
The following message is displayed:
Received verification code
7. Click OK and then click Save.
The Google Drive Queue component is ready to use.
8. Open file explorer and go to the folder that you had specified as the Token Folder. You will see a new file called StoredCredential is created. This is the granted token file for your Google Drive account.
Supported Actions
 
Action
Description
Connect
Establishes a connection to the authorized Google Drive account.
Disconnect
Disconnects from the authorized Google Drive account.
GetMessage
Downloads a file from the authorized Google Drive account to either a DJMessage or a local file.
PutMessage
Uploads a file to the authorized Google Drive account from a DJMessage or local file.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
Name of a DJMessage to which the file downloaded file will be written if a local file was not specified.
PutMessage
Message
Name of a DJMessage from which data will be uploaded if a local file was not specified.
Supported Action Properties
 
Action
Property
Description
GetMessage
Google Drive File
Name of the file to retrieve from Google Drive.
You can specify the full path to the file if it is a folder within one or more directories. For example, if the accounts.txt file is within the data directory, which is in a another directory called company, then the path to the file is company/data/accounts.txt.
Local File
Click Browse and specify a local file, to which the downloaded Google Drive file will be written. Leave blank to write the downloaded content to this step's DJMessage.
Bytes Limit
Number of bytes to download from the selected Google drive file. This is useful for sampling. Leave this blank if you want to download the entire object.
Delete File After Retrieval
If set to True, then after successfully downloading the file, delete the file from Google drive. The default value is False.
PutMessage
Google Drive File
Name of the Google Drive file to be written. This can be overridden through the DJMessage property DriveFile.
Local File
Click Browse and specify the local file that will be copied to the specified Google Drive file. Leave it blank if you want to upload the content of the DJMessage body.
MIME Type
Specify the MIME type of the content that will be uploaded to Google Drive. This is a useful option if the content is being uploaded from the body of a DJMessage. It can be overridden through the DJMessage property MimeType.
Note:  MIME Type will automatically be determined when uploading from a local file.
A limited list of MIME types is available for selection. However, you can specify any MIME type supported by Google Drive. The available MIME Types are:
text/xml
text/html
text/plain
application/json
application/xml
application/octet-stream
Replace Existing File
If set to True, the existing file is replaced on Google Drive. The default value is False.
Override Queue Properties
You can override the values provided for the object name and local file name in the PutMessage action using the following DJMessage properties:
DriveFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Google Drive File option.
LocalFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Local File option. If both are missing, then the uploaded data is read from the DJMessage body.
MimeType: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the MIME Type option.
ReplaceFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Replace Existing File option.
Errors
The following table describes the error codes generated for the Google Drive Queue component.
 
Action
Code
Description
Connect
ERR_OK
Connect completed successfully without any errors
ERR_OPENERR
Connect failed. Typically associated with LT_ERROR log level.
Disconnect
ERR_OK
Disconnect completed successfully.
ERR_CLOSERR
Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
ERR_DELETERR
If the component is configured to delete uploaded files from Google Drive during connect, then this error code is generated when one or more files that are deleted resulted in error. This is associated with warning-level log message.
GetMessage
ERR_OK
File retrieved and saved (to file or DJMessage) without any errors.
ERR_BADOPTIONVALUE
Google Drive file name is not set. This is associated with error-level log message.
ERR_READERR
Local file or referenced file already exists. This is associated with error-level log message.
ERR_DELETERR
Error in copying response content to the provided DJMessage. This is associated with error-level log message.
Too many errors generated by the Google Drive SDK produces this error. The error code will include a message appropriate to the error.
PutMessage
ERR_OK
File uploaded to the Google Drive account without any errors.
ERR_BADOPTIONVALUE
Google Drive file name property must be set. This is associated with error-level log entry
ERR_WRITERR
Error copying data from DJMessage.
Google-generated error occurred when sending or writing data to the remote Google Drive account. A number of messages can be generated and all these messages are associated with error-level log entry.
JMS Queue
JMS queues are components based on the Java Message Service. The components allows you to connect to JMS-compliant queues. The available JMS queue component is JMS Queue 2.0.0. If you want to use client jars with this component, you must use JMS 2.0.0. The JMS Queue 2.0.0 is a Java component that directly implements the JMS API.
Note:  The JMS 1.0.0 component is deprecated.

Make sure to set the Java ClassPath option in the cosmos.ini file when using JMS Queue.
Use the following information to migrate from JMS 1.0.0 to JMS 2.0.0.
 
Component Version
Display Name
Actual Option Name Used in Process
Option Type
1.0.0 and 2.0.0 (component configuration)
 
User Name
com.pervasive.di.queue.connection.userid
text
Password
com.pervasive.di.queue.connection.password
password
java.naming.factory.initial
contextfactory
text
javax.jms.connectionfactory
connectionfactory
text
java.naming.provider.url
providerurl
text
PropertiesFile
propertiesfile
file
OutputMessageType
msgtype
enum
MessageSelector
msgselector
text
Timeout
timeout
number
1.0.0 (component configuration)
 
Server Name
com.pervasive.di.queue.connection.server
text
JMSCorrelationID
jmscorrelationid
text
JMSReplyTo
jmsreplyto
text
JMSType
jmstype
text
2.0.0
(component configuration)
JarLocation
jarlocation
text
2.0.0
(configured with PutMessage action in step)
JMSCorrelationID
jmscorrelationid
text
JMSReplyTo
jmsreplyto
text
JMSType
jmstype
text
Note:  Possible values for msgtype are-javax.jms.TextMessage, javax.jms.BygtesMessage, javax.jms.StreamMessage
JarLocation is a single directory that contains all client jars.
JMS Queue Property Descriptions
 
Property
Default Value
Description
User Name
 
User name used to connect with host.
Password
 
Password used to connect with host.
Messaging Styles
PTP
Select one:
PTP (Point-to-Point)
Publish-Subscribe
java.naming.factory.initial
 
Fully qualified class name of the initial context factory for the JNDI service provider.
javax.jms.connectionfactory
 
JNDI lookup name of the JMS Queue connection factory.
java.naming.provider.url
 
URL for the JNDI provider-specific configuration information.
PropertiesFile
 
Specifies the name of a properties file that contains the JMS Queue configuration information. You can either specify the three properties above separately, or place them in a properties file.
OutputMessageType
javax.jms.TextMessage
Type of JMS Queue message to create when writing messages.
javax.jms.TextMessage (default)
javax.jms.BytesMessage
javax.jms.StreamMessage
MessageSelector
 
JMS Queue message selector to use for reading messages.
JarLocation
 
Location of the .jar file.
Timeout
0
Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.
Supported Actions
 
Action
Description
Connect
Establishes a connection to the server.
Disconnect
Disconnects from the server.
GetMessage
Retrieves message into a message object.
PutMessage
Writes a message object into the queue.
BeginTransaction
Starts a JMS Queue transaction.
CommitTransaction
Commits a JMS Queue transaction.
RollbackTransaction
Rolls back a JMS Queue transaction.
Supported Action Parameters
 
Action
Parameter
Description
Connect
Queue
Name of the queue used to connect to.
Supported Action Properties
There are no action properties displayed for JMS Queue component.
Note:  For more information on JMS, see the JMS API documentation.
Kafka Micro-batch Queue
Kafka is a distributed, partitioned, and replicated commit log service. Kafka maintains feeds of messages in categories called topics. The processes that publish messages to a Kafka topic are called producers. The processes that subscribe to topics and process the feed of published messages are called consumers. Kafka is run as a cluster comprised of one or more servers. Each server is called a broker. The producers send messages over the network to the Kafka cluster, which in turn sends them up to consumers.
The Kafka writer provides the ability to send data to Kafka from batch or real-time invoked processes without the expectation of handling big data volumes. Kafka provides a client Producer API for the user to write data to Kafka. The Producer API version used is 2.2.1.
The following features are supported:
Reading messages from Kafka topic: The GetMessage action of the component and the associated action properties such as Group ID, Topic, and so on is used to read the messages from the Kafka topic. The messages read from the topic will be inserted into the body of the DJMessage object and the JSON file (if specified in File Name property). The JSON format or structure is as shown.
where:
recordCount: Represents number of messages/records fetched from the Kafka topic
topic: Name of the topic from where the messages are fetched.
records: Records array contains message objects. Each message object consists of key, offset, partition, timestamp, value of the message fetched from Kafka topic.
Writing messages to Kafka topic: The PutMessage action of the component is used to write the message to a topic in a Kafka queue. The messages to be written must be attached to the body of the DJMessage or must be provided in the JSON file (specified in File Name property). If the JSON file is specified, then this file will be used to write the messages to the Kafka topic. Otherwise, the body or contents of the DJMessage object will be used.
JSON File or contents (body) of the DJMessage object must be in the same format as shown in the preceding image. The only property required is Value, which contains the message to be written to Kafka Topic. The other propertied are not required. The structure is as shown.
Kafka Micro-batch Queue Properties
 
Property
Default Value
Description
Bootstrap Servers
 
List of host or port pairs to use for establishing the initial connection to the Kafka cluster.
The format is host1:port1,host2:port2.
Supported Actions
 
Action
Description
Connect
Establishes a connection to the server.
Disconnect
Disconnects from the server.
GetMessage
Reads a message object from the queue.
PutMessage
Writes a message object into the queue.
Supported Action Parameters
 
Action
Parameter
Description
GetMessage
Message
The message that must be read.
PutMessage
Message
The message that must be written.
Supported Action Properties
Action
Property
Description
GetMessage
Group ID
Consumer group ID to which the consumer belongs.
Topic
Kafka message topic name from where the messages must be read.
Partition Key
Partition key in the Topic from where the messages must be read. If you do not know this, leave it as blank.
Poll Duration
Maximum time to block in milliseconds. This is the time required to reach the specified Kafka Server. It blocks the process execution until the fetch request reaches the Kafka Server and records are fetched. The value should not be greater than Long.MAX_VALUE milliseconds. Default value is 5000.
Note:  Long.MAX_VALUE is equal to 2^64 minus 1, which is 9,223,372,036,854,775,807 milliseconds.
Max Poll Records
Maximum number of records to read. Default value is 1000.
Auto Offset Reset
The offset to start reading messages from when a Group ID is specified, for which there is no committed offset with Kafka Server. The options are:
earliest: Starts reading messages from the beginning of the queue.
latest: Starts reading messages from the end of the queue (if the queue is not receiving any messages, the records or messages fetched will always be 0 in this case).
none: Throws an exception if no previous offset is found for the given Group ID.
Default value is earliest.
Enable Auto Commit
If set to True, then the latest offset of messages fetched will be committed at the interval specified by the Auto Commit Interval. Default is True.
Auto Commit Interval
Auto commit the offset of the last read message or record to the Kafka Server (in milliseconds). Default is 5000.
Key Deserializer
Format in which the bytes returned by the server for Key is deserialized. Default value is String Deserializer.
Value Deserializer
Format in which the bytes returned by the value for Key should be deserialized. The options are:
String Deserializer
Kafka Avro Deserializer
JSON Deserializer
Default value is String Deserializer.
Schema Registry URL
Specify this option only if you are using Confluent Stream Platform as Kafka Server and the selected Value Deserializer is Kafka Avro Deserializer.
File Name
Filename with the path, where the messages pulled from Kafka Queue must be populated. A JSON file having format mentioned in the description above.
PutMessage
Topic
Kafka message topic name to where the messages must be sent or written.
Partition Key
Partition key in the Topic to where the messages must be sent.
Acks
Number of acknowledgments the producer requires the leader to have received before considering a request complete.
The options are:
0: Producer does not wait for any acknowledgment from the server.
1: Leader writes the record to its local log but responds without awaiting full acknowledgement from all followers.
all: Leader waits for the full set of in-sync replicas to acknowledge the record.
Default value is 0.
Buffer Memory
Total bytes of memory the producer can use to buffer records waiting to be sent to the server. Default value is 33554432
Retries
If the value is greater than zero, the client resends any record when the previous send has failed with a potentially transient error. Default value is 1
Batch Size
Producer attempts to batch records together into fewer requests whenever multiple records are being sent to the same partition. Default value is 16384.
Key Serializer
Format in which the key of the message to be written to the server must be serialized. Default value is String Deserializer
Value Serializer
Format in which the message written to the server must be serialized. The options are:
String Deserializer
JSON Deserializer
Default value is String Deserializer.
File Name
Filename with path from where the messages must be read. A JSON file having format mentioned in the description above.
Error Conditions
The following table describes error codes generated while sending a message to Kafka.
 
Error Condition
Code
Description
No error
0
The status is OK. There is no error. The information is returned successfully.
Missing license
46
Component Kafka micro-batch component is not licensed. The component requires DataConnect professional license. Contact your Actian sales representative to obtain the licensing information.
Missing broker list
19
Error detected when establishing a connection. The Bootstrap Servers property is required.
Error during connect
19
Exception 'xxx' occurred while attempting to connect to Kafka server 'xxx'.
Missing topic
33
Error detected when sending or receiving a message to or from a topic. A topic is required.
Error when sending a message
8
Exception 'xxx' occurred while attempting to put message to a topic 'xxx'.
Missing Group ID
33
Error in GetMessage. Group ID is required
Missing Schema Registry URL
33
Error in GetMessage. Schema Registry URL is required when using 'Kafka Avro Deserializer' as 'Value Deserializer'.
Note:  For more information about Kafka, see the Kafka documentation available at http://kafka.apache.org.
Kafka Writer Queue
The Kafka writer provides the ability to send data to Kafka from batch or real-time invoked processes without the expectation of handling big data volumes. Kafka provides a client Producer API for the user to write data to Kafka. The Producer API version used is 2.2.1.
The following features are supported:
All Kafka producer configuration properties except for some properties that are not related to data write properties such as metric.reporters, metrics.num.samples, metrics.sample.window.ms.
Separate actions for producer connect, data write, and producer disconnect. Therefore, you can use one connection for multiple data write actions.
Allows you to change topics for each data write action.
Kafka Writer Queue Properties
 
Property
Default Value
Description
Bootstrap Servers
 
List of host or port pairs to use for establishing the initial connection to the Kafka cluster.
The format is host1:port1,host2:port2.
Acks
1
Number of acknowledgments the producer requires the leader to have received before considering a request complete.
The allowed values are:
0 - Producer does not wait for any acknowledgment from the server.
1 - Leader writes the record to its local log but responds without awaiting full acknowledgement from all followers.
2 - Producer requires the given number of acknowledgments but this is generally less useful.
all - Leader waits for the full set of in-sync replicas to acknowledge the record.
Buffer Memory
33554432
Total bytes of memory the producer can use to buffer records waiting to be sent to the server.
Compression Type
none
Compression type for all data generated by the producer.
The allowed values are:
none
gzip
snappy
Retries
3
If the value is greater than zero, the client resends any record when the previous send has failed with a potentially transient error.
Batch Size
16384
Producer attempts to batch records together into fewer requests whenever multiple records are being sent to the same partition.
Client Id
 
User-specified string sent in each request to help trace calls.
Linger
0
Producer groups together any records that arrive in between request transmissions into a single batched request. The value is in milliseconds.
Max Request Size
1048576
Maximum size (records) of a request.
Receive Buffer
32768
Size of the TCP receive buffer to use when reading data.
Send Buffer
131072
Size of the TCP send buffer to use when sending data.
Timeout
30000
Configuration controls the maximum amount of time the server waits for acknowledgments from followers to meet the acknowledgment requirements the producer has specified with the acks configuration. The value is in ms.
Block On Buffer Full
True
When the memory buffer is exhausted, it should either stop accepting new records (block) or throw errors.
Metadata Fetch Timeout
60000
When data is sent to a topic first time, the metadata about that topic is required to know the servers that host the topic's partitions. This configuration controls the maximum amount of time to wait for the metadata fetch to succeed before throwing an exception back to the client. The value is in ms.
Metadata Max Age
300000
Amount of time in milliseconds after which force a refresh of metadata even if any partition leadership changes are not seen to proactively discover any new brokers or partitions. The default value is in ms.
Reconnect Backoff
10
Amount of time to wait before attempting to reconnect to a host when a connection fails. The value is in ms.
Retry Backoff
100
Amount of time to wait before attempting to retry a failed produce request to a particular topic partition. The value is in ms.
Supported Actions
 
Action
Description
Connect
Establishes a connection to the server.
Disconnect
Disconnects from the server.
PutMessage
Writes a message object into the queue.
Supported Action Parameters
 
Action
Parameter
Description
PutMessage
Message
The message that must be written.
Supported Action Properties
 
Action
Property
Description
PutMessage
Topic
Kafka message topic
Partition Key
Message partition key
Error Conditions
The following table describes error codes generated while sending a message to Kafka.
 
Error Condition
Code
Description
No error
0
The status is OK. There is no error. The information is returned successfully.
Missing license
46
Component Kafka writter is not licensed. The component requires DataConnect professional license. Contact your Actian sales representative to obtain the licensing information.
Missing broker list
4
Error detected when establishing a connection. The Bootstrap Servers property is required.
Error during connect
4
Exception 'xxx' occurred while attempting to connect to Kafka server 'xxx'.
Missing topic
4
Error detected when sending a message to a topic. A topic is required.
Error when sending a message
8
Exception 'xxx' occurred while attempting to put message to a topic 'xxx'.
Note:  For more information about Kafka, see the Kafka documentation available at http://kafka.apache.org.
Websphere MQ Queue
WebSphere MQ Queue allows a message to be put into the queue or retrieved from a queue to a message object. Use the PutMessage action to write to the queue. Use the GetMessage action in a Queue step to retrieve messages. Session properties control which message objects are read and the names of the messages that are written, as well as whether to preserve the message properties.
Supported versions:
DataConnect version <= 11.3.x supports WebsphereMQ client 7.0.0-7.5.x.x using the Websphere MQ 1.0.0 component
DataConnect version >= 11.4.x supports WebsphereMQ client 8.x.x (only) using the Websphere MQ 1.0.0 component
DataConnect version >= 11.5.2 supports WebsphereMQ (IBM MQ) >= 9.x.x using the Websphere MQ 2.0.0 component
MQ client 8.0.0.2 distributes libraries for versions 2005 and 2012. You can install this version of client then update their path to use either library, depending on which version of DataConnect you need to use.
For more information about the provision of both 2005 and 2012 versions of the native API client libraries in WebSphere MQ 8.0.0.2 and higher, see the following link:
https://www.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.pro.doc/q130070_.htm
Websphere MQ Queue Properties
 
Property
Default Value
Description
Format
MQFMT_NONE
WebSphere MQ message format.
Timeout
0
Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.
Browse
False
Retrieve messages from a queue without removing them.
Convert
False
Converts the message data to a specific character set or numeric encoding. Requires CCSID or Encoding value.
CCSID
MQCSSI_DEFAULT
Character set identifier of the message data.
Encoding
 
Encoding for numeric data. The representation used for binary integers, packed-decimal integers, and floating-point numbers.
Supported Actions
Action
Description
Connect
Establishes a connection to the WebSphere MQ server.
Disconnect
Disconnects from the WebSphere MQ server.
GetMessage
Retrieves message into a message object.
PutMessage
Writes a message object into the queue.
Begin Transaction
Starts a WebSphere MQ transaction.
Commit Transaction
Commits a WebSphere MQ transaction.
Rollback Transaction
Rolls back a WebSphere MQ transaction.
Supported Action Properties
Action
Property
Description
PutMessage
GetMessage
Format
WebSphere MQ message format, with allowable values defined by WebSphere MQ
PutMessage
GetMessage
Msgid
Message ID of a specific message put in or retrieve from a queue
PutMessage
GetMessage
Correlationid
Correlation ID of messages to put in or retrieve from a queue
PutMessage
GetMessage
Groupid
Group ID of message to put in or retrieve from a queue
PutMessage
GetMessage
CCSID
Character set that you can convert to. Used with the Convert option
PutMessage
GetMessage
Encoding
Numeric encoding you can convert to. Used with the Convert option
GetMessage
SequenceNumber
Sequence number of a specific message to retrieve from a queue
PutMessage
Offset
Segment offset
PutMessage
ReplyToQ
Reply queue
PutMessage
ReplyToQMgr
Reply queue manage
PutMessage
UserID
User identifier
PutMessage
AccountingToken
Accounting token
PutMessage
ApplIdData
Application identity data
PutMessage
PutAppIName
Application that put the message
PutMessage
PutDate
Date the message was put
PutMessage
PutTime
Time the message was put
PutMessage
ApplOriginData
Application origin data
PutMessage
MsgType
Message type
PutMessage
Expiry
Message lifetime
PutMessage
Feedback
Feedback code
PutMessage
Priority
Message priority
PutMessage
Persistence
Message persistence
PutMessage
BackoutCount
Backout counter
PutMessage
PutApplType
Put application type
PutMessage
MsgFlags
Message flags
PutMessage
OriginalLength
Length of original message
WebSphere MQ Environment Variables
If you use WebSphere MQ environment variables, to ensure compatibility with queue sessions, use the following format:
MQServer=ChannelName/TransportType/IPAddress
Example
MQServer=mymq1/TCP/123.123.123.123
Queue Manager Parameter
Specify the Queue Manager as the Server parameter in the queue session as shown in this example:
Server=<servername> (123.123.123.123) Port=default QueueManager=<queuemanagername> queue=tester1 ServerChannel=serverch1
Server Parameter
The MQServer, MQCHLLIB, and MQCHLTAB environment variables can be changed at runtime or before the GUI tools are launched to test writing to different servers.
In the queue session, there is a Server parameter where you specify a Queue Manager (if you need one).
If you use the MQCHLLIB or MQCHLTAB variable, you must specify the Server parameter.
If you just use the MQServer environment variable, it is not necessary to specify the Server parameter. Instead, just change the MQServer environment variable at run time to specify the new Queue Manager to which you are writing. The MQServer environment variable has the channel, protocol, server, and port specified, which is all that is needed to connect to a different queue manager.
Macro Values for the Server Parameter
You can use macro values for the Server parameter of the session. If you use the MQCHLLIB and MQCHLTAB environment variables, you can create a macro to specify the Queue Manager. The Queue Name cannot use a macro value. Change this before deployment, or use a variable that drives Decision steps to do a Get or a Put message to the proper queue.
It is recommended that you change the MQServer environment variable at runtime, since no Queue Manager is needed in the Server parameter of the queue session. You only need to change the queue name.
Key Points About Websphere MQ and the DJComponent Object
IBM Websphere MQ requires a separate queue manager connection for each thread, and does not permit cross-thread operations.
For processes, three scenarios are possible:
Set MaxConcurrentThreads = 1. All process steps are executed on the same thread, so queue operations can be spread across multiple steps.
Complete all queue operations in a single step. This includes connecting to a queue, getting or putting messages, and completing a transaction. This guarantees that all queue operations are executed in a single thread. Multiple steps can still contains queue operations, but each step is a separate transactional unit of work.
Confine all queue operations to the Start and Stop steps. In multithreaded processes, the Start and Stop steps always execute in the main thread. The Start step can be used to create a connection, begin a transaction and retrieve messages from the queue. The Stop step can be used to write messages to the queue and complete the transaction.
Note:  These threading restrictions apply only to DJComponent objects, not DJMessage objects. The DJMessage objects are completely independent from the messaging systems. So, in scenario 3 above, you can pull a message from a queue in the Start step. Use that message object in multiple concurrent Script or Transformation steps, and complete the transaction in the Stop step. Likewise, you can create DJMessage objects in multiple steps and write all of the message objects to a queue in the Stop step. The maximum size limit for DJMessage is 512 MB.
IBM MQ Queue
IBM MQ Queue allows a message to be put into the queue or retrieve from a queue to a message object. Use the PutMessage action to write to the queue. Use the GetMessage action in a Queue step to retrieve messages. Session properties control which message objects are read and the names of the messages that are written. Also, if the message properties must be to preserved.
Note:  IBM MQ Component supports IBM MQ Version 1.0.0.
IBM MQ Queue Properties
Property
Default Value
Description
Host
 
Host name where IBM MQ server is installed.
Port
1414
Port number to connect to.
Channel
 
Channel name through which the communication occurs.
Queue Manager
 
Name of the Queue Manager to connect to.
User ID
 
User ID that has access to the specified Host.
Password
 
Password of the above specified User ID.
Supported Actions
Action
Description
Connect
Establishes a connection to the IBM MQ server.
Disconnect
Disconnects from the IBM MQ server.
GetMessage
Retrieves message into a message object.
PutMessage
Writes a message object into the queue.
Begin Transaction
Starts a IBM MQ transaction.
Commit Transaction
Commits a IBM MQ transaction.
Rollback Transaction
Rolls back a IBM MQ transaction.
Supported Action Properties
Action
Property
Description
PutMessage
Include JMS Headers
Set the property to FALSE if you want MQ-style message body, without a JMS MQRFH2 header.
PutMessage
GetMessage
CCSID
Coded Character Set Identifier of character data within the message body.
PutMessage
 
Encoding
Encoding for numeric data. The representation used for binary integers, packed-decimal integers, and floating-point numbers.
PutMessage
GetMessage
MsgId
Message ID of a specific message put in or retrieved from a queue.
PutMessage
GetMessage
CorrelationId
Correlation ID of messages to put in or retrieve from a queue.
PutMessage
GetMessage
GroupId
Group ID of message to put in or retrieve from a queue.
GetMessage
SequenceNumber
Sequence number of a specific message to retrieve from a queue.
GetMessage
Timeout
Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.
GetMessage
Browse
Retrieve messages from a queue without removing them.
PutMessage
Offset
Segment offset.
PutMessage
ReplyToQ
Reply to queue (this field must not be blank if the MsgType field has the value MQMT_REQUEST).
PutMessage
ReplyToQMgr
Set the name of the queue manager to which reply message must be sent.
PutMessage
UserID
UserIdentifier specifies the user identifier of the application that originated the message.
PutMessage
AccountingToken
Accounting token of the message to be sent.
PutMessage
ApplIdData
Set ApplIdentityData to provide additional information about the message or its originator.
PutMessage
PutAppIName
Application that put the message.
PutMessage
PutDate
Date the message was put.
PutMessage
PutTime
Time the message was put.
PutMessage
ApplOriginData
Set ApplOriginData to provide additional information about the origin of the message.
PutMessage
MsgType
Message type.
PutMessage
Expiry
Message lifetime expressed in tenths of a second.
PutMessage
Feedback
Set Feedback to indicate the nature of the report (used with a message of type MQMT_REPORT).
PutMessage
Priority
Message priority.
PutMessage
Persistence
Persistence indicates whether the message survives system failures and restarts of the queue manager.
IBM MQ Environment Variables
You can use IBM MQ environment variable MQServer, to connect to the server. If the Host is not defined in Queue properties, then DataConnect attempts to read the MQServer environment variable to connect. Use the following format to define MQServer environment variable:
MQServer=ChannelName/TransportType/IPAddress(Port)
Example
MQServer=mymq1/TCP/123.123.123.123(1415)
Or
MQServer=mymq1/TCP/123.123.123.123
 
If you do not provide (Port), then the default port number 1414 will be used to connect.
Note:  Press Test Connection button after defining Queue (Component) Properties, if the Test Connection fails, then make sure that you can successfully ping the server/host where IBM MQ is deployed from the machine where DataConnect is installed. You will need to set appropriate authentication rules (CHLAUTH) at the IBM MQ Server which allows the access to the Server/Queue Manager/Queue you are trying to connect to. For more information, see IBM MQ documentation — https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.2.0/com.ibm.mq.sec.doc/q132560_.htm
MSMQ Queue
MSMQ Queue allows a file folder to be treated like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a queue step. Message objects can also be written to the folder as files using the PutMessage action. The properties of the session control which files are read and the names of the files that are written, as well as whether to preserve the message properties.
MSMQ Queue Properties
Property
Default Value
Description
User Name
 
User name to connect to the MSMQ server.
Password
 
Password to connect to the MSMQ server.
Server Name
 
Name of the MSMQ server.
Timeout
0
Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages
BrowseMode
False
Retrieve messages from a queue without removing them
Supported Actions
Action
Description
Connect
Establishes a connection to the MSMQ server.
Disconnect
Disconnects from the MSMQ server.
GetMessage
Retrieves message into a message object.
PutMessage
Writes a message object into the queue.
BeginTransaction
Starts a MSMQ transaction.
PrepareTransaction
Prepares a MSMQ transaction.
CommitTransaction
Commits a MSMQ transaction.
RollbackTransaction
Rolls back a MSMQ transaction.
Supported Action Parameters
Action
Parameter
Description
Connect
Server
Name of the MSMQ server.
 
Queue
Name of the queue.
GetMessage
Queue
Name of the queue.
 
Message
Message content.
PutMessage
Queue
Name of the queue.
 
Message
Message content.
Supported Action Properties
 
Action
Parameter
Description
PutMessage
Ack
Specifies the type of acknowledgement messages that MSMQ posts. The available options are:
MQMSG_ACKNOWLEDGEMENT_NONE (default)
MQMSG_ACKNOWLEDGEMENT_FULL_REACH_QUEUE
MQMSG_ACKNOWLEDGEMENT_FULL_RECEIVE
MQMSG_ACKNOWLEDGEMENT_NACK_REACH_QUEUE
MQMSG_ACKNOWLEDGEMENT_NACK_RECEIVE
MQMSG_ACKNOWLEDGEMENT_POS_ARRIVAL
MQMSG_ACKNOWLEDGEMENT_POS_RECEIVE
MQMSG_ACKNOWLEDGEMENT_NEG_ARRIVAL
MQMSG_ACKNOWLEDGEMENT_NEG_RECEIVE
AppSpecific
Specifies application generated information such as single Integer values or application defined message classes. The default value is:
0
AuthLevel
Specifies whether the message must be authenticated when it arrives at the target queue. The available options are:
MQMSG_AUTH_LEVEL_NONE (default)
MQMSG_AUTH_LEVEL_NONE_ALWAYS
CorrelationId
Identify the message using a 20-byte correlation identifier.
Delivery
Specifies how MSMQ delivers the message. The available options are:
MQMSG_DELIVERY_EXPRESS (default)
MQMSG_DELIVERY_RECOVERABLE
PutMessage
EncryptAlgorithm
Specifies the inception algorithm used by MSMQ to encrypt private messages. The available options are:
MQMSG_CALG_RC2 (default)
MQMSG_CALG_RC4
HashAlgorithm
Specifies the hash algorithm used by MSMQ when authenticating messages. The available options are:
MQMSG_CALG_MD4
MQMSG_CALG_MD5 (default)
MQMSG_CALG_SHA
MQMSG_CALG_SHA256
MQMSG_CALG_SHA512
MQMSG_CALG_MAC
MQMSG_CALG_RSA_SIGN
MQMSG_CALG_DSS_SIGN
MQMSG_CALG_RC2
MQMSG_CALG_RC4
MQMSG_CALG_SEAL
Journal
This property is used for queues (MSMQQueueInfo objects) and non-transaction messages (MSMQMessage objects). The available options are:
MQMSG_JOURNAL_NONE (default)
MQMSG_JOURNAL_DEADLETTER
MQMSG_JOURNAL
Label
Specifies a description of the queue or message.
 
MaxTimeToReachQueue
Specifies a time limit (in seconds) for the message to reach the queue. The available option is:
LONG_LIVED (default)
PutMessage
MaxTimeToReceive
Specifies a time limit (in seconds) for the message to be retrieved from the target queue. The available option is:
INFINITE (default)
Priority
Specifies the privacy level of a queue or message. The default value is:
3
PrivLevel
Specifies the privacy level of the message to be sent. The available option is:
MQMSG_PRIV_LEVEL_NONE (default)
MQMSG_PRIV_LEVEL_BODY_BASE
MQMSG_PRIV_LEVEL_BODY_ENHANCED
SenderCertificate
Provides an array of bytes that represents the security certificate.
SenderIdType
Specifies the type of sender identifier found by MSMQ. The available options are:
MQMSG_SENDERID_TYPE_SID (default)
MQMSG_SENDERID_TYPE_NONE
Trace
Specifies where report messages are sent when tracing a message. The available options are:
MQMSG_TRACE_NONE (default)
MQMSG_SEND_ROUTE_TO_REPORT_QUEUE