DC 11.6 | Queue Step

User Guide > Designing and Executing Processes > Process Steps and Associated Components > Queue Step

Was this helpful?

Queue Step

Queues provide access to applications or middleware that produce and consume data in the form of messages. Queues encapsulate the information necessary to establish a connection with the messaging system. Queue components can access data and services provided by message queuing systems. This includes the ability to read and write messages to and from a queue as well as transactional support. Queue components can provide access to common message queuing systems such as FTP Queue, AWS S3 Queue, File Folder Queue, Email Queue, JMS Queue, Oracle AQ, IBM MQ, and MSMQ. They can also be used to provide a queuing interface to a nonmessage-based system, like a file system. Typical queuing systems perform a destructive read. In other words, once a message is read, it is removed from the queue.

For each type of queue component, you create a corresponding Queue step. The Queue step allows you to select a specific instance of a component and to specify the Action for the component to execute.

Queue Step Properties

After adding a /download/attachments/24976145/Queue_step.png?version=1&modificationDate=1490033775548&api=v2

Queue step from the Palette to the canvas, you can double-click the step on the canvas and specify the following step properties in the Properties tab displayed at the bottom.

Name	Description
Name	Unique name for the step.
Description	Description of the step (optional).
Enabled	Select this option to enable the step for execution. If it is not enabled, then the step on the canvas is greyed-out. This option is useful when troubleshooting a process, especially when there are numerous steps in the process.
Error Handling	Select either of the following: • Abort Process - All the steps after the current step are skipped or the entire process is stopped (depending on how you have set the error logging options) if this step stops due to an error. • Ignore Error - The remaining steps in the current process are executed even if this step stops due to an error. This is selected by default.
Select Queue	Select a Queue component instance. These instances are displayed when you add message components in the Configuration tab. For more information, see Adding Message Components in Configuring Processes.
Action	Select the action for the queue instance.
Parameter	Parameters for the selected action. Specify values for the parameters.
Property	Properties for the selected action. Specify values for the properties.

Note: The Actions, Parameters, and Properties are specific to the component type. For information about this, based on the component instance selected in the Select Queue drop-down list, click the component type in the Queue Step Components section below.

Queue Step Components

• AWS S3 Queue

• Azure Storage Queue

• POP3 Email Queue

• IMAP Email Queue

• File Folder Queue

• FTP Queue

• Google Cloud Storage Queue

• Google Drive Queue

• JMS Queue

• Kafka Micro-batch Queue

• Kafka Writer Queue

• Websphere MQ Queue

• IBM MQ Queue

• MSMQ Queue

AWS S3 Queue

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. You can use it to store and protect any amount of data for websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. For more information on Amazon's Simple Storage Service, see https://aws.amazon.com/s3/.

DataConnect provides AWS S3 Queue component that can be used to connect to AWS S3. This component supports the following features:

• Specifying the AWS S3 bucket name that stores the objects being written and read.

• Specifying the Region that houses the bucket.

• Selecting common AWS authentication methods

• Uploading files or DJMessage content to S3 buckets

• Downloading objects from S3 buckets to DJMessages or files

• Removing objects from the S3 bucket when they are no longer needed in the process

• Creating a temporary bucket to be used in the current process. When the process calls the disconnect method of the component, the temporary bucket is deleted.

You can use the S3 to Avalanche template for loading data to Avalanche. It leverages the S3 component. For more information about this template, see Integration Manager documentation.

AWS S3 Queue Properties

Property	Default Value	Description
AWS Region		Region that hosts the specific S3 bucket. The component will be configured to use this bucket. For more information about AWS regions, availability zones, and other related topics, see https://aws.amazon.com/about-aws/global-infrastructure/regions_az/
S3 Bucket Type	Use Existing Bucket	Select one of the following: • Use Existing Bucket: Select this option to use an existing S3 bucket. The following additional options are displayed: – S3 Bucket Name: Name of the AWS S3 bucket. The value must match the name of an existing AWS S3 bucket for which you have the appropriate privileges. This bucket will be used to upload and retrieve stored objects. For more information about AWS S3 buckets, see Working with Amazon S3 Buckets. – Delete Objects When Finished: If the value is set to True, then the component retains a list of all files that it uploaded during its life cycle in the process. When the component's disconnect method is called by the process, the uploaded files are deleted. The default value is False. • Create Temp Bucket: Select this option to create a temporary S3 bucket. It is used while the component instance remains active in the process. The following additional options are displayed: – Temp S3 Bucket Prefix: Specify a custom prefix for the temporary bucket name that will be created by the component. The prefix must be consistent with AWS S3's bucket naming guidelines as defined in the Rules for Bucket Naming section in Bucket Restrictions and Limitations. If you do not provide this name, the component uses the default prefix for all unique temporary bucket names that it creates. The default prefix is actian-dc-tempbucket-.
Authentication Type	Default Credential Provider Chain	Means by which identity is provided to AWS • Default Credential Provider Chain: Uses the "default" credentials provider chain that is provided in the AWS SDK. For information about the standard mechanism that AWS client-side automation tools use to obtain user credentials, see New and Standardized Way to Manage Credentials in the AWS SDKs. • Access Keys: Access keys are long-term credentials for an IAM user or the AWS account root user. Access keys are created in pairs that consist of the following (displayed as additional options): – Access Key ID: Provide the access key ID. – Secret Access Key: Provide the secret access key. Note: If the Access Keys authentication type is selected, the component will not search for additional means by which the credentials are provided.

Supported Actions

Action	Description
Connect	Creates an authenticated S3 client that will be used to perform all subsequent operations.
PutMessage	Provide a DJMessage or file name. The component will write this file or message to the S3 bucket specified in the S3 Queue properties. The component uses the authenticated S3 client to upload data to an S3 bucket and saves it using a provided "key". Since the data is represented in S3 as an object, the key is the "object name" or "file name".
GetMessage	Specify a DJMessage or a referenced file through which content can be downloaded. Uses the authenticated S3 client to download data from an S3 bucket. The data in the bucket is referenced by a key.
Disconnect	Removes the authenticated S3 client. If the Create Temp Bucket option is enabled, then the temporary bucket (along with all uploaded content) is removed. If Delete Objects When Finished option is selected, all uploaded content will be deleted.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Specify a DJMessage variable or file through which content can be downloaded.
PutMessage	Message	Specify a DJMessage variable or file through which content can be uploaded.

Supported Action Properties

The following action properties are applicable.

Action	Property	Description
GetMessage	Object Key	Object key that must be downloaded from the S3 bucket. The key is unique within the S3 bucket and may include separator characters, such as slash (/) to emulate folders. This option is mandatory.
	Local File Name	Local file name, where the component writes the data retrieved from the S3 bucket. If you do not specify this file name, then the data read from the bucket is inserted into the DJMessage parameter.
	Bytes Limit	Limit the number of bytes that are downloaded from the object in the S3 bucket. For example, the value 10000 means that only the first 10000 bytes are downloaded from the source object.
	Delete Object After Retrieval	If this value is set to True, then the object is deleted from the S3 bucket after it is successfully retrieved. The object will not be deleted if an error occurs during download. The default value is False. Note: This option is displayed only if you have selected Use Existing Bucket for the component properties and have set Delete Object When Finished to False.
PutMessage	ObjectKey	Key that is assigned to the object after it is uploaded to the S3 bucket. The key is unique within the S3 bucket and may include separator characters, such as slash (/) to emulate folders. If this value is not provided, then the component expects to find an override property called ObjectKey within the DJMessage.
PutMessage	Local File Name	Local file name from which the component reads the data that it uploads to the S3 bucket. The value of this option (whether it is blank or populated) is overridden by the LocalFileName DJMessage property. If neither value is set, then the data that will be uploaded is read from the DJMessage body.

Override Properties

You can override the values provided for the object key and local file name in the PutMessage action with DJMessage properties:

• ObjectKey DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Object Key option.

• LocalFileName DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Local File Name option. If both are missing, then the uploaded data is read from the DJMessage body.

Errors

The following table describes error codes generated for AWS S3 Queue component.

Action	Code	Description
Connect	ERR_OK	Connect completed successfully with no errors
Connect	ERR_OPENERR	Connect failed. Typically associated with LT_ERROR log level. Associated error message describes the specific issue.
Disconnect	ERR_OK	Disconnect completed successfully
	ERR_CLOSERR	Associated with warning-level log message. Disconnect is called again after it is already disconnected.
	ERR_DELETERR	• Attempt to delete non-temporary bucket. Associated with warning-level log message. • Attempt to delete one or more objects resulted in error. Associated with warning-level log message.
GetMessage	ERR_OK	Object retrieved and saved (to file or DJMessage) with no errors
	ERR_BADOPTIONVALUE	• Object key not set. Associated with error-level log message. • Local file referenced file already exists. Associated with error-level log message.
	ERR_READERR	• Error copying response content to the provided DJMessage. Associated with error-level log message. • Wide variety of errors produced by Amazon AWS S3 API will produce this error. The error code will include a message appropriate to the error. Typically associated with error-level log entry.
	ERR_DELETERR	Delete after download option is set, but the component failed to delete the object from the S3 bucket. Associated with warning-level log entry
PutMessage	ERR_OK	Object uploaded to S3 bucket with no errors
	ERR_BADOPTIONVALUE	Target object key property must be set. Associated with error-level log entry.
	ERR_WRITERR	• Error copying data from DJMessage. Associated with error-level log entry. • AWS-generated error occurred while sending or writing data to the remote S3 bucket. A number of messages could be produced. All associated with error-level log entry.

Azure Storage Queue

‎Azure Data Lake Storage (ADLS) Gen2 is a set of capabilities built on Azure Blob storage. Data Lake Storage Gen2 is the result of converging the capabilities of the two existing storage services—Azure Blob storage and Azure Data Lake Storage Gen1. Features from Azure Data Lake Storage Gen1, such as file system semantics, directory, and file level security and scale are combined with low-cost, tiered storage, high availability or disaster recovery capabilities from Azure Blob storage. For more information on Azure Data Lake Storage, see Introduction to Azure Data Lake Storage Gen2.

DataConnect provides the Azure Storage Queue component that can be used to connect to ADLS Gen2. This component supports the following features:

• Storage account’s name and shared access key (for authentication)

• Container’s specification within the storage account that hosts the data

• Selecting Data Lake Storage Service (Blob Storage Service or File Storage Service)

• Uploading files or DJMessage content

• Downloading objects to DJMessages or files

You can use the ABFS to Avalanche Azure template for loading data to Avalanche. It leverages the Azure Storage Queue component. For more information about this template, see Integration Manager documentation.

Azure Storage Queue Properties

Property	Default Value	Description
Azure Storage Service Type		Select any of the following options for interacting with the selected container: • Blob Storage Service: Component uses Microsoft Azure's Blob-specific APIs to interact with the selected storage account and container. For more information, see Introduction to Azure Blob Storage. • ADLS File System Service: Component uses Microsoft Azure's Data Lake Storage Gen2 hierarchical namespace-specific APIs to interact with the selected storage account and container. For more information, see Azure Data Lake Storage Gen2 hierarchical namespace.
Storage Account Name		Name of a pre-existing storage account that is created within a Microsoft Azure account. For more information, see Storage account overview.
Container Name		Name of a pre-existing container within the selected storage account.
Delete Objects When Finished	False	If set to True, all files that are uploaded by the component during the process execution are deleted when the disconnect action is called. Note: The process engine automatically calls the disconnect action when the process is completed.
Azure Storage Authentication Type		Select the authentication scheme that the component will use: • Storage Shared Access Keys: Azure generates two 512-bit storage account access keys when you create a storage account. These keys can be used to authorize access to data in your storage account through Shared Key authorization. The following additional option is displayed: – Access Key: Provide one of the two 512-bit storage account access keys that was automatically generated by Microsoft Azure when the storage account was created. • Azure Active Directory Service Principal With Client Secret: Customer applications, which embed the Azure Storage Component, can be integrated with the Microsoft identity platform by registering it with an Azure Active Directory tenant. After this, a client secret can be generated from the registered application and it can be used for authentication. For more information, see Microsoft’s How to page. Actian Avalanche COPY VWLOAD directly supports this type of registration and authentication. The following additional options are displayed: – Application (Client) ID: Identity assigned to the application after registration. It is the service principal assigned to the application. – Directory (tenant) ID: Provide the tenant ID. A tenant is a representation of an organization in Azure Active Directory. Applications are registered relative to a tenant within Azure Active Directory. – Client Secret: Specify the client secret value. After an application or service principal is created, you can upload a certificate or create a client secret to be used for authentication.

Supported Actions

Action	Description
Connect	Creates an authenticated Azure storage client that is used to perform all subsequent operations.
PutMessage	Uses the authenticated Azure Storage client for uploading data to an Azure Storage container and saves it using the provided blob (or file) name. The content that must be uploaded can be provided to the PutMessage action through DJMessage or a referenced file.
GetMessage	Uses the authenticated Azure storage client to download data from the specified container. The downloaded content can be written to a file or injected into a DJMessage object that is provided to the GetMessage action.
Disconnect	Removes the authenticated Azure storage client. If you have chosen the option to delete uploaded objects, then all the uploaded files are deleted.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Specify a DJMessage variable or file through which content can be downloaded.
PutMessage	Message	Specify a DJMessage variable or file through which content can be uploaded.

Supported Action Properties

The following action properties are applicable.

Action	Property	Description
GetMessage	Object Name	Object name that you intend to download from the configured Azure Storage container. If the component is interacting with the container using the blob service, then the name represents the blob name. If the component is interacting with the container through the file system service, then the name represents the full path to the file including directory elements separated by the "/" character. Note: This option is required within the GetMessage action.
	Local File Name	Local file name, where the component writes the data retrieved from the Azure Storage container. If you do not specify this file name, then the data read from the container is inserted into the DJMessage parameter.
	Bytes Limit	Limits the number of bytes downloaded from the selected blob or file to the specified value. If you want to download the entire blob or file, do not specify this value.
	Delete Object After Retrieval	If set to True, the specified blob or file is deleted from the Azure Storage container after the download is successful. The default value is False. Note: This option is displayed only if the Delete Objects When Finished component option's value is False.
PutMessage	Object Name	Name of the blob or file that is assigned to the object after it is uploaded to the Azure Storage container. • If the component is configured to use the File System Service for interacting with the selected Azure Storage container, then the name can include the full path (including directories) to the file. If the path includes missing directories in the container, then the component creates the missing directories. • If the object name is not specified, then the component tries to find an override property called ObjectName within the DJMessage. It is an error condition if the option is left blank and the override property is missing from the DJMessage.
PutMessage	Local File Name	If a value is provided, then the component reads the data that it uploads from the referenced file. An error will be generated if the file cannot be resolved or is inaccessible. When this option is left blank or populated, it is overridden by the LocalFileName DJMessage property. If neither value is set, then the data that is uploaded is read from the DJMessage body.

Override Properties

You can override the values provided for the object name and local file name in the PutMessage action with DJMessage properties:

• ObjectName DJMessage property: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Object Name option.

Errors

The following table describes error codes generated for the Azure Storage Queue component.

Action	Code	Description
Connect	ERR_OK	Connect completed successfully without any errors.
Connect	ERR_OPENERR	Connect failed. Typically associated with LT_ERROR log level.
Disconnect	ERR_OK	Disconnect completed successfully.
	ERR_CLOSERR	Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
	ERR_DELETERR	If the component is configured to delete uploaded objects during connect, this error code is generated when one or more objects that are deleted resulted in error. This is associated with warning-level log message.
GetMessage	ERR_OK	Object retrieved and saved (to file or DJMessage) without any errors.
	ERR_BADOPTIONVALUE	Object name is not set. This is associated with error-level log message.
	ERR_READERR	Local file or referenced file already exists. This is associated with error-level log message.
	ERR_DELETERR	• Error in copying response content to the provided DJMessage. This is associated with error-level log message. • Too many errors generated by the Azure Storage SDK produces this error. The error code will include a message appropriate to the error. This is associated with error-level log message.
PutMessage	ERR_OK	Object uploaded to Azure Storage container without any errors.
	ERR_BADOPTIONVALUE	Target object name property must be set. This is associated with error-level log entry.
	ERR_WRITERR	• Error copying data from DJMessage. This is associated with error-level log entry • Azure-generated error occurred when sending or writing data to the remote Azure Storage container. A number of messages can be generated and all these messages are associated with error-level log entry.

POP3 Email Queue

POP3 Email Queue supports receiving text and binary attachments. DataConnect supports POP3 Email Queue versions 1.0.2 and 1.3.0.

The following table provides the supported content types.

Content Type	Details
text/*	• text • HTML
image/*	• .gif • .jpg
application/*	• All applicable applications, such as .xls, .zip, and .doc

The email components support encoding in all character sets when an email is sent and received. The message header and body are encoded in the character set specified by the MIME. In addition, the email components support only Base64 and other MIME encoding. Enriched encoding is not supported.

Note: The POP3 Email Queue component implicitly trust the certificates presented by the email servers that require SSL or TLS connections, regardless of the issuer.

POP3 Email Queue Properties

Property	Default Value	Description
User Name		User name to connect to the POP3 email server.
Password		Password to connect to the POP3 email server.
Server Name		Name of the POP3 email server
Port	110	Port to be used to connect to the server. Note: If the default port is not used, then the component assumes a secure (SSL or TLS) connection is required.
ConnectionTimeout	90	Connection time out value in seconds.
Retry Count	3	Number of attempts the server tries to establish connection when the connection times out or fails.
From Contains		From email address
To Contains		To email address
Subject Contains		Subject value
Browse Mode	FALSE	Select from TRUE or FALSE: • TRUE - If this is selected, the email message remains on the server even after it is download. • FALSE - If this is selected, the email message is deleted from the server after it is download.
Download	UNREAD	Download the read and/or unread messages from the server: • ALL - Download both read and unread messages from the server. • READ - Download only read messages from the server. • UNREAD - Download only unread messages from the server. Note: This property is displayed only for POP3 Email Queue version 1.0.2.
Encoding String	ISO-8859-1	Encoding string required for the email from the list.
Mark Failed Messages as Seen	FALSE	Select from TRUE or FALSE: • TRUE - Marks unprocessed messages as seen. This value provides a method to bypass messages that the component cannot handle by marking them as read. • FALSE - Leaves unprocessed messages as unseen. Note: This property is displayed only for POP3 Email Queue version 1.3.0.
Always Create Attachment Messages	FALSE	Select from TRUE or FALSE: • TRUE - Component encodes any primary content that may be an email attachment, and then includes it into a new DJMessage attachment. The original content normally has a MIME type indicating that it is binary data. • FALSE - Component places any primary content that may be an email attachment in the main DJMessage. The original content normally has a MIME type indicating that it is text data. Note: If the email is not multi-part MIME- formatted, then the content is included in the main DJMessage regardless of the MIME type.This property is displayed only for POP3 Email Queue version 1.3.0.

Tip... You can set filters or flags using the From Contains, To Contains, and Subject Contains properties. However, the filters do not work if the server does not support the same.

Supported Actions

Action	Description
Connect	Opens a persistent connection to the database.
GetMessage	Retrieves message into a message object.
Disconnect	Closes an existing database connection and performs any clean up actions if required.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Name for the incoming message

Supported Action Properties

There are no supported action properties.

Errors

Errors are generated when email messages cannot be sent or received due to various reasons. Typical errors include the following:

• Connection timeout

• Unable to disconnect from the email server

• Errors while sending email messages

• Errors while Receiving email messages

• Miscellaneous or unknown errors

All errors are written to an error log, which you can read to diagnose and troubleshoot the problem. The error messages specify whether a problem occurred while receiving or sending.

The error log also contains error codes that help you to identify and understand the cause of the error.

The following table describes error codes generated while sending or receiving email.

Code	Description
19	Unable to connect to the mail server
27	Unable to disconnect from the mail server
8	Unable to send email
4	Unable to receive email
50	Unknown error
1	End of file
12	No more messages to download
0	Messages sent or received successfully

IMAP Email Queue

The IMAP Email Queue supports receiving text and binary attachments.

The supported content types are outlined here.

Content Type	Details
text/*	• text • HTML
image/*	• .gif • .jpg
application/*	• All applicable applications, such as .xls, .zip, and .doc

Note: The IMAP Email Queue component implicitly trust the certificates presented by the email servers that require SSL or TLS connections, regardless of the issuer.

Instance Properties

IMAP Email Queue has the following instance properties.

Property	Default Value	Description
User Name		User name to connect to the IMAP email server.
Password		Password to connect to the IMAP email server
Server Name		Name of the IMAP email server
Port	143	Port to be used to connect to the server. Note: If the default port is not used, then the component assumes a secure (SSL or TLS) connection is required.
ConnectionTimeout	90	Connection time out value in seconds.
Retry Count	3	Number of attempts the server tries to establish connection when the connection times out or fails.
From Contains		From email address
To Contains		To email address
Subject Contains		Subject value
Folder Name		Folder that resides in the IMAP server. If you specify a folder name, all the emails are delivered to the specified folder. Typically, an email is downloaded to the INBOX folder unless you specify another location. Examples of specific folders are INBOX.myemail, INBOX.general, and so on.
Browse Mode	TRUE	Select from TRUE or FALSE: • TRUE - If this is selected, the email message remains on the server even after it is download. • FALSE - If this is selected, the email message is deleted from the server after it is download.
Download	UNREAD	Download the read and/or unread messages from the server: • ALL - Download both read and unread messages from the server. • READ - Download only read messages from the server. • UNREAD - Download only unread messages from the server.
Encoding String	ISO-8859-1	Encoding string required for the email from the list.
Mark Failed Messages as Seen	FALSE	Select from TRUE or FALSE: • TRUE - Marks unprocessed messages as seen. This value provides a method to bypass messages that the component cannot handle by marking them as read. • FALSE - Leaves unprocessed messages as unseen.
Always Create Attachment Messages	FALSE	Select from TRUE or FALSE: • TRUE - Component encodes any primary content that may be an email attachment, and then includes it into a new DJMessage attachment. The original content normally has a MIME type indicating that it is binary data. • FALSE - Component places any primary content that may be an email attachment in the main DJMessage. The original content normally has a MIME type indicating that it is text data. Note: If the email is not multi-part MIME- formatted, then the content is included in the main DJMessage regardless of the MIME type.

Tip... You can set filters or flags using the From Contains, To Contains, Subject Contains, or Download properties. It is recommended to download only unread messages. However, filters or flags do not work if the server does not support the same.

Supported Actions

Action	Description
Connect	Opens a persistent connection to the database.
Disconnect	Closes an existing database connection and performs any clean up actions if required.
GetMessage	Retrieves message into a message object.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Name for the incoming message

Supported Action Properties

There are no supported action properties.

Errors

Errors are generated when email messages cannot be sent or received due to various reasons. Typical errors include the following:

• Connection timeout

• Unable to disconnect from the email server

• Errors while sending email messages

• Errors while Receiving email messages

• Miscellaneous or unknown errors

All errors are written to an error log, which you can read to diagnose and troubleshoot the problem. The error messages specify whether a problem occurred while receiving or sending. The error log also contains error codes that help you to identify and understand the cause of the error.

This table describes error codes generated while sending or receiving email.

Code	Description
19	Unable to connect to the mail server
27	Unable to disconnect from the mail server
8	Unable to send email
4	Unable to receive email
50	Unknown error
1	End of File
12	No more messages to download
0	Messages sent or received successfully

File Folder Queue

File Folder Queue allows a file folder to be treated like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a Queue step. Message objects can also be written to the folder as files using the PutMessage action. The properties of the session control the files that are read and the names of the files that are written, as well as whether to preserve the message properties.

File Folder Queue Properties

Property	Default Value	Description
User name	-	User name to log in to the system where the file folder is available.
Password	-	Password to log in to the system.
Server Name	-	Name of the system where the file folder resides.
Directory		Folder for reading and writing files.
Working Directory	-	Directory name used to store temporary files that are created when retrieving files with the GetMessage action. If this property is not specified, the connector creates a directory called DJFTFile Folder Queue in the directory specified for the Directory property. This new directory is used to store temporary files.
Deserialize	FALSE	If a message object was written to a folder with the Serialize property set, the message properties are stored in a separate file with an .hdr extension. When reading a file from a folder using the GetMessage action, the Deserialize property indicates that the message properties must be read from the .hdr file.
Serialize	FALSE	When a message object is written to a file with the PutMessage action, only the message body is written. The Serialize property indicates whether to write the message properties to a separate file with an .hdr extension.
Pattern	*	Pattern used to match files to be read using the GetMessage action.
GetMsgClass	dat	The file extension that is appended to the file name when reading a file with the GetMessage Action.
PutMsgClass	dat	File extension that is appended to the file name generated by the FileFormat property when writing a message object to a file with the PutMessage action.
MessageTimeOut	5	Indicates the time to wait (in seconds) to retrieve the first file from the folder when retrieving files with the GetMessage action. The files to be retrieved are determined by the Pattern property. The MessageTimeOut property works with the SleepInterval property . The File Folder Queue tries to read the first file then sleep for the specified sleep interval. This continues based on the value in the MessageTimeOut option.
PollingTimeOut	5	Indicates the time to wait to retrieve all of the files from the folder when retrieving files with the GetMessage action. The files to be retrieved are determined by the Pattern property. The PollingTimeOut property works with the PollingTimeOutType property.
PollingTimeOutType	Minutes	Type of the polling timeout value. The options available are: • Years • Months • Days • Hours • Minutes • Seconds
SleepInterval	1000	Indicates the time to sleep when no messages are available to process. The sleep interval is in milliseconds.
BrowseMode	FALSE	If this property is set to TRUE, the messages read from the folder are not removed. Else, the files are deleted after processing.
FilenameFormat	MessageFile%n	When writing a message object to a file using the PutMessage action, this format is used to create the file name. The following format specifications can be used to build the format. Each specification begins with a percent sign (%). The specifiers represent properties on the message object that are created by the File Folder Iterator. Any or all of the specifications can be included in any order. Note: The format must generate unique names for the files. The supported formats are: • MessageFile%n • %i TransferId • %p PieceNumber • %s SourceName • %h TargetHostName • %t TargetName • %d Transport Start Date/Time • %n Number starting with 0 (zero) • %c Current date/time stamp displaying number of seconds elapsed since midnight GMT January 1, 1970 (value always contains 10 digits) Examples TargetName: TargetFFQFile PieceNumber: 5 FileFormat: %t%p MessageClass: dat Filename: TargetFFQFile5.dat
		Best Practice — The %n and %c format specifications enable you to create unique file names for generic messages. These should be used together to ensure greatest efficiency. If only %n is used in the file name specification, for example, MessageFile%n, the connector tries to assign the file name "MessageFile0" when a message is created. If this is unique it is used, but if it already exists in the specified directory, %n is incremented by 1 and this is checked against existing files. This continues until a unique file name is found. This means that the more messages that are created, the longer it takes for a unique name to be found. If only %c is used, the process is limited to writing one file per second as the file name it tries to assign for the new message is not unique until the timestamp has advanced. Therefore, use a combination of %n and %c. For example, MessageFile%c%n creates as many messages each second as required and %n is reset to 0 each time the timestamp changes.
Source Encoding	ENC_OEM or 0	Character set of the source data. This indicates using the default character set for the system. The ENC_ names can be used in expressions for setting the encoding value. On the integration engine command line, the number must be used.
Binary Indicator	FALSE	Indicates whether the files being read must be opened in binary mode.

Supported Actions

Action	Description
Connect	Opens a persistent connection to the database.
Disconnect	Closes an existing database connection and performs any clean up actions if required.
GetMessage	Retrieves message into a message object.
PutMessage	Writes a message object into the queue.

Supported Action Parameters

Action	Parameter	Description
GetMessage PutMessage	Message	Name for the incoming message

Supported Action Properties

There are no supported properties for the actions.

FTP Queue

The FTP Queue enables you to send and receive files using a secure or non-secure FTP connection. It allows a folder on an FTP site to be considered like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a Queue step.

Message objects can also be written to the folder as files using the PutMessage action. The properties control the files that are read and the names of the files that are written.

The primary functions of FTP Queue are:

• Connect to and disconnect from a secure or unsecure FTP server

• Retrieve a file from an FTP server and insert it in a message object

• Put a message object into the selected file on an FTP server

This component has the following versions:

• FTP Queue 1.0.0 - Provides an non-secure connection. While version 4.1.0 is recommended, 1.0.0 is supported for backward compatibility with existing processes.

• FTP Queue 4.1.0 - Provides either a secure or non-secure connection using one of the three FTP types:

– FTP (non-secure)

– SFTP (secure FTP over SSH)

– FTPS (secure FTP over SSL (certificate-based authentication))

Active and Passive Modes

FTP Queue 1.0.0 tries passive mode first. If it fails to connect, then it tries active mode.

FTP Queue 4.1.0 uses active mode for processing.

Encoding

In secure connections, FTP Queue 4.1.0 uses ISO-8859-1 to encode binary data.

FTP Queue 1.0.0 Properties

Property	Default Value	Description
User Name		User name to connect to the server.
Password		Password to connect to the server.
FTPSite		Name of the server to connect to. Type the FTP server name or IP address, port number, and directory in the following format: FTP_server:port/directory_name The port number and directory name are optional. If you specify the port number, the component uses the default port number for the selected FTP server type.
Serialize	False	When a message object is written to a file with the PutMessage action, only the message body is written. The Serialize property indicates whether to write the message properties to a separate file with an .hdr extension.
Deserialize	False	If a message object was written to a folder with the Serialize property set, the message properties are stored in a separate file with an .hdr extension. When reading a file from a folder using the GetMessage action, the Deserialize property indicates that the message properties must be read from the .hdr file.
RetryCount	0	Number of times to retry if there is a failure to connect to the FTP site.
FilenameFormat	MessageFile%n	When writing a message object to a file using the PutMessage action, this format is used to create the file name. The following format specifications can be used to build the format. Each specification begins with a percent sign (%). The specifiers represent properties on the message object that are created by the File Folder Iterator. Any or all of the specifications can be included in any order. Note: The format must generate unique names for the files. The supported formats are: • MessageFile%n • %i TransferId • %p PieceNumber • %s SourceName • %h TargetHostName • %t TargetName • %d Transport Start Date/Time • %n Number starting with 0 (zero) • %c Current date/time stamp displaying number of seconds elapsed since midnight GMT January 1, 1970 (value always contains 10 digits) Examples TargetName: TargetFFQFile PieceNumber: 5 FileFormat: %t%p MessageClass: dat Filename: TargetFFQFile5.dat
		Best Practice — The %n and %c format specifications enable you to create unique file names for generic messages. These should be used together to ensure greatest efficiency. If only %n is used in the file name specification, for example, MessageFile%n, the connector tries to assign the file name "MessageFile0" when a message is created. If this is unique it is used, but if it already exists in the specified directory, %n is incremented by 1 and this is checked against existing files. This continues until a unique file name is found. This means that the more messages that are created, the longer it takes for a unique name to be found. If only %c is used, the process is limited to writing one file per second as the file name it tries to assign for the new message is not unique until the timestamp has advanced. Therefore, use a combination of %n and %c. For example, MessageFile%c%n creates as many messages each second as required and %n is reset to 0 each time the timestamp changes.
GetMsgClass	dat	The file extension that is appended to the file name when reading a file with the GetMessage Action.
PutMsgClass	dat	File extension that is appended to the file name generated by the FileFormat property when writing a message object to a file with the PutMessage action.
Working Directory		Directory name used to store temporary files that are created when retrieving files with the GetMessage action.
Pattern	*	Pattern used to detect files to be read, such as File* or *95. To retrieve a specific file, type the exact file name without the extension. The extension is read from the File Extension option.
MessageTimeOut	5	Indicates the number of times to query the FTP folder to get the desired message. Default is 5. This option works with the SleepInterval option.
PollingTimeOut	5	Indicates the maximum total time to take to retrieve all files from the folder. Use this option with the Polling TimeOut Type option.
PollingTimeOutType	Minutes	Type of the PollingTimeOut value. The available options are: • Years • Months • Days • Hours • Minutes • Seconds
SleepInterval	1000	Time interval, in milliseconds, to wait between queries if there are no messages in the queue to process. This option works with the Message TimeOut option.
BrowseMode	False	Indicates whether the file on the FTP server from which the message is read is deleted. If Browse Mode is true, the file is not deleted.
SourceEncoding	0	Character set of the source data. This indicates using the default character set for the system. The ENC_ names can be used in expressions for setting the encoding value. On the integration engine command line, the number must be used.
Binary Indicator	False	Indicates whether the files being read should be opened in binary mode.

Supported Actions for FTP Queue 1.0.0

Action	Description
Connect	Establishes a connection to an FTP server, which is useful when you are testing your configuration.
Disconnect	Breaks the connection with an FTP server, which is useful when you need to disconnect before a process is complete.
GetMessage	Connects to an FTP server, retrieves the contents of a file into a message object, then disconnects. The GetMessage action sets the SourceName message property to the name of the file from which it retrieved the message.
PutMessage	Connects to an FTP server, writes a message object to the specified file, then disconnects.

Supported Action Parameters for FTP Queue 1.0.0

Action	Parameter	Description
GetMessage, PutMessage	Message	Name of the message.

FTP Queue 4.1 Properties

Property	Default Value	Description
User Name		User name to connect to the server.
Password		Password to connect to the server.
Server Name	localhost	Name of the server to connect to. Type the FTP server name or IP address, port number, and directory in the following format: FTP_server:port/directory_name The port number and directory name are optional. If you do not specify the port number, the component uses the default port number for the selected FTP server type.
FTP Type	FTP	Specify the type of FTP: • FTP • SFTP • FTPS
FTP Connection Mode		Connection mode is either Active (the default) or Passive. Note: This is not applicable for SFTP.
Message Payload Type		Select File Contents (the default) or File Location: • File Contents: If the actions are: – If GetMessage is the , contents of the retrieved file are stored in the DJMessage. – If PutMessage, contents of the DJMessage are sent to the remote FTP site and stored as a file. • File Location: If actions are: – GetMessage - File retrieved from the remote FTP site is stored directly as a file. – PutMessage - Local file contents are sent to the remote FTP location.
Encoding Type	ISO8859-1	Select ISO8859-1 or UTF-8. Note: This is not applicable for SFTP.
Binary Indicator	False	Indicates whether the files being read must be opened in binary mode.

Supported Actions for FTP Queue 4.1.0

Action	Description
Connect	Establishes a connection to an FTP server, which is useful when you are testing your configuration.
Disconnect	Breaks the connection with an FTP server, which is useful when you need to disconnect before a process is complete.
GetMessage	Connects to an FTP server, retrieves the contents of a file into a message object, then disconnects. The GetMessage action sets the SourceName message property to the name of the file from which it retrieved the message.
PutMessage	Connects to an FTP server, writes a message object to the specified file, then disconnects.

Supported Action Parameters for FTP Queue 4.1.0

Action	Parameter	Description
GetMessage, PutMessage	Message	Name of the message.

Supported Action Properties for FTP Queue 4.1.0

Action	Property	Default Value	Description
GetMessage	Target Directory		Target directory for storing the retrieved file on the local machine.
	Pattern	*	Pattern used to detect files to be read. To retrieve a specific file, type the exact file name with or without the extension. The extension is read from the File Extension option. The pattern is case sensitive. The Pattern properties supports the use of regular expressions. Default is . For example, File, 95, File?, ?95('' matches any character sequence and '?' matches any character).
	File Extension		File extension to append to the file name when retrieving a file and inserting into a message object. Use values such as txt and asc. Do not enter the dot (.), the extension string.
	Browse Mode	False	Indicates whether the file on the FTP server from which the message is read is deleted. If Browse Mode is true, the file is not deleted.
	Message Timeout	5	Indicates the number of times to query the FTP folder to get the desired message. Default is 5. This option works with the SleepInterval option.
	Sleep Interval	1000	Time interval, in milliseconds, to wait between queries if there are no messages in the queue to process. This option works with the Message TimeOut option.
	Polling TimeOut	5	Indicates the maximum total time to take to retrieve all files from the folder. Use this option with the Polling TimeOut Type option.
	Polling TimeOut Type	Minutes	Type of the PollingTimeOut value. The available options are: • Years • Months • Days • Hours • Minutes • Seconds
PutMessage	Remote Directory		Remote location for storing the file on the FTP Server.
	Filename Format		Format for the file name written to the message object. Various format specifications are supported. Each specification begins with a %. Any or all specifications can be placed consecutive to each other. However, you may not place text after a format specification. For example, MessageFile%n%s is valid, but MessageFile%nTest is invalid. Make sure that the format generates a unique name for each file. The default format is MessageFile%n, which produces the file name MessageFile0. The supported format specifications are: • %n - Inserts a number, starting with 0 (zero). • %c - Inserts current date/time stamp, displaying number of seconds elapsed since midnight GMT January 1, 1970. The value has 10 digits. • %s - Inserts the source file name, which can be specified as a message property. • %t - Inserts the target file name, which can be specified as a message property.
	File Extension		File extension to append to the file name when writing a message object to a file. Use values such as txt and asc. Do not enter the period, only the extension string.
	Use Temporary Files
	Local File		Local path to the location where the file exists.

Errors

The following table provides possible FTP Queue error codes.

Error Code	Description
4	Error occurs while trying to get a message from the FTP server.
8	Error occurs while trying to add a message on the FTP server.
12	Error occurs because the GetMessage or PutMessage actions cannot find the specified file.
19	Error occurs while trying to connect to the FTP server.
27	Error occurs while trying to disconnect from the FTP server.

The process log may also contain messages related to FTP Queue.

Error Code	Description
28047	An error occurred executing a queue step. This run-time error can result from a change in the server certificate after you deploy your integration application. View the process log for the following message: "Error connecting to FTP Server: Could not validate server certificate. Please make sure you run TestConnect and save the process." To resolve this error, do one of the following. • Obtain a new certificate and add it manually to the client trust store file. • In the process file, use the Test Connect option to connect to the FTP server so that it prompts you to accept the new certificate.

Error Code

Description

28047

An error occurred executing a queue step.

This run-time error can result from a change in the server certificate after you deploy your integration application. View the process log for the following message:
"Error connecting to FTP Server: Could not validate server certificate. Please make sure you run TestConnect and save the process."

To resolve this error, do one of the following.

• Obtain a new certificate and add it manually to the client trust store file.

• In the process file, use the Test Connect option to connect to the FTP server so that it prompts you to accept the new certificate.

Limitations

• The file transfer is bound by the amount of physical memory and heap size.

• If you are transferring binary files, you must encode the data in the message object in Base64. When you receive files with GetMessage in binary mode, the files will be binary, and will not be encoded in Base64.

Google Cloud Storage Queue

The Google Cloud Storage Queue component facilitates integration with Google Cloud Platform (GCP) Storage services. The key component features are:

• Specify the name of the GCP Storage bucket which will store the objects being written and read. Optionally, create a bucket if missing.

Note: A missing bucket will be created only if the component uploads content to the bucket. A new bucket will not be created if the component is downloading content.

• Specify the storage location (region) of the bucket.

• Select authentication methods:

• Supports Service account keys contained within a file, which is provided by the GCP Storage account owner

• Supports the following schemes for finding the key file:

– Explicitly specify the file path

– Implicitly discover the file through the GOOGLE_APPLICATION_CREDENTIALS environment variable

• Upload files or DJMessage content to GCP Storage buckets

• Download "objects" from GCP Storage buckets to DJMessages or files

For GCP overview, see https://cloud.google.com/. For information about GCP Storage services, see https://cloud.google.com/storage.

Google Cloud Storage Queue Property Descriptions

Property	Default Value	Description
Bucket Location		Location for storing your object data when you create a bucket. The location types are: • Region: Specific geographic place, such as London. • Dual-region: Specific pair of regions, such as Finland and the Netherlands. • Multi-region is a large geographic area, such as the United States, that contains two or more geographic places. Objects stored in a multi-region or dual-region are geo-redundant. Cloud Storage stores object data in the selected location in accordance with the Service Specific Terms. For more information about locations, see https://cloud.google.com/storage/docs/locations.
Bucket Name		GCP Storage bucket that will be used to upload and retrieve stored objects. The value must match the name of an existing GCP Storage bucket for which the service account has appropriate privileges. If the name does not match an existing bucket, then an error occurs unless the Create missing bucket option is set to true. For more information about GCP Storage buckets, see https://cloud.google.com/storage/docs/key-terms.
Create Missing Bucket	False	If set to True, the component creates the specified bucket (if it is missing) when attempting to upload (PutMessage action) content to the specified bucket. This setting has no effect if the component is only downloading content. The default value is False.
Bucket Storage Class	Standard Storage	When creating a missing bucket, the component must know the GCP Storage class to be used. The options are: • Standard Storage • Nearline Storage • Coldline Storage • Archive Storage For more information about storage classes, see https://cloud.google.com/storage/docs/storage-classes.
Authentication Type	Use path in environment variable (GOOGLE_APPLICATION_CREDENTIALS)	Select any of the following authentication type: • Use path in environment variable (GOOGLE_APPLICATION_CREDENTIALS): Instructs the component to configure the GCP client SDK in a way, which instructs it to implicitly find the service account key file using the GOOGLE_APPLICATION_CREDENTIALS environment variable. If the environment variable is set, then its value is expected to be the path to the file. If the environment variable is not set, then an error occurs. • Provide path to service account key file: The component will display the Service Account Key File option, which allows you to specify the complete path of the service account key file.

Supported Actions

Action	Description
Connect	Creates an authenticated GCP Storage client, which will be used to perform all subsequent operations. The client is stateless, so it is not actually "connected."
Disconnect	Disconnects from the authenticated client.
GetMessage	Uses the authenticated client to download data from a GCP Storage bucket. The data in the bucket is referenced using an object name. The downloaded content can be written to a file or injected into a DJMessage object which is provided to the GetMessage action.
PutMessage	Uses the authenticated client to upload data to a GCP Storage bucket, saving it using the provided object name. This object name is equivalent to an S3 "key" or local "file name". The content to be uploaded can be provided to the PutMessage action via DJMessage or a referenced file.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Name of a DJMessage to which the downloaded file will be written if a local file was not specified.
PutMessage	Message	Name of a DJMessage from which data will be uploaded if a local file was not specified.

Supported Action Properties

Action	Property	Description
GetMessage	Object Name	Name of the object to retrieve from GCP Storage bucket. The name is unique within the bucket and may include separator characters, such as slash ("/") to emulate folders. This option is required.
	Local File Name	Click Browse and specify a local file, to which the component will write the data retrieved from the bucket.
	Bytes Limit	Number of bytes to download from the object in the bucket. This is useful for sampling. Leave this blank if you want to download the entire object.
PutMessage	Object Name	Name of the GCP Storage object to be written. The name is unique within the bucket and may include separator characters, such as slash ("/" ) for emulating folders. If left blank, then the component expects to find an override property called "ObjectName" within the DJMessage. It is an error condition if the option is left blank and the override property is missing from the DJMessage.
	Local File Name	Click Browse and specify the local file from where the component will read the data that it uploads to the bucket. The value of this option (whether it is blank or populated) is overridden by the LocalFile DJMessage property. If neither value is set, then the data that will be uploaded is read from the DJMessage body.
	MIME Type	MIME type that will be assigned to the object after it is uploaded to the bucket. This is a useful option if the content is being uploaded from the body of a DJMessage. It can be overridden through the DJMessage property MimeType. Note: MIME Type will be automatically determined when uploading from a local file. A limited list of MIME types is available for selection. However, you can specify any MIME type supported by GCP Storage. The available MIME Types are: • text/xml • text/html • text/plain • application/json • application/xml • application/octet-stream
	Replace Existing Object	If set to True, the existing object is replaced on GCP Storage. The default value is False.

Override Queue Properties

You can override the values provided for the object name and local file name in the PutMessage action using the following DJMessage properties:

• ObjectName: This header property can be set in the DJMessage argument for the PutMessage action. If provided, it overrides any value that is set in the Object Name option.

• LocalFile: If the DJMessage argument for the PutMessage action has a header property with this name, then its value will override any value that is set in the Local File Name option. If both are missing, then the upload data will be read from the DJMessage body.

• MimeType: This header property can be set in the DJMessage argument for the PutMessage action. If provided, it overrides the value set in the MIME Type option.

• ReplaceFile: If the DJMessage argument for the PutMessage action has a header property with this name, then its value will override any value that is set in the Replace Existing Object option. If neither value is set, then the default value is False.

Errors

The following table describes the error codes generated for the Google Cloud Storage Queue component.

Action	Code	Description
Connect	ERR_OK	Connect completed successfully without any errors
Connect	ERR_OPENERR	Connect failed. Typically associated with LT_ERROR log level.
Disconnect	ERR_OK	Disconnect completed successfully.
	ERR_CLOSERR	Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
GetMessage	ERR_OK	File retrieved and saved (to file or DJMessage) without any errors.
	ERR_BADOPTIONVALUE	GCP storage file name is not set. This is associated with error-level log message.
	ERR_READERR	Local file or referenced file already exists. This is associated with error-level log message.
PutMessage	ERR_OK	File uploaded to the GCP Storage bucket without any errors.
	ERR_BADOPTIONVALUE	Object name property must be set. This is associated with error-level log entry
	ERR_WRITERR	Error copying data from DJMessage. Google-generated error occurred when sending or writing data to the remote GCP Storage account. A number of messages can be generated and all these messages are associated with error-level log entry.

Google Drive Queue

Google Drive is a personal cloud storage service from Google that allows you to store and synchronize digital content across computers, laptops and mobile devices, including Android-powered tablets and smart phone devices. Once permissions are granted by the owner of a Google Drive account, the component facilitates uploading or downloading files to the account.

Google Drive Queue Property Descriptions

Property	Default Value	Description
Token Folder		Local directory that will be used to store an OAuth token after the owner of the Google Drive account has granted the requested permissions to the component. Click Browse and select a folder. You can also right-click and use the Paste Macro option to specify the token folder. After specifying this folder, you must test the connection and obtain a valid token for connecting to Google Drive. For the steps, see Obtaining Valid Token for Connecting to Google Drive. The connection is successful if a valid token file is created in the Token Folder.
Delete Uploaded Files When Finished	False	Automatically deletes all the uploaded files after the process completes. The options are: • True • False

Obtaining Valid Token for Connecting to Google Drive

The Test Connection option helps you to obtain a valid token for connecting to Google drive.

To test the connection and obtain a valid token, perform the following:

1. In the Google Drive queue component properties, in the Token Folder, specify a path. This is the folder where the granted OAuth token file will be created. For example, C:\Temp.

2. Click Test Connection.

The Google Sign in page opens with the following message:

Choose an account to continue to Actian DataConnect

3. Select your Google or Gmail account that you want to connect with Actian DataConnect.

The Grant Actian DataConnect permission dialog is displayed asking for permission to view, edit, create, and delete all of your Google Drive files.

4. Click Allow.

The following message is displayed:

View and manage metadata of files in your Google Drive files

5. Click Allow.

The following message is displayed:

Confirm your choices

6. Click Allow.

The following message is displayed:

Received verification code

7. Click OK and then click Save.

The Google Drive Queue component is ready to use.

8. Open file explorer and go to the folder that you had specified as the Token Folder. You will see a new file called StoredCredential is created. This is the granted token file for your Google Drive account.

Supported Actions

Action	Description
Connect	Establishes a connection to the authorized Google Drive account.
Disconnect	Disconnects from the authorized Google Drive account.
GetMessage	Downloads a file from the authorized Google Drive account to either a DJMessage or a local file.
PutMessage	Uploads a file to the authorized Google Drive account from a DJMessage or local file.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	Name of a DJMessage to which the file downloaded file will be written if a local file was not specified.
PutMessage	Message	Name of a DJMessage from which data will be uploaded if a local file was not specified.

Supported Action Properties

Action	Property	Description
GetMessage	Google Drive File	Name of the file to retrieve from Google Drive. You can specify the full path to the file if it is a folder within one or more directories. For example, if the accounts.txt file is within the data directory, which is in a another directory called company, then the path to the file is company/data/accounts.txt.
	Local File	Click Browse and specify a local file, to which the downloaded Google Drive file will be written. Leave blank to write the downloaded content to this step's DJMessage.
	Bytes Limit	Number of bytes to download from the selected Google drive file. This is useful for sampling. Leave this blank if you want to download the entire object.
	Delete File After Retrieval	If set to True, then after successfully downloading the file, delete the file from Google drive. The default value is False.
PutMessage	Google Drive File	Name of the Google Drive file to be written. This can be overridden through the DJMessage property DriveFile.
	Local File	Click Browse and specify the local file that will be copied to the specified Google Drive file. Leave it blank if you want to upload the content of the DJMessage body.
	MIME Type	Specify the MIME type of the content that will be uploaded to Google Drive. This is a useful option if the content is being uploaded from the body of a DJMessage. It can be overridden through the DJMessage property MimeType. Note: MIME Type will automatically be determined when uploading from a local file. A limited list of MIME types is available for selection. However, you can specify any MIME type supported by Google Drive. The available MIME Types are: • text/xml • text/html • text/plain • application/json • application/xml • application/octet-stream
	Replace Existing File	If set to True, the existing file is replaced on Google Drive. The default value is False.

Override Queue Properties

You can override the values provided for the object name and local file name in the PutMessage action using the following DJMessage properties:

• DriveFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides any value that is set in the Google Drive File option.

• LocalFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Local File option. If both are missing, then the uploaded data is read from the DJMessage body.

• MimeType: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the MIME Type option.

• ReplaceFile: This header property can be set in the DJMessage argument to the PutMessage action. If provided, it overrides the value set in the Replace Existing File option.

Errors

The following table describes the error codes generated for the Google Drive Queue component.

Action	Code	Description
Connect	ERR_OK	Connect completed successfully without any errors
Connect	ERR_OPENERR	Connect failed. Typically associated with LT_ERROR log level.
Disconnect	ERR_OK	Disconnect completed successfully.
	ERR_CLOSERR	Associated with warning-level log message. Disconnect action is called after the component is already disconnected.
	ERR_DELETERR	If the component is configured to delete uploaded files from Google Drive during connect, then this error code is generated when one or more files that are deleted resulted in error. This is associated with warning-level log message.
GetMessage	ERR_OK	File retrieved and saved (to file or DJMessage) without any errors.
	ERR_BADOPTIONVALUE	Google Drive file name is not set. This is associated with error-level log message.
	ERR_READERR	Local file or referenced file already exists. This is associated with error-level log message.
	ERR_DELETERR	Error in copying response content to the provided DJMessage. This is associated with error-level log message. Too many errors generated by the Google Drive SDK produces this error. The error code will include a message appropriate to the error.
PutMessage	ERR_OK	File uploaded to the Google Drive account without any errors.
	ERR_BADOPTIONVALUE	Google Drive file name property must be set. This is associated with error-level log entry
	ERR_WRITERR	Error copying data from DJMessage. Google-generated error occurred when sending or writing data to the remote Google Drive account. A number of messages can be generated and all these messages are associated with error-level log entry.

JMS Queue

JMS queues are components based on the Java Message Service. The components allows you to connect to JMS-compliant queues. The available JMS queue component is JMS Queue 2.0.0. If you want to use client jars with this component, you must use JMS 2.0.0. The JMS Queue 2.0.0 is a Java component that directly implements the JMS API.

Note: The JMS 1.0.0 component is deprecated.

Make sure to set the Java ClassPath option in the cosmos.ini file when using JMS Queue.

Use the following information to migrate from JMS 1.0.0 to JMS 2.0.0.

Component Version	Display Name	Actual Option Name Used in Process	Option Type
1.0.0 and 2.0.0 (component configuration)	User Name	com.pervasive.di.queue.connection.userid	text
	Password	com.pervasive.di.queue.connection.password	password
	java.naming.factory.initial	contextfactory	text
	javax.jms.connectionfactory	connectionfactory	text
	java.naming.provider.url	providerurl	text
	PropertiesFile	propertiesfile	file
	OutputMessageType	msgtype	enum
	MessageSelector	msgselector	text
	Timeout	timeout	number
1.0.0 (component configuration)	Server Name	com.pervasive.di.queue.connection.server	text
	JMSCorrelationID	jmscorrelationid	text
	JMSReplyTo	jmsreplyto	text
	JMSType	jmstype	text
2.0.0 (component configuration)	JarLocation	jarlocation	text
2.0.0 (configured with PutMessage action in step)	JMSCorrelationID	jmscorrelationid	text
	JMSReplyTo	jmsreplyto	text
	JMSType	jmstype	text

Note: Possible values for msgtype are-javax.jms.TextMessage, javax.jms.BygtesMessage, javax.jms.StreamMessage
JarLocation is a single directory that contains all client jars.

JMS Queue Property Descriptions

Property	Default Value	Description
User Name		User name used to connect with host.
Password		Password used to connect with host.
Messaging Styles	PTP	Select one: • PTP (Point-to-Point) • Publish-Subscribe
java.naming.factory.initial		Fully qualified class name of the initial context factory for the JNDI service provider.
javax.jms.connectionfactory		JNDI lookup name of the JMS Queue connection factory.
java.naming.provider.url		URL for the JNDI provider-specific configuration information.
PropertiesFile		Specifies the name of a properties file that contains the JMS Queue configuration information. You can either specify the three properties above separately, or place them in a properties file.
OutputMessageType	javax.jms.TextMessage	Type of JMS Queue message to create when writing messages. • javax.jms.TextMessage (default) • javax.jms.BytesMessage • javax.jms.StreamMessage
MessageSelector		JMS Queue message selector to use for reading messages.
JarLocation		Location of the .jar file.
Timeout	0	Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.

Supported Actions

Action	Description
Connect	Establishes a connection to the server.
Disconnect	Disconnects from the server.
GetMessage	Retrieves message into a message object.
PutMessage	Writes a message object into the queue.
BeginTransaction	Starts a JMS Queue transaction.
CommitTransaction	Commits a JMS Queue transaction.
RollbackTransaction	Rolls back a JMS Queue transaction.

Supported Action Parameters

Action	Parameter	Description
Connect	Queue	Name of the queue used to connect to.

Supported Action Properties

There are no action properties displayed for JMS Queue component.

Note: For more information on JMS, see the JMS API documentation.

Kafka Micro-batch Queue

Kafka is a distributed, partitioned, and replicated commit log service. Kafka maintains feeds of messages in categories called topics. The processes that publish messages to a Kafka topic are called producers. The processes that subscribe to topics and process the feed of published messages are called consumers. Kafka is run as a cluster comprised of one or more servers. Each server is called a broker. The producers send messages over the network to the Kafka cluster, which in turn sends them up to consumers.

The Kafka writer provides the ability to send data to Kafka from batch or real-time invoked processes without the expectation of handling big data volumes. Kafka provides a client Producer API for the user to write data to Kafka. The Producer API version used is 2.2.1.

The following features are supported:

• Reading messages from Kafka topic: The GetMessage action of the component and the associated action properties such as Group ID, Topic, and so on is used to read the messages from the Kafka topic. The messages read from the topic will be inserted into the body of the DJMessage object and the JSON file (if specified in File Name property). The JSON format or structure is as shown.

where:

• recordCount: Represents number of messages/records fetched from the Kafka topic

• topic: Name of the topic from where the messages are fetched.

• records: Records array contains message objects. Each message object consists of key, offset, partition, timestamp, value of the message fetched from Kafka topic.

• Writing messages to Kafka topic: The PutMessage action of the component is used to write the message to a topic in a Kafka queue. The messages to be written must be attached to the body of the DJMessage or must be provided in the JSON file (specified in File Name property). If the JSON file is specified, then this file will be used to write the messages to the Kafka topic. Otherwise, the body or contents of the DJMessage object will be used.

JSON File or contents (body) of the DJMessage object must be in the same format as shown in the preceding image. The only property required is Value, which contains the message to be written to Kafka Topic. The other propertied are not required. The structure is as shown.

Kafka Micro-batch Queue Properties

Property	Default Value	Description
Bootstrap Servers		List of host or port pairs to use for establishing the initial connection to the Kafka cluster. The format is host1:port1,host2:port2.

Supported Actions

Action	Description
Connect	Establishes a connection to the server.
Disconnect	Disconnects from the server.
GetMessage	Reads a message object from the queue.
PutMessage	Writes a message object into the queue.

Supported Action Parameters

Action	Parameter	Description
GetMessage	Message	The message that must be read.
PutMessage	Message	The message that must be written.

Supported Action Properties

Action	Property	Description
GetMessage	Group ID	Consumer group ID to which the consumer belongs.
	Topic	Kafka message topic name from where the messages must be read.
	Partition Key	Partition key in the Topic from where the messages must be read. If you do not know this, leave it as blank.
	Poll Duration	Maximum time to block in milliseconds. This is the time required to reach the specified Kafka Server. It blocks the process execution until the fetch request reaches the Kafka Server and records are fetched. The value should not be greater than Long.MAX_VALUE milliseconds. Default value is 5000. Note: Long.MAX_VALUE is equal to 2^64 minus 1, which is 9,223,372,036,854,775,807 milliseconds.
	Max Poll Records	Maximum number of records to read. Default value is 1000.
	Auto Offset Reset	The offset to start reading messages from when a Group ID is specified, for which there is no committed offset with Kafka Server. The options are: • earliest: Starts reading messages from the beginning of the queue. • latest: Starts reading messages from the end of the queue (if the queue is not receiving any messages, the records or messages fetched will always be 0 in this case). • none: Throws an exception if no previous offset is found for the given Group ID. Default value is earliest.
	Enable Auto Commit	If set to True, then the latest offset of messages fetched will be committed at the interval specified by the Auto Commit Interval. Default is True.
	Auto Commit Interval	Auto commit the offset of the last read message or record to the Kafka Server (in milliseconds). Default is 5000.
	Key Deserializer	Format in which the bytes returned by the server for Key is deserialized. Default value is String Deserializer.
	Value Deserializer	Format in which the bytes returned by the value for Key should be deserialized. The options are: • String Deserializer • Kafka Avro Deserializer • JSON Deserializer Default value is String Deserializer.
	Schema Registry URL	Specify this option only if you are using Confluent Stream Platform as Kafka Server and the selected Value Deserializer is Kafka Avro Deserializer.
	File Name	Filename with the path, where the messages pulled from Kafka Queue must be populated. A JSON file having format mentioned in the description above.
PutMessage	Topic	Kafka message topic name to where the messages must be sent or written.
	Partition Key	Partition key in the Topic to where the messages must be sent.
	Acks	Number of acknowledgments the producer requires the leader to have received before considering a request complete. The options are: • 0: Producer does not wait for any acknowledgment from the server. • 1: Leader writes the record to its local log but responds without awaiting full acknowledgement from all followers. • all: Leader waits for the full set of in-sync replicas to acknowledge the record. Default value is 0.
	Buffer Memory	Total bytes of memory the producer can use to buffer records waiting to be sent to the server. Default value is 33554432
	Retries	If the value is greater than zero, the client resends any record when the previous send has failed with a potentially transient error. Default value is 1
	Batch Size	Producer attempts to batch records together into fewer requests whenever multiple records are being sent to the same partition. Default value is 16384.
	Key Serializer	Format in which the key of the message to be written to the server must be serialized. Default value is String Deserializer
	Value Serializer	Format in which the message written to the server must be serialized. The options are: • String Deserializer • JSON Deserializer Default value is String Deserializer.
	File Name	Filename with path from where the messages must be read. A JSON file having format mentioned in the description above.

Error Conditions

The following table describes error codes generated while sending a message to Kafka.

Error Condition	Code	Description
No error	0	The status is OK. There is no error. The information is returned successfully.
Missing license	46	Component Kafka micro-batch component is not licensed. The component requires DataConnect professional license. Contact your Actian sales representative to obtain the licensing information.
Missing broker list	19	Error detected when establishing a connection. The Bootstrap Servers property is required.
Error during connect	19	Exception 'xxx' occurred while attempting to connect to Kafka server 'xxx'.
Missing topic	33	Error detected when sending or receiving a message to or from a topic. A topic is required.
Error when sending a message	8	Exception 'xxx' occurred while attempting to put message to a topic 'xxx'.
Missing Group ID	33	Error in GetMessage. Group ID is required
Missing Schema Registry URL	33	Error in GetMessage. Schema Registry URL is required when using 'Kafka Avro Deserializer' as 'Value Deserializer'.

Note: For more information about Kafka, see the Kafka documentation available at http://kafka.apache.org.

Kafka Writer Queue

The following features are supported:

• All Kafka producer configuration properties except for some properties that are not related to data write properties such as metric.reporters, metrics.num.samples, metrics.sample.window.ms.

• Separate actions for producer connect, data write, and producer disconnect. Therefore, you can use one connection for multiple data write actions.

• Allows you to change topics for each data write action.

Kafka Writer Queue Properties

Property	Default Value	Description
Bootstrap Servers		List of host or port pairs to use for establishing the initial connection to the Kafka cluster. The format is host1:port1,host2:port2.
Acks	1	Number of acknowledgments the producer requires the leader to have received before considering a request complete. The allowed values are: • 0 - Producer does not wait for any acknowledgment from the server. • 1 - Leader writes the record to its local log but responds without awaiting full acknowledgement from all followers. • 2 - Producer requires the given number of acknowledgments but this is generally less useful. • all - Leader waits for the full set of in-sync replicas to acknowledge the record.
Buffer Memory	33554432	Total bytes of memory the producer can use to buffer records waiting to be sent to the server.
Compression Type	none	Compression type for all data generated by the producer. The allowed values are: • none • gzip • snappy
Retries	3	If the value is greater than zero, the client resends any record when the previous send has failed with a potentially transient error.
Batch Size	16384	Producer attempts to batch records together into fewer requests whenever multiple records are being sent to the same partition.
Client Id		User-specified string sent in each request to help trace calls.
Linger	0	Producer groups together any records that arrive in between request transmissions into a single batched request. The value is in milliseconds.
Max Request Size	1048576	Maximum size (records) of a request.
Receive Buffer	32768	Size of the TCP receive buffer to use when reading data.
Send Buffer	131072	Size of the TCP send buffer to use when sending data.
Timeout	30000	Configuration controls the maximum amount of time the server waits for acknowledgments from followers to meet the acknowledgment requirements the producer has specified with the acks configuration. The value is in ms.
Block On Buffer Full	True	When the memory buffer is exhausted, it should either stop accepting new records (block) or throw errors.
Metadata Fetch Timeout	60000	When data is sent to a topic first time, the metadata about that topic is required to know the servers that host the topic's partitions. This configuration controls the maximum amount of time to wait for the metadata fetch to succeed before throwing an exception back to the client. The value is in ms.
Metadata Max Age	300000	Amount of time in milliseconds after which force a refresh of metadata even if any partition leadership changes are not seen to proactively discover any new brokers or partitions. The default value is in ms.
Reconnect Backoff	10	Amount of time to wait before attempting to reconnect to a host when a connection fails. The value is in ms.
Retry Backoff	100	Amount of time to wait before attempting to retry a failed produce request to a particular topic partition. The value is in ms.

Supported Actions

Action	Description
Connect	Establishes a connection to the server.
Disconnect	Disconnects from the server.
PutMessage	Writes a message object into the queue.

Supported Action Parameters

Action	Parameter	Description
PutMessage	Message	The message that must be written.

Supported Action Properties

Action	Property	Description
PutMessage	Topic	Kafka message topic
PutMessage	Partition Key	Message partition key

Error Conditions

The following table describes error codes generated while sending a message to Kafka.

Error Condition	Code	Description
No error	0	The status is OK. There is no error. The information is returned successfully.
Missing license	46	Component Kafka writter is not licensed. The component requires DataConnect professional license. Contact your Actian sales representative to obtain the licensing information.
Missing broker list	4	Error detected when establishing a connection. The Bootstrap Servers property is required.
Error during connect	4	Exception 'xxx' occurred while attempting to connect to Kafka server 'xxx'.
Missing topic	4	Error detected when sending a message to a topic. A topic is required.
Error when sending a message	8	Exception 'xxx' occurred while attempting to put message to a topic 'xxx'.

Note: For more information about Kafka, see the Kafka documentation available at http://kafka.apache.org.

Websphere MQ Queue

WebSphere MQ Queue allows a message to be put into the queue or retrieved from a queue to a message object. Use the PutMessage action to write to the queue. Use the GetMessage action in a Queue step to retrieve messages. Session properties control which message objects are read and the names of the messages that are written, as well as whether to preserve the message properties.

Supported versions:

• DataConnect version <= 11.3.x supports WebsphereMQ client 7.0.0-7.5.x.x using the Websphere MQ 1.0.0 component

• DataConnect version >= 11.4.x supports WebsphereMQ client 8.x.x (only) using the Websphere MQ 1.0.0 component

• DataConnect version >= 11.5.2 supports IBM MQ >= 9.x.x using the IBM MQ 1.0.0 component

• MQ client 8.0.0.2 distributes libraries for versions 2005 and 2012. You can install this version of client then update their path to use either library, depending on which version of DataConnect you need to use.

For more information about the provision of both 2005 and 2012 versions of the native API client libraries in WebSphere MQ 8.0.0.2 and higher, see the following link:
https://www.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.pro.doc/q130070_.htm

Websphere MQ Queue Properties

Property	Default Value	Description
Format	MQFMT_NONE	WebSphere MQ message format.
Timeout	0	Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.
Browse	False	Retrieve messages from a queue without removing them.
Convert	False	Converts the message data to a specific character set or numeric encoding. Requires CCSID or Encoding value.
CCSID	MQCSSI_DEFAULT	Character set identifier of the message data.
Encoding		Encoding for numeric data. The representation used for binary integers, packed-decimal integers, and floating-point numbers.

Supported Actions

Action	Description
Connect	Establishes a connection to the WebSphere MQ server.
Disconnect	Disconnects from the WebSphere MQ server.
GetMessage	Retrieves message into a message object.
PutMessage	Writes a message object into the queue.
Begin Transaction	Starts a WebSphere MQ transaction.
Commit Transaction	Commits a WebSphere MQ transaction.
Rollback Transaction	Rolls back a WebSphere MQ transaction.

Supported Action Properties

Action	Property	Description
PutMessage GetMessage	Format	WebSphere MQ message format, with allowable values defined by WebSphere MQ
PutMessage GetMessage	Msgid	Message ID of a specific message put in or retrieve from a queue
PutMessage GetMessage	Correlationid	Correlation ID of messages to put in or retrieve from a queue
PutMessage GetMessage	Groupid	Group ID of message to put in or retrieve from a queue
PutMessage GetMessage	CCSID	Character set that you can convert to. Used with the Convert option
PutMessage GetMessage	Encoding	Numeric encoding you can convert to. Used with the Convert option
GetMessage	SequenceNumber	Sequence number of a specific message to retrieve from a queue
PutMessage	Offset	Segment offset
PutMessage	ReplyToQ	Reply queue
PutMessage	ReplyToQMgr	Reply queue manage
PutMessage	UserID	User identifier
PutMessage	AccountingToken	Accounting token
PutMessage	ApplIdData	Application identity data
PutMessage	PutAppIName	Application that put the message
PutMessage	PutDate	Date the message was put
PutMessage	PutTime	Time the message was put
PutMessage	ApplOriginData	Application origin data
PutMessage	MsgType	Message type
PutMessage	Expiry	Message lifetime
PutMessage	Feedback	Feedback code
PutMessage	Priority	Message priority
PutMessage	Persistence	Message persistence
PutMessage	BackoutCount	Backout counter
PutMessage	PutApplType	Put application type
PutMessage	MsgFlags	Message flags
PutMessage	OriginalLength	Length of original message

WebSphere MQ Environment Variables

If you use WebSphere MQ environment variables, to ensure compatibility with queue sessions, use the following format:

MQServer=ChannelName/TransportType/IPAddress

Example

MQServer=mymq1/TCP/123.123.123.123

Queue Manager Parameter

Specify the Queue Manager as the Server parameter in the queue session as shown in this example:

Server=<servername> (123.123.123.123) Port=default QueueManager=<queuemanagername> queue=tester1 ServerChannel=serverch1

Server Parameter

The MQServer, MQCHLLIB, and MQCHLTAB environment variables can be changed at runtime or before the GUI tools are launched to test writing to different servers.

In the queue session, there is a Server parameter where you specify a Queue Manager (if you need one).

• If you use the MQCHLLIB or MQCHLTAB variable, you must specify the Server parameter.

• If you just use the MQServer environment variable, it is not necessary to specify the Server parameter. Instead, just change the MQServer environment variable at run time to specify the new Queue Manager to which you are writing. The MQServer environment variable has the channel, protocol, server, and port specified, which is all that is needed to connect to a different queue manager.

Macro Values for the Server Parameter

You can use macro values for the Server parameter of the session. If you use the MQCHLLIB and MQCHLTAB environment variables, you can create a macro to specify the Queue Manager. The Queue Name cannot use a macro value. Change this before deployment, or use a variable that drives Decision steps to do a Get or a Put message to the proper queue.

It is recommended that you change the MQServer environment variable at runtime, since no Queue Manager is needed in the Server parameter of the queue session. You only need to change the queue name.

Key Points About Websphere MQ and the DJComponent Object

IBM Websphere MQ requires a separate queue manager connection for each thread, and does not permit cross-thread operations.

For processes, three scenarios are possible:

• Set MaxConcurrentThreads = 1. All process steps are executed on the same thread, so queue operations can be spread across multiple steps.

• Complete all queue operations in a single step. This includes connecting to a queue, getting or putting messages, and completing a transaction. This guarantees that all queue operations are executed in a single thread. Multiple steps can still contains queue operations, but each step is a separate transactional unit of work.

• Confine all queue operations to the Start and Stop steps. In multithreaded processes, the Start and Stop steps always execute in the main thread. The Start step can be used to create a connection, begin a transaction and retrieve messages from the queue. The Stop step can be used to write messages to the queue and complete the transaction.

Note: These threading restrictions apply only to DJComponent objects, not DJMessage objects. The DJMessage objects are completely independent from the messaging systems. So, in scenario 3 above, you can pull a message from a queue in the Start step. Use that message object in multiple concurrent Script or Transformation steps, and complete the transaction in the Stop step. Likewise, you can create DJMessage objects in multiple steps and write all of the message objects to a queue in the Stop step. The maximum size limit for DJMessage is 512 MB.

IBM MQ Queue

IBM MQ Queue allows a message to be put into the queue or retrieve from a queue to a message object. Use the PutMessage action to write to the queue. Use the GetMessage action in a Queue step to retrieve messages. Session properties control which message objects are read and the names of the messages that are written. Also, if the message properties must be to preserved.

Note: IBM MQ Component supports IBM MQ Version 9.2.0.

IBM MQ Queue Properties

Property	Default Value	Description
Host		Host name where IBM MQ server is installed.
Port	1414	Port number to connect to.
Channel		Channel name through which the communication occurs.
Queue Manager		Name of the Queue Manager to connect to.
User ID		User ID that has access to the specified Host.
Password		Password of the above specified User ID.

Supported Actions

Action	Description
Connect	Establishes a connection to the IBM MQ server.
Disconnect	Disconnects from the IBM MQ server.
GetMessage	Retrieves message into a message object.
PutMessage	Writes a message object into the queue.
Begin Transaction	Starts a IBM MQ transaction.
Commit Transaction	Commits a IBM MQ transaction.
Rollback Transaction	Rolls back a IBM MQ transaction.

Supported Action Properties

Action	Property	Description
PutMessage	Include JMS Headers	Set the property to FALSE if you want MQ-style message body, without a JMS MQRFH2 header.
PutMessage GetMessage	CCSID	Coded Character Set Identifier of character data within the message body.
PutMessage	Encoding	Encoding for numeric data. The representation used for binary integers, packed-decimal integers, and floating-point numbers.
PutMessage GetMessage	MsgId	Message ID of a specific message put in or retrieved from a queue.
PutMessage GetMessage	CorrelationId	Correlation ID of messages to put in or retrieve from a queue.
PutMessage GetMessage	GroupId	Group ID of message to put in or retrieve from a queue.
GetMessage	SequenceNumber	Sequence number of a specific message to retrieve from a queue.
GetMessage	Timeout	Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages.
GetMessage	Browse	Retrieve messages from a queue without removing them.
PutMessage	Offset	Segment offset.
PutMessage	ReplyToQ	Reply to queue (this field must not be blank if the MsgType field has the value MQMT_REQUEST).
PutMessage	ReplyToQMgr	Set the name of the queue manager to which reply message must be sent.
PutMessage	UserID	UserIdentifier specifies the user identifier of the application that originated the message.
PutMessage	AccountingToken	Accounting token of the message to be sent.
PutMessage	ApplIdData	Set ApplIdentityData to provide additional information about the message or its originator.
PutMessage	PutAppIName	Application that put the message.
PutMessage	PutDate	Date the message was put.
PutMessage	PutTime	Time the message was put.
PutMessage	ApplOriginData	Set ApplOriginData to provide additional information about the origin of the message.
PutMessage	MsgType	Message type.
PutMessage	Expiry	Message lifetime expressed in tenths of a second.
PutMessage	Feedback	Set Feedback to indicate the nature of the report (used with a message of type MQMT_REPORT).
PutMessage	Priority	Message priority.
PutMessage	Persistence	Persistence indicates whether the message survives system failures and restarts of the queue manager.

IBM MQ Environment Variables

You can use IBM MQ environment variable MQServer, to connect to the server. If the Host is not defined in Queue properties, then DataConnect attempts to read the MQServer environment variable to connect. Use the following format to define MQServer environment variable:

MQServer=ChannelName/TransportType/IPAddress(Port)

Example

MQServer=mymq1/TCP/123.123.123.123(1415)

MQServer=mymq1/TCP/123.123.123.123

If you do not provide (Port), then the default port number 1414 will be used to connect.

Note: Press Test Connection button after defining Queue (Component) Properties, if the Test Connection fails, then make sure that you can successfully ping the server/host where IBM MQ is deployed from the machine where DataConnect is installed. You will need to set appropriate authentication rules (CHLAUTH) at the IBM MQ Server which allows the access to the Server/Queue Manager/Queue you are trying to connect to. For more information, see IBM MQ documentation — https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.2.0/com.ibm.mq.sec.doc/q132560_.htm

MSMQ Queue

MSMQ Queue allows a file folder to be treated like a message queue. The files in the folder can be retrieved into message objects by using the GetMessage action in a queue step. Message objects can also be written to the folder as files using the PutMessage action. The properties of the session control which files are read and the names of the files that are written, as well as whether to preserve the message properties.

MSMQ Queue Properties

Property	Default Value	Description
User Name		User name to connect to the MSMQ server.
Password		Password to connect to the MSMQ server.
Server Name		Name of the MSMQ server.
Timeout	0	Maximum number of seconds to wait for a message to arrive on a queue when retrieving messages
BrowseMode	False	Retrieve messages from a queue without removing them

Supported Actions

Action	Description
Connect	Establishes a connection to the MSMQ server.
Disconnect	Disconnects from the MSMQ server.
GetMessage	Retrieves message into a message object.
PutMessage	Writes a message object into the queue.
BeginTransaction	Starts a MSMQ transaction.
PrepareTransaction	Prepares a MSMQ transaction.
CommitTransaction	Commits a MSMQ transaction.
RollbackTransaction	Rolls back a MSMQ transaction.

Supported Action Parameters

Action	Parameter	Description
Connect	Server	Name of the MSMQ server.
	Queue	Name of the queue.
GetMessage	Queue	Name of the queue.
	Message	Message content.
PutMessage	Queue	Name of the queue.
	Message	Message content.

Supported Action Properties

Action: PutMessage

Parameter	Description
Ack	Specifies the type of acknowledgement messages that MSMQ posts. The available options are: • MQMSG_ACKNOWLEDGEMENT_NONE (default) • MQMSG_ACKNOWLEDGEMENT_FULL_REACH_QUEUE • MQMSG_ACKNOWLEDGEMENT_FULL_RECEIVE • MQMSG_ACKNOWLEDGEMENT_NACK_REACH_QUEUE • MQMSG_ACKNOWLEDGEMENT_NACK_RECEIVE • MQMSG_ACKNOWLEDGEMENT_POS_ARRIVAL • MQMSG_ACKNOWLEDGEMENT_POS_RECEIVE • MQMSG_ACKNOWLEDGEMENT_NEG_ARRIVAL • MQMSG_ACKNOWLEDGEMENT_NEG_RECEIVE
AppSpecific	Specifies application generated information such as single Integer values or application defined message classes. The default value is: 0
AuthLevel	Specifies whether the message must be authenticated when it arrives at the target queue. The available options are: • MQMSG_AUTH_LEVEL_NONE (default) • MQMSG_AUTH_LEVEL_NONE_ALWAYS
CorrelationId	Identify the message using a 20-byte correlation identifier.
Delivery	Specifies how MSMQ delivers the message. The available options are: • MQMSG_DELIVERY_EXPRESS (default) • MQMSG_DELIVERY_RECOVERABLE
EncryptAlgorithm	Specifies the inception algorithm used by MSMQ to encrypt private messages. The available options are: • MQMSG_CALG_RC2 (default) • MQMSG_CALG_RC4
HashAlgorithm	Specifies the hash algorithm used by MSMQ when authenticating messages. The available options are: • MQMSG_CALG_MD4 • MQMSG_CALG_MD5 (default) • MQMSG_CALG_SHA • MQMSG_CALG_SHA256 • MQMSG_CALG_SHA512 • MQMSG_CALG_MAC • MQMSG_CALG_RSA_SIGN • MQMSG_CALG_DSS_SIGN • MQMSG_CALG_RC2 • MQMSG_CALG_RC4 • MQMSG_CALG_SEAL
Journal	This property is used for queues (MSMQQueueInfo objects) and non-transaction messages (MSMQMessage objects). The available options are: • MQMSG_JOURNAL_NONE (default) • MQMSG_JOURNAL_DEADLETTER • MQMSG_JOURNAL
Label	Specifies a description of the queue or message.
MaxTimeToReachQueue	Specifies a time limit (in seconds) for the message to reach the queue. The available option is: • LONG_LIVED (default)
MaxTimeToReceive	Specifies a time limit (in seconds) for the message to be retrieved from the target queue. The available option is: • INFINITE (default)
Priority	Specifies the priority of the message. The default value is: 3
PrivLevel	Specifies the privacy level of the message to be sent. The available options are: • MQMSG_PRIV_LEVEL_NONE (default) • MQMSG_PRIV_LEVEL_BODY_BASE • MQMSG_PRIV_LEVEL_BODY_ENHANCED
SenderCertificate	Provides an array of bytes that represents the security certificate.
SenderIdType	Specifies the type of sender identifier found by MSMQ. The available options are: • MQMSG_SENDERID_TYPE_SID (default) • MQMSG_SENDERID_TYPE_NONE
Trace	Specifies where report messages are sent when tracing a message. The available options are: • MQMSG_TRACE_NONE (default) • MQMSG_SEND_ROUTE_TO_REPORT_QUEUE

Last modified date: 08/02/2023