11.4.0 Release Notes
What’s New in DataConnect 11.4
Installation
Known Issues
Eclipse Known Issues
Change Log
Change Log for 11.4.0 (Build 11.4.0-27)
Change Log for 11.4.0 (Build 11.4.0-12)
Change Log for 11.4.0 (Build 11.4.0-11)
Introduction
DataConnect Architecture
Key Features
Integration Design Maintenance Flow
Comparison of v9 and v11 Features
About This Manual
Intended Audience
Videos
Actian Support
Online Product Documentation
Technical Support
Installing DataConnect
Installing DataConnect Studio IDE on Windows
Minimum System Requirements for Windows
Operating System and Software Requirements
Hardware Requirements
Runtime Engine Requirements
DataConnect Development Studio Requirement
Installing DataConnect Studio IDE on Windows
Uninstalling DataConnect Studio IDE on Windows
Installing DataConnect Studio IDE Site Plug-in for Eclipse on Windows
Installing DataConnect Studio IDE Site Plug-in on Windows
Post DataConnect Studio IDE Site Plug-in Installation Tasks on Windows
Upgrading DataConnect Studio IDE Site Plug-in Versions on Windows
Installing DataConnect Studio IDE on Linux
Minimum System Requirements for Linux
Operating System and Software Requirements
Hardware Requirements
Runtime Engine Requirements
DataConnect Development Studio Requirement
Installing DataConnect Studio IDE on Linux
Uninstalling DataConnect Studio IDE on Linux
Installing DataConnect Site Plug-in for Eclipse on Linux
Install DataConnect Site Plug-in on Linux
Post Plug-in Installation Tasks on Linux
Installing DataConnect Runtime Engine
Installing DataConnect Runtime Engine on Windows
Installing DataConnect Runtime Engine on Linux
Uninstalling DataConnect Runtime Engine on Linux
Getting Started
Integration Toolset
Project
Maps
Processes
Portability
Launching DataConnect Studio IDE
Launching DataConnect Studio IDE On Windows
Launching DataConnect Studio IDE on Linux
Uploading License File
DataConnect License File Information
Welcome Page Links
Importing Samples or Prior Version Artifacts
DataConnect Compatibility Mode
Understanding DataConnect Studio IDE User Interface
Perspectives
List of Perspectives
Choosing a Perspective
UI Components in DataConnect Studio IDE Window
Editors
Menus
Navigators
Views
Toolbars
Cosmos.ini Settings
Mandatory Settings
Optional Settings
Importing Prior Version Projects and Artifacts
Importing Version 9 Workspace
Importing Version 10 Artifacts
Importing Artifacts from Folder
Setting Preferences
Enabling Data Integrator v9 Compatibility Mode
Setting General Preferences
Setting Workspace Preferences
Setting Execution Preferences
Setting Logging Preferences
Setting Error Handling Preferences
Setting Classpath Preferences
Setting Process Preferences
Setting Map Preferences
Setting Schema Preferences
Setting EZscript Preferences
Setting color preference for EZscript editor
Setting general preferences for EZscript Editor
Setting Macro Preferences
Managing Workspaces, Projects, and Integration Artifacts
Managing Workspaces
Creating Projects
DataConnect Integration Artifacts and Editors
Adding Integration Artifacts
Creating a Map
Creating a Process
Creating EZscript Library
Creating General EZscript Libraries
Creating EZscript Library Using the Lookup Wizard
Creating a Structured Schema
Creating a Folder
Viewing and Editing Properties for a Project or Artifact
Viewing and Editing Project Properties
Viewing and Editing Artifact Properties
Opening an Artifact
Importing Projects and Artifacts
Importing an Archive File
Importing Existing Projects into Workspace
Importing from File System
Importing Preferences
Import Projects from a Folder or Archive
Importing Projects or Artifacts from Prior Versions
Importing Samples
Exporting Projects or Artifacts
Exporting Projects to Another Workspace
Exporting Artifacts to Another Project
Moving Artifacts Between Projects
Copying Artifacts Between Projects
Copying Artifacts Within Projects
Deleting Projects and Artifacts
Choosing a Perspective
Searching Artifacts
Searching and Replacing Text
Search Results
Managing Macro Sets and Macros
About Macro Definition Files
Specifying Macro Definition File
Managing Macro Sets
Creating Macro Sets
Duplicating Macro Set
Deleting Macro Sets
Viewing All Macro Sets
Reordering Macro Sets for Project, Map, or Process
Managing Workspace-level Macros
Creating Workspace-level Macros
Duplicating Workspace-level Macros
Copying Workspace-level Macros
Moving Workspace-level Macros
Deleting Workspace-level Macros
Adding Project-level Macro Sets
Managing Local Macros for a Map or Process
Execution Service Macros
Testing Execution Service Macros During Design
Using Macros
Pasting Macro in Map or Process
Logging Macro Values
Designing and Executing Processes
Exploring Process Editor
Step Builder Canvas Tab
Process Toolbar
Toolbox Palette
Step Navigator
Thumbnail
Process Properties
Building Processes
Creating a Process File
Setting Execution and Logging Options
Specifying EZscript Library, Global Variables, or Engine Profile
Adding Steps
Copying and Pasting Steps
Deleting Steps
Enabling and Disabling Steps
Adding Component Instances or SQL Sessions
Adding Component Instances
Adding SQL Sessions
Associating Instances or Sessions with Steps
Linking Steps
Adding Macros
Testing Steps
Saving a Process
Configuring Processes
Configuring Macros
Managing EZscript Library Files
Adding EZscript Library Files
Deleting EZscript Library Files
Managing Global Variables
Managing Message Components
Adding Message Components
Editing Message Components
Deleting Message Components
Manually Deploying Message Components
Configuring SQL Sessions
Adding SQL Sessions
Editing SQL Sessions
Duplicating SQL Sessions
Importing SQL Sessions
Deleting SQL Sessions
Configuring Execution Options
Configuring Logging Options
Configuring Engine Profiler
Process Steps and Associated Components
List of Process Step Components
Aggregators
Invokers
Iterators
Queues
Transformers
Validators
Adding Annotations
Aggregator Step
Aggregator Step Properties
Aggregator Step Components
EDI X12 Functional Group Aggregator
File Transfer Aggregator
JDBC WebRowSet Aggregator
Application Step
Application Step Properties
Decision Step
Evaluating Decision Step Expressions
Decision Step Properties
Example
Invoker Step
Invoker Step Properties
Invoker Step Components
DataFlow Invoker
Orchestration Invoker 1.0.0
Dynamic Map Invoker
SMTP Email Invoker
Query to WebRowSet Invoker
REST API Invoker
Salesforce Login Invoker
Salesforce Changed Data Query Invoker
Salesforce Bulk Loader
Siebel Login Invoker
JDBC Stored Procedure Invoker
WebRowSet Update Invoker
Schematron Validation Invoker
XQuery Invoker
Zip Invoker
Iterator Step
Iterator Step Properties
Iterator Step Components
EDI X12 Batch Transaction Iterator
EDI X12 Functional Group Iterator
EDI X12 Transaction Set Iterator
File Transfer Iterator
JDBC WebRowSet Iterator
XPath Iterator
Process Step
Process Step Properties
Queue Step
Queue Step Properties
Queue Step Components
POP3 Email Queue
IMAP Email Queue
File Folder Queue
FTP Queue
JMS Queue
Kafka Writer Queue
Websphere MQ Queue
MSMQ Queue
Script Step
Script Step Properties
SQL Step
SQL Step Properties
Start Step
Start Step Properties
Stop Step
Stop Step Properties
Transformation Step
Transformation Step Properties
Transformer Step
Transformer Step Properties
Transformer Step Components
File to Message Transformer
Message to File Transformer
Automatic Mapping Transformer
Doc Validator Step
Doc Validator Step Properties
Doc Validator Type Properties
Generating an Error File
Validator Step
Validator Step Properties
Validator Step Components
TA1 Validator
XML (DTD) Validator
XML Schema Validator
XSLT Step
XSLT Step Properties
Health Care Connectivity
About Health Care Integration
Before You Begin
Health Care Step Types
Health Care Components
HL7 Integration Schemas
Using Scripts in Processes
Using EZscript Libraries in Processes
Building Decision Step Expressions
Managing Queue Message Objects
Creating and Using Message Variables
Creating New DJMessage Variable
Assigning Result of DJComponent GetMessage Function to Message Variable
Using FindMessage Function
Validating and Running Processes
Validating Process Step
Validating Complete Process
Running the Process
Designing and Executing Map
Exploring Map Editor
Viewing Map Properties
Saving a Map as RTC File
Setting Up Source Data
Specifying Connector, Parts, and Properties
Confirming Source Schema
Defining and Applying Filters to Source Data
Defining Sort Options
Defining Sample
Browsing Source Data
Clearing Source Connection
Saving Source Configuration
Setting Up Target Data
Specifying Connector, Parts, and Properties
Confirming Target Schema
Defining and Applying Filters to Target Data
Defining Update and Delete Key Options
Defining Sample
Browsing Target Data
Clearing Target Connection
Saving Target Configuration
Managing Multiple Targets
Adding Multiple Targets
Editing Target
Deleting Target
Setting Up Targets Using Multimode Connectors
Using a Multimode Target Connector in JSON Map
Using a Multimode Target Connector in XML Map
Managing Constants
Adding Constants
Searching Constant
Deleting Constant
Example
Managing Intermediate Targets
Managing Lookups
Lookup Types
Adding Lookups
Deleting Lookups
Managing Temporary Targets
Adding Temporary Targets
Specifying New Schema for Temporary Target
Deleting Temporary Targets
Setting Up Reject Data
Confirming Reject Schema
Using Reject Data Browser
Clearing Reject Connection
Saving Reject Configuration
Using Map Navigator
Configuring Map
Configuring Macros
Using Script Library Files
Adding EZscript Library File
Removing EZscript Library File
Defining Global Variables
Configuring Map Events
Adding Map Event
Modifying Map Event
Deleting Map Event
Configuring Logging Options
Configuring Engine Profiler
Configuring Execution Options
Mapping Target from Source Data
Mapping Methods
Mapping Fields Using Simple View
Mapping Fields Using Advanced View
JSON Map Advanced View - Nodes and Actions
XML Map Advanced View - Nodes and Actions
Common Tasks in Simple View and Advanced View
Viewing Data
Custom Search Navigator
Managing Record Types in Advanced View
Adding Record Type
Deleting Record Type
Duplicating Record Type
Managing All Record Instances in Advanced View
Specifying Record Instance Properties
Deleting Root Record Reference
Managing Groups in Advanced View
Adding a Group
Deleting a Group
Managing Field Records in Advanced View
Adding Field Record
Common Options
Dragging and Dropping Fields From Source to Target
Mapping Fields Automatically
Target File Mapping Scenarios
Managing Events and Actions in Advanced View
Events and Actions
Events and Actions for XML Maps (When v9 Compatibility Mode is Enabled)
Managing Events and Actions in Advanced View
Managing Record Rules in Advanced View for XML Map
Managing Recognition Rulesets and Rules
Adding Target Record Write Condition Rules in Advanced View
Validating and Running Map
Connectors
Types of Connections
Native API Connections
Physical Data Connections
Raw Sequential Connection Process
Physical File Format Connection Process
Intermediate Import and Export Format Connections
Importing Through Intermediate Data
Exporting Through Intermediate Data
ODBC and Other Middleware Connections
Driver Installation
Text Box to Enter Driver Options
SQL Connections
Relational Database Management Systems (RDBMS)
Creating SQL Table Indexes and Primary Keys
Calling PL/SQL Stored Procedures
Passing Variables
Generating a Random Unique Identifier
Authentication
Extensible Markup Language (XML) Support
Unknown Application or File Format Connections
Unknown File Format
Binary Data Types
Intermediate Methods to Connect to Data Formats
Using an Intermediate File Format
Alternative Connection Methods
Connecting Through Content Extract Designer/CXL Scripts
Custom Report Reading Scripts
Additional Connectivity Details
Batch Response File
Data File Formats
Structured Formats
Semi-structured Formats
Unstructured Formats
Delimited ASCII
Relational Database Management Systems
Determining Field Width in Characters or Bytes
Field Width in Characters
Field Width in Bytes
Cursors
Installing an ODBC Driver
Data Direct ODBC Drivers
ODBC Connectivity Tips
Connecting to a Table
ODBC Driver Setup Error
Oracle Connectivity Guidelines
Oracle Client
Path
TNS name
Version
Upgrades
Table Names
Issues and Their Solutions
Oracle Database Global Transaction Support
XML Schema
XML Schema (MS XDR Biztalk)
Element and Attribute
XML Schema (W3C XSDL)
Hex Values
Encoding Reference
Constant Values for Encoding Parameters in EZscript
Customizing Character Mappings
International Components for Unicode
Determining Which Connector to Use
Flat File Connectors
Database Connectors
CRM Connectors
Components
Encoding and Sybase Databases
Encoding and Oracle Databases
Customer Example
Encoding EZscript Functions and Objects
Encoding Functions
File Functions
EZscript DJMessage Objects
Understanding Unicode and Other Encoding Types
UTF Format Encoding in Applications
Non-Unicode Connectivity
Unicode and Encoding Checklist
Installing Language Sets
Installing Character Sets
Encoding Tips
Troubleshooting Encoding Issues
Unicode 4.0 Limitations
Unicode 4.0 Character Encodings
Customizable ICU Encodings
Customizing Non-ICU Japanese Mappings
Byte Order Marks
HDFS Connectivity
Source and Target Connectors
General Connector Topics Explained
Connector-Specific Notes
Property Options
Source and Target Connectors A-G
AccountMate
Actian Ingres (ODBC 3.x)
Actian PSQL v11
Actian PSQL v11 Multimode
Actian Vector
Actian Vector Multimode
Actian Vectorwise (Delimited)
Actian Vectorwise (ODBC 3.x)
Actian Vectorwise (ODBC 3.x Multimode)
Acucobol (ODBC 3.x)
ACUCOBOL-GT Version 7
Adabas (NatQuery)
Alpha Four
Apache Common Logfile Format
Archive Queue
ASCII (Delimited)
ASCII (Fixed)
Avro
BAF
Binary
Binary (International)
Binary Line Sequential
Btrieve 10
Clarion
Clipper
Cloudscape (ODBC)
COBOL Flat File
Common Logfile Format Webserver
Content Extractor
Data Junction Log File
DataEase
DataFlex (ODBC 3.x)
dBASE II, III+
dBASE IV
Derby
Derby (Client)
Derby (Embedded)
DIALOG
DIF
Dodge Reports
EDI (X12)
EDIFACT
Enable
Excel
Excel 2000
Excel 2007, Excel 2010, and Excel (xlsx)
Excel XP
FIX
Folio Flat File
Foxbase+
FoxPro
GoldMine
GoldMine Import File (DBF)
GoodData
Source and Target Connectors H-O
HIPAA
HIPAA (eDoc)
Hitachi Hi-RDB (ODBC)
HL7
HL7 (eDoc)
HTML
IBM DB2 9.7
IBM DB2 9.7 Multimode
IBM DB2 10
IBM DB2 10 Multimode
IBM DB2 Loader
IBM DB2 Universal Database 9.5
IBM DB2 Universal Database 9.5 Multimode
IBM Informix Dynamic Server 10
IBM Informix Dynamic Server 11
Informix (ODBC 3.x)
Informix-Online DS
Informix-Online DS Multimode
Integration Log File
JSON Connector
LDIF
Lotus 123
Lotus Notes Structured Text
MAILERS+4 (dBASE)
Microsoft Dynamics AX 4
Microsoft Dynamics CRM 4.0
Microsoft Dynamics GP 9 and 10
Microsoft IIS Extended Logfile Format
Microsoft SQL Azure
Microsoft SQL Azure Multimode
MUMPS (ODBC)
MySQL (ODBC 3.5)
Navision Financials (ODBC 3.x)
Netezza
NetSuite
NonStop SQL MX (ODBC)
Null
OData
ODBC 3.5
ODBC 3.5 Mass Insert
ODBC 3.5 Multimode
ODBC 3.x
ODBC 3.x Mass Insert
ODBC 3.x Multimode
Oracle 10g, 11g, and 12c
Oracle 10g, 11g, and 12c Multimode
Oracle CRM On Demand
Oracle Direct Path 10g, 11g, and 12c
Oracle Service Cloud
Oracle Siebel CRM On Demand 12 and 14
Oracle SQL Loader
Source and Target Connectors P-S
PayChex (DJF) Import
Personal Librarian
Pivotal Business Server 5.9
PostgreSQL
PostgreSQL Multimode
PostgreSQL (ODBC 3.x)
Process and Transformation LogFile Parser
Progress (ODBC 3.x)
RBase (ODBC 3.x)
Red Brick
Rich Text Format
RightNow CX
Salesforce
SAP Business One
SAP IDoc (eDoc)
SAP IDoc
SAS Transport Format
SBT Acct (FoxPro)
Sequential Binary
SGML
SPLUS
SPSS
SQL Script
SQL Server 7 (OLEDB)
SQL Server 7 Mass Insert (OLEDB)
SQL Server 7 Multimode (OLEDB)
SQL Server 2000
SQL Server 2000 Mass Insert
SQL Server 2000 Multimode
SQL Server 2005
SQL Server 2005 Mass Insert
SQL Server 2005 Multimode
SQL Server 2008
SQL Server 2008 Mass Insert
SQL Server 2008 Multimode
SQL Server 2012
SQL Server 2012 Mass Insert
SQL Server 2012 Multimode
SQL Server 2016
SQL Server 2016 Mass Insert
SQL Server 2016 Multimode
SQL Server BCP
SWIFT
SWIFT (eDoc)
Property Options
Sybase Adaptive Server 12.x and 15.x
Sybase Adaptive Server 12.x and 15.x Multimode
SYSTAT
Source and Target Connectors T-Z
Tape Drive Sequential
Teradata (Fastload)
Text (Delimited - EDI)
Text (Delimited - EDIFACT)
Unicode (Delimited)
Unicode (Fixed)
USMARC
Variable Sequential (MVS)
Variable Sequential (Record-V UniKix)
Variable Sequential (SyncSort)
Variable Sequential (User Defined)
Velocis (ODBC 3.x)
Visual dBASE 5.5
Visual FoxPro
Visual FoxPro (ODBC)
XDB (ODBC 3.x)
XML
XML-DMS
Deprecated Connectors
Managing Schemas
Creating Schemas
Creating Schema Manually
Creating Schema Using Metadata Connector
Types of Metadata Connectors
Creating Schema Using External Connection
Creating Schema From Another Schema
Creating Schema Using Data Parser
Using Data Parser
Data Parser Display Mode Options
Hex Values Reference Chart
Exploring Schema Editor
Nodes in Schema Design Editor
Nodes in v9 Compatible XML Schema
Viewing Schema File Properties
Managing Records
Adding Records in JSON Schema
Duplicating Records in JSON Schema
Deleting Records in JSON Schema
Adding Records in XML Schema
Duplicating Records in XML Schema
Reordering Records in XML Schema
Deleting Records in XML Schema
Managing Fields in Records
Adding Fields to a Record in JSON Schema
Deleting Fields in a Record in JSON Schema
Reordering Fields in a Record in JSON Schema
Adding Fields to a Record in XML Schema
Deleting Fields in a Record in XML Schema
Reordering Fields in a Record in XML Schema
Managing Rules for a Record
Managing Recognition Rules in JSON Schema
Managing Recognition Rule Sets for Hierarchical Connector based JSON Schema
Managing Recognition Rules in XML Schema
Adding Validation Rule in XML Schema
Updating Schemas
Managing Document Schemas
About E-Documents
E-Document Types
EDI X12
HIPAA
EDIFACT
SWIFT
Document Schema Designer Workflow
Creating Document Schema File
Viewing or Editing Document Schema File Properties
Viewing or Editing Document Schema File Properties Using Properties tab
Viewing Document Schema File Properties Using Right-click Menu
Viewing Document Schema File in Schema Viewer tab
Editing Document Schema File
Editing Values for Node Properties
Inserting Node
Deleting Node
Undo/Redo Insert or Delete Node
Copying/Pasting Node
Linking Library Files
Linking Library Files With Document Schema File
Importing Segments from Segment Library File
Editing Library Files
Editing Segment Library Files
Editing Field Library Files
Editing Codeset Library Files
Creating Sample Data File
Creating Schema From Data Instance
Creating Abstract Message Definition
Scripting
EZscript Scripting Language
EZScript Editor
Accessing EZscript Editor
Exploring EZscript File in Tab View
Exploring EZscript Expressions Window
Using Content Assist
Adding or Removing Bookmarks
EZscript Libraries
Using DJImport/DJExport Object Connection Wizard
Creating New DJImport or DJExport Object
Using Existing DJImport or DJExport Object
Writing Expressions
Basic Elements of an Expression
Operands
Operators
Parentheses
Functions
Statements
Scripting Keywords
Precedence in Evaluating Expressions
Commenting Out Code
Quotation Marks in Expressions
Specifying Field Names in Expressions
Continuing Lines in Expressions
Regular Expressions
Special Characters
Literal Values of Special Characters
Functions and Operators
Escaping Hex Values
Literal Values in Expressions
Using Scripts to Perform Specific Tasks
Parsing a Name Field Into Separate Fields
Example of Rearranging Order of a Name
Parsing a Source Field by Character
Specifying Ranges in Expressions
Converting Dates
Using Masks to Convert Two-Digit Dates
Mask Type for Two-Digit Dates, "YYdddd"
Filtering by Date
Setting Up Picture Masks
Formatting Boolean Data
Formatting Date/Time Data
Validating Data Types in Expressions
Calling External DLLs
Calling DLLs using a Declare statement
Examples of Calling External DLLs
Capturing Transformation Metadata
Handling Invalid Data
Retrieving Source and Target Metadata
Using DJRecord and DJField Objects to Access Metadata
Retrieving and Setting LogFile and LogMask Properties
Example
Setting the Scope of Functions
Script Functions
Script Functions A-F
Abort Function
Abs Function
Add1Fmt Function
Add1Part Function
Add2Part Function
Asc Function
B64Decode Function
B64Encode Function
Bin Function
CBool Function
CByte Function
CDate Function
CDbl Function
CDec Function
ChangeDirectory Function
ChangeFromUnicode Function
ChangeToUnicode Function
CharCount Function
Choose Function
Chr, Chr$ Functions
CInt Function
Clean Function
ClearMacro Function
CLng Function
Compare Function
CreateObject Function
CSng Function
CStr Function
CurDir Function
Date, Date$ Functions
DateAdd Function
DateConvert Function
DateDiff Function
DatePart Function
DateSerial Function
DateValMask Function
DateValue Function
Day Function
DefineMacro Function
DeleteMessage Function
Discard Function
Environ Function
Eval Function
Exp Function
Extract Function
FieldAt Function
FileAppend Function
FileCopy Function
FileDateTime Function
FileDelete Function
FileExists Function
FileLen Function
FileList Function
FileRead Function
FileReadLine Function
FileRename Function
FileWrite Function
Filter Function
FindMessage Function
Fix Function
FlushLog Function
Format, Format$ Function
Script Functions G-M
GetDirectory Function
GetObject Function
GetThreadID Function
GSub Function
Hex Function
Hour Function
IIf Function
ILAddrow Function
ILBuild Function
ILClear Function
ILFirstKey Function
ILNextKey Function
ILookup Function
ILRemoveRow Function
InitCaps Function
InputBox, InputBox$ Functions
InStr Function
InStrRev Function
Int Function
IsArray Function
IsDate Function
IsDirectory Function
IsFile Function
IsMacroDefined Function
IsNull Function
IsNumeric Function
IsObject Function
Join Function
LBound Function
LCase, LCase$ Functions
Left, Left$ Functions
LeftB Function
Len Function
Log10 Function
Log Function
LogMessage Function
LTrim, LTrim$ Functions
MacroExpand Function
MacroValue Function
MakeDirectory Function
Mid, Mid$ Functions
Minute Function
Month Function
MsgBox Function
Script Functions N-Z
NamePart Function
Now Function
Null Function
Oct Function
Parse Function
ParseXMLFile Function
ParseXML Function
Print Function
RecordAt Function
Reject Function
RemoveDirectory Function
Right, Right$ Functions
RightB Function
Rnd Function
Round Function
RTrim, RTrim$ Functions
Second Function
Serial Function
Sgn Function
Shell Function
SignFlip Function
Sleep Function
SortArray Function
Space, Space$ Functions
Split Function
Str, Str$ Functions
StrComp Function
String, String$ Functions
StrReplace Function
StrReverse Function
Sub Function
Switch Function
Terminate Function
TestMask Function
Time, Time$ Functions
TimeSerial Function
TimeValue Function
TLookup Function
Transliterate Function
Trim, Trim$ Functions
UBound Function
UCase, UCase$ Functions
UTF8Decode Function
UTF8Encode Function
Val Function
ValMask Function
Weekday Function
Xlate Function
Year Function
Script Operators
Arithmetic Operators
Multiply (*) Operator
Add (+) Operator
Difference (-) Operator
Integer Division (\) Operator
Numeric Division (/) Operator
Exponent (^) Operator
Mod Operator
Assignment (=) Operator
Assign (=) Operator
Comparison Operators
Less Than (<) Operator
Greater Than (>) Operator
Less Than or Equal To (<=) Operator
Greater Than or Equal To (>=) Operator
Does Not Contain (!~) Operator
Equals (==) Operator
Does Not Equal (<>) Operator
Concatenation (&) Operator
Concatenate (&) Operator
Logical Operators
And Operator
Contains (~) Operator
Or Operator
Not Operator
Like Operator
Important Pattern Matching Rules
Recognition Rule Operators
Any Value Operator
Between Operator
Like Operator
Regular Expression Operator
Script Statements
Using DJX to Create SQL Statements
Dynamic SQL Statements
DJXBegin and DJXEnd
Source Query Statement
Flow Control Statements
ElseIf Statement
For...Next Statement
For Each...Next Statement
If...Then...Else Statement
Option Explicit Statement
Option FieldExplicit Statement
Option FieldImplicit Statement
Option Implicit Statement
Select Case...End Case Statement
While...Wend Statement
Option Math Statements
Option Math Fast Statement
Option Math Precise Statement
Option Math Strict Statement
Run-Time Error Trapping Statements
GoTo Statement
On Error GoTo Statement
On Error Resume Next Statement
Resume Label Statement
Resume Next Statement
Resume Statement
Declare Statement
Dim Statement
Function Statement
Global Statement
Include Statement
Option Base Statement
ReDim Statement
Return Statement
Variables
Scoping Variables
Scope Levels
Local Scope Notes
Public Scope Notes
Guidelines for Naming Variables
Variant Data Type
Array Variables
Referencing Elements in Arrays
Declaring Arrays
Storing Data in Arrays
Retrieving Data from Arrays
Clearing Variables and Array Elements
Managing Variables
Adding Global Variables
Editing Global Variables
Importing Global Variables
Deleting Global Variables
Script Objects
Object Properties
Read Write Abilities of Object Properties
Object Operators
Object Variables
Declaring Object Variables
Internal Object Variables for Mapping
Err Object Variable
DJRecord Object Type
Properties
Methods
Operators
DJField Object Type
Properties
DJImport and DJExport Object Types
Properties
Methods
Operators
Examples of DJImport Object Usage
DJExport Object Type Notes
Example of DJExport Object Type Usage
DJRowSet Object Type
Properties
Methods
Operators
DOMDocument Object Type
Properties
Methods
Operators
DOMNode Object Type
Properties
Methods
Operators
DJComponent Object Type
Messaging Support
Properties
Methods
Operators
DJMessage Object Type
Methods
Properties
Operators
Using a DJMessage Object with a Map
Creating Map Project Files
Creating a Process with a DJMessage Object
Intializing a DJMessage Object
Adding a Transformation Step to a Process
Linking Steps in a Process
Creating a Configuration and Running a Process
Using DJMessage and DJComponent Objects in Expressions
Examples
RFH Header Support: Websphere MQ
RFH Header Guidelines and Properties
RFH2 Header Guidelines and Properties
Specifying RFH and RFH2 Headers
Websphere MQ Header Support
Supported Header Types
Examples
Java Objects in Scripts
Data Types
Class Objects
Transformation Object
Project Object
Properties
Errors
Examples
Script Troubleshooting
Novice Troubleshooting Suggestions
Use the most efficient function for your task
Understand the difference between syntax errors and errors in logic
Use comments to document your expressions
Intermediate to Advanced Suggestions
Assign a variable in the code to speed up your transformations
Use the Option Explicit statement
Use functions that signal errors in processing
Use Resume and Resume Next appropriately
Use the correct operator for the job
Learn about flow control for faster processing
Stack Overflow
Example 1 (Incorrect)
Example 2 (Correct)
64-Parameter Limitation in Function Arguments
Expression Tester Utility
Using the Expression Tester
Using Flags
Debugger Commands
Validating Data
Validating Data In Map Editor
Validating Data In Process Editor
Validating Data Using DjValidator Standalone Utility
Return Codes
Error Message Format
Using Engine Profiler
Creating Profile Output File
Viewing Profile Output File
Statistics Tab
Iteration and Time Histogram Tabs
Call Graph Tab
File Format of the Profile Output File
Question
Node Info Blocks
Using Package Manager
Creating a Package (.djar File)
Using .djar Package
Using Runtime Engine
Invoking Runtime Engine Using CLI
Running Multiple DataConnect Versions
Runtime Engine Command Line Interface
Command Line Syntax and Usage
Running a Map
Running a Process
Running Multiple Maps or Processes
Running a Map or Process in djar
Running a Map or Process (XML)
Running Multiple Maps and Processes (XML)
Specification Files
Additional Information
Quotes and Escape Characters
Command Line Options
Global Options
Map Options
Process Options
Command Line Options (XML Files)
Error Codes and Messages
Using Content Extraction Language
Types of Text Files Supported by CXL
Types of Operations that CXL Can Perform
Use of Content Extraction in the Integration Platform
Script Examples
CXL Quick Reference
CXL Scripts
Basic Syntax
Script Structure
Script Execution
Script Licensing
A Simple Script Example
Language Elements
CXL Character Set
Keywords
Numeric and String Constants
Variables and Fields
User-Defined Variables
Built-in Variables
Field Variables
Variable Typing and Conversion
Operators
Arrays
One-Dimensional Arrays
Associative Arrays
Multidimensional Arrays
Array Iteration
Functions
Built-in Functionality
User-Defined Functions
Rules and Limitations for Variables in a User Defined Function
Variables Definitions
Return
Pattern Matching and Action Statements
CXL Patterns
BEGIN { statements }
END { statements }
CATCHUP { statements }
Expressions in Patterns
Regular Expression
/regular expression/ { statements }
Metacharacters
Comparison Expressions
CXL Comparison Operators used in Comparison Expression Patterns
Actions and Statements
Expressions in Statements
Comment Statements
Reinitialize Statement
Assignment Statements
Catchup Statements
Control Statements
Output Statements
Integrating with Map Designer
Map Designer Setup
Source Type
Source Data File
CXL Script
Map Designer Results
Connect
Process Script
Refresh Script
Source Data Browser and Source Schema
DJXLPACK - Script Compressing and Encoding
Script Errors and Debugging Tips
Syntax Errors
Logic Errors
Debugging Techniques
Trace
Comments
Logmessage
Source Data Browser
Language Syntax and Examples
!=
!~
# comment
$
$n
&&
*
+
+=
-
.
/
/ regular expression /
<
<=
=
= =
>
>=
?
{ and }
[ and ]
\
^
||
~
in
|
r1r2
abort;
accept explist;
asc (string);
BEGIN { statements }
break;
catchup on;
catchup off;
CATCHUP { statements }
Chr (n)
cleararray ( array );
clearvars;
EBCDIC
ENCODING
END { statements }
Fix (number)
for ( index in array ) { statements }
gsub ( reg, sub, var );
if ( condition ) { statements } else { statements }
IFS
ILS
InitCaps (string)
Int (number)
IsNumeric (value)
LCase (strexpr)
Left (strexpr, n)
length ( var );
logmessage ( warningtype, format, parm1, parm2 );
ltrim ( var );
NamePart ("[h] [f] [m] [mi] [l] [t]", "string")
Part Description
NF
NR
readahead ( num );
REGPOS
REGLEN
reject;
Right (strexpr,n)
rtrim ( var );
Sgn (number)
shift;
skip ( num );
Space (number)
split ( var, array, fs );
String (number, string)
sub ( reg, sub, var );
substr ( var, pos, len );
TABS
TRACE ON
TRACE OFF
Transliterate (source_char_set, target_char_set, string)
trim ( var );
UCase (strexpr)
Val (string)
var [ num ]
var ( j k )
while ( condition ) { statements }
Script Examples
Example Script 1: Fixed-Format Report
Example Script 2: Multicolumn Record Reports
Example Script 3: Tagged Reports
Example Script 4: One-To-Many Item Reports
Example Script 5: Multirecord Type Hierarchical Reports
Example Script 6: User Defined Functions, Arrays, and Loops
Best Practices
Architect the Solution
Business Requirements
Technical Requirements
Design for Reuse
Set Up Collaborative Workspaces
Create DataConnect Templates
Use Orchestration
Use Naming Conventions
Workspaces
Projects
Artifacts
Variables and Object Names
Macros and Constants
Translate Business Processes to DataConnect
Data Governance
Processes
Transformation Maps
Schema
Determine Infrastructure Requirements
Identify Connectivity Requirements
Message Component Framework SDK
Develop Your Templates
Leverage In-line Documentation
Process Designs
Artifact Properties
Script Comments
Search and Replace
Secure Your Integrations
Common Integration Techniques
Macros
Reusable Scripts
Constants
djMessage Objects
Parallelism and Concurrency
Data Validation
Event Condition Action rules
Data Enrichment
Temporary Targets
Lookups
Aggregating Data
Transformation and Process Objects
Web Service APIs
Custom Java objects
Time Zone Considerations
Important Date Functions
Localization Issues
Scripting Tips, Tricks, and Techniques
Performance
Accuracy
ECA Loops
Complex Text Parsing
Plan for Customization
Bulk or Batch Pattern Integrations
Files vs. Message Objects
Large File Splitting
Wide Files
Concurrency
RDBMS Transformation Maps
Source Data
Target Data
RDBMS Process Designs
SQL step
Application Step
Sessions
Message Pattern Integrations
DataConnect Studio API Invoker
Payload Size
Compression
Orchestration
RDBMS Considerations
API Limitations
Batch Size
Call Allocation
Concurrency
Optimize and Harden the Solution
Optimize
Use Sample Data
Manage Log Files
Analyze with Engine Execution Profiler
Test
Package and Deploy
Execute
Debug
Iterate
Samples
Default Samples Installation Location
dc-samples Content
Importing Sample Projects When Creating a New Workspace
Importing or Copying Samples Projects
Using the Correct Macros When Working With Samples
Fundamentals Samples
Error Codes and Message
Types of Errors and Log Messages
Error Log Messages
Error Model for the Integration Components
Scenario 1
Scenario 2
Scenario 3
Viewing the Error and Event Log File
Transformation Log Codes
Transformation Engine Error Codes
Error Messages Related to Transformations
Transformation Engine Error Return Codes
Base Transformation Processing Return Codes
Transformation Engine Error Return Codes (Deprecated)
Process Engine Error Codes
Error Handling in the Process Engine
Process Engine Error Message Return Codes
Scripting Language Error Codes
Run-Time Error Values
Integration Language Run-Time Errors
Engine Command Line Interface Error Codes
Types of Errors Returned by the Command Line Interface
Runtime Engine Command Line Interface Error Messages
Message Component Framework Engine Error Codes
Troubleshooting
Troubleshooting Tips
Problems and Solutions
Unable to launch DataConnect or displays an error while creating or opening artifacts.
Not able to view the default views and editors
DataConnect project and artifact menus are not displayed in the new toolbar drop-down
Artifacts do not open in the correct editor
A big red X is displayed in an editor. Also, when you go to create an artifact, either nothing happens or artifact is invalid
Performance has deteriorated for both maps and processes
Unable to view Macro resolution
Unable to connect to a source or target
Huge log file size
Error during command line execution
Temporary projects appear in the Project Explorer
Unable to run a process that runs a transformation
Frequently Asked Questions (FAQs)
Can I use previous versions including Data Junction or Data Integrator?
What APIs are available?
Where can I find the Java SDK?
Has the scripting language changed?
Does the engine still support version 9 djengine commands?
Are 32-bit operating systems still supported?
What version of Java is supported?
Is multi-threading supported?
Does the Expression Results control panel still work?
Is the engine execution profiler still available?
What are the deployment options for runtime?
Does Change Source and Change Target still work?
Can the metadata be accessed and edited directly?
Are there any new features from Data Integrator version 9?
Are the e-Document connectors like EDI, HIPAA, HL7, and SWIFT still supported?
Is there a Developer Community?
Are third party source control systems supported?
What different types of life cycle management or versioning features are available?
Is Integration Hub (iHub) still supported?
When will DataConnect version 10 integrations be supported?
Are Joins and the Join Designer still supported?
Does Content Extractor that is Extract Schema Designer work in version 11?
Appendix A - Uniform Resource Identifier
Non-routable Addresses
URL Notes
Escape Characters
Connector Types Supported
Websphere MQ Messaging Middleware Connector
File Functions That Support URI Schemes
Objects That Support URI Schemes
Connector Types That Support URI
Procedure
Parameters
Alternatives to Accessing Data from URLs
Scripting Languages to Use in Conjunction with Integration Product Tools
CLR Script
REBOL/Core
Limitations
Appendix B - CXL Quick Reference
Basics
Variables and Fields
Built-in Variables
Field Variables
Built-in Functions
Operations
CXL Patterns
Regular Expression Summary
Statements in Actions
11.4.0 Release Notes
Statements in Actions