12.5.0-19

First release for DataConnect 12.5. See Change Log for 12.5 (Build Number 12.5.0-19).

Data Profile Engine change

The Data Profile Engine now uses Apache Spark to process rules. Spark improves data processing performance through fast, in-memory computation and provides a unified, scalable platform for batch, streaming, and SQL workloads, with strong fault tolerance and seamless integration with existing data systems.

 

As a result, the Data Rush scripting language for Execute Expression and Assert rules now uses SQL Script. These rules support SQL expressions only. If a profile uses these rules, any existing scripts must be rewritten to comply with SQL syntax.

 

Data Quality Rules

The following new rules have been added:

See Rule and Parameter Reference.

Data Quality Rules

The following rules are not supported anymore. The result of many of these rules are available in data discovery:

See Deprecated Rules.

Data Quality

The Data Profiler .dp artifact is not supported in DataConnect 12.5 and therefore cannot be opened for editing. It has been replaced by a .dq (Data Quality) file and a .dq.rtc (Runtime Config) file. Please recreate the profile as .dq and .dq.rtc artifacts. You can right-click on the .dp file to view profile content in a JSON editor.

EZscript

Clear Property - A new clear property/method has been added, when called, will set the body of the message object to an empty string (""), and will remove all properties that have been set on the message. It is useful when reusing the message object, as it ensures that the second use does not contain anything from the first use. See DJMessage Object Type and djMessage Objects.

Function Statement - The Function Statement now supports declaring an object return type using the AS typename clause. This clause is only used when the function returns an object type. See Function Statement.

Return Statement - The Return Statement can now be optionally used to return a newly constructed object from within a function. This allows the function to immediately create and return a new object instance without first assigning it to a local variable. See Return Statement.

JavaVM Objects - EZscript has been enhanced to allow calling static methods of any class through the JVM. See JavaVM Objects.

Iterator

XPath Iterator - Version 1.4.0 is supported now. See XPath Iterator.

Invoker

Encryption Invoker - A new Invoker component has been added, enabling encryption and decryption of data within processes. See Encryption Invoker.

Map events and actions

A new action, OutputSubTree, is now available. By adding the RecordStarted event together with the OutputSubTree action, it is now possible to write all records within a subtree of the hierarchical target in a single operation, eliminating the need for a separate action for each record. See Actions and Action Parameters.

Deprecated Components

Processes that use a deprecated component can now be executed, edited, and saved. Previously, this was not allowed. However, deprecated components still cannot be used when creating new processes.

Component Affected: Data Profiler

Updated File: lz4-java-1.8.0.jar

 

Component Affected: Data Profiler

Updated File: hadoop-hdfs-native-client-3.3.6.jar

 

Component Affected: Kafka

Updated File: kafka-clients-3.9.1.j

 

Component Affected: DataProfiler, Json, Json2, Microsoft Dynamics 365-1.0.0

Updated File: jackson-core-2.18.2.jar

 

Component Affected: aws_s3, azure_storage

Updated File: netty-codec-http-4.2.9.Final.jar

 

Component Affected: aws_s3, azure_storage

Updated File: netty-codec-http2-4.2.9.Final.jar

 

Component Affected: Kafka

Updated File: avro-1.12.0.jar

Component Affected: Data Profiler

Updated File: jersey-client-3.0.16.jar

 

Component Affected: Data Profiler

Updated File: netty-codec-http-4.1.118.Final.jar

 

Component Affected: AWS S3 Components

Updated File: netty-codec-http2-4.1.126.Final.jar

 

Component Affected: AWS S3 Components

Updated File: netty-codec-http-4.1.126.Final.jar

 

Component Affected: salesforceBulkLoader2

Updated File: geronimo-ws-metadata_2.0_spec 1.1.3

 

Component Affected: webservice

Updated File: geronimo-jta_1.1_spec 1.1.1

 

Component Affected: Google Cloud Components

Updated File: grpc-netty-shaded-1.68.0.jar

 

Component Affected: Azure Storage Components

Updated File: netty-codec-http-4.2.6.Final.jar

 

Component Affected: llm-prompt-invoker, AWS S3 Components

Updated File: netty-codec-http-4.1.126.Final.jar

 

Component Affected: Rest Invoker, XML DMS, Odata

Updated File: commons-httpclient-3.1.jar

 

Component Affected: ibm_mq

Updated File: bcpkix-jdk15to18-1.76.jar

 

Component Affected: JDBC-1.0.0 connector

Updated File: commons-dbcp2-2.6.0.jar