Data Products¶

Introduction to Data Products and Data Contracts¶

A data product is a reusable, active, and standardized data asset designed to deliver measurable value to its users, whether internal or external, by applying the rigorous principles of product thinking and management. It includes one or more data artifacts, such as datasets, models, and pipelines. It is enriched with metadata, including governance policies, data quality rules, data contracts, and, where applicable, a software bill of materials (SBOM) to document its dependencies and components. Ownership of a data product is aligned to a specific domain or use case, ensuring accountability, stewardship, and continuous evolution throughout its lifecycle.

Adhering to FAIR principles (findable, accessible, interoperable, and reusable), a data product is designed to be discoverable, scalable, reusable, and aligned with both business and regulatory standards, driving innovation and efficiency in modern data ecosystems.

A data product includes the following components:

• Input ports: An input port is a standardized interface through which a data product receives data from upstream sources. It defines how external data enters the data product. Input ports enable the controlled, traceable ingestion of data, facilitating lineage tracking and quality checks before the data is transformed and served to consumers.

• Output ports: An output port is a standardized interface through which a data product exposes its data to consumers. It defines how the data can be accessed (for example, through APIs, SQL tables, or event streams), along with its format, schema, and protocols. Output ports ensure that data products are interoperable, discoverable, and easy for other teams or systems to consume, while enforcing access controls and contracts. Each output port is associated with a data contract.

• Internal components: Internal components include datasets and processes required to produce the output ports but not intended for end-users. These components are not represented in the current version of the platform.

A data contract is a formal agreement between a data product owner (also known as a producer) and its consumers. It defines the structure, meaning, quality expectations, and access terms of the exposed data. A data contract includes a schema definition and can also include data quality rules, Service Level Agreements (SLAs), ownership, rights, and other related information. Data contracts ensure reliable data consumption, prevent breaking changes, and promote accountability between domains.

Data Products in the Actian Data Intelligence Platform¶

The Actian Data Intelligence Platform supports data products and data contracts natively. It enables organizations to manage, govern, and maximize the value of their data assets as products:

• Define your data products and data contracts with YAML descriptors that include all relevant information for discovery and consumption (for example, name, description, terms and conditions, and custom properties).
• Synchronize data products and their data contracts from your CI/CD pipelines by using the dedicated Data Product API.
• Manage data products and their components in the Studio to enrich their documentation and publish them to the enterprise marketplace.
• Search, find, and understand data products using the graph-powered search engine and an optimized layout dedicated to the discovery of these new item types.
• Request access to data products directly in Zeenea Explorer and manage these requests in Zeenea Studio to allow an efficient and governed consumption of the data products.

The following are the key benefits of implementing data products in the Actian Data Intelligence Platform:

• Enhance Data Discovery & Accessibility

Data products are supported as native items, enabling the modeling of simple and complex data products. You can define one or more output ports for each data product to create more use-case-oriented data that meets business needs.

A dedicated and optimized search experience, powered by the knowledge graph, enables users to efficiently discover, understand, and consume data products.

• Governance by Design

By supporting data contracts, the platform encourages organizations in their efforts to shift left metadata management. Organizations can design data contracts early and integrate them into their CI/CD pipelines to ensure that business expectations from the data contract are met when deploying new data. Synchronizing data contracts ensures that metadata remains up to date.

• Scale Data Management and Governance

Together with the federated catalog, each domain can design and manage its own data products.

The platform breaks data silos and supports a data mesh approach by allowing domains to publish their data products into the Enterprise Marketplace.

Create Data Products with the API¶

Define and Upload Data Products¶

The Actian Data Intelligence Platform leverages these standards managed by Bitol (a Linux Foundation project):

• Open Data Contract Standard (ODCS)
• Open Data Product Standard (ODPS)

These YAML files can be uploaded to the platform through a dedicated REST API. This API can be called from external tools or the CI/CD pipelines, for example, from a GitHub Action, as illustrated in the following diagram:

The list of output ports is defined in the data product descriptor. Each output port must have a name and must be linked to a data contract ID (UUID) with its version.

outputPorts:
- name: rawtransactions
  version: 1.0.0
  contractId: c2798941-1b7e-4b03-9e0d-955b1a872b32

Before calling the API, bundle the data product and all referenced data contract YAML files into a single ZIP file so that the platform can process them together.

Manage Output Ports¶

Output Port Lifecycle¶

When updating a data product through the API, newly defined output port versions replace existing ones. An output port is updated only if it references the same data contract ID and version.

Output ports are read-only items in the catalog. Their documentation is fully managed through the associated data contracts, which act as the single source of truth.

Output ports that are not included in the last version of the data product descriptor are automatically deleted.

Link Output Ports to Imported Datasets¶

The data contract must define both the model of the output port (the schema section in ODCS) and the location of the related datasets (the servers section in ODCS).

schema:
  - name: Transactions
    physicalName: transactions
    description: Real-time transaction data.
    properties:
      - name: Transaction ID
        physicalName: transaction_id
        required: true
        logicalType: string
        description: Unique transaction identifier
      - name: Account ID
        physicalName: acct_id
        required: true
        logicalType: string
        description: Account Id

When uploading a data product or importing a dataset from a scanner, the platform uses the name and physicalName and the servers section to automatically link output ports with their physical implementations. Only connectors built on the public SDK V2 are supported.

Following is an example of a servers section for Snowflake:

servers:
  - id: snowflake-server-pse
    server: Snowflake server
    type: snowflake
    host: ABCDEFGH-IJ12345.snowflakecomputing.com
    port: 8080
    account: IJ12345
    database: DATABASE_1
    warehouse: SMALL_COMPUTE_WH
    schema: SCHEMA_1

You can link the same dataset to multiple output ports and to different data products.

After a link is created, the related dataset appears in the Datasets tab of the output port in the Studio and Explorer.

If not all datasets defined in the data contract are linked to the output port, a banner indicates that the data contract is not fully implemented.

Manage Actian-Specific Attributes for Data Products and Output Ports¶

Manage Properties¶

You can use the data product and data contract YAML files to create or update properties of data products and output ports in the Data Intelligence Platform.

Properties must be defined using the customProperties syntax in ODPS/ODCS, with the property name zeeneaProperties.

The platform automatically determines the property type based on the syntax of the file:

• Short text: If defined as a key/value.
• Tag: If defined as an array.
• Date: If defined using a predefined format (ISO 8601).
• URL: If defined as a valid URL.

For example:

customProperties:
  - property: zeeneaProperties
    value:
      Sensitivity: Very High
  - property: zeeneaProperties
    value:
      Compliance: [BSA, FinCEN]
  - property: zeeneaProperties
    value:
      Documentation: https://www.acme.com
  - property: zeeneaProperties
    value:
      Approval date: 2025-10-27

The example above creates the following properties:

• A Short-text property Sensitivity and apply the value Very High to the item.
• A Tag property Compliance and apply the values BSA and FinCEN to the item.
• A URL property Documentation and apply the value https://www.acme.com to the item.
• A Date property Approval date and apply the value 2025-10-27 to the item.

Properties are managed as source documentation, similar to source properties of a dataset. They are not mapped to the data product template.

Properties created through the data product API can be used as filters or displayed in the catalog as any other source property, depending on their type, by customizing their search and display options in the Catalog Design section of the Studio.

Manage Contacts¶

You can use the data product and data contract YAML files to create or update contacts on data products and output ports in the Data Intelligence Platform using the ODPS/ODCS syntax for teams.

For example:

team:
 members:
   - username: ceastwood@acme.com
     name: Clint Eastwood
     role: data-owner
   - username: daustin@acme.com
     name: David Austin
     role: data-steward

The example above creates contacts with a name, email address, and source responsibilities such as data-owner and data-steward. The role value must be an existing source responsibility code. If the code does not exist, the platform creates it automatically using the provided value.

If the provided code corresponds to a user-defined responsibility code (created from the Catalog Design section), the contact is not added, and the API returns a non-blocking error.

When team members are specified, the platform attempts to match an existing contact by email (using the username field).
If the username is not a valid email address, then the platform attempts to match an existing contact by name (using the name field, and then the username field as a last fallback).

If no existing contact matches the specified contact, the platform creates a new contact.

Manage Links with Glossary Items¶

You can use the data product YAML files to create implementation links between glossary items and data products.

Links must be defined using the customProperties syntax in ODPS/ODCS, with the property name zeeneaGlossaryRefs.

customProperties:
  - property: zeeneaGlossaryRefs
    value:
      - "business-terms/Transaction Type"
      - “business-terms/Account Type”
      - “business-terms/Account ID”
      - “business-terms/Customer ID

Ensure that an implementation rule is configured at the metamodel level between the Data product item type and the targeted Glossary item type in the Catalog Design section. Otherwise, the link will not be created.

Manage Links with Custom Items¶

You can use the data product YAML files to create links between custom items and data products.

Links must be defined using the customProperties syntax in ODPS/ODCS, with the property name zeeneaCustomItemRefs.

customProperties:
  - property: zeeneaCustomItemRefs
    value:
      - regulation/Bank-Secrecy-Act
      - regulation/USA-PATRIOT-Act
      - regulation/FinCEN
      - "data-domain/Finance | Application"

Manage Access Request Policies for Output Ports¶

You can use the data contract YAML files to apply an access request policy to the output port. This allows data consumers to request access to the output port.

Access request policies must be defined using the customProperties syntax in ODPS/ODCS, with the property name accessRequestPolicy.

customProperties:
  - property: zeeneaProperties
    value:
      accessRequestPolicy: data-product-access-request-policy

The specified access request policy must already exist in the platform.

Define the Catalog of a Data Product and its Output Ports¶

If multiple catalogs are defined, you can specify the targeted catalog for your data products and their output ports in the payload of the data product API request.

You can also move a data product from its original catalog to another using the Studio. All output ports are automatically moved to the same catalog. Linked datasets are not moved automatically.

ODPS and ODCS Mapping Details in the Actian Data Intelligence Platform¶

The platform supports ODPS version 1.0.0 and ODCS version 3.1.0.

Uploaded YAML files must specify the ODPS or ODCS version used and comply with the standard version syntax and contain all required attributes as defined by the standard version. Files that do not meet these requirements are not processed.

ODPS Mapping¶

The following table shows how ODPS sections are mapped with the platform internal concepts and which sections are required:

• Fully mapped: All attributes are mapped to platform concepts.
• Partially mapped: Some attributes are mapped, but not the entire list.
• Not mapped: None of the attributes in this section are mapped to platform concepts.

The following sections provide detailed attribute-level information for the fully and partially mapped sections.

Fundamentals¶

Product Information¶

Team¶

Ancillary Objects¶

Custom Properties¶

Authoritative Definitions¶

ODCS Support¶

The following table shows how ODCS sections are mapped with the platform internal concepts and which sections are required:

• Fully mapped: All attributes are supported and mapped to Actian concepts.
• Partially mapped: Some attributes are supported, but not the entire list.
• Not mapped: None of the attributes in this section are mapped to Actian concepts.

The following sections provide detailed attribute-level information for the fully and partially mapped sections.

Fundamentals¶

Schema¶

Schema (Top-level)¶

Applicable to Elements (either Objects or Properties)¶

Applicable to Objects¶

This section is not required and is not mapped.

Applicable to Properties¶

Some keys are more applicable when the described property is a column.
This section is not required and is not mapped.

Logical Type Options¶

Additional metadata options to more accurately define the data type.
This section is not required and is not mapped.

Team¶

Infrastructure & Servers¶

Each server type can be customized with different properties such as host, port, database, and schema, depending on the server technology in use. Refer to the specific documentation for each server type for additional configurations.
If your server is not in the list, use the type Custom Server.

Custom and Other Properties¶

Custom Properties¶

Authoritative Definitions¶

Manage Data Product Documentation in Studio¶

Data Product Template¶

You can manage the data product template and responsibilities as any other item type in the Catalog Design section. It allows curators to provide metadata in addition to that harvested from the source with the API.

You can also configure data products to implement glossary items in the Glossary metamodel section.

Data Product and Output Port Attributes¶

Data Product Attributes¶

General Information¶

Data products have the following common attributes:

• Name / Source name
• Description / Source description
• Properties / Source properties
• Contacts / Source contacts
• Glossary items
• Links with custom items
• Catalog

In Studio, curators can manage data product properties defined in the template from their side panel and details page. They can also manage these properties using bulk actions and file import.

Input Ports Tab¶

In the Input ports tab of a data product, all the sources consumed by the data product's input ports are listed. The input ports themselves are displayed only in the lineage.

Output Ports Tab¶

In the Output ports tab of a data product, all the output ports to be consumed by the end-users are listed.

Data Quality Status¶

The data quality status of a data product is calculated from the data quality status of its output ports.

Attachments¶

In Explorer, you can download the YAML descriptor of the data product.

Output Port Attributes¶

General Information¶

Output ports have the following common attributes:

• Source name
• Source description
• Source properties
• Source contacts
• Catalog (inherited from the parent data product)

Datasets Tab¶

The Datasets tab lists all the datasets and their fields that compose the output port.

Data Model Tab¶

The Data model tab shows the relationships between the datasets of the output ports.

Data Quality Tab and Status¶

The Data quality tab lists all checks performed for all the datasets that compose the output port. For each check, a link allows the user to open the side panel of the corresponding dataset.

The data quality status of the output port is calculated from the quality status of its datasets.

Attachments¶

In Explorer, you can download the YAML descriptor of the data contract attached to the output port.

Delete a Data Product¶

You can delete a data product from the Studio. When you delete a data product, its output ports are also automatically deleted. The dataset linked to the output ports is not deleted.

Share a Data Product in Marketplace¶

Data products can be shared in the marketplace when the federated catalog option is activated. When you share a data product, its output ports and the datasets that compose them are shared automatically.

Move a Data Product to Another Catalog¶

In Studio, you can move a data product to another catalog when the federated catalog option is activated. When you move a data product, its output ports and the datasets that compose them are moved automatically.

You cannot move a data product to another catalog using the data product API. Attempting to do so generates a non-blocking error.

Search for Data Products¶

In Explorer and Studio, you can search data products using their own attributes or the attributes of their output ports. Output ports are not displayed in search results. Datasets linked to output ports are displayed in search results.

Request Access to a Data Product¶

In Explorer, users can request access to data products at the output port level.

To enable access requests for data product output ports, create an access request policy in Administration. Then the access request policy must be provided in the data contract.