Adding a Google Data Catalog Connection¶

Prerequisites¶

• A user with sufficient permissions is required to establish a connection with Google Data Catalog.
• Zeenea traffic flows towards the data source must be open.

Supported Versions¶

The Google Data Catalog connector has been successfully tested with the Web version.

Installing the Plugin¶

The Google Data Catalog plugin can be downloaded here: Zeenea Connector Downloads.

For more information on how to install a plugin, please refer to the following article: Installing and Configuring Connectors as a Plugin.

Declaring the Connection¶

Creating and configuring connectors is done through a dedicated configuration file located in the /connections folder of the relevant scanner.

Read more: Managing Connections

In order to establish a connection with Google Data Catalog instance, specifying the following parameters in the dedicated file is required:

User Permissions¶

In order to collect metadata, the running user's permissions must allow them to access and read databases that need cataloging.

Role¶

To extract metadata, the technical account must have the following predefined role:

• Data Catalog Viewer (roles/datacatalog.viewer)

Collected Metadata¶

Inventory¶

Will collect the list of tables and views accessible by the user.

Dataset¶

A dataset can be a table or a view.

• Name
• Source Description
• Technical Data:
  • Project Id
  • Dataset Id
  • Table Id
  • Origin
  • Dataset Type

Field¶

Dataset field.

• Name
• Source Description
• Type
• Can be null: Depending on the field settings.
• Multivalued: true for array fields, false otherwise.
• Primary Key: Not supported. Default value false.

Tags¶

It's possible to get all the tags applied to your Data Catalog objects through a specified configuration in your dedicated scanner. You just need to define the tag model you want to get into your Zeenea Data Catalog in the application.conf file of your scanner.

Here is an example of a configuration file application.conf that retrieves select tags:

connector.google_data_catalog.tags = """
[
    {
        "id": "data_governance",
        "name": "Data Governance",
        "fields": [
            {
                "id": "data_governor",
                "name": "Data Governor",
                "type": "STRING"
            },
            {
                "id": "data_classification",
                "name": "Data Classification",
                "type": "ENUM"
            },
            {
                "id": "approved_by_governance",
                "name": "Approved By Governance",
                "type": "BOOL"
            },
            {
                "id": "approved_by_governance_date",
                "name": "Date of Governance Approval",
                "type": "DATETIME"
            }
        ]
    }
]"""

Object Identification Keys¶

An identification key is associated with each object in the catalog. In the case of the object being created by a connector, the connector builds it.

More information about how it works can be found here: Identification Keys.