# Universal Filters

This document describes Zeenea's scanner universal filtering system, a powerful tool designed to give you precise control over which assets are imported from your data sources.

With filters, you can create a set of rules to include or exclude specific items based on their names, ids, or other attributes. This is the perfect way to keep your data catalog clean and relevant, for example by:

* Including only production assets.
* Excluding temporary tables or test reports.
* Focusing on assets from specific teams or projects.
* Rooting items to specific catalogs

This feature is available for **Scanner version 79 and above**.

It is available for the following plugins: 

* PowerBI V2
* Tableau V2
* DBT Cloud

## Core Concepts

Understanding two core concepts is key to mastering filters:

1. **Rules are Processed in Order:** The filter is a list of rules that you define. When scanning a source, Zeenea checks an item (like a table or a report) against your rules from top to bottom.
2. **The First Match Wins:** As soon as an item matches a rule's conditions, Zeenea applies the corresponding `ACCEPT` or `REJECT` action and immediately stops processing. No further rules are checked for that item.

If an item goes through the entire list without matching any rule, it will be accepted by default. For this reason, we strongly recommend that your **last rule is a "catch-all" reject rule** to prevent unwanted assets from being imported.

A rule is defined by several key properties that work together. You must specify its `id` (a unique name), its `action` (`ACCEPT` or `REJECT`), and an optional `catalog` to which it is being rooted. If `catalog` is not defined, thr default catalog will be used. The conditions themselves are defined within a `rules` block, where you list all the criteria an item must meet for the rule to apply.

### Example: Include Only a Specific Workspace

The logic is:

1. **Rule 1:** If an item is in the "Zeenea" workspace, `ACCEPT` it and root it to "Zeenea Catalog".
2. **Rule 2:** If it is not in the "Zeenea" workspace, `REJECT` it.

This filter will accept items from the "Zeenea" workspace and reject all others.

```
filters = [
  {
    id = "accept_zeenea_workspace"
    action = ACCEPT
    catalog = "Zeenea Catalog"
    rules {
      workspace_name = "Zeenea"
    }
  },
  {
    id = "default_reject"
    action = REJECT
  }
]
```
## Building Your Rules

Each rule you create is an object with four parts:

*   `id`: A unique name you give the rule for easy identification.
*   `action`: The action to perform: `ACCEPT` (to keep the item) or `REJECT` (to discard it).
*   `catalog`: The catalog identifier where items are rooted (optional)
*   `rules`: A block where you define the conditions for the match. If this block is empty or missing, it matches everything.

The attributes you can use as keys inside the `rules` block (for example, `schema`, `table`, `workspace_name`) depend entirely on the technology being scanned. A database connector will offer keys like `schema` and `table`, while a business intelligence tool connector will have keys like `report_name` or `dashboard_id`. Therefore, it is essential to consult the documentation for the specific connector you are configuring to see the complete list of available filter keys.

### Defining Conditions: Matching Values

The `rules` block is where you set the conditions based on an item's attributes (like `schema`, `table`, `workspace_name`, etc.). You can match values in three powerful ways.

1. **Exact Matching**<br />
     This is the default. If you provide a value without any special characters, Zeenea will look for an exact match.

     **Example:**<br />
     This condition only matches if the schema is exactly `production`.

      ```
      rules {
        schema = "production"
      }
      ```
2. **Pattern Matching with Wildcards (`*` and `?`)**<br />
     You can use wildcards for more flexible matching. This is the most common way to match patterns.
   
       *  `*` matches any number of characters.
       *  `?` matches any single character.

       **Example:**<br />
       This condition matches any table starting with `FACT_` in the `analytics` schema.
      ```
      rules {
        schema = "analytics"
        table = "FACT_*"
      }
      ```
3. **Advanced Pattern Matching (Regular Expressions)**<br />
     For maximum power and precision, you can use regular expressions by adding the `regex:` prefix. This is an advanced feature for complex matching needs.
   
     * **Example 1: Match one of several prefixes**<br />
         The pattern `(HR|IT|MARKET)_` matches anything that starts with `HR_`, `IT_`, or `MARKET_`.

         **Example:**<br />
         This condition accepts tables from the HR, IT, or Market departments.
         ```
         rules {
           table = "regex:(HR|IT|MARKET)_.*"
         }
         ```
         
     * **Example 2: Exclude a prefix**<br />
         The pattern `^(?!TMP_).*` is a very useful one that means "does not start with `TMP_`".
         
         **Example:**<br />
         This condition matches tables that DO NOT start with `TMP_`.
         ```
         rules {
           table = "regex:^(?!TMP_).*"
         }
         ```