Parameter | Expected value |
|---|---|
name | The name that will be displayed to catalog users for this connection. |
code | The unique identifier of the connection on the Zeenea platform. Once registered on the platform, this code must not be modified or the connection will be considered as new and the old one removed from the scanner. |
connector_id | The type of connector to be used for the connection. Here, the value must be local-filesystem and this value must not be modified. |
inventory.root | Path of the root directory to explore |
Optional parameters for customizing dataset detection | |
inventory.partition | Regex to identify partition folders |
inventory.skippedDirectory | Regex on the name of the folders to ignore while keeping the content taken into account. The content will be scanned as if it were at the root of the parent folder. No folder is ignored by default. |
inventory.ignoredDirectory | Regex on the name of the folders to ignore: their content will also be ignored. No folder is ignored by default. |
inventory.ignoredFile | Regex in the name of the files to ignore, default value: \..* | _.* | .*\.crc |
inventory.extension.csv | For CSV files detection. Default value: csv, tsv, csv.gz, tsv.gz, csv.zip, tsv.zip |
inventory.extension.parquet | For Parquet files detection. Default value: parquet |
inventory.extension.avro | For Avro files detection. Default value: avro |
inventory.extension.orc | For Orc files detection. Default value: orc |
inventory.extension.xml | For XML files detection. Default value: xml, xml.gz, xml.zip |
inventory.extension.json | For JSON files detection. Default value: json, json.gz, json.zip |
inventory.csv.header | Used for configuring csv files header detection pattern. Select always to force recognizing the schema on the first line of csv files. Possible values are: never, always, and only string. |
xml.namespace_identification | Used for configuring XML fields identification. Value uri. Note: To keep the compatibility with a scanner previous to version 43, it is necessary to use the value legacy (default value). |
xml.fields_ordering | Starting from version 67. Allows ordering the list of retrieved fields. Possible values are: - alphabetical: Fields are ordered alphabetically - "", legacy, or unordered: Fields are ordered as they are read. |
Object | Identifier Key | Description |
|---|---|---|
Dataset | code/path/dataset name | - code: Unique identifier of the connection noted in the configuration file - path: Full path - dataset name |
Field | code/path/dataset name/field name | - code: Unique identifier of the connection noted in the configuration file - path: Full path - dataset name - field name |