Configuring the Google Cloud Storage Listener

For System Administrators > System Administration Guide > Authorizing and Configuring the File Folder Listener Service > Configuring the Google Cloud Storage Listener

Was this helpful?

The Google Cloud Storage Listener monitors Google Storage buckets for files and will execute a pre-defined Job Configuration (config-id) when triggered.

Notes:

• Make sure you have already completed Authorize the File Folder Listener Service.

• After making any configuration changes to the application.properties, file-folder-listener.properties, or file-folder-listener-listeners.yml file, you must restart the File Folder Listener Service for those changes to take effect.

application.properties:

Add the appropriate Service Account Key file location to <ProgramDataDirectory>\Actian\FileFolderListener\conf\application.properties.

For more information about how to create/obtain this key, see https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication

Property	Default Value	Description
gcp.storage.service-account- key	—	Complete path to the Service account key that can be used as credentials for the Storage client

file-folder-listener.properties:

# GCP Storage Connection Info (key file)
gcp.storage.service-account-key=C:\ProgramData\AccessKeys\gcp-account-name-1-935e6EXAMPLE.json

file-folder-listener-listeners.yml:

Configure these properties in <ProgramDataDirectory>\Actian\FileFolderListener\conf\file-folder-listener-listeners.yml:

Property	Default Value	Description
id	—	A unique identifier for the listener
listener-type	—	Available listener types: local, aws, gcp, azure
config-id	—	The Job Configuration ID to run in Integration Manager
active	true	Whether the listener is active
source- bucket-name	—	The GCP bucket name to monitor for new files
source- bucket- region	—	Region of the GCP bucket Note: GCP region codes are slightly different from AWS S3.
include- pattern	—	Includes files if the file name matches the regular expression pattern you specify (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet)
exclude- pattern	—	Excludes files if the file name matches the regular expression pattern you specify (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet)
filename- override	—	This value will override the filename passed to Integration Manager, regardless of the original source file name. The original source file name will always be used for backup and error files.

Example:

Note: Indentation is critical for YAML syntax!

listeners:
  - id: gcp-bucket-listener-accounts
    config-id: 90378
    listener-type: gcp
    active: true
    source-bucket-name: listener-bucket-us-east1-accounts
    source-bucket-region: us-east1
    include-pattern: ^Accounts.txt
    exclude-pattern: ^skipme.txt
  - id: gcp-bucket-listener-contacts
    config-id: 90379
    listener-type: gcp
    active: true
    source-bucket-name: listener-bucket-us-east1-contacts
    source-bucket-region: us-east1
    include-pattern: ^Contacts.txt
    exclude-pattern: ^skipme.txt

Last modified date: 03/14/2022