Configuring the AWS S3 Bucket Listener

For System Administrators > System Administration Guide > Authorizing and Configuring the File Folder Listener Service > Configuring the AWS S3 Bucket Listener

Was this helpful?

The AWS S3 Bucket Listener monitors S3 buckets for files and will execute a pre-defined Job Configuration (config-id) when triggered.

Notes:

• Make sure you have already completed Authorize the File Folder Listener Service.

• After making any configuration changes to the application.properties or file-folder-listener-listeners.yml file, you must restart the File Folder Listener Service for those changes to take effect.

application.properties:

Add the appropriate AWS Access and Secret Key to <ProgramDataDirectory>\Actian\FileFolderListener\conf\application.properties.

For more information on how to create/obtain these keys, see: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys

Property	Default Value	Description
aws.s3.accesskey	—	required
aws.s3.secretkey	—	required

Example:

# AWS S3 Connection Info
aws.s3.accesskey=AKIAIOSFOXNN7EXAMPLE
aws.s3.secretkey=wJalrXUlnFEMI/K7MDEMG/bPxRfiCYEXAMPLEKEY

file-folder-listener-listeners.yml:

Configure these properties in <ProgramDataDirectory>\Actian\FileFolderListener\conf\file-folder-listener-listeners.yml

Property	Default Value	Description
id	—	A unique identifier for the listener
listener-type	—	Available listener types: local, aws, gcp, azure
config-id	—	The Job Configuration ID to run in Integration Manager
active	true	Whether the listener is active
source- bucket-name	—	The S3 bucket name to monitor for new files.
source- bucket- region	—	Region of the S3 bucket Note: AWS S3 region codes are slightly different from GCP.
include- pattern	—	Includes files if the file name matches the regular expression pattern you specify. (See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet)
exclude- pattern	—	Excludes files if the file name matches the regular expression pattern you specify. (See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Cheatsheet)
filename- override	—	This value will override the filename passed to Integration Manager, regardless of the original source file name. The original source file name will always be used for backup and error files.

Example:

Note: Indentation is critical for YAML syntax!

listeners:
  - id: aws-bucket-listener-accounts
    config-id: 90378
    listener-type: aws
    active: true
    source-bucket-name: listener-bucket-us-east-1-accounts
    source-bucket-region: us-east-1
    include-pattern: ^Accounts.txt
    exclude-pattern: ^skipme.txt
  - id: aws-bucket-listener-contacts
    config-id: 90379
    listener-type: aws active: true
    source-bucket-name: listener-bucket-us-east-1-contacts
    source-bucket-region: us-east-1
    include-pattern: ^Contacts.txt
    exclude-pattern: ^skipme.txt

Last modified date: 03/14/2022