AWS credentials grant access to data on S3, and Azure OAuth credentials grant access to data in Azure Datalake Storage (ADL), so it is important to keep them safe. Referencing the credentials in the target URI from the command line (when using vwload, for example) or in environment variables can leave them easily accessible in logs, command histories, and Hadoop configuration.

The Hadoop credential provider framework allows secure “credential providers” to keep the credentials outside Hadoop configuration files, storing them in encrypted files in local or Hadoop file systems.