
Component | Purpose and Function | Cloud Service Examples |
|---|---|---|
Kubernetes (K8s) Cluster | Hosts Data Observability Platform services. Allows for scaling based on usage, simplified operations (auto-recovery), and ease of upgrades with no downtime. | Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS) |
Elastic Search (ES) | Stores data profiling data. Enables fast data retrieval via API and UI. | Managed or hosted ES on K8s |
Spark Engine | Handles the analysis of large datasets at scale with optimized cost and performance. Used to run Data Observability Platform data scan jobs. | Azure DataBricks, Databricks or Dataproc (GCP), Databricks or EMR (AWS) |
Postgres | Stores results and configurations. | Azure Database, Cloud SQL (GCP), AWS RDS |
Key Vault | Used to store encryption keys. | Azure Key Vault, Secret Manager (GCP/AWS) |
Container Repository | Stores Data Observability Platform binaries. Used for deploying Data Observability Platform images. | Azure Container, GCP Artifact Registry, AWS ECR |
Authentication Backend | Manages authentication, often utilizing existing services, i.e Okta. | Okta |
Data Type | Description | Sensitivity |
|---|---|---|
Customer Data | Data (sensitive and non-sensitive) that Actian monitors. This may be the original records or data decomposed into individual values for monitoring. | Varies (Sensitive or Non-sensitive) |
Derived Data | Results (numbers) from statistical calculations performed on Customer Data. | Non-sensitive |
Data Metrics | Derivatives used for analyzing trends in data. Examples include percentage of complete records or number of records. | Non-sensitive |
Metadata | Meta information about data, such as data source names, attribute names, and create/update dates. | Non-sensitive |
Sensitive User Data | Personally Identifiable Information (PII), such as usernames and passwords. | Sensitive |
Non-Sensitive User Data | User identification and roles . | Non-sensitive |
Data Type | Retention Policy | Notes |
|---|---|---|
Customer Data | Can be configured to be purged as soon as metrics are calculated (typically 1 hour). | If no explicit retention policy is set by the customer, this data is stored for up to 30 days and then permanently deleted . |
Derived Data | Stored for up to 30 days. |
|
Data Metrics | Stored for up to 360 days. | Used for long-term trend analysis. |
Metadata and User Data (Sensitive & Non-Sensitive) | Stored indefinitely. | This data is retained until explicitly requested for deletion. |