Non-functional Test Criteria
Typical non-functional test criteria might include areas such as:
• What target data volumes need to be supported for production and for evaluation?
• What target data loading rates need to be supported?
• What ETL tools, if any, will be used to load the data, and what operating systems will they run on? VectorH supports clients on Windows and Linux.
• Where is the data coming from, and how often will data need to be loaded—that is, what is the target data latency from the originating source system?
• How many users need to be supported, and therefore what query concurrency level is needed? Typically we have seen a 3-5% “active query per connected user” ratio.
• What proportion of my production workload do I need to test with to create a scalability model that I can trust for sizing the production environment?
• What service level is needed for the production service, and how can the Actian Analytics Platform support that Service Level Agreement?
• What happens if the master node fails?
• What happens if a slave node fails?
• Will VectorH be expected to compete with other services (for example, Hive, Impala)?
• What does hardware (CPU, memory, disk, network) usage look like during the performance tests? How balanced and saturated is the cluster?
• Are there any restrictions on the movement of data or programs (such as downloading software from esd.actian.com) that need to be addressed?