Data Loading
Typical Data Loading Practices
Databases for reporting and data analysis are typically loaded through a controlled process. Traditionally, large incremental data sets are added daily, weekly, or even monthly, and additional data is mostly inserted rather than updated or deleted. Now, however, data loads are often much more frequent and, in some cases, are a constant stream of inputs. With the increase in data load frequency, data volumes have grown as well.
Vector works very well for scenarios with large incremental volumes of data that are predominantly inserts. Streaming data can often be collected and presented in incremental batches every few seconds (depending on the query requirements), which works well if incremental batches have a relatively high volume. Ideally, multiple data blocks are filled for every incremental data load.
Last modified date: 06/28/2024