Management of In-memory Updates
Batch updates are stored in a special highly-optimized differential data structure, called the Positional Delta Tree (PDT), in main memory. Data is also written to disk immediately to ensure recoverability.
Batch updates that reside in memory can potentially consume a significant percentage of useful system memory. As a result, if the user is performing batch operations, Vector automatically propagates the changes buffered in memory to the disk-resident table. Propagation is triggered by factors such as table sizes and the number of DML operations buffered.
The volume of batch updates in such a propagation cycle should fit in main memory.
Update propagation can be a relatively expensive operation if there are many modified rows (especially in case of even distribution of updates across the table). Vector tries to rewrite only the affected blocks unless it would result in excessive table fragmentation.
The best practice is to
avoid propagation to large tables (see
How to Avoid Propagation) and use appends where possible.
The amount of memory reserved for Positional Delta Trees and the amount of differences after which propagation is triggered can be configured with system configuration parameters. For more information, see PDT Parameters. In cases with high update rates, the memory required for PDTs can become too high, in which case you will have to work with raw tables or use bulk operations, such as the MODIFY...TO COMBINE statement.
Last modified date: 11/09/2022