A. Command Reference : vwload Command--Load Data into a Table : vwload in Parallel Mode : Requirements for Parallel vwload
 
Share this page                  
Requirements for Parallel vwload
If parallel mode is enabled, all processing steps including reading input files from disk, UTF8 verification or character set conversion, parsing, and data type conversion are done by the server. Therefore, files must be accessible by the server, not the vwload client (as in normal mode).
Vwload cannot split one big input file to load it in parallel. Multiple input files must be specified to benefit from parallelization. Parallelization works best when input files have (almost) the same size.
Linux: To use the vwload command in parallel mode, /proc/sys/vm/overcommit_memory (see Virtual Address Space Allocation (Linux)) must be set to 1 or [memory] max_overalloc (see max_overalloc) must be 0.