Batch Updates using Spark SQL
Loading or updating data using Spark SQL through the Spark-Vector Connector is a bulk operation by default.
To force a batch insert, the SET INSERTMODE ROW command must be specified with the loadpreSQL option. For example:
sqlContext.sql("""CREATE TEMPORARY TABLE vector_table
USING com.actian.spark_vector.sql.DefaultSource
OPTIONS (
host "vectorhost",
instance "VI",
database "databasename",
table "vector_table",
loadpreSQL "set insertmode row"
)""");
sqlContext.sql("INSERT INTO vector_table SELECT * FROM spark_table");
Last modified date: 01/30/2023