Was this helpful?
How External Tables Work
For the external table functionality, X100 leverages Apache Spark's extensive connectivity through the Spark Connector.
External Tables architecture is composed of two main components:
Spark Provider
X100 component of Actian X
X100 receives queries operating on External Tables from the user, rewrites them into JSON requests for external data, which are sent to the Spark Provider. The Spark Provider is a Spark application that behaves as a multi-threaded Spark server. It receives requests from Actian X, translates them into Spark jobs, and launches them. These jobs typically issue queries (to SparkSQL) like "INSERT INTO x100_table SELECT * FROM external_resource" for reading external data or "INSERT INTO external_resource SELECT * FROM x100_table" for writing to external systems. Finally, these jobs use the Spark Connector to push and pull data in and out of Actian X.
Last modified date: 08/14/2024