How External Tables Work
For the external table functionality, X100 leverages Apache Spark's extensive connectivity through the Spark Connector.
External Tables architecture is composed of two main components:
• Spark Provider
• X100 component of Actian X
X100 receives queries operating on External Tables from the user, rewrites them into JSON requests for external data, which are sent to the Spark Provider. The Spark Provider is a Spark application that behaves as a multi-threaded Spark server. It receives requests from Actian X, translates them into Spark jobs, and launches them. These jobs typically issue queries (to SparkSQL) like "INSERT INTO x100_table SELECT * FROM external_resource" for reading external data or "INSERT INTO external_resource SELECT * FROM x100_table" for writing to external systems. Finally, these jobs use the Spark Connector to push and pull data in and out of Actian X.
Last modified date: 08/14/2024