Building DataFlow Applications > Building DataFlow Applications > Building DataFlow Applications Using RushScript > Additional Scripting Features > Setting Engine Configuration Properties
Was this helpful?
Setting Engine Configuration Properties
The dr variable contains several functions for setting engine configuration values. Engine settings will then be used on the next graph execution.
A list of functions for setting engine configuration values are enumerated in the following table.
Function Name
Input Parameters
Returns
Description
batchSize
int: batch size (optional)
Previous batchSize value
Sets the ports.batchSize engine configuration to the specified value. Returns the previous value of the ports.batchSize setting. See Engine Configuration Settings for more information.
cluster
• String: cluster host name
• int: port number
Previous cluster specification
Sets the cluster specification on the current engine configuration. The next execution invocation will execute on the defined cluster if it exists. Returns the previous value of the cluster setting. See Engine Configuration Settings for more information.
socksProxy
• String: proxy host
• int: proxy port
void
Sets the socket provider to a SocksSocketProvider with the specified host and port. Can be used to contact a DataFlow cluster via a socks proxy. See Engine Configuration Settings for more information.
dumpfilePath
String: local path name (optional)
Previous dumpfilePath setting
Sets the dumpfilePath engine configuration to the specified value. Returns the previous value of the dumpfilePath setting. See Engine Configuration Settings for more information.
maxMerge
int: maxMerge value (optional)
Previous maxMerge setting
Sets the join.maxMerge engine configuration setting. Returns the previous value of the join.maxMerge setting. See Engine Configuration Settings for more information.
maxRetries
int: maxRetries value (optional)
Previous maxRetries setting
Sets the maxRetries engine configuration setting. Returns the previous value of the maxRetries setting. See Engine Configuration Settings for more information.
monitored
boolean: monitored value (optional)
Previous monitored value
Sets the monitored engine configuration setting. Returns the previous value of the monitored setting. See Engine Configuration Settings for more information.
parallelism
int: parallelism value (optional)
Previous parallelism value
Sets the parallelism engine configuration setting. Returns the previous value of the parallelism setting. See Engine Configuration Settings for more information.
sizeByReaders
boolean: sizeByReaders value (optional)
Previous sizeByReaders value
Sets the ports.sizeByReaders engine configuration setting. Returns the previous value of the ports.sizeByReaders setting. See Engine Configuration Settings for more information.
sortBuffer
String: sortBuffer value (optional)
Previous sortBuffer value
Sets the sort.sortBuffer engine configuration setting. Returns the previous value of the sort.sortBuffer setting. This value is set using a text value to represent the size. Use "k", "m" and "g" suffixes to represent kilobytes, megabytes and gigabytes, respectively. See Engine Configuration Settings for more information.
sortIOBuffer
String: sortIOBuffer value (optional)
Previous sortIOBuffer value
Sets the sort.sortIOBuffer engine configuration setting. Returns the previous value of the sort.sortIOBuffer setting. This value is set using a text value to represent the size. Use "k", "m" and "g" suffixes to represent kilobytes, megabytes and gigabytes, respectively. See Engine Configuration Settings for more information.
spoolThreshold
int: spoolThreshold value (optional)
Previous spoolThreshold value
Sets the ports.spoolThreshold engine configuration setting. Returns the previous value of the ports.spoolThreshold setting. See Engine Configuration Settings for more information.
storageManagementPath
String: storageManagementPath value
Previous storageManagementPath value
Sets the storageManagementPath engine configuration setting. Returns the previous value of the storageManagementPath setting. See Engine Configuration Settings for more information.
writeAhead
int: writeAhead value (optional)
Previous writeAhead value
Sets the ports.writeAhead engine configuration setting. Returns the previous value of the ports.writeAhead setting. See Engine Configuration Settings for more information.
enabledModules
String: module to use
Previous value
Sets the moduleConfiguration engine configuration setting. See Engine Configuration Settings for more information.
The following code example demonstrates setting the engine configuration values.
Example of setting engine configuration values
// Define the schema wanted from the data generator
var schema = dr.schema().DOUBLE('dblField').STRING('stringField');

// Create a data set of randomly generated data
var data = dr.generateRandom({rowCount:1000, outputType:schema});

// Log the data set
dr.logRows(data, {logFrequency:0, disableParallelism:true});

// Set engine configuration values.
dr.parallelism(8);      // explicitly set parallelism to 8
dr.writeAhead(4);       // set write ahead (queue size)
dr.monitored(true);     // enable monitoring
dr.sortBuffer('2m');    // sort buffer to 2 Megabytes

// Execute the created application using the set engine configuration values.
dr.execute('SimpleApp');
Last modified date: 06/14/2024