User Guide : Map Connectors : Source and Target Map Connectors : ODBC 3.x Mass Insert
 
Share this page             
ODBC 3.x Mass Insert
The term ODBC stands for "Open Database Connectivity". This is a standard promulgated by Microsoft to allow client-side Windows applications to access various backend Databases through a single common interface. With the ODBC 3.x Mass Insert connector, the integration platform writes to target tables.
Note:  This connector only supports 64-bit ODBC drivers.
ODBC 3.x Mass Insert is a rapid way of inserting records into a relational database. It bypasses the transactional layer of the database and adds information directly to the storage tables. ODBC 3.x Mass Insert is a useful option if you are dealing with large tables and performance is of paramount importance, but you do not have the option of a native connection.
Connector-Specific Notes
SQL Server: The use of Regional settings in ODBC DSN connections to SQL Server databases is not supported in this release. In order for currency, numbers, dates and times to be interpreted correctly, ensure that the Use regional settings when outputting currency, numbers, dates and times checkbox in the SQL Server DSN settings is not checked. Then in the ODBC connector DriverOptions property, type Regional=No.
Property Options
You can set the following source (S) and target (T) properties.
Property
S/T
Description
AutoCommit
T
If set to True, it allows you to automatically commit changes as they are made by each SQL statement, instead of waiting until the end of the transaction.
Also, if this option is set to True, you cannot roll back changes after they are done. It will overwrite the CommitFrequency value, which means that the changes are committed by each SQL statement irrespective of the value set for CommitFrequency.
The default value is False.
BulkOperations
T
Use bulk operations for faster insert. This property determines if an insert statement is executed for each record or a bulk add is executed for each record. The default is true. To cause an insert statement to be executed for each record, this should be set to false, the slower setting.
CommitFrequency
T
Allows you to control how often data is committed to the database when the AutoCommit property is set to False.
The default value is zero that is, the data is committed at the end of the transformation, allowing rollback on error. This is the slowest setting. When performing large transformations, this is not practical as it may produce too many transaction log entries.
Specifying a nonzero value indicates that data is committed to the database after inserting or updating specified number of records.
ConstraintDDL
T
Allows you to specify some additional SQL data definition language statements to be executed after their target table is created. This is similar to the support we provide for SQL pass-through in the SQL import connectors. Each line must be a valid ODBC DDL statement.
For example, you could have the statements:
CREATE UNIQUE INDEX index1 ON mytable (Field1 ASC)
CREATE INDEX index2 ON mytable (Field2, Field3)
These statements would create two indices on the table "mytable". The first one does not allow duplicates and the index values are stored in ascending order. The second index is a compound index on fields Field2 and Field3.
The ConstraintDDL is executed only if the replace mode is used for the target. If there are any errors, the errors are written to the error and event log file. During the transformation, the transformation error dialog box appears. If you want to ignore the DDL errors, they can continue the transformation.
ConstraintDDL also supports an escaping mechanism that allows users to specify DDL in the native SQL of the DBMS. Any statement that is preceded by an '@' is sent straight to the DBMS.
The statement:
@CREATE INDEX pk_mytable ON mytable (Field1, Field2) WITH PRIMARY
is a DDL statement for creating a primary key for the table mytable. Some ODBC drivers do not support the SQL extensions needed to create a primary key with the ODBC variant of the SQL CREATE statement. So to create primary keys, use native SQL.
Note:  This option works only in REPLACE mode.
DriverCompletion
T
Allows you to control whether or not the driver prompts you for information. The options are Prompt, Complete (default), Complete Required, and No Prompt. The Prompt option prompts you for every individual bit of information. Complete prompts you for any information they forgot to enter. The Complete Required option prompts you only for information that is essential to complete the connection. The No Prompt option does not prompt you for any information.
DriverOptions
T
Enter any valid ODBC connect string options here.
DSNType
T
Data sources or drivers to connect to in the Data Source Name (DSN) list. DSNs are listed in Control Panel > Administrative Tools > ODBC Data Source Administrator window. The options are:
User: Created for a specific user. DSN information is stored in the registry.
System: Can be used by anyone who has access to the machine. DSN information is stored in the registry.
User & System: This is the default option.
For more information, see Configuring ODBC Data Source.
IdentifierQuotes
T
All databases have what are called quoted identifiers. You use quoted identifiers to make the SQL statement parseable and distinguish between columns and character data in SQL statements. For example, Oracle uses double quotes for column and table names in SQL statements and uses single quotes for character data. In a SQL statement, you should enclose identifiers containing special characters or match keywords in identifier quote characters; (also known as delimited identifiers in SQL-92). For example, the Accounts Receivable identifier is quoted in the following SELECT statement:
SELECT * FROM "Accounts Receivable"
If you do not use identifier quotes, the parser assumes there are two tables, Accounts and Receivable and return a syntax error that they are not separated by a comma.
IdentifierQuotes has four options:
Default
None
"
'
MaxDataLength
T
The maximum data length for long data types. Default is 1 MB. You can reset this number as you choose based on your available memory capabilities and target requirements.
Some ODBC drivers have limitations concerning the maximum data length they can handle. If you choose an Application (ODBC) Source or target connector and the default setting is not 1 MB, then the integration platform sets the default for the capabilities of that particular ODBC driver. It is not recommended that the MaxDataLength property be set any higher under those conditions.
ModifyDriverOptions
T
Allows you to store the ODBC connection. The default is true. If you set this to false, you are prompted for your connection information each time that you run your transformation.
PrimaryKey
T
The PrimaryKey property allows you to specify a list of field names that are used to make the primary key. The field names are delimited by commas. If the PrimaryKey property contains one or more field names, these names are included in the SQL CREATE statement when the connector is in replace mode.
There is one additional requirement for using the PrimaryKey property. The ODBC driver must support integrity enhancement facility (IEF). Only the more advanced ODBC drivers support this.
Synonyms
T
If set to true, this property allows you to see synonyms. The alias names appear in the table list along with the tables. Default is false.
SystemTables
T
The SystemTables property is only applicable if you are logged onto the database as the database administrator. Only the DBA has access to SystemTables. If set to true, this property allows you to see all the tables created by the DBA. The system table names appear in the table list. The default is false.
TransactionIsolation
T
The Translation Isolation option allows you to specify any one of five different isolation levels when reading from or writing to a database table with ODBC. The default is Serializable.
The ANSI SQL 2 standard defines three specific ways in which serializability of a transaction may be violated: P1 (Dirty Read), P2 (Nonrepeatable Read), and P3 (Phantoms).
The following lists the four supported isolation levels.
READ_UNCOMMITTED - Permits P1, P2, and P3.
READ_COMMITTED - Permits P2 and P3. Does not permit P1.
REPEATABLE_READ - Permits P3. Does not permit P1 and P2.
SERIALIZABLE - Does not permit any of P1, P2, and P3.
For more information, see the Microsoft ODBC SDK documentation.
UseCursors
T
The UseCursors property allows you to turn cursor support on and off. The default is false. If you set the property to true and the specified ODBC driver does not support cursor inserts, the integration platform uses the SQL INSERT mode of adding records.
For exports, cursor support is supposed to enhance the performance of inserting records. This appears to be the case for desktop databases. For database servers, there is no noticeable change in insert speed. They seem to execute prepared queries about as quickly as they handle cursor inserts
Another complication of the cursor inserts is that some drivers require that the target table be indexed, otherwise positioned updates (cursors) are not allowed. Two additional properties in the ODBC export connector address this issue: PrimaryKey and ConstraintDDL (see details above).
ArraySize
T
Determines the number of rows to be sent to the server at one time. The default value is 1000, meaning each row is individually sent to the server. Larger values will buffer multiple rows and send them all at once. While this improves the speed, it affects error reporting (a server error won't be detected/reported until the next batch of records is sent to the server).
The maximum value allowed for this property is 100000. While the connector allows the value to be set that high, many drivers have lower limits. The connector will log a message indicating if the driver is forcing a lower value for the array size. In addition, the connector does not support arrays when there is a LOB-type field in the table, or when the (maximum) length of a character-type field is longer than 32767 characters. In these cases, a message will be logged indicating the array size has been reduced to 1.
Due to the way the connector attempts to support older drivers, the array support requires BulkOperations and UseCursors is set to True or both must be set to False. If BulkOperations is False and UseCursors is True, then the array size is ignored and a message is logged indicating this condition.
Views
T
If set to True, this property allows you to see the view names in the table list along with the table names. Default is True.
Note:  This property supports only Append and DeleteAndAppend output modes and does not support the Replace output mode.
Encoding
T
Sets the character encoding for ODBC 3.x Mass Insert source and target files.
Note:  Shift-JIS encoding is used only in Japanese operating systems. UCS-2 is no longer a valid encoding name, but you may use UCS2 by changing the string value in the XML file.
Data Types
The only available data type for an ODBC connection is Text.
Length
These are field lengths in your data. If you need to change field lengths, reset them in the schema.