Data Integrity
The following features support concurrent access while ensuring the integrity of your files in a multi-user environment:
Record Locks
Applications can explicitly lock either one record at a time (single record lock) or multiple records at once (multiple record lock). When an application specifies a record lock, the application can also apply a wait or no-wait condition. When an application requests a no-wait lock on a record that is currently unavailable (either the record is already locked by another application or the whole file is locked by an exclusive transaction), the MicroKernel Engine does not grant the lock.
When an application requests a wait lock on a record that is unavailable, the MicroKernel Engine checks for a deadlock condition. You can configure the MicroKernel Engine to wait before returning a deadlock detection status code. Doing so improves performance in multi-user situations by allowing the MicroKernel Engine to wait internally, rather than forcing the application to retry the operation.
Transactions
If you have a number of modifications to make to a file and you must be sure that either all or none of those modifications are made, include the operations for making those modifications in a transaction. By defining explicit transactions, you can force the MicroKernel Engine to treat multiple operations as an atomic unit. Other users cannot see the changes made to a file until the transaction ends. The MicroKernel Engine supports two types of transactions: exclusive and concurrent.
Exclusive Transactions
In an exclusive transaction, the MicroKernel Engine locks the entire data file when you insert, update, or delete a record in that file. Other applications (or other instances of the same application) can open the file and read its records, but they cannot modify the file. The file remains locked until the application ends or aborts the transaction.
Concurrent Transactions
In a concurrent transaction, the MicroKernel Engine can lock either records or pages in the file, depending on the operation you are performing. The MicroKernel Engine enables multiple applications (or multiple instances of the same application) to perform modifications inside concurrent transactions in different parts of the same file simultaneously, as long as those modifications do not affect other previously locked portions of the file. The record or page remains locked until the application ends or aborts the transaction. Concurrent transactions are available only for 6.0 and later files.
Exclusive vs. Concurrent
Clients can still read records from a file even if a concurrent transaction has locked the requested record. However, these clients cannot be operating from within an exclusive transaction. Also, they cannot apply a lock bias to their read operation if the file containing the requested record is currently locked by an exclusive transaction, or if a concurrent transaction has locked the requested record.
When a client reads a record using an exclusive lock, the MicroKernel Engine locks only the individual record; the rest of the page on which the record resides remains unlocked.
*Note: Simply opening a file from within a transaction does not lock any records, pages, or files. In addition, the MicroKernel Engine does not lock files that you flag as read-only or files that you open in read-only mode.
When you use exclusive transactions, the MicroKernel Engine causes other clients to implicitly wait on the locked file unless the No Wait bias is added to the Begin Transaction (19 or 1019) operation. The application seems to hang during this implicit wait state. If these exclusive transactions are short lived, you may not notice the wait time. However, the overall effect of many clients involved in implicit waits results in using a large amount of CPU time. Additionally, multiple position blocks in the same file share locks.
Exclusive transactions involved in implicit waits also waste network bandwidth. The MicroKernel Engine waits about one second before returning to the requester. The requester recognizes a wait condition and returns the operation to the MicroKernel Engine. Thus, exclusive transactions also can cause extra network traffic.
The amount of extra CPU cycles and network traffic increase exponentially with the number of clients waiting on the locked file combined with the length of time involved in the exclusive transaction.
Transaction Durability
You can configure the MicroKernel Engine to guarantee Transaction Durability and atomicity by logging all operations to a single transaction log. Transaction durability is the assurance that the MicroKernel Engine finishes writing to the log when a client issues the End Transaction operation and before the MicroKernel Engine returns a successful status code to the client. Atomicity ensures that if a given statement does not execute to completion, then the statement does not leave partial or ambiguous effects in the database, thereby ensuring the integrity of your database by keeping it in a stable state.
If you want atomicity without the overhead of Transaction Durability, you can use Transaction Logging feature in PSQL V8 and later releases. See Advanced Operations Guide for more information on Transaction Logging.
In a default installation, the transaction log is in C:\ProgramData\Actian\PSQL\logs. The log must exist on the same machine as the PSQL engine. You can change the location using the transaction log directory configuration option in PCC by right-clicking the MicroKernel Engine, selecting Properties, and clicking Directories.
The MicroKernel Engine maintains the transaction log in one or more physical files, called log segments. The MicroKernel Engine starts a new log segment when the current log segment reaches a user-defined size limit, no files have pending changes, and the MicroKernel Engine has finished a system transaction.
All transaction log segments have a .log extension. The MicroKernel Engine names log segments with consecutive, 8-character hexadecimal names, such as 00000001.log, 00000002.log, and so on.
To improve performance on specific files, you can open a file in Accelerated mode. (Version 6.x MicroKernel Engine accepted Accelerated open requests, but interpreted them as Normal open requests.) When you open a file in Accelerated mode, the MicroKernel Engine does not perform transaction logging on the file.
*Note: If a system failure occurs, there will be some log segments that are not deleted. These segments contain changes that did not get fully written to the data files. Do not delete these log segments. You do not know which files are represented in these log segments. No action is necessary because those data files will automatically get rolled forward the next time they are opened.
System Data
PSQL uses a 7.x transaction log file format. In order for the MicroKernel Engine to log transactions on a file, the file must contain a log key, which is a unique (non-duplicatable) key that the MicroKernel Engine can use to track the record in the log. For files that have at least one unique (non-duplicatable) key, the MicroKernel Engine uses one of the unique keys already defined in the file.
For files that do not have any unique keys, the MicroKernel Engine can include system data upon file creation. The MicroKernel Engine includes system data in a file only if the file uses the 7.x file format or later and if at the time the file is created, the MicroKernel Engine is configured to include system data in files upon creation. System data is defined as an 8-byte binary value with the key number 125. Even if a file has a unique, user-defined key, you may want to use system data (also called the system-defined log key), to protect against a user dropping an index.
The database engine turns on data compression for the file if the file uses system data and the record length exceeds the limit shown in Table 12.
The MicroKernel Engine adds system data only at file creation. If you have existing files for which you want to add system data, turn on the System Data configuration option, then use the Rebuild utility.
*Note: When the MicroKernel Engine adds system data, the resulting records may be too large to fit in the file’s existing page size. In such cases, the MicroKernel Engine automatically increases the file’s page size to the next accommodating size.
Shadow Paging
The MicroKernel Engine uses shadow paging to protect 6.0 and later files from corruption in case of a system failure. When a client needs to change a page (either inside or outside a transaction), the MicroKernel Engine selects a free, unused physical location in the data file itself and writes a new page image, called a shadow page, to this new location. During a single MicroKernel Engine operation, the MicroKernel Engine might create several shadow pages all the same size as the original logical pages.
When the changes are committed (either when the operation is complete or the transaction ends), the MicroKernel Engine makes the shadow pages current, and the original pages become available for reuse. The MicroKernel Engine stores a map, which is the Page Allocation Table, in the file to keep track of the valid and reusable pages. If a system failure occurs before the changes are committed, the MicroKernel Engine does not update the PAT, thereby dropping the shadow pages and reverting to using the current pages, which are still in their original condition.
*Note: This description simplifies the shadow paging process. For improved performance, the MicroKernel Engine does not commit each operation or user transaction individually, but groups them in a bundle called a system transaction.
When a client operates inside a transaction, the shadow pages corresponding to the original logical pages are visible only to that client. If other clients need to access the same logical pages, they see the original (unchanged) pages—that is, they do not see the first client’s uncommitted changes. Shadow paging thus enhances reliability because the original file is always valid and internally consistent.
Backing Up Your Files
Backing up your files regularly is an important step in protecting your data.
For information on backing up your files, please see the following topic in Advanced Operations Guide: Logging, Backup, and Restore.