Using DataExchange
A Guide to Using DataExchange in Daily Operations
The following topics provide in-depth information for using DataExchange on a daily basis. They include discussions about fundamental concepts, as well as procedural information for dealing with some of the common scenarios you may encounter during replication.
Activation
Activation is the process of creating a new database for replication, or enabling an existing one, and defining it as belonging to a site within a replication network so that it can replicate. Because activation associates a database with a site, activating a database and activating a site mean the same thing.
The Deployment Wizard and dxdeploy command-line utility both activate replication databases automatically as part of deployment. The dxact command-line utility provides a manual method of activation as well as the flexibility to specify options such as preprocessing of starter data. For this discussion, we use dxact to activate the sample database, but first it’s important to first understand a few concepts in the activation process:
Activation and Data Source Names
When you activate a database, DataExchange creates an internal system data source name (DSN) to point to the activated copy. In addition, DataExchange installation creates a database called PSQL DataExchange Database Network Analysis (PDDNA), which is the internal replication database. This database appears in PSQL Explorer with the DNA icon in its Replication node.
Caution Do not alter or otherwise directly attempt to use the internal DNA database (PDDNA). Doing so can corrupt the DataExchange installation.
Activating First and Partner Sites
The First Site special because it contains information needed to manage the replication network. The First Site must be activated before any of the Partner Sites so that they each can obtain copies of the information. Once the First Site is activated, you may activate your Partner Sites in any order. You can use the Deployment Wizard, dxdeploy.exe or dxact.exe on each Partner Site machine to activate it.
Tip The replication engine must be running to activate a database.
If possible, each Partner Site machine should have a network connection to allow it to communicate with the First Site so that the initial replication session can start immediately. This is not mandatory, however. You can perform the initial replication at a later time if you prefer. In the Data Synchronization edition, a Partner Site cannot replicate with other Partner Sites until it performs an initial replication with the First Site.
Once you’ve successfully performed activation, PSQL Control Center displays the database as activated, with the Replication node icon showing a letter A.
Once the initial replication has succeeded, the newly installed site is fully aware of all administrative data in that network, including all replication schedules.
For more information
Specific information about using each of the DataExchange deployment utilities is available in Using the DataExchange Utilities and Services. The following table provides specific reference information.
 
1
2
dxact demodata /FI
This example activates the demodata database on the First Site using the dxact.exe utility. By default, messages from dxact.exe are written to the dxact.log file, not to the screen. To view the messages on the screen while dxact.exe runs, use the /LOGFILE= option. To do so, you would enter a command similar to the following:
dxact demodata /Logfile= /FI
The /FI option specifies you are activating the First Site, but you should always run dxact.exe from the site you want to activate – remote activations are not available in DataExchange. For the complete list of dxact.exe options available, see dxact.
Now that we’ve covered the basics of activation, the next topics are deactivation and reactivation.
Deactivation
Deactivation is the disabling of an activated database in a replication network. Deactivation uses the dxdeact utility as explained in Using the DataExchange Utilities and Services.
If you deactivate a database on the First Site, PSQL Control Center displays the First Site database as a design template, since it was used to design the database for the replication network. When the First Site is deactivated, the Replication node icon displays a letter T.
When a Partner Site is deactivated, in PSQL Control Center on that site the Replication node icon for the database no longer displays a letter A for activation.
Other sites do not detect a deactivated site until replication occurs, which fails for the deactivated site. This is normal and generates an error message similar to the following in the dre.log file:
E 019f 0301-15:46:25 Partner site Partner_Site_2 (site number: 00LFLU) has been removed from the replication network - it cannot replicate with any other site.
After the failed replication, the error no longer occurs because all sites in the replication network have detected the deactivated site.
1
2
dxdeact demodata
This example deactivates the demodata database using the dxdeact utility. By default, messages from dxdeact are written to the dxdeact.log file, not to the screen. To view the messages on the screen while dxdeact runs, use the /LOGFILE= option just as you did earlier with dxact.exe. It’s helpful to note that all the DataExchange Utilities provide the /LOGFILE= option to control logging output. For the complete list of dxdeact.exe options available, see dxdeact.
Reactivation
Reactivation is the reenabling of a deactivated site for replication. A database must be deactivated before it can be reactivated. Reactivation is, in essence, the same as activation in that you use the dxact utility in the same manner as you did when you activated.
If you deactivate a First Site, you must perform a manual replication if you reactivate the First Site. After you reactivate the First Site, you must manually initiate a replication session from each Partner Site to the First Site. The replication sessions ensure that all available sites appear on the Initiate Replication dialog.
Note Manually initiating a replication session is not required if you deactivate/reactivate a Partner Site. It applies only to a First Site.
If you deactivate a First Site, you must perform a manual replication if you reactivate the First Site. After you reactivate the First Site, you must manually initiate a replication session from each Partner Site to the First Site. The replication sessions ensure that all available sites appear on the Initiate Replication dialog. This action is not required if you deactivate/reactivate a Partner Site. It applies only to a First Site.
Adding Dynamic Tables to an Activated Database
You applications may add tables to a PSQL database that did not exist at the time database replication was set up in DataExchange. DataExchange provides a way to include these new tables in – or exclude certain tables from – the set of replicated tables for an activated database. Only table files should be included because only tables may be replicated. Any type of file may be excluded, but whether or not you exclude them, files other than tables are not replicated.
The new tables are identified by a pattern-matching algorithm, which uses glob-style patterns in both regular and dir expressions. You specify which file patterns you want the algorithm to use. Blank by default, the patterns are the only input required from you. DataExchange does the rest automatically.
DataExchange first compares file names for the include pattern then compares file names against the exclude pattern. Only files whose names pass both comparisons are included. For example, suppose you specify c:\mydata\*.d?? as an include pattern. If you specify an exclude pattern of c:\mydata\*.ddf, then all files in c:\mydata will be included except for ones with a file extension of “ddf.”
Tip For a detailed discussion about the types of expressions you can use as include and exclude patterns with dxdynpath, see Types of Expressions.
The inclusion or exclusion of tables begins when a table is successfully opened with the database open API. DataExchange then determines whether the table is identified as one to be replicated (that is, is the table new to replication). If the table is new and its name matches only with the exclude pattern, DataExchange takes no replication action on the table. DataExchange includes a new table if its name does not match with the exclude pattern but does match with the include pattern. Initially, after a successful open, the table is simply identified to DataExchange as being new. Data in the new table is not immediately replicated.
The next replication process (for example, from a scheduled replication) prepares the new table for replication and synchronizes the replication information about it on all sites. The second replication process includes the new table. Note, then, that the actual replication of data to or from a new table requires two replication cycles.
Note Inclusion of dynamically created tables applies only to Real-Time Backup replication. Also, the dynamically created tables must be on the machine where the database application is running. That is, the tables must be on the machine that serves as the source location from which data is replicated.
Before you Begin
If you’re unfamiliar with expressions or have had little exposure to working with them, we strongly recommend you first review Types of Expressions to gain a basic understanding of how to work with the different types of expressions.
Caution Any file used with the dxdynpath utility requires system keys, which are needed by PSQL for transaction logging purposes.
Include File Patterns
In our example database Demodata, we’ll automatically generate a set of tables every day that contain the attendance records for each campus. This is important data that must be included in the replication. We’ll add an include pattern using dxdynpath so these tables get replicated as well. The table name uses a naming convention of attend<campusid>.mkd, where <campusid> is the three letter campus code.
Caution This topic applies only if your application dynamically creates new tables for an activated database. This process is only recommended as a solution for tables that are added dynamically.
1
dxdynpath /t=include /pa="dir attend*.mkd" append demodata
2
dxdynpath /t=both show demodata
A message should display that lists the name and version of the dynamic table configuration and test utility, and additional information such as the following:
INCLUDE pattern is "dir attend*.mkd"
EXCLUDE pattern is ""
You may want to manually initiate replication after you specify new files to include. This completes the first cycle of replication required to prepare the dynamic tables for replication. The new tables are then ready for the second cycle (for example, a scheduled replication) in which data is replicated.
Exclude File Patterns
Now let’s fine tune our pattern expressions for our example database. We’ve included all of the attend*.mkd files, but there is a cumulative file (attendall.mkd) that gets created too. Since we have all the individual files that are included in the cumulative one, we don’t really need that file replicated. So, we’ll exclude it from our pattern matching.
1
dxdynpath /t=exclude /pa="dir attendall.mkd" append demodata
2
dxdynpath /t=both show demodata
A message should display that lists the name and version of the dynamic table configuration and test utility, and additional information such as the following:
INCLUDE pattern is "dir attend*.mkd"
EXCLUDE pattern is "dir attendall.mkd"
These simple examples illustrate how dxdynpath uses pattern matching to determine what files are replicated. For more detail on using dxdynpath and the types of expressions you can use, see dxdynpath.
Changing a Table Structure in an Activated Database
You may have a situation where a table that is currently part of a replication design changes structurally, or its system keys are updated, and these changes need to be synchronized with othersites. You’ll need to be sure that these changes are captured in your replication design, are synchronized across sites, and that you are able to successfully replicate with the changes in place.
Before you Begin
We strongly recommend that you confirm the following items before beginning this procedure:
Caution It is important that no database user or application is active while tables are being modified because any changes to the database cannot be captured while databases are deactivated.
The high-level steps needed to complete this procedure are as follows:
Clone PDC Tables
Although it is not required to use a blank copy of the PDC table, doing so will:
1
2
3
Tip For information on using the butil utility to clone tables, see PSQL Advanced Operations Guide.
Activate the First Site and Sync Tables
1
dxact /FI <Database_DSN>
For example, to activate the Demodata sample database, the command is dxact /FI demodata.
2
dxsynctables <TableName> <DSN>
Note This process may take some time to run, depending on the number of records contained in the user table. Tests show that in cases where there are several million records or more, disabling L2 caching can reduce the population times and speed up this process significantly. Make sure to reenable L2 after the user is finished with all the tables.
Copy Files and Activate the Partner Site
1
2
Activate each Partner Site using dxact.exe and the /pa[rtneraddress]=<string> option to designate the site with which you will replicate.
For example, if the First Site address is SiteA, you would enter a command similar to the following:
dxact /pa=SiteA <Database_DSN>
Verify the Replication Session
1
This directory is C:\ProgramData\Actian\PSQL\Replication\LogFiles.
2
3
You may now allow your users to return to the system.