The following sections describe how to specify the data file format for table columns. The format specifies how each is written and delimited in the data file.

The character formats are the ones most commonly used to read and write ordinary text (human-readable) data files.

The basic character formats are BYTE, C, CHAR, and TEXT. TEXT(0) is a variable-length form. BYTE(n), Cn, CHAR(n), and TEXT(n) are fixed-length forms. An optional delim may follow to specify a delimiter.

The subtle differences between the various character formats are described in COPY Format Details.

The BYTE VARYING, LONG BYTE, LONG VARCHAR, LONG NVARCHAR, NVARCHAR(n), and VARCHAR formats are "counted" formats: each data file value is preceded by a character count. The character count defines the length of the data value; the actual field length as defined by a fixed-length specifier or a delimiter may be larger. On input (COPY FROM), extra field characters beyond those included by the embedded character count are ignored. On output (COPY INTO), any extra field length after the actual value is filled with padding, as defined by the specific format.

The fixed-length forms are BYTE VARYING(n), NVARCHAR(n), and VARCHAR(n). The variable-length forms are BYTE VARYING(0), LONG BYTE(0), LONG VARCHAR(0), LONG NVARCHAR(0), and VARCHAR(0). An optional delim may follow to specify a delimiter.

For all fixed-length counted formats: the field length N does not include the preceding length specifier. For example, a VARCHAR(1) field takes 6 bytes. When reading data (COPY FROM), if the character count found in the data is larger than the defined length, a runtime conversion warning is issued and the row is not loaded.

These counted formats are the only ones that can be used with LONG table columns. The two LONG formats can only be used with LONG BYTE or LONG VARCHAR table columns. See COPY Format Details.

The D (dummy) format describes a data file column that does not map to any table column. On input (COPY FROM), a D format column describes file data to be skipped and discarded. On output (COPY INTO), a D format column describes constant data to be sent to the data file.

The column name given for any dummy column is not matched to any table column. The Dn form for COPY INTO uses the column name as the value to output; all other uses of the dummy format ignore the column name completely.

The Unicode formats are NCHAR, NVARCHAR, and LONG NVARCHAR and they can only be used with NCHAR or NVARCHAR table columns. Fixed-length forms are NCHAR(n) and NVARCHAR(n). Variable-length forms are NCHAR(0), NVARCHAR(0), and LONG NVARCHAR(0).

Fixed-length NCHAR(n) and NVARCHAR(n) formats read and write using the two-byte UCS-2 encoding. The variable-length NCHAR(0), NVARCHAR(0), and LONG NVARCHAR(0) forms read and write using the variable-length UTF8 encoding.

The field length n for NCHAR(n) and NVARCHAR(n) should be specified as character lengths, not byte (octet) lengths. However, the embedded length specifier used by the NCHAR(0) and NVARCHAR(0) formats should give the number of bytes, not characters. (The reason is that NCHAR(0) and NVARCHAR(0) use the UTF8 encoding, which encodes Unicode code points into a variable number of bytes. COPY needs the byte count to know how many bytes to read and decode from UTF8.)

In a UTF8 installation, if using CHAR(n) as a copy format and the column is a Unicode column, then n must be 4 bytes for each UCS-2 character or the data may be truncated.

For example:

A field length of 16 is required to safely copy out the 4 UCS-2 characters.

The formatted COPY statement supports binary formats that match the binary types used to store data in tables. These are the BOOLEAN, DATE, DECIMAL, FLOAT, INTEGER, and MONEY formats (and size variants such as BIGINT, SMALLINT, REAL, and so on). Most data files are text, not binary, so these binary formats are not often needed.

This section describes specifying the format of fields in the data file. When specifying data file formats for COPY INTO, be aware of the following points:

The following table explains the details for the various COPY list formats. Most non-binary formats can be followed by an optional delim to specify a delimiter (see Delimiters in the Data File). In some cases, a delimiter is required

For user-defined data types (UDTs), use CHAR or VARCHAR.

Delimiters are characters in the data file that separate fields and mark the end of records. Valid delimiters are listed in the following table:

When a single character is specified as the delimiter, enclose that character in quotes. If the data type specification is C or D, the quotes must enclose the entire format. For example, 'd0%' specifies a dummy column delimited by a percent sign (%). If the data type specification uses parentheses around the length, quote only the delimiter. For example, char(0)'%' specifies a char field delimited by a percent sign.

Be careful using the sp (space) or null delimiters, especially with COPY FROM. Spaces or nulls are used as padding characters by many of the COPY formats. If a pad character is improperly treated as a delimiter, the COPY FROM will get out of sync with the input, eventually producing an error. When designing a data file format, use delimiters that will not appear in the data or padding, or use CSV or SSV forms.

The CSV and SSV delimiters allow COPY to read and write files that contain comma separated values (CSV).

The rules for a CSV delimited field are:

The SSV delimiter works exactly the same as the CSV delimiter, with semicolon in place of comma.

CSV and SSV delimiters are only allowed with BYTE(0), C0, CHAR(0), and TEXT(0). They are not allowed with the “counted” formats (VARCHAR(0) and so on); the count defines the value exactly and there is no need for quoting. (If delimiting is desired, use the comma or nl delimiters on counted formats.)

COPY FROM: Some CSV file variants use quote escaping (\") instead of quote doubling ("") to indicate a quote inside a quoted string. The C format handles\- escaping, so use the C0CSV format and delimiter to handle this type of file. (CSV with COPY INTO always writes quote doubling—never quote escaping—when needed.)

The WITH NULL clause allows the data file to contain NULL data. There are two methods of indicating a NULL: an indicator byte (no value given), or a special user-defined marker value. The indicator byte method does not depend on a particular marker value, but it is a binary form not suited to human readable text files. Also, the indicator byte method is not available for variable-length data fields. The user-defined marker value method works with any file format, but depends on a user chosen value that must not otherwise appear in the data.

When copying data from a table to a file, the WITH NULL (value) clause directs COPY to put the specified value in the file when a null is detected in the corresponding column. If a null is detected and there is no WITH NULL clause, a runtime error occurs, and aborts the COPY statement.

When copying data from a file to a table, the WITH NULL (value) clause specifies a marker value to be interpreted as a null. When COPY encounters this value in the file, it writes a null to the corresponding table column. The table column must be nullable; if it is not, a runtime error occurs, and aborts the COPY statement.

To prevent conflicts between valid data and null entries, choose a value that does not occur as part of the data in your table. The value chosen to represent nulls must be compatible with the format of the field in the file: character formats require quoted values, and binary numeric formats require unquoted numeric values. For example:

This example of a value is incorrect:

because the value specified for nulls (numeric zero) conflicts with the character data type of the field. However, this example is correct:

because the null value is character data, specified in quotes, and does not conflict with the data type of the field. Do not use the keyword null, quoted or unquoted, for a numeric format.

When copying from a table to a file, be sure that the specified field format is at least as large as the value specified for the WITH NULL clause. If the column format is too small, the DBMS Server truncates the null value written to the data file to fit the specified format.

For example, in the following statement the string, 'NULL,' is truncated to 'N' because the format is incorrectly specified as one character:

The correct version specifies a 4-character format for the column.

If WITH NULL is specified but value is omitted, COPY uses a trailing indicator byte in the file to determine whether a file value is a null. COPY INTO writes a zero trailing byte if the value is not NULL; it writes a nonzero trailing byte to indicate a NULL. (The value written prior to the indicator byte is undefined if the indicator shows NULL.) COPY FROM reads and interprets a trailing indicator byte in the same manner, zero for not null and nonzero for null. Indicator bytes are not allowed for variable-length formats such as char(0); variable-length formats must specify a value clause.