When working with text files with special encoding characters, use a Unicode connector instead of an ASCII connector. Even if you do not have Unicode data, you have the option of setting the encoding property in the Unicode connectors. The encoding property options include non-Unicode and Unicode formats.

Setting the correct character encoding is important. If you are not sure which encoding to use, select UTF-8. It is the most common Unicode scheme and handles most issues. If needed, you can also use ASCII encoding.

When connecting a Unicode source data file with the Unicode connector, the encoding scheme should automatically change from OEM to the proper character set after the source file is specified. For instance, your file may specify UCS-2 when used as a source file. However, if the file does not having leading byte characters as described below, you may have to set the encoding scheme (to UCS-2 or UTF-16, for example).

When outputting to a Unicode target data file with the Unicode connector, the encoding scheme defaults to OEM. You should change the encoding to reflect the correct Unicode character set such as UTF-8 or UTF-16. Note that choosing UCS-2 in the Map window provides the same results as choosing UTF-16. For the purposes of data exchange, UTF-16 and UCS-2 are the same. Both are 16-bit and have the same code unit representation.

Choosing OEM can result in an output file containing replacement characters, such as "?" or "." in place of characters that were not transformed to the OEM code page.

To fix the issue, change the encoding to the correct Unicode character set of your target file.

If some characters still fail to display correctly, do the following:

The XML connector encoding is determined by the encoding property in the XML file.