User Guide > Designing and Executing Data Profile > Adding Rules from Rules Tab > Derived Fields
Was this helpful?
Derived Fields
Derived fields are not part of the original source data file or table. Instead, they are dynamically generated in memory using specific rules or scripts. These fields are created using function or conversion rules applied to existing fields, allowing you to generate new output fields derived from the original field.
Derived fields can be leveraged for further rule creation (see Add Rules from Derived Fields) or used in calculations within scripts.
By default derived fields are non output fields and are listed under -NonOutputFields- (see Managing Rules). Non-output-fields are fields that exist in memory, have rules defined on them, but are not written to the target (output). However you can use the Add source/derived fields to output option to write them to target (see Add Source/Derived Fields to Output).
The following are some examples of derived field usage:
Example 1:
If your source data file contains a string field called DateOfBirth and you need it to be a date field for further calculations, you can use the StringToConversion rule to convert it into a date data type derived field. This will generate a derived field called d_DateOfBirth_StringToConversion. You can then create additional rules for the derived field (see Add Rules from Derived Fields) or write it to the target (see Add Source/Derived Fields to Output).
Note:  In the Add Rules dialog, or in the Expression column of the Rules tab > Field/Rule pane, the derived field, is listed below all the source fields.
Example 2:
If your field is a string but should be a numeric field, and you need to test that the field values are not greater than 2000, you can follow these steps:
First, use the StringToConversion rule to convert the string field into a numeric data type derived field. This will generate a derived field called d_FieldName_StringToConversion. This derived field will now be available for further use.
Next, apply the CompareToConstant rule to this derived field. Set the Operator to ">" and specify the Constant value as 2000.
Add Rules from Derived Fields
You can add rules for a derived field in the same way as you add rules for a source field. See Add New Rule.
To add a new rule using derived fields:
1. Open an existing profile and select the Rules tab.
2. Click and select Add a New Rule from the context menu.
The Add Rules dialog is displayed. This dialog allows you to add rules for a single derived field. To add rules to multiple derived fields, launch the dialog multiple times. The rule parameters can be configured once back in the editor.
3. Select a derived field from the Select Field Name drop-down list.
The derived fields are listed at the bottom of the drop-down list and their names are preceded with a “d_” unless you changed the default name. Based on the selected derived field, the available rules get listed in the Profiling, Remediation, and Data Prep tabs.
4. Select a rule and click Finish.
The Add Rules dialog (for details see Add New Rule) closes and focus returns back to the Rules tab, and the new rule is displayed on the Field/Rule pane. The rule details are displayed on the Rule Definition pane. A default rule name is provided which follows the <FieldName>_<RuleType> format. For example, City_IsNotBlank. You can edit the name or click Reset to revert to the default name.
Note:  The underscore (_) character is the only special character allowed in the name. Rule names cannot begin with a digit. If a field or column in the source data starts with a digit, 'r_' will be prepended to any rules created based on that field.
5. Specify the rule parameters (see Rule and Parameter Reference).
Note:  The parameters may be different for different Rules.
6. Save the profile.
 
Add Source/Derived Fields to Output
By default, all source fields are considered as output fields, are written to the target, and are listed under OutputFields section, unless you choose to delete them from the OutputFields section. Source fields that are not written to target are listed as NonOutputFields. All Derived Fields are also by default not written to the target and are listed as NonOutputFields.
Thus NonOutputFields are fields that exist in memory, have rules defined on them, but are not written to the target (output). However you can use the Add source/derived fields to output menu option to write them to target.
See Also: Managing Rules
To add a source field or derived field to output:
1. Open an existing profile and select the Rules tab.
2. Click and select Add source/derived fields to output from the context menu or right-click anywhere in the Field/Rule pane and choose Add source/derived fields to output.
The Choose source fields to add back dialog appears.
3. Select the fields to add to output.
The fields are listed under Field/Key names. Derived field names are preceded with a “d_” unless you changed the default name.
Note:  You can select or deselect all available fields using Select All and Unselect All.
4. Click Ok.
The Choose source fields to add back dialog closes, and focus returns back to the Rules tab. The field you selected is moved from NonOutputFields to OutputFields.
If the fields were moved to NonOutputFields using the Delete Fields > Keep option, the fields and the mapping in the Expression field, as well as associated rules are displayed in the OutputFields list. You can also drag and drop the field (with its associated rules) into the OutputFields list, if needed. When a field is dragged from NonOutputFields to OutputFields, it is inserted above the target field.
If the fields were moved to NonOutputFields using the Delete Fields > Delete option, only the fields and the mapping in the Expression field is displayed in the OutputFields list.
5. Save the profile.
 
Last modified date: 09/22/2025