Rule and Parameter Reference
You can create profiling Rules of different types to identify data quality issues within a data set. Each profile rule contains one or more conditions, allowing you to evaluate data against those conditions. The records that adhere to the profiling rules are written to the “Pass Target” and the records that do not adhere to the profiling rules are written to the “Fail Target”. Since data is written into two separate files, each target can be consumed or processed in separate workflows once the profile has been run. For example, the data in the Pass Target, is considered “clean” and can be consumed immediately, the data in the Fail Target is considered “dirty” and can be routed separately for remediation.
Note: If your data contains whitespaces, the rules may not work correctly as rules consider whitespace as a part of the data. Hence if a rule is not working as expected, check for whitespaces in the source data. You can workaround the problem with some of the rules by adding spaces to the values in the rule parameters.
Listed below are the different Rule Types:
Last modified date: 10/30/2024