Data Profiler allows you to create data quality rules using Data Profile Wizard or Data Profile Editor, that are specific to an individual use case or project. You can configure Profiler rules to identify specific data patterns, formats, and values. You can also configure rules to identify null, blank, or duplicate values in fields. This helps you to quickly determine quality levels of the source data, identify the types of problems, and reduce issues resulting from propagating bad data to down-stream systems and applications.

In Data Profiling, the data is profiled (identified) based on its conformance to configured rules (conformance to identified data quality characteristics of accuracy, completeness, consistency, timeliness, validity, uniqueness, etc.). Once data has been profiled, data quality can be quantified (calculated), and an insight into specific data quality issues (what types, how many) can be determined. Then the records that adhere to the profiling rules are written to a “pass” output file and the records that do not adhere to the profiling rules are written to a “fail” output file. Since the data is written into two separate files, the “passing” data can be processed and the “failing” data can be rectified of its data quality problems.

You can optionally assign data profile test rules to data quality dimensions, which give you more detailed insights about the quality of your dataset. Data quality dimensions are industry standard data quality characteristics, such as Accuracy, Consistency, and Timeliness. Each dimension generates a data quality score, as well as additional unique views of your data. Importance levels can be set for rules by assigning rule weights, which are figured into scores. For information about dimensions, see Data Quality Dimensions.

You can also enable notifications for monitoring how data quality is changing/evolving in your dataset. For more information, see Configuring Execution Options.

In addition to the Data Profile Editor, there is a Data Profiler Invoker which enables you to execute profiles from within the Process Designer workflows and also from the command line interface. For more information, see Data Profiler Invoker.