4 VALIDATION ARCHITECTURE

4.1 Validation Rules

This section defines the business validation rules architecture that was established by the Validation Working Group. It contains introductory and background information about the purpose of validation, a guide to how to make use of the documents published by the Validation Working Group, and serves as a reference point for implementers.

The main documents produced by the Validation Working Group is the set of validation rules, expressed in English:

• Equity Derivative Rules: Business constraints for EQD definitions.
• Interest Rate Derivative Rules: Business constraints for IRD definitions.
• Credit Derivatives Rules: Business constraints for CD definitions.
• FX Rules: Business constraints for FX definitions.
• Syndicated Loans Rules: Business constraints for Syndicated Loans definitions.
• Pricing and Risk Rules: Business constraints for Pricing and Risk definitions.
• Business Process Rules: Business constraints for Business Process messages.
• Shared Elements Rules: Supplementary business constraints in addition to the specific assets.
• ID / IDREF Rules: Business constraints for ID / IDREF relationships.
• Test Cases (zip file): Download file containing normative test cases.

Previous versions of the validation rules, expressed in English:

• Validation Rules 4.1 - July 19, 2004: http://www.fpml.org/spec/2004/FpML-4-1/validation/
• Validation Rules 4.0 - April 2, 2004: http://www.fpml.org/2003/FpML-4-0/validation/

The Validation Working Group's charter includes a mandate to formalize the validation rules from their plain English representation to a suitable implementation language. The purpose of this decision was to disambiguate the English language representation, to serve as a point of reference for implementers looking for clarification, and to validate the test cases produced by the working group. Reference implementations play an important part in providing quality guarantees to the working group and it is hope that their publication will be useful to implementers.

The reference implementations are non-normative and should not be considered publications of the Validation Working Group. Instead, they are contributions provided by members of the working group or other implementers.

Although the working group does not have the means to provide official certification, it expects implementers to demonstrate that they produce the correct results - valid or invalid - for the normative test cases of each implemented rule. There is no requirement to implement all rules, but reference implementations will state clearly which rules are implemented by publishing a list of rule identifiers.

The Validation Working Group also publishes a set of test cases with each rule set release (available above). For each rule, a number of valid examples and a number of invalid counter-examples are given. The test cases for a release will have been validated both manually and against at least one reference implementation to check that they are correct. The test cases have the following properties:

• All test cases are valid against the schema
• Every valid test case is valid against all rules (business valid), but is guaranteed to carry the information that the particular rule it is designed to test against makes use of.
• Every invalid test case for a rule is guaranteed to violate the rule, but may in addition violate other rules.
• Each invalid test case must state which instance of the context is invalid. For example an invalid document with more than one matching context must state which contexts are invalid, and the other contexts must be valid.
• There is no guarantee of completeness. The test cases are designed for disambiguation and test the most common violations, they are not an exhaustive set of boundary cases.

This document should be useful to:

• Architects that need to consider the impact of semantic validation on the overall workflow or processing pipeline. Relevant section: Validation Architecture Concepts
• Implementers of FpML that need to understand how to make use of the validation rules in implementations. Relevant section: Validation Rules and Reference Implementations
• Analysts or other interested parties who wish to use the validation rules to further their understanding of FpML. Relevant section: Background

FpML has been designed as a very rich language, with a large amount of optionality in its element and attribute definitions. This richness, which arises from the complexity of the information that FpML is designed to carry, has made it easier for implementers to fit FpML into existing architectures, and has facilitated the reuse of common element types across different asset classes.

Partly due to this flexibility, and partly due to the rich structure of FpML, it is unrealistic to assume that a schema-validated FpML document is necessarily correct in a business sense. The Validation Working Group was established early in 2003 to consider the implications of this problem and breaks down the correctness of an FpML document into three parts:

• Well-formed XML: The document meets the basic XML well-formedness constraints defined by W3C.
• Schema Compliance: The document is valid against the Schema that the document references.
• Business Rules Compliance: The document is valid against the validation rules published by the Validation Working Group.

It is useful to consider some simple example to illustrate the different levels of validity. The following is a fragment of a well-formed FpML document, but violates the schema due to an invalid start date:

<calculationPeriodDates>

  <effectiveDate>

    <unadjustedDate>xxxx-xx-xx</unadjustedDate>

    ..

  </effectiveDate>

  <terminationDate>

    <unadjustedDate>2004-02-29</unadjustedDate>

    ..

  </terminationDate>

  ..

</calculationPeriodDates>

The following version of the fragment is syntactically valid, because the contents of the effective date now match the schema definition of a valid date.

<calculationPeriodDates>

  <effectiveDate>

    <unadjustedDate>2004-08-29</unadjustedDate>

    ..

  </effectiveDate>

  <terminationDate>

    <unadjustedDate>2004-02-29</unadjustedDate>

    ..

  </terminationDate>

  ..

</calculationPeriodDates>

The fragment of FpML inserted above is however not semantically valid, because the effective date of the trade precedes the termination date. As a consequence, the trade is not meaningful and in fact violates one of the validation rules for IRD interest rate streams. Many other types of constraints will not be validated by the schema, which is designed to designate structure, not semantics:

• Constraints between the contents of multiple elements, like as in example given above
• Relationships between elements in distant parts of the document
• Complex data type operations that require processing, for example checking cash flows against the parametric representation
• Reference data validity, or other checks against external documents or databases

The validation working group has considered these problems and their implications, and has attempted to address a number of questions. How do we extract the knowledge about what constitutes a correct trade from business experts? What, if any, guarantees will we give that our work is complete? What is the normative status of the validation rules? How can parties override or tailor the rule set? What infrastructure support is necessary in the FpML architecture to enable rule-based checking? What support is necessary in the new messaging framework? The answers to these questions are addressed in the remainder of this section.

Since the Validation Working Group was established after most product working groups, it still has a lot of work to complete before the rules span the whole product range, and indeed are reasonably complete for any particular product type. The latest set of validation rules is thus likely to change more frequently than the specification, and the process of rule publication has been decoupled from the specification.

The validation working group undergoes the following process in preparing its releases:

• Comments from previous publications are reviewed and logged as issues.
• The working group collaborates with selected experts from product working group to establish an initial set of rules, expressed in plain English.
• The rules are reviewed and extended, by performing a structured walk-through of the schema for the product type.
• The chairman of the Validation Working Group calls a freeze on the rule sets when a substantial set of new rules is available, or critical issues need to be urgently resolved.
• Test cases are written for each rule and ambiguities in the language addressed. This process usually leads to the elimination of some rules, and discovery of additional ones.
• The test cases must be validated by at least one reference implementation, and further ambiguities eliminated.

At the end of this process, a new URI is allocated for the revised rule set, and the rule set is announced and published as the new normative rule set on the FpML web site. We expect to publish the process to take on average three months for the foreseeable future.

The rule sets and test cases are the normative content published by the Validation Working Group. The reference implementations, see below, are informative and not part of the normative process. This normative characteristic has the implication that applications that construct FpML documents have to be designed so that their output meets the full set of validation constraints.

Applications that receive FpML for processing may want to check the validation rules to ensure that the data they are receiving represent a valid FpML document. Processing applications are not obliged to reject invalid messages. Instead, the implications of rule normativity are that the receiver gains the right to reject invalid messages. In other words, the sender has to either be prepared for rejection, or the rule set has to be tailored.

The validation working group is concerned that there may be legal implications due to this definition in situations where trades are executed between counter-parties. While we believe that the definition given above, providing a right for the receiver to reject trades and placing the burden on the sender, is the least controversial, we would like to obtain additional feedback on this issue before the publication of the final recommendation.