FpML 5 XQuery rules

[andykey]

I am aware of XQuery based validation rules for FpML 4.x. Are there equivelents for the various FpML 5.x views? If not, is there a plan and/or timetable for creating them? If not, what is the gold standard, executable specification of these rules? What other executable specifications exist? Thanks {{{ Andy

[matthew]

Dear Andy – ISDA have asked me to respond to you. My authority to respond is that I was responsible for the big refactoring of the rules over the past few years to the point where they’re now correct, complete, and precise. And I am the author of the XPath Specification and the XQuery Implementation of the validation rules. I am responding in my personal capacity as a longstanding participant in FpML – part of the community. My response is nothing to do with previous, current, or future employers. The specification of the validation rules was never upgraded to work with FpML 5.x. The ‘views’ mechanism introduced in 5.x subsets the schema to different business contexts. This should subset the applicable rules and the contents of the rules. This was noted as an issue by ISDA nearly 3 years ago, in Issue 611: http://www.fpml.org/issues/view.php?id=611. The last activity recorded on this issue was over a year ago. There is no published plan or timetable for addressing this issue. The issue looks readily tractable to me, and could be resolved by someone with the standing to lead the consensus in the community. Before an implementation of 5.x can be considered the specification in the Standard for 5.x must first be resolved. Any implementation of 5.x validation rules I would personally regard as speculative until the Standard is revised to handle ‘views’ for the rules. The dilemma I faced as an implementer was whether to implement the Standard as specified, including what I believed to be bugs in the Standard, or whether to implement as I believed it should have been specified. Both approaches carry risk. My usual practise was to implement rules I believed to be correct, and not to implement broken rules. Other implementations took different approaches. The very best specification of the validation rules, the “gold standard” is the XPath Description of each rule in the Standard. This specification is always precise and complete and you can evaluate the XPath inside a range of tools, for example such as MarkLogic, XML Spy and Saxon. In contrast the English Description is in places ambiguous and incomplete, and will not evaluate in any tool. The Standard contains a Comment, which is nothing more than an informative non-normative guide to intent of the rule. The XQuery implementation is part of FpML, but not part of the Standard. The XQuery implementation is straight from the XPath Description of the Standard with only some formatting added. The Validation Working Group do not maintain the XQuery implementation. I’d use the term “evaluate” rather than “execute” for a description language. The only other implementation I know of (that evaluates using the official specification), is a bank has that a XSDL 1.1 version using xs:assert statements. This is a private implementation based on the XPath descriptions; though it could be reproduced for FpML. This would be another implementation, and not part of the Standard which uses XSD 1.0. In short, the problem is readily tractable, and would take only a few man-weeks to address for all versions. The first step is to address the specification, the second step is to address the implementation of that specification. – Matthew Rawlings

[andrew]

FpML 5.x currently has two defined views, confirmation and reporting. The confirmation view contains strictly defined product and business process like those in FpML 4.x and therefore it makes sense to support the full set of validation rules on this view. The case for rules on the reporting view is currently less clear. Most of our rules are product specific and intended to ensure that appropriate and consistent business data values have been defined. The product models in the reporting view are intended to express a summary of a trade and have very loose content models. We could add additional guards to the product model rules to allow them to be evaluated in the reporting view but as products in reporting messages are provided only for context the benefit of changes is limited. Business process rules are view specific and mostly dont apply in the reporting view. There is an argument for allowing some of the referential integrity rules to be applied in the reporting view. The choice of formal representation for the validation rules has been discussed many times within the working group. The XPath/XQuery style has both pros and cons. It is directly executable but as its an interpreted language I suspect its quite slow compared to other technologies (No one has ever published performance figures for an XQuery implementation). It can only implement a subset of the total number of rules as some require the coding of algorithms to manipulate dates according to common financial markets practice. These rules are in no sense ‘broken’. Probably the biggest issue with XPath/XQuery based rules however is that while the definition of the rule may be precise it is often utterly incomprehensible. For this reason the VWG favours structured English rule descriptions as these are more accessible to our user community. I doubt that FpML will ever move an XSD 1.1 Schema with embedded XQuery rules. Users of the big implementations based on FpML (e.g. MarkIt, DTCC, SWIFT) like rule checking to be separate from XML validation so that they can have looser rule sets and the ability to process syntactically correct but semantically incorrect documents if they want. The notion of a “Gold Standard” is subjective. Other implementations of the rules may implement the rule sets more completely, more efficiently and for more versions of FpML. They may also be more easly integrated into an enterprise messaging framework. They just don’t have thier source code linked into specification. – Andrew Jacobs (Validation Working Group co-chair)

[andykey]

Thanks for the replies above… I understand there may be more value in rules on the confirmation view than on the reporting view. Does the VWG still plan to produce XQuery rules for any FpML 5-x views? If so, upon what timescale? IMHO, the priority order should be : precise > maintainable > optimised As (only one) member of the user community, I support the idea that the normative specification is written in a formally precise language and that any english text is non-normative and is designed to aid understanding. XPath/XQuery/xs:assert might not roll off the tongue, but at least they’re readily accessible open standards. If the specification is precise, I (and vendors) stand a fair chance of being able to implement alternative optimised versions. I thought XQuery is a full Turing-complete language, thus allowing the expression of potentially complex algorithms. Is the difficulty around “common financial markets practice” related to access to external reference data or transactional state? {{{ Andy

[Author]

Posts