I was asked to clarify my previous blog posting from yesterday with some examples, so I'll give it a go.
A simple example is an OASIS Universal Business Language (UBL) invoice's so-called 'calculation model' (an example of rules showing how calculations in the invoice are to be made). The invoice has all kinds of totals and amounts so the way you calculate totals from the amounts is important to get right.
In TAML (Test Assertions Markup Language) I can write the rules as test assertions but using XPath so they can be executed against the XML invoice:
...
<taml:testAssertion id="IN1" name="Invoice" enable="true">
<taml:normativeSource>U2ICMDraft5Rule1:
"To be a conforming UBL 2 invoice the document MUST be valid according to a standard UBL 2 Invoice schema."</taml:normativeSource>
<taml:target type="document" idscheme="'document'">/</taml:target>
<taml:predicate>count(//in:Invoice) ge 1</taml:predicate>
<taml:prescription level="mandatory"/>
<taml:report label="failed" message="Not a standard UBL 2 invoice">The file does not contain a standard UBL 2
invoice.</taml:report>
</taml:testAssertion>
<taml:testAssertion id="INTOT1" name="LineExtensionAmount (1)" enable="true">
<taml:normativeSource>U2ICMDraft5Rule2:
"The 'LineExtensionAmount' in the invoice 'LegalMonetaryTotal' SHOULD equal the sum of all 'LineExtensionAmount's in all of the invoice lines."
</taml:normativeSource>
<taml:target type="total"
idscheme="'invoice-total'">
/in:Invoice/cac:LegalMonetaryTotal</taml:target>
<taml:prerequisite>(count(distinct-values(//*/@currencyID)) eq
1)
</taml:prerequisite>
<taml:predicate>
number(./cbc:LineExtensionAmount) eq
sum(/in:Invoice/cac:InvoiceLine/cbc:LineExtensionAmount)
</taml:predicate>
<taml:prescription level="preferred"/>
<taml:report label="failed" message="Error in Line Extension Amount">
The line extension total is not the sum of the invoice lines'
line extension amounts.
</taml:report>
</taml:testAssertion>...
These are just two rules in an example set of rules.
If I use XML Schema (XSD) 1.1 to apply the assertions (effectively as test cases) by combining them with a schema (see previous blog posting) I run into some immediate problems:
1) Ideally I need to use a schema which targets a UBL invoice but a) my UBL invoice already has a schema b) my UBL invoice schema has some design rules which might make it tricky changing a globally defined element into a locally defined one
2) Assuming I can write my own schema in XSD 1.1, and can find a way to define some elements globally, if the assertion(s) targeting it allow this, and some elements locally if the assertions targeting those elements demand
it (e.g. have relevance only to certain contexts for that element)
3) Does every assertion map to one or more elements? I need to be able to turn any failed test of one assertion into a report refering to the one TA (by its ID). Can I do that?
4) How do I apply prerequisites? I could add them to the predicate perhaps but it makes it quite complex. The logic needs to be that if (count(distinct-values(//*/@currencyID)) eq 1)then the predicate applies, else it does not apply so it might be a little more complex than I'd like. Now this doesn't give me a mapping to my test assertion so I need to add the TA id somewhere - in a report or annotation, say (can I do that in XML Schema 1.1 ?).
That's how I'd like it to work but I'm told there is a hitch to this. The XML Schema 1.1 assert cannot lookup values from another part of the document. Duh! Still, I can handle it perhaps. I need to take all the values from the UBL that I want to test, calculate the appropriate totals, do the appropriate lookups and put the results in a special XML file - I might call it the provisional report file. It is then this file's markup, specially designed to support my test assertions, which I define using XML Schema 1.1.
e.g.
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="InvoiceCalculationModelReport">
<xs:complexType>
<xs:sequence>
<xs:element name="IN1">
<xs:complexType>
<xs:sequence>
<xs:element name="InvoiceCount">
<xs:simpleType>
<xs:restriction base="xs:int">
<xs:assertion test="$value ge 1"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="INTOT1">
<xs:complexType>
<xs:sequence>
<xs:element name="DistinctCurrencyCount">
<xs:simpleType>
<xs:restriction base="xs:int">
<xs:assertion test="$value eq 1"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="LegalMonetaryTotalLineExtensionAmount">
<xs:simpleType>
<xs:restriction base="xs:int"/>
</xs:simpleType>
</xs:element>
<xs:element name="SumOfLineExtensionAmounts">
<xs:simpleType>
<xs:restriction base="xs:int"/>
</xs:simpleType>
</xs:element>
</xs:sequence>
<xs:assert test="SumOfLineExtensionAmounts eq LegalMonetaryTotalLineExtensionAmount"/>
</xs:complexType>
</xs:element>
<!-- ... -->
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Then it is a relatively simple matter to extract values using code, database or XSLT (with help from the TA XPaths perhaps if the latter is used) from the target UBL invoice into a provisional report file which is validated by this schema. The report file would, in this example with just a few test assertions, look like the following:
<?xml version="1.0" encoding="UTF-8"?>
<InvoiceCalculationModelReport>
<IN1>
<InvoiceCount>1</InvoiceCount>
</IN1>
<INTOT1>
<DistinctCurrencyCount>1</DistinctCurrencyCount>
<LegalMonetaryTotalLineExtensionAmount>100</LegalMonetaryTotalLineExtensionAmount>
<SumOfLineExtensionAmounts>100</SumOfLineExtensionAmounts>
</INTOT1>
<!-- ... -->
</InvoiceCalculationModelReport>
The added advantage is that I can then get results into my provisional report file whatever the target system, even for a non-software system. The final output is the final report file which reports pass/fail/irrelevant of the like for each assert or better still, for each TA.
In the open source tool for executing test assertions written in XPath, called Tamelizer (hosted on Google Code) there is a similar way to handle such scenarios. The Test Assertions (TAs) are written in TAML and there is a provisional markup written describing the features of the targeted product. The executable TAs are written in terms of the markup used to describe the product features and executed against that marked up description document.
Here's a shot at how it would work for a target which is a mechanical widget with some features as follows
1) red button on top
2) number of batteries = 2
3) voltage = 6V
4) alarm sound = continuous
5) country of destination = UK
The spec might have some TAs written for it such as
TA1: if there is a 6V battery then the button on top MUST be red
TA2: there MUST be at least 1 battery
TA3: if there are two batteries then their voltage MUST be 3V
TA4: if the country of destination for the widget is US then the
button on the widget MUST be blue
The features can be marked up with some XML. The test cases are written as the TA expressions but in terms of the markup as XPaths
Now here the XSD 1.1 comes in. The markup for the features is defined now in XSD 1.1 so that the test cases can be added into the asserts of the respective elements.
The markup might be
<widget>
<buttonColour>red</buttonColour>
<batteries>
<number>2</number>
<voltage units="Volts">6</voltage>
</batteries>
<destination>UK</destination>
...
</widget>
The expession of a TA in terms of the above might be
<TA id="TA2"><predicate language="XPath">/widget/batteries/number gt; 1</predicate>...</TA>
The schema definition for batteries might be
<xsd:element name="batteries">
...
<xsd:assert>number > 1</xsd:assert>
...
</xsd:element>
Now the point is that the assertion needs to meet some criteria of TAs. One is that the report needs to refer to exactly one TA by its Id. There might be several TAs applying to the same element so the XSD 1.1 might or might not meet this requirement. This relates to the fact that a TA should itself map to a single requirement in the spec. The TA needs to be self-contained too so that this mapping is unambiguous.
If there is a prerequisite for a TA then there needs to be a corresponding prerequisite for the assertion in the schema.
It gets complex when I have a single prerequisite for many TAs together but I can just apply the same prerequisite logic to each and every assertion individually. If I do though the XPaths of testing might take longer. If I structure the schema though and the if I have control over the features markup too I might be able to put my prerequisites on ancestor elements so that descendants' assertions are irrelevant if those higher assertions fail. Now this is where portability might apply. I really want to make each assertion composable and composability makes portability desirable. Say I have many profles for my spec: I might want some TAs to be transferred between those profiles. This might mean I have to make the assertions in the schema portable for those TAs so that I can move them from a schema for one profile into a schema for another one. Having those higher level assertions might hinder this: It might be better to have any prereqs added to every assertion where it applies so that I can always copy that element with its prerequisites and predicate expressed all in the one assertion. That is really my main point. Plus I tend to think the TAs are going to be each self-contained and atomic so the assertions in the schema relating to them might need to be too.
Now I can go back to the real world example of the UBL invoice test assertions and instead of writing an XSD 1.1 schema again for UBL, I can write a schema for a list of elements to take values extracted from a UBL invoice - the tax totals, line totals, etc. I can derive it from an invoice using XSLT, say, or just programming code. Now the schema for this totals document can itself be written in XML Schema 1.1 and can be modelled along the lines of my set of test assertions. It can then take XPath Boolean expressions derived from my Test Assertions (if the latter are already written in XPath I might only want to combine the prerequisite and predicate expressions to get my XSD 1.1 assert expression). Then I can execute the schema against the totals in their XML markup and, using say a tool like Saxon which can read and execute the XSD 1.1, obtain a list of any deviations from the rules.
Not exactly keeping it simple but sometimes it has to be just a little complex to accomplish what we need, just no more complex than it needs to be, hopefully.
I have to say though, even when compared to Schematron, another assertion-based schema language based on XPath and executed using a two-step XSLT approach, I do rather prefer Tamelizer, the Google Code project, with XPath expressions in executable TAML test assertions. Maybe it won't get as wide use as XML Schema 1.1 but it is just right, I think, for test cases for XML document targets. It takes more skill to write all of the reasoning into the TAML XPath expressions but they execute against the actual target XML so for this kind of target it is worth that little more struggle getting the XPaths right. The prerequisite facility makes it all the more satisfactory in my view.
No comments:
Post a Comment