Testing XML don't use xUnit

[Ihe Onwuka <ihe.onwuka@gmail.com>]

Since it's the season for this sort of thing.

Anybody who has ever worked on the QA side of  test automation  (that
would be me) will instantly spot the flaw.

Some background. In the early 90's I was engaged to write scripts in
something called Automator/QA to automate a GUI interface application.
After writing a few of these things you begin to notice the pattern.

1. Automate the actions required to navigate to the screen where the
test will be executed.
2. Automate the actions required to Execute test.

The sort of things that went into 1t were - select some menu option,
enter some data, press enter, press a function key, select some
widget, tab enough times
The sort of things that went into 2 was entering the data pressing
enter and somehow capturing the response.

You soon notice that the navigational aspect of the test changes alot
and every time it does the test stopped working and had to be amended.
See the flaw yet.

The sequence of navigational instructions in my AutomatorQA scripts is
completely analagous to the XPath exposed in things like XMLUnit. When
you write that

assertXpathExists("//planet/somepath/[   ]/elem/*[@name='Earth']",
mySolarSystemXML);

anything that changes be it along the path or in the predicate is
going to break the test.

What I did back then was to abstract the navigation instructions into
a spreadsheet which was read and translated into actions by the test
execution tool. This mitigated the problem of changing navigation from
changing the program to just changing the spreadsheet. Obviously I
wasn't the only one that had this idea Years later somebody with
better marketing skills than I coined the phrase keyword driven
testing for this practice
https://en.wikipedia.org/wiki/Keyword-driven_testing and books have
been published about it
http://www.amazon.com/Just-Enough-Software-Test-Automation/dp/0130084689/ref=sr_1_1?ie=UTF8&qid=1365593600&sr=8-1&keywords=just+enough+software+test+automation

An experienced test automation dev would never expose navigation in
his test script, but that is exactly what tools like XMLUnit do. It
doesn't strike me that keyword driven option is appropriate here as
xpath entails different levels of complexity than encoding imperative
gui actions.

OK so things like XMLUnit fit well with the Java ecosystem but test
cases have a primary and dual purpose.

1. Highlight where a bug exists by failing a test.
2. Pass tests where bugs do not exist...... i.e NOT to flag up false
positives because they need to be investigated and that wastes time.

Tools that expose xpaths are not good at 2.

There is a ready made fix for this problem - write your assertions
with a tool that doesn't force you to expose your xpaths - Schematron.
This last point has been pretty much universally accepted and widely
known in the QA community since the turn of the millenium.