Re: [xml-dev] Testing XML don't use xUnit

[Michael Sokolov <msokolov@safaribooksonline.com>]

On 04/11/2013 06:37 AM, Stephen D Green wrote:
>
> I also do a lot of work with testers writing tests for HTML pages 
> using the
> very well-known integration test tool Selenium IDE (for Firefox) and that
> makes heavy use of XPath even though the target is HTML. Here we end
> up with many tests which include XPaths which break every time a
> significant change is made to the HTML. So we have to keep mending the
> XPath expressions (even though we try to write them with //, etc, to avoid
> this). So I know the problem. It doesn't stop people favouring Selenium
> IDE for website integration tests though. It's well worth the hassle 
> to many.
> I guess it might be different if we were using XPath for unit tests but I
> don't know about that. I guess modern unit tests get associated with TDD
> and Agile, so maybe having separate test assertions (written in prose)
> would be a 'no no' for many unit test writers as it might be seen as going
> against Agile mantras. You'd have to update the assertions less 
> frequently
> than the unit tests but having to update something other than the tests
> and the XML / code would be just too much for many Agile teams. So
> if you wanted to inut test your XML you'd probably be stuck with 
> having to
> keep revising a load of XPath expressions. It goes with the territory.

The problem with Selenium tests is not the tool: it's the markup.  HTML 
is not semantic markup, which means: it's not meaningful.  We can 
extract meaning from it, but its not easy because the underlying 
application logic (which is usually what we want to test) is not what 
people are thinking about when they are crafting the markup.   They're 
thinking about how to get pixels to line up on four different browser 
platforms.

Selenium IDE is great for testing layout.  Not as good for testing 
behavior, but in some cases it's all we have.  We've had good success by 
working with the layout developers to get them to include meaningful 
identifiers in the markup *so that it can be tested*.

If you want to make sure the price calculation is displayed to the user 
properly on a checkout screen, it's much more stable and testable if 
there is an id="price" attribute on the price display box.  Otherwise, 
if you end up having to write XPath like 
//div[@class="blue-box"]/ul/li[3]/span to select the price, you've lost 
the game.

No amount of tooling support is going to allow you to write stable tests 
if the ground is constantly shifting underneath you.

-Mike