xml-dev - RE: Improved writing -- who's going to pay for it?

[Jonathan.Robie@SoftwareAG-USA.com]

Title: RE: Improved writing -- who's going to pay for it?

Linda van den Brink wrote:
>
> >
> > Is it therefore not sensible to do
> > "usability testing" (involving real users)
> > on documents?

Earlier this year, I did a survey on Schema, and one of the questions had to do with the readability of the specifications. With 30 respondents, the results were as follows:

<snip source="http://www.ibiblio.org/xql/tally.html">
Readability

Readability seen as a significant problem for the Structures specification. One half of all respondents (15) agreed with the statement that "Readability seriously interferes with the purpose of this document. This really needs to be rewritten before it is released." No respondent found it "well written and helpful". About one third felt either that it was either usable with some shortcomings (5) or that it does the job (5). Several comments stated that respondents had tried to read structures but had been unable to do so. Almost two thirds (18) found that this specification was not readable compared to other technical specifications. Comments on the structures specification showed a level of frustration. The following comments are drawn from different individuals: "Don't know, since I cannot read Part 1", "I don't really know (whether the important details are spelled out). It's hard for me to find them.", "The complexity of the document is an indication that certain things aren't well designed. Removal of certain features and refinement of schema for schemas could remove a whole lot of the complexity." Those actually developing software tended to feel that the important details actually are spelled out, but hard to find and hard to understand. One individual writing such software said, "Although this document was extremely difficult to digest, after re-reading it about 5 times I finally feel that I have a good handle on how things work...I don't mind complex documents, but the language obscures the meaning in these sections." Another person writing schema-based software said, "A lot of detail is there, but sometimes there seems to be a framework guiding the spec but it is mysterious to me. Perhaps it is a reflection of the theoretical immaturity of this area of work, but it would be nice to have more answers to 'why', and not just what and how." Another developer had to admit defeat (though he did not specify the structures specification), saying "I have *attempted* to read the WDs - I have retreated to experimenting against the online validator". These kinds of comments indicate to me that the Structures specification may not yet be mature enough to given to the development community.

The Primer and Datatypes specifications are significantly more readable. Only one respondent felt that the Primer seriously needed to be written, and only two felt that the Datatypes specification seriously needed to be rewritten. Most felt that the Primer was either well written and helpful (8) or does the job (9), and a further 8 felt it was usable with some shortcomings. A few less than half (12) felt that the datatypes specification was usable with shortcomings, and another third felt that it was either well written and helpful (3) or at least does the job (6).

</snip>

So I think we already *know* that there are big problems with readability for the Structures draft.

The Schema WG is working to do significant rewriting of the Structures specification to deal with the readability problem, but this work is to be done during the Candidate Recommendation phase. This is unfortunate, because it interferes with the usefulness of the Candidate Recommendation period. If people can't read it, or if implementation takes too long, we won't get good feedback.

Incidentally, I don't think that all specifications need to be readable for *users*. I didn't find the XSLT documentation particularly good as an introduction - I like Michael Kay's Saxon documentation much better for that - but the XSLT documentation does tell implementors what they need to know to produce an implementation.

Jonathan