Re: [xml-dev] It is okay for things to break in the future!

[Norman Gray <norman.gray@glasgow.ac.uk>]

Liam and all, hello.

On 5 Sep 2022, at 21:07, Liam R. E. Quin wrote:

> On Mon, 2022-09-05 at 19:46 +0100, Norman Gray wrote:
>>
>> Fine -- then where I said '12', read 'up to 15' (which Wikipedia
>> suggests is the ITU maximum). 
>
> And Wikipedia is never wrong. Although i gave a longer example earlier
> in the thread, which also contained (necessary) alphabetic characters.

Argh.  Forget about phone numbers!  The underlying point is not about phone numbers!

I stuck with the phone number example, even though I noted it was a poor one.  And it's a poor one because...

>>  The original point is not phone numbers, or about validation, but
>> the general point about a hard-coded maximum length not necessarily
>> being bad design.
>
> But you were willing to use 12 digits, which would exclude some
> legitimate values.
>
> If the purpose is to store telephone numbers and be standards compliant
> and not be blamed, go ahead, you found a standard that lets you use 12
> digits.

...It's easy to skim the Wikipedia page too quickly and produce a poor implementation.

The only relevance of phone numbers, as I see it, is that it demonstrates the existence of _an_ argument for a fixed size limit: the ITU says that phone numbers will be a maximum of 12^W 15 digits in length (so YAGNI).  Whether that's a good engineering argument is a separate question -- we have all rehearsed our favourite examples of stupid schema constraints and idiotic normalisations, which suggest that in the service of a variety of interesting use-cases, an ITU-inspired 15 character limit wouldn't be a good constraint.

If we go back to the post at the start of this thread [1], however, Roger selected and highlighted a quote from John Carmack, which I might as well requote here:

> I'm kind of fond in a lot of cases of static array size declarations. I went through this period where we should just make everything variable length because I had this history in the early days where Doom had some fixed limits on it and then everybody started making crazier and crazier things and they kept bumping up the different limits -- this many lines, this many sectors -- and it seemed like a good idea that we should just make it completely generic so it can go up to whatever. There are cases where that's the right thing to do, but the other aspect of the world changing around you is it's good to be informed when the world has changed more than you thought it would. If you've got a continuously growing collection, you're never going to find out. You might have this quadratic slowdown on something where you thought “Oh, I'm only ever going to have a handful of these,” but something changes and there's a new design style and all of a sudden you've got 10,000 of them. So I kind of like in many cases picking a number, some nice round power of two, and setting it up in there and having an assert saying “Hey, if you hit this limit, I need to know.” When that occurs, you should probably think: “Are the choices that I've made around all of this still relevant if somebody's using 10 times more than I thought they would? This code was originally written with this kind of world view, with this kind of set of constraints, and I was thinking of the world in this way.” If something breaks that means I’ve got to rethink the initial stuff.

That is an interesting point, which I don't recall having seen written down before.  That's all.

Liam, again:

> The "lex" lexical analyzer generator used to generate code that read
> input a line at a time, with a 200 character line length limit; longer
> lines caused a crash at runtime.  I think it's easy to argue
> convincingly this was unhelpful: general tools can't predict the
> context in which they will be used, so should not have limits.

And this is an example where this Carmack argument doesn't work.  It doesn't require a change in one's world-view to imagine input lines longer than 200 characters, so an implementation which doesn't accept arbitrary-length inputs is  probably a poor one.

But if there is a bit in your code where there is effectively a queue (I'm struggling to find a good concrete example here), you might decide to implement the queue insertion with what is effectively a O(N^2) algorithm, because it's bulletproof, and N is never going to be more than a few.  The argument is saying 'Go ahead and do that -- don't beat yourself up -- _but_ it would be wise to think briefly about how big N can be, and wise to fix N, and to ensure the program deliberately fails with 'rewrite me!' when that limit is breached.'

Actually rereading the original post, I see that Roger did in fact mention validation there (I'd forgotten that, because the Carmack quote caught my attention).  I think this limits argument will be less often persuasive in the context of schemas, to the extent that schemas are about intention rather than implementation.

I wouldn't want to claim the argument can never ever work with schemas.  Roger remarks: 'Your XML Schema was originally written with one world view, if validation starts breaking that means you have got to rethink the initial stuff.'  That's true, and it is indeed useful to know when a world-view is wrong, but in most cases it would be the implementers who should discover and complain about this, rather than the schema designer.

And I see I've gone on at too much length, again...!

Best wishes,

Norman


[1] http://lists.xml.org/archives/xml-dev/202208/msg00076.html

-- 
Norman Gray  :  https://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK