Re: ASN.1 and XML

["Simon St.Laurent" <simonstl@simonstl.com>]

On 24 May 2001 20:11:56 +0100, Al B. Snell wrote:
> On 24 May 2001, Simon St.Laurent wrote:
> 
> > > well, the ASN.1 type notation is a
> > > bit gnarly to parse right now, but those parsers are not needed in
> > > programs that use ASN.1 encodings; just in the codec generators in the
> > > toolkits used at compile time, and there is some interest in defining a
> > > notation for ASN.1 types in ASN.1 itself, much like XML Schema replacing
> > > DTDs. I think this would be neat.
> > 
> > And is that process something you'd like to explain to a large number of
> > Web developers with scripting skill but no grasp of codecs?  Is creating
> > and processing ASN.1 (in your mind) nearly as accessible as creating and
> > processing XML?
> 
> Which process do you refer to?

All of the parts you described above.  "Gnarly parsing", codecs, and
compile time...

> The design process of specifying ASN.1
> types as ASN.1 values? The only thing I see in my paragraph that would
> need explaining to the scripting people is how to invoke the codec in
> their scripting language, which will work pretty much like how they access
> the XML parser from their scripting language.

And how do you plan to address the people who work in XML on a regular
basis but aren't actually programmers?  People who don't mind getting
their hands dirty in the data, maybe even applying regular expressions
to it, but aren't planning on writing scalable systems for large-scale
data interchange?

While I'm happy to admit that ASN.1 is better optimized for certain
situations, a lot of see that optimization as coming with costs that
aren't worth bearing for a lot of work.

> I don't mean that making it accessible is bad, just that of all the
> technologies they choose to put effort into, we could have chosen a good
> one rather than one full of holes people will keep tripping over and
> reinventing wheels. 

Well, I guess markup has been called worse.  Personally, I'm happy to
work with XML because I do real work using it at practically zero cost.
I can explore it in a text editor, serve it on a Web server, generate it
with tools built for HTML, and do all kinds of stuff with tools that
aren't even designed with XML in mind.

> All these debates about object/data/document, whether
> XML schemas are good or bad, the meaning of namespace URIs, whether a
> binary format is worthwhile - they've all been solved long ago with ASN.1.
> You can get on with defining useful schemas and improving the toolkit
> implementations or porting them to new languages, and stop wasting your
> time with figuring out how to adapt a document markup language to a data
> interchange format!

It might be a waste of my time, except that data interchange formats
aren't really what I do.  I'm happy to apply the XML knowledge I picked
up on the docment side to data problems, and I'm especially interested
in blurring the boundaries between documents and data.

It sounds like you're purely focused on data.  I'm happy to see that
document tools have proven useful for data communities, but haranguing
me (and probably much of this list) about the need to build
highly-optimized data interchange formats probably isn't going to win
you a whole lot of converts.

> Dude, read my emails! I just explained that you only need a disassembler
> if you choose to use a non-textual encoding!
>
> Yet still everyone using ASN.1 DOES choose non-textual encoding. Textual
> encodings tend to be used for examples in books and debugging log output,
> as far as I can tell, and that's about it.

I'll take the second paragraph to negate the first.  Textual encodings
in XML are used equally abundantly in examples and in reality. 

> Exciting?!?! This isn't here to amuse you! PER is a tight, compact,
> encoding. It was defined because it was needed. XML doesn't give you a
> choice. It limits you for no good reason.

Your limitation, my freedom.  If I really wanted a "tight, compact
encoding" I think I might actually find PER exciting.  Since I have
other priorities - readability among them - I find PER limiting for no
good reason.

> No, but how much of what is happening today with XML is heavily inspired
> by SGML? See the email before this one for more on that.

Actually, lots.  Repurposing technologies doesn't necessarily mean that
all the experience that led to the creation of those technologies is
discarded.
 
> I've not studied the history that hard, but XML seems to have gained hype
> since the promises of the Semantic Web struck at the hearts of marketing
> types, AFAICT.

That would explain why so many people on this list keep looking at the
Semantic Web sideways.  I'd suggest that you might want to study the
history again.

> The idea of representing computer-processable data over the
> Internet seemed novel to them, making new products possible. And so the
> "big players" took it on. ASN.1 is marketted as a way of representing
> information, nobody ever claimed it would revolutionise the way you do
> anything; it's just a tool. Now we know that the Semantic Web probably
> won't revolutionise anything either, but the marketting people don't see
> things that way :-)

Is the fault in ASN.1's stars, or in itself?  XML seems to hit people
and make them jump up and down - usually happily.  I think there's more
going on than marketing.  (Though I'll be the first to admit that there
is a cruel learning curve beyond the fundamentals.)

> Where it not for the hype, XML would be no better - the accessability
> stuff is a symptom of the hype, that's all; it's the hype that annoys me
> :-)

Well, I see the hype as a symptom of the accessibility, so round and
round...

> > Yeah, well, a lot of us don't have much admiration for the finer things
> > in life, and would rather devote our lives to the advancement of
> > technologies which focus on humans as much as on machines. 
> 
> That's fine, but don't try to use those technologies with machines. For
> machine stuff, use machine technologies. If you want to focus on humans,
> develop user interfaces or baby foods. If you're designing a way for
> machines to communicate, then do it properly!

I'm not sure declaring that some information is meant for humans and
some is meant for machines is such a wise decision.  A lot of the
excitement around XML comes from the ways in which it explicitly
violates that separation.

> > Tools-available reflects accessibility and motivation.  XML got its
> > initial start from a group of deeply motivated and pretty thoroughly
> > experienced developers who seem to have wanted to create something far
> > more accessible to people with less experience than themselves.  
> 
> They could have made ASN.1 accessible to people with less experience than
> themselves (it's no more complex to use than XML, darnit - the users don't
> need to know about the details of BER or PER; you could use ASN.1 with
> textual encodings in exactly the same way people use XML, except that the
> standards would be stabilised and the wrinkles ironed out already!).

They just didn't do it.  And telling people they "don't need to know
about the details" doesn't always go over very well.  With XML, people
can get into the details all the way down quickly and easily.  (Once
they start to pile things up, of course, it gets weird.)
 
> The existance of BER and PER doesn't mean that BER and PER are vital to
> ASN.1. Think of the ASN.1 encodings like SGML application profiles - it's
> a pretty good analogy. People can stick to XML without going into wierd
> profiles that use = and + instead of < and >!

Getting rid of application profiles was a lot of what XML 1.0 was about.

> You can do textual encoding with ASN.1. 

Yes, and you _can_ encode binary information in XML as Base 64.  People
don't tend to use either practice when they can avoid it.

> And, if you wish to, you can
> invest the extra effort to use compact binary encodings when you want to.
> If you don't want to, you can ignore them.

Just like people _could_ ignore all of the features in SGML, if they
wanted to.  The mere existence of ignorable features adds significantly
to the perception of complexity.

> Perhaps writing everything in
> ASCII *is* the best way of transferring data over the Internet. That's not
> a problem.

Replace ASCII with Unicode - that was another large move XML made.

> But the same tools can deal with compact binary data from
> embedded systems if you just plug in the appropriate decoder/encoder
> module. A good toolkit will give you the option, but not force you to use
> it. I explained that!

But why would I want to get into that?  If you write an ASN.1 parser
which emits SAX events and an ASN.1 writer which consume SAX events, I
might be willing to use ASN.1 in those cases where I absolutely have to
work with ASN.1 streams.  Otherwise, I'd really rather stick with what
I've got, thanks!


> No! It's not! XML lets you do less than ASN.1 does, and is about equally
> complex (the fact that there are several ASN.1 encodings does *not* make
> it more complex, you can ignore the ones that aren't applicable to your
> application, or even ignore the issue altogether and let the toolkit use
> its default, which will probably be BER. Tell it to choose XER if you
> want a textual format, but in practice this will probably only be used to
> present data to a human, and never used as *input*.).

That's the longest parenthetical phrase I've seen in a long while
telling me why something isn't complex.  There's a usable subset of XML
that I could probably explain in the same number of characters if you
really want to push.
 
> The user does not need to understand encodings at all with ASN.1. With
> XML, they do need to be able to read and write XML. With ASN.1, your
> toolkit handles all this stuff for you, like your TCP/IP stack handles the
> networking and the filesystem handles disk allocation, and your DBMS
> handles indexing.

"The user does not need" does not sound enticing to me.  If I want
access, I want it yesterday.  As far as users needing to be able to read
and write XML, there's nothing which says they need to use Notepad or vi
- they can wrap their work in toolkits the same way you describe for
ASN.1.

> To the script author, using an ASN.1 toolkit will be almost identical to
> using an XML parser, if not easier since it will use the schema
> information to map things to types in the language you're using rather
> than just giving it all to you as a DOM tree of strings.

And if I found types deeply interesting, I might be excited about that.
To the extent I do find them interesting, I can get the same information
out of the PSVI - far more information than I want in any event.  It's
not difficult to encode the same information in XML for applications
where it's appropriate, nor do I expect it to be more difficult for
script authors to reach that information through code.

> If you don't have a schema, then it's not the end of the world. You can
> use a SAX/DOM like interface such as the IAAPI mentioned on
> http://www.oss.com/products/tools.html. Then it will be as awkward as XML
> processing, but only then. Normally, it's easier.

What you see as awkward seems to be precisely what I find convenient and
useful.

> If you pay the money for a decent ASN.1 toolkit (as opposed to getting a
> free XML one - yeah, I know, but I'm working on that :-) then you will get
> more than XML provides for less learning effort. ASN.1 is designed for
> this kind of problem, and solves it pretty well. Of course it's not
> perfect, but it's ahead of XML in what it can do!

I'm sorry, but I really just plain don't believe that claim after the
discussion above and prior encounters with ASN.1.  It may well be true
that ASN.1 is a more optimal solution for particular kinds of problems.
I find the costs of that optimization to substantially outweigh the
benefits.