Re: ASN.1 and XML

["Al B. Snell" <alaric@alaric-snell.com>]

On 24 May 2001, Simon St.Laurent wrote:

> > well, the ASN.1 type notation is a
> > bit gnarly to parse right now, but those parsers are not needed in
> > programs that use ASN.1 encodings; just in the codec generators in the
> > toolkits used at compile time, and there is some interest in defining a
> > notation for ASN.1 types in ASN.1 itself, much like XML Schema replacing
> > DTDs. I think this would be neat.
> 
> And is that process something you'd like to explain to a large number of
> Web developers with scripting skill but no grasp of codecs?  Is creating
> and processing ASN.1 (in your mind) nearly as accessible as creating and
> processing XML?

Which process do you refer to? The design process of specifying ASN.1
types as ASN.1 values? The only thing I see in my paragraph that would
need explaining to the scripting people is how to invoke the codec in
their scripting language, which will work pretty much like how they access
the XML parser from their scripting language.

Something like:

$contact = asn_load_module("http://www.warhead.co.uk/oid/contact.asn1");
$x = contact->decode("MobilePhone",...source of ASN.1 encoded data...);

echo "<p>" . $x->number ."</p>";

...in a scripting language, I guess.

> > Point to something you can do in XML that you can't do in ASN.1 :-) XML is
> > only competition for ASN.1 because it's being pushed by hype; there's
> > nothing it can usefully do that couldn't be done as well or better before.
> 
> There's really nothing technical in data representation that hasn't been
> done before.  Making data representation accessible to a much wider
> range of developers is pretty clear new, however, and worth the
> attention you appear to dismiss as hype.

I don't mean that making it accessible is bad, just that of all the
technologies they choose to put effort into, we could have chosen a good
one rather than one full of holes people will keep tripping over and
reinventing wheels. All these debates about object/data/document, whether
XML schemas are good or bad, the meaning of namespace URIs, whether a
binary format is worthwhile - they've all been solved long ago with ASN.1.
You can get on with defining useful schemas and improving the toolkit
implementations or porting them to new languages, and stop wasting your
time with figuring out how to adapt a document markup language to a data
interchange format!

> One of the big wins of XML over ASN.1 is that you need much less than
> the facilities you describe above. You don't need a schema to get real
> work done.  You don't need a disassembler to inspect your information.

Dude, read my emails! I just explained that you only need a disassembler
if you choose to use a non-textual encoding!

Yet still everyone using ASN.1 DOES choose non-textual encoding. Textual
encodings tend to be used for examples in books and debugging log output,
as far as I can tell, and that's about it.

> You don't need to ponder encoder modules or compile-time choices much of
> the time.

(sigh)

> > BER is like XML without the element/attribute names, just numbers in
> > place. PER is like XML without any identifiers, but usage of the schema to
> > know what's expected in what order, and a small number field of a few bits
> > where there's a choice.
> 
> Thanks, but XML without the element/attribute names sounds like a
> roadmap with no roads on it to me.  And PER sounds almost as exciting as
> the PSVI.

Exciting?!?! This isn't here to amuse you! PER is a tight, compact,
encoding. It was defined because it was needed. XML doesn't give you a
choice. It limits you for no good reason.

> > Another win of ASN.1 over XML is 15 years of practical experience using it
> > in the real world. That's hard to beat :-)
> 
> Uh... SGML goes back way before my time.  1986 for ISO, 1960s for early
> development.  XML wasn't built of whole cloth.

No, but how much of what is happening today with XML is heavily inspired
by SGML? See the email before this one for more on that.

> > ASN.1 is easy to learn... there are books out there that cover it all
> > quite nicely. And you don't even need to read all of them.
> 
> And, uh, why exactly is it so lacking in hype if it's so easy to learn.
> Microsoft does use ASN.1 for a lot of different projects, so I don't
> think it's that...

I've not studied the history that hard, but XML seems to have gained hype
since the promises of the Semantic Web struck at the hearts of marketing
types, AFAICT. The idea of representing computer-processable data over the
Internet seemed novel to them, making new products possible. And so the
"big players" took it on. ASN.1 is marketted as a way of representing
information, nobody ever claimed it would revolutionise the way you do
anything; it's just a tool. Now we know that the Semantic Web probably
won't revolutionise anything either, but the marketting people don't see
things that way :-)

> Wow.  So you really think ASN.1 is the way, even though that paragraph
> demonstrates how far behind ASN.1 is as far as reaching a wide audience
> of developers.  Sounds like your annoyance has a lot more to do with
> accessibility than with hype.

Where it not for the hype, XML would be no better - the accessability
stuff is a symptom of the hype, that's all; it's the hype that annoys me
:-)

> > However, there is still hope; we're working on techniques for
> > interoprating ASN.1 with XML, so it will be possible to implement "XML
> > systems" that actually use superior ASN.1 encodings and abstract syntax
> > (=schema) internally, yet still interoperates with the heathens by just
> > seeing XML as another encoding. 
> 
> Great.  It's nice to know we're heathens.

Pleasure :-)

> > Seeing intelligent people getting all excited over XML makes me worry. Am
> > I and the sympathisers I communicate with really mad to find XML somewhat
> > disappointing compared to existing technologies that do the same
> > thing? What is the world coming to? Why do I bother? I devote my life to
> > the study of fundamental platform/tool technologies, exploring the state
> > of the art, and people go and use *XML*...
> 
> Yeah, well, a lot of us don't have much admiration for the finer things
> in life, and would rather devote our lives to the advancement of
> technologies which focus on humans as much as on machines. 

That's fine, but don't try to use those technologies with machines. For
machine stuff, use machine technologies. If you want to focus on humans,
develop user interfaces or baby foods. If you're designing a way for
machines to communicate, then do it properly!

> Remember, we're heathens.

Hey! Don't take it personally! I was being "wry" :-(

> > I really have never seen anything you can do with XML that you can't do
> > better in ASN.1, apart from the tools-available issue, which is silly!
> 
> Tools-available reflects accessibility and motivation.  XML got its
> initial start from a group of deeply motivated and pretty thoroughly
> experienced developers who seem to have wanted to create something far
> more accessible to people with less experience than themselves.  

They could have made ASN.1 accessible to people with less experience than
themselves (it's no more complex to use than XML, darnit - the users don't
need to know about the details of BER or PER; you could use ASN.1 with
textual encodings in exactly the same way people use XML, except that the
standards would be stabilised and the wrinkles ironed out already!).

The existance of BER and PER doesn't mean that BER and PER are vital to
ASN.1. Think of the ASN.1 encodings like SGML application profiles - it's
a pretty good analogy. People can stick to XML without going into wierd
profiles that use = and + instead of < and >!

> I don't think XML was ever about solving data problems perfectly, but
> rather about creating a flexible system in which programmers and even
> document authors could walk up, kick the tires, look under the hood, and
> get work done without huge barriers between them and the data.  Textual
> markup does really well at that...

You can do textual encoding with ASN.1. And, if you wish to, you can
invest the extra effort to use compact binary encodings when you want to.
If you don't want to, you can ignore them. Perhaps writing everything in
ASCII *is* the best way of transferring data over the Internet. That's not
a problem. But the same tools can deal with compact binary data from
embedded systems if you just plug in the appropriate decoder/encoder
module. A good toolkit will give you the option, but not force you to use
it. I explained that!

> Personally, I get excited about the potential of doing more with less.
> Seems to be a common tendency on the Web.

No! It's not! XML lets you do less than ASN.1 does, and is about equally
complex (the fact that there are several ASN.1 encodings does *not* make
it more complex, you can ignore the ones that aren't applicable to your
application, or even ignore the issue altogether and let the toolkit use
its default, which will probably be BER. Tell it to choose XER if you
want a textual format, but in practice this will probably only be used to
present data to a human, and never used as *input*.).

The user does not need to understand encodings at all with ASN.1. With
XML, they do need to be able to read and write XML. With ASN.1, your
toolkit handles all this stuff for you, like your TCP/IP stack handles the
networking and the filesystem handles disk allocation, and your DBMS
handles indexing.

To the script author, using an ASN.1 toolkit will be almost identical to
using an XML parser, if not easier since it will use the schema
information to map things to types in the language you're using rather
than just giving it all to you as a DOM tree of strings.

If you don't have a schema, then it's not the end of the world. You can
use a SAX/DOM like interface such as the IAAPI mentioned on
http://www.oss.com/products/tools.html. Then it will be as awkward as XML
processing, but only then. Normally, it's easier.

If you pay the money for a decent ASN.1 toolkit (as opposed to getting a
free XML one - yeah, I know, but I'm working on that :-) then you will get
more than XML provides for less learning effort. ASN.1 is designed for
this kind of problem, and solves it pretty well. Of course it's not
perfect, but it's ahead of XML in what it can do!

ABS

-- 
                               Alaric B. Snell
 http://www.alaric-snell.com/  http://RFC.net/  http://www.warhead.org.uk/
   Any sufficiently advanced technology can be emulated in software