[
Lists Home |
Date Index |
Thread Index
]
Note the copied email from Don Brutzman. There
are application solutions, in this case, one for the X3D
language. Also note as has been noted on
this list before, the encumbered MPEG solution
which relies on the Schema as well.
len
From: Mike Champion [mailto:mc@xegesis.org]
So, I agree that nobody has shown a good solution here, but I don't agree
that it means there there's no problem. There may not BE a good solution
outside a given application (another point I believe Sean has made) because
the parameters and constraints on them (message size, markup overhead,
whether or not the vocabulary is known in advance, whether the "SOAP
profile" of XML is in use or whether entity expansion must be supported,
what kind of bandwidth and processor speed can be assumed, etc.) are so
variable. But that doesn't mean that possible solutions shouldn't be
explored, IMHO.
**************************************************************************
NPS has been working for a few years on a binary encoding of general
XML documents/languages, indended for rapid prototyping of network protocols
in support of large-scale virtual environments. This message summarizes
the XFSP technical approach and proposes we adapt it to support an
extensible binary encoding for X3D.
Originally called Dynamic Behavior Protocol, our approach was to define
a packet-description language in XML. Such a language might easily be
used to define the packet payloads used by network protocols, then
parsed by a network handler which did the necessary work of reading &
writing protocols conforming to the format. We got this working last
fall.
An important insight was then realized by Andrzej Kapolka, Ekrem Serin
and Don McGregor of NPS. It turns out that XML Schema already supports
most of the data-structure information needed for a packet description
language. We mapped out an approach for doing so, renaming this work
the Cross-Format Schema Protocol (XFSP). It was implemented in
November and demonstrated at the IITSEC conference in December 2002.
Ekrem performed a valuable test when he took the IEEE DIS Specification,
encoded the Entity State PDU (ESPDU) as an XML schema, and created an
XFSP reader/writer all within one day. Such DIS implementation work
took about a year, originally.
A brief XFSP description and detailed slideset are available at
http://www.movesinstitute.org/xmsf/IITSEC2002/demos/serin/description.html
The current version of XFSP source and Ekrem's draft thesis chapter
describing this work is on Sourceforge. Detailed finite state machines
on serialization/deserialization are included. Comments are welcome.
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/npsnetv/xfsp
It appears that this work is suitable for generation of a binary protocol
(and hence binary file format) for X3D. We're proposing it as such a
basis. After some scrutiny, we can consider whether how to reopen the
deferred RFP for X3D compression.
_X3D Binary Encoding based on XFSP_
Goals:
- smaller X3D files
- faster loading
- streaming
- all other points in the deferred X3D Binary Requirements and RFP
http://www.web3d.org/TaskGroups/x3d/binary/X3dBinaryRFP.html
Basic principles:
- XFSP reads an XML schema for an arbitrary tagset
- XFSP thereby produces a packet reader/writer which can handle
any valid document corresponding to the XML schema
- Packets are in binary form, using tokens for elements/attributes
and serialization of payload data
- Unambigous efficient tokenization/serialization/deserialization
- This algorithm can be tuned for X3D binary encoding.
Further/future benefits:
- Since the documents are in XML, they are well structured and suitable
for further compression. http://www.research.att.com/sw/tools/xmill
has some interesting studies on this topic. Arbitrary documents may
grow when converted to XML, but then compress better than the gzipped
original.
- XML format allows simulataneous use of XML Encryption and XML Signature
(i.e. Authentication), both in W3C Recommendation (i.e. approved) status.
http://www.w3.org/Encryption http://www.w3.org/Signature
- XML Encryption/Signature provide default royalty-free (RF) algorithms
but also allow specification of alternates. This technique might again
be applied in an X3D binary encoding to allow both open and extensible
algorithms for geometry compression, quantization, encryption etc.
- XML format allows integration of other metadata, such as Resource
Description Framework (RDF) which is of likely interest to CAD3D group
and others. http://www.w3.org/RDF
- Properly adapted, the binary encoding appears to be suitable both as a
file format and also for network delivery (e.g. scene streaming,
progressive rendering, event delivery, etc.)
- Major performance speedups are expected over gzip during binary scene
loading, since the binary structures are directly ready for immediate
insertion into scene-graph data structures in memory. No further
character-based parsing is needed for numeric fields.
- Fixing the token set values assigned to X3D nodes and fields permits
immediate streamed deserialization upon commencing loading/receipt.
- Suitable for implementation in any network- or file-capable programming
language. Example XFSP implementation offered is in Java. Even better
performance might be achieved through a tuned X3D-binary implementation,
rather than an XFSP implementation capable of general XML.
- Tuned X3D-XML implementations might be adapted to handle VRML encoding
directly. Alternatively, 1-1 translation to X3D's XML encoding
allows corresponding usage by VRML encoding also.
- Offered by NPS under royaltee-free terms in accordance with the Web3D
Consortium Intellectual Property Rights (IPR) Policy
http://www.web3d.org/aboutus/ipr.html
Design approach:
- XML elements are represented both by open-tag and close-tag tokens.
This allows rapid regeneration of the original tree-like document
structure.
- XML attributes represented by single tokens, in same namespace as
elements.
- Payload data for attributes and element content is type dependent,
occupies predictable lengths between tokens. MF array types are
preceded by field-length integers.
- Payload compression is type specific, e.g. long integers, floats,
doubles, etc.
- Further typing may be applied for special geometry-related types
like Color, Normal, coordinates, indices, pixelTexture, enumerations,
UUIDs (e.g. for CAD3D), etc. This would support specialty compression
components on a geometric-type by geometric-type basis.
- Quantization tables will likely be allowed.
- Separate namespaces for prototype/field names and DEF/IDs.
- Use of XML validation and Canonical XML form can ensure that
scenes are compressed consistently, e.g. independent of arbitrary
whitespace, apostrophe/quotation mark attribute delimitations, etc.
http://www.w3.org/TR/xml-exc-c14n
Streaming:
- Need to investigate whether to support just field payloads, or
also stream nodes.
- Multiple modes may be necessary for uncorrupted updates of field data:
(a) direct replacement
(b) progressive, appending to arrays
(e.g. pass continuous motion-capture data)
(c) buffer until delivered, then swap
- Event delivery likely contains regular binary segment consisting of
node ID, field token and payload data, along with (optionally honored)
timestamp.
So, there is a lot here but it appears that a compatible composition of
all known X3D binary encoding goals might be achieved.
Thanks for discussion and review by Alan Hudson and Justin Couch of
Yumetech. http://www.yumetech.com http://www.xj3d.org
Thanks too for long-running stellar efforts by Don, Andrzej and Ekrem
of the NPS MOVES Institute. http://www.movesinstitute.org
Comments welcome. Primary discussion forum is x3d-contributors@web3D.org
|
