Four fine text-based data formats ... liberate yourself from one(silo) d

["Costello, Roger L." <costello@mitre.org>]

Hi Folks,

Here are four fine text-based data formats. There are all well supported. 

1. XML: obviously you know about this data format and its support.

2. JSON: data that is in this format can be readily queried and manipulated in a JavaScript program, and support for JavaScript is growing at a breathtaking rate. From Simon St. Laurent: There are also piles of public APIs using JSON.  Programmable Web and similar places keep showing growth in JSON-based APIs.  See, for example:

http://blog.programmableweb.com/2012/12/17/leading-apis-say-bye-xml-in-new-versions/ 

3. CSV: data in the form of comma-separated-values (CSV) can be readily queried and manipulated in Excel. There are many tools that support CSV, here's one from Google:

http://code.google.com/p/csvfix/ 

4. Plain text: of all the data formats, this one is by far the most widely supported. Every computer on the planet has at least one text editor (probably several). There are many, many powerful tools, such as vi and emacs, that can readily query and manipulate plain text files. 

Which data format is best? They are all fine; each has advantages and disadvantages.

Can we interoperate when data is in different formats? Yes! David Lee's tool, xmlsh [1], allows conversion from XML to JSON and vice versa; from XML to CSV and vice versa.

What about semantics; is one data format semantically richer than another? No! All of the data formats are just syntax. Semantics must be applied to the syntax. How are semantics applied to syntax? Easy, write a software program that does something when it sees certain syntax. The software gives life to the syntax - it gives semantics to the syntax.

Shouldn't we define standards - using a particular data format - for data exchanges? No! Define standards at the semantic level, not the syntax level. Let everyone use their own syntax. Convert syntax where necessary (see David Lee's tool). From Simon St. Laurent: As it turns out, the world needs infinitely more formats for private (and internal) data interchange than it needs standards for universal interchange.  Most of XML's advantages for creating such standards, most notably its endless obsession with schemas, turn out to be dead weight for private interchange applications.

Comments?

/Roger

[1] David Lee's tool, xmlsh, may be found here: http://www.xmlsh.org/HomePage