The Social Phenomena that Leads to a Diversity of Data Formats andthe En

["Costello, Roger L." <costello@mitre.org>]

Hi Folks,

This is fascinating [1]:

The way software is developed has led to the situation today where there are various data formats. Programs are very often written speculatively, that is, without any advance understanding of how important they will become. Given this situation, little effort is expended on data formats since it remains easier to program the I/O in the most straightforward way possible with the programming tools in use. Even something as simple as using an XML-based data format is harder than just using the native I/O libraries of a programming language.

In time, however, it is realized that a software program is important because either many people are using it, or it has become important for business or organizational needs to start using it in larger scale deployments. At that point it is often too late to go back and change the data formats. For example, there may be real or perceived business costs to delaying the deployment of a program for a rewrite just to change the data formats, particularly if such rewriting will reduce the performance of the program and increase the costs of deployment.

The result? Many applications, each with their own data format, e.g. binary, text-not-XML, and text-XML.

Comments?

/Roger

[1] Section 1.1 of http://www.ogf.org/documents/GFD.174.pdf