OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help



   RE: [xml-dev] Understanding Scaling - Critical for a Net-Centric Vision

[ Lists Home | Date Index | Thread Index ]

A controversial observation is that systems based on the promotion of top-performers don't scale.  One can,
of course, argue the qualities of 'top-performance' but it is my experience that where elites are promoted,
competitive capability diminishes as the information dissemination has less reach.  I can think of counter-examples
based on systems that have need-to-know built in, but these are operational issues and as exceptions don't
change the general example.

From: Roger L. Costello [mailto:costello@mitre.org]
Hi Folks,
A few musings about "scaling" which I thought might be of interest ...
Note: In this message I use the word "technology" in its broadest sense.  It can mean a specific tool, or it can mean an approach or strategy.
A key aspect of the Web is its ability to scale.  Web technologies flourish or die depending on their ability to scale.
What does it mean for a technology to scale?  What does it mean that a technology does not scale?
To have a sensible net-centric vision it is critical to have a deep, intuitive understanding of this concept of scaling.  As a company moves to a net-centric environment it must adopt technologies that scale well.  A technology will either scale or it will die.
Let's consider some examples.
Roger Ebert is a professional movie critic.  He reviews a movie and then provides information about it:
   - a short description
   - a rating (1 - 4 stars)
[Note that what Ebert provides is "metadata" about the movie]
Over the years, the number of movies has multiplied (independent movies, blockbuster movies, etc).  It is no longer possible to Ebert to review all them.
Having a single professional movie critic review movies doesn't scale.  Even a group of professional movie critics would be soon overloaded.
As well as not scaling, there is a problem with "truth".  A movie review represents someone's interpretation, not facts.  Everyone has their own opinions about a movie.  Ebert may love the movie, but the rest of the world may hate it.
An alternative to using a professional movie critic to review movies is to have each movie producer generate a movie review.  This approach scales well - there is one movie reviewer for each movie.  Of course, the problem with this approach is that the movie review will be heavily biased.
A third alternative is to record the comments of the movie-goers.  This approach is highly scalable - there are many reviewers for each movie.  Further, since there are many reviewers there is a diversity of opinions, which give a variety of perspective to someone who is trying to decide whether to view a movie.
Thus, a scalable movie review "technology" is one that can grow as the number of movies to be reviewed increases.
The New York Times has a small group of book reviewers.  This group reviews books and then publishes their reviews.
Just as with the problem of reviewing movies, using professional reviewers does not scale.  The New York times group is unable to review the exponentially growing number of new and old books.
An alternative approach is for each book author to elicit a review from a fellow author.  This scales well, but suffers from bias.
A third alternative is to let the users (the readers) provide a review of the books.  This scales well and provides for a diversity of opinions.
A scalable book review "technology" is one that can grow as the number of books to be reviewed increases.
There are some lessons to be learned from the above examples. There are three approaches to solving a problem:
1. The high priests approach: Solve the problem using a small, centralized set of
    skilled professionals (the professionals may be geographically separate but
    functionally they are centralized)
2. The authors approach: Solve a problem using the authors.
3. The users approach: Solve the problem using the large body of distributed users.
The high priests approach is not scalable, although it may produce precise, reliable results.
The author approach is scalable, but it may produce dubious results.
The user's approach is scalable and produces multiple results which are then shared. 
A successful web technology is one that:
    - scales and
    - shares
Technologies that scale invite participation by the users.  When the users become the professionals then the technology has succeeded: 
- When your users style the XML data to their own liking and share their stylesheets with others, then your technology for making appealing looking data has succeeded.
- When your users markup data using tags that make sense to them and share their tags with others, then your markup technology has succeeded.
- When your movie goers write reviews of movies and post them for others to view, then your movie review technology has succeeded.
- When your book readers write review of books and post them for others to view, then your book review technology has succeeded.
A long time ago (Oct. 28, 2003) John Cowan made the below remark on xml-dev.  I now understand it.
Roger L. Costello scripsit:

> BTW John, I really liked your paradigm shift on the telephone operator story.
> Can you think of a similar shift with programming?  /Roger

Yes, in fact.  Perhaps ten years ago, I was sitting next to a stranger
on the New York to Boston shuttle.  In casual conversation, she asked
me what I did.  "I'm a computer programmer", said I.  She said, not
unusually, "Oh, I don't know anything about programming", but then
added in the same breath "except for some Excel and Basic."  I had
an epiphany: the world had changed.
As always, comments are welcome. /Roger


News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS