Hi
Folks,
A few musings about "scaling" which I thought might
be of interest ...
Note: In this
message I use the word "technology" in its broadest sense.
It can mean a specific tool, or it can mean an approach or
strategy.
A key aspect
of the Web is its ability to scale. Web technologies flourish or die
depending on their ability to scale.
What does it mean
for a technology to scale? What does it mean that a technology does not
scale?
To have a
sensible net-centric vision it is critical to have a deep, intuitive understanding of
this concept of scaling. As a
company moves to a net-centric environment it must adopt
technologies that scale well. A technology will
either scale or it will die.
Let's consider
some examples.
A SCALABLE MOVIE
REVIEW TECHNOLOGY
Roger Ebert is a
professional movie critic. He reviews a movie and then provides
information about it:
- a
short description
- a
rating (1 - 4 stars)
[Note that what
Ebert provides is "metadata" about the movie]
Over the years,
the number of movies has multiplied (independent movies, blockbuster movies, etc). It is no
longer possible to Ebert to review all them.
Having a
single professional movie critic review movies doesn't scale. Even
a group of professional movie critics would be soon
overloaded.
As well as not
scaling, there is a problem with "truth". A movie review represents
someone's interpretation, not facts. Everyone has their own opinions
about a movie. Ebert may love the movie, but the rest of the world
may hate it.
An alternative to
using a professional movie critic to review movies is to have each movie
producer generate a movie review. This approach scales well - there
is one movie reviewer for each movie. Of course, the problem with this
approach is that the movie review will be heavily biased.
A third
alternative is to record the comments of the movie-goers. This approach
is highly scalable - there are many reviewers for each movie.
Further, since there are many reviewers there is a diversity of opinions,
which give a variety of perspective to someone who is trying to decide whether
to view a movie.
Thus,
a scalable movie review "technology" is one that can grow as the number
of movies to be reviewed increases.
A SCALABLE BOOK REVIEW
TECHNOLOGY
The New York Times
has a small group of book reviewers. This group reviews books and then
publishes their reviews.
Just as with the
problem of reviewing movies, using professional reviewers does not
scale. The New York times group is unable to review
the exponentially growing number of new and old
books.
An alternative approach is for each
book author to elicit a review from a fellow author. This scales
well, but suffers from bias.
A third alternative is to let the users
(the readers) provide a review of the books. This scales well and
provides for a diversity of opinions.
A scalable book review "technology" is one
that can grow as the number of books to be reviewed
increases.
LESSONS
LEARNED
There are
some lessons to be learned from the above examples. There are three approaches to solving a
problem:
1. The high
priests approach: Solve the problem using a small, centralized set of
skilled professionals (the professionals may be geographically separate but
functionally they are centralized)
2. The authors
approach: Solve a problem using the authors.
3. The users
approach: Solve the problem using the large body of distributed
users.
The high priests
approach is not scalable, although
it may produce precise, reliable results.
The author
approach is scalable, but it may produce dubious results.
The user's
approach is scalable and produces multiple results which are then
shared.
A SUCCESSFUL WEB
TECHNOLOGY
A successful web
technology is one that:
- scales and
- shares
Technologies that
scale invite participation by the users. When the users become the
professionals then the technology has succeeded:
- When your
users style the XML data to their own liking and share their stylesheets with
others, then your technology for making appealing looking data
has succeeded.
- When your
users markup data using tags that make sense to them and share their tags with
others, then your markup technology has succeeded.
- When your
movie goers write reviews of movies and post them for others to view, then
your movie review technology has succeeded.
- When your
book readers write review of books and post them for others to view, then your
book review technology has succeeded.
A long time ago
(Oct. 28, 2003) John Cowan made the below remark on xml-dev. I now understand
it.
Roger L. Costello
scripsit:
> BTW John, I really liked your paradigm shift on the
telephone operator story.
> Can you think of a similar shift with
programming? /Roger
Yes, in fact. Perhaps ten years ago, I
was sitting next to a stranger
on the New York to Boston shuttle. In
casual conversation, she asked
me what I did. "I'm a computer
programmer", said I. She said, not
unusually, "Oh, I don't know
anything about programming", but then
added in the same breath "except for
some Excel and Basic." I had
an epiphany: the world had
changed.
As always, comments are welcome.
/Roger
