[
Lists Home |
Date Index |
Thread Index
]
Another interesting bit in Gray's article was:
"Publish/subscribe systems contribute further to the challenge by inverting
the traditional data/query ratios, requiring that incoming data be compared
against millions of queries instead of queries being used to search through
millions of records."
What we're seeing here is completion of the "database" problem to
include prospective search in addition to the traditional retrospective
search implemented by today's relational database systems. A retrospective
system such as a traditional database is focused on evaluating queries
against historical collections of data. A retrospective system answers the
question: "What is known?" It is focused on the past. On the other hand, a
prospective system stores queries instead of data and it evaluates each
newly discovered data object against stored queries. A prospective engine
typically answers questions of the form "Tell me whenever x happens!" and is
focused on searching the future -- not the past. Traditionally, database
engines have had prospective search grafted onto them via mechanisms like
triggers.
With the exception of financial market applications which deal with
tremendous volumes of data and have very strict timeliness requirements,
there hasn't been much pressure to improve on prospective matching
capabilities until recently. But today, we're seeing high transaction
volumes and increasing timeliness requirements across all application areas.
The old, tacked-on trigger mechanisms are no longer good enough. This has
resulted in a great deal of "stream processing" research at places like
Stanford, Brown, MIT, Berkeley, U Wisconsin, etc. as well as the
implementation of Internet services like that of PubSub ( http://pubsub.com
) or companies like Stonebraker's new StreamBase.com
Just as SQL wasn't able to handle the challenges of "object" data
without extensions, I think we'll have to recognize that SQL -- a language
designed for retrospective search -- is simply not an adequate query
language for prospective search. More extensions will be needed. The same
is, of course, true of XQuery since that language was also designed for use
in retrospective systems and didn't take prospective system requirements
into consideration in its design.
Gray's paper opens the discussion on a good number of issues -- not
the least of which are the issues related to the prospective search or
matching which is typical of Publish/Subscribe systems. Let's hope it's a
fun and useful discussion.
bob wyman
|
