[
Lists Home |
Date Index |
Thread Index
]
perhaps we could not confuse pipes with pipelines and other plumbing
devices (from memory first attributed to ken thompson).
a pipe connects the output (stdout) of one program to the input (stdin)
of another. a pipeline is series of programs whose in inputs and outputs
are connected by pipes.
pipes do a little more than just move the data from one program to
another. they provide some buffering and synchronisation - the producer
has to wait at times for the consumer to consume before producing more.
the death of the consumer leads to a notification to the producer which
can take some sort of recovery action. the death of the producer usually
just looks like the end of the data stream. the consumer has to detect
unnatural deaths itself. validations etc are the responsibility of the
producers and consumers. in the end the pipe just sees bit stream.
by definition asynchronous processes cannot take part in a pipeline and
in general the kinds of transactions envisaged and implemented by soa
and other distributed processing models do not involve pipes. they
really are some sort of rpc. also workflow (sometimes to a lesser
extent) and soa architectures share another distinction from pipelines:
the fate of the data is not necessarily known by any one agent. a
pipeline is by definition deterministic in action and destination.
in the network model the "socket" concept can be used to create network
pipes, but again this assumes the processes at each end can complete
their processing without manual intervention.
i think the original request is for something closer to a workflow where
each process completes its modifications before handing the message as a
whole to the next process(es) in the queue.
rick
david.lyon@computergrid.net wrote:
>Hi Ron,
>
>Normally a pipe wouldn't modify the contents of what it
>is transporting. As you say, Unix pipes transport the contents
>to the destination where one assumes they arrive unchanged.
>
>On these XML pipelines, when data or a document comes
>out of a pipe, it is then modified by a process or
>program. That's where I got confused. I thought the
>modifications were being done in the pipe. It seems
>that they are actually being done once the data
>comes out, and that seems logical to me.
>
>And this pretty much sums up the paradigm that most
>people are using to build xml applications. A set of
>custom pipes, put together, with custom processes do
>stuff with whatever comes out of the pipe. Validation,
>transformation etc.
>
>It's a conventional paradigm where the programmer handcrafts
>a process and a set of messages into a tightly coupled
>set to do some job of work.
>
>Pipe sizes vary, and all have a multitude of different
>wrappings.
>
>There is little or no conformity.
>
>Every pipe that appears in a business (customer request?)
>must be custom crafted by a programmer so that the data that
>pops out is useful.
>
>So we have lots of custom tools and thousands of
>combinations of everything so that there is never a
>shortage of programming work when trying to do something
>with xml. This is supposed to be good I guess.
>
>But I would suggest the ability to build every different
>size of pipe with "any connector you could possibly want"
>is an impediment to interoperability.
>
>XML Grids on the other hand are very different. Instead
>of a custom made connector at the end of each pipe to
>process each type of message, the approach is to use a
>mass-produced connector and make hundreds or thousands of
>copies of it, and make an array of those. With the
>array online, you have an automatic grid.
>
>The key concept of the Grid approach is to use a simpler
>type of message server but make it in such a way that it can be
>duplicated many times over.
>
>So this is just a slightly different paradigm I guess
>and possibly something for the future.
>
>Best Regards
>
>David
>
>
>
>
>
>Quoting Ronald Bourret <rpbourret@rpbourret.com>:
>
>
>
>>I'm not sure why a pipeline wouldn't modify the contents. Isn't that the
>>point of pipes? For example, UNIX pipes are used to pass a character
>>stream from program A to program B, which modifies that stream and
>>passes it to program C, and so on. An XML pipeline, it seems, would do
>>the same thing, with the data stored in a document and the programs
>>being XML aware.
>>
>>-- Ron
>>
>>david.lyon@computergrid.net wrote:
>>
>>
>>>ok, I know nothing...
>>>
>>>but it doesn't sound like an xml pipeline to me..
>>>
>>>one would normally think that the pipes wouldn't
>>>be modifying the contents. You wouldn't normally
>>>want that in a pipeline.
>>>
>>>It might be a relay, or a daisy-chain, but it
>>>doesn't really sound like what you need is
>>>a pipeline...
>>>
>>>
>>
>>-----------------------------------------------------------------
>>The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
>>initiative of OASIS <http://www.oasis-open.org>
>>
>>The list archives are at http://lists.xml.org/archives/xml-dev/
>>
>>To subscribe or unsubscribe from this list use the subscription
>>manager: <http://www.oasis-open.org/mlmanage/index.php>
>>
>>
>>
>>
>
>
>
>
>----------------------------------------------------------------
>This message was sent using IMP, the Internet Messaging Program.
>
>-----------------------------------------------------------------
>The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
>initiative of OASIS <http://www.oasis-open.org>
>
>The list archives are at http://lists.xml.org/archives/xml-dev/
>
>To subscribe or unsubscribe from this list use the subscription
>manager: <http://www.oasis-open.org/mlmanage/index.php>
>
>
>
begin:vcard
fn:Rick Marshall
n:Marshall;Rick
email;internet:rjm@zenucom.com
tel;cell:+61 411 287 530
x-mozilla-html:TRUE
version:2.1
end:vcard
|
