RE: [ebxml-msg] Bundling concerns, continued.

Comments inline.

 

Pim

From: Moberg Dale [mailto:] 

Sent: 01 October 2009 01:15
To: 

Subject: [ebxml-msg] Bundling concerns, 
continued.

There are several design principles 
and conventions at work in our current approach to bundling that don’t quite 
hang together cleanly -- or so it seems to me. Let me describe some of the major 
ideas and then discuss some of the tensions in getting them to work 
together.

 

Modularity: Each user 
message should behave in a bundle just as it would by being sent on its 
own.

Partly true.  For routing, transport and security (except 
receipts) and reliability the secondary units are being processed like the 
primary unit.

 

MSHDiscretion: A MSH 
should be free to be statically or dynamically configured to send messages in a 
bundle or individually. 

 

RootConvention: The P-Mode 
of the SOAP body user message (href attribute omitted in PartInfo) is 
primary.

No, 
the primary message is the first eb3:UserMessage in the eb3:Messaging container, 
which is a SOAP header.  It doesn't matter where the payloads of that 
primary message are stored (in the SOAP Body, as an attachment).  And 
perhaps the eb3:UserMessage doesn't even have payloads, as the minOccurs of 
eb3:Payloadinfo is 0.  

 

SingleSignature: When a signed message is indicated 
by the P-Mode of the Root the signature must cover all user messages. 

The PMode of the primary unit specifies what (if anything) is 
signed. If it says: sign the eb3:Messaging unit, the SOAP Body, any other SOAP 
headers per WS-I RSP, and all payloads, then the effect is that all user 
messages are signed. 

 

 

OneSharedReliabilityId (sequence 
number in WS-RX)

 

 

 

Tensions in 
Modularity Root Convention, and Single 
Signature

 

Suppose one non-root body part has a 
P-mode with enveloped encryption. The assembly of parts is to proceed as it 
would for individually sent messages even when in a bundle. But the P-mode of 
the root is signed only. Should the non-root part be encrypted and then signed? 
Or should encryption be omitted and the receiving end indicate a 
Value-Inconsistent error? See section 3.2.2 of the latest draft.  Whether or not to 
encrypt payloads is determined by the primary unit's PMode, since this is 
specified in the  
Pmode[1].Security PMode section. This is true for both 
sending and receiving MSH, so there is no Value Inconsistency error.  
Or when bundled should each non-root part be processed by ignoring 
the defined P-mode for this user message?   

 

On the face 
of it Modularity can easily conflict with RootConvention when SingleSignature is 
operative. In fact whenever the root P-mode conflicts with a P-mode of a 
non-root user-message part, then the normal message receiver-side error checks 
must be disengaged. Likewise I think during sender message assembly, the access 
to the P-mode associated with the user message’s AgreementRef will need to be 
overridden. This is a lot of special coding and almost amounts to a new mode of 
operation.  Bundling is 
functionality for the ebMS module, the security and reliability modules don't 
need to know much if anything about it.  Suppose you have an existing ebMS3 
implementation that you want to adapt to support bundling.  The only thing 
is to check that it picks the right (=first) UserMessage for PMode setting 
(agreement) validation, since the only new thing is that there may be more of 
them. The security and reliability modules just use a different key to the 
agreement/pmode store than the ebMS module. Please explain why this would 
be a lot of special coding.  

 

So I agree with Sander that it might 
be better to restrict user message bundling to messages with the “same” P-mode, 
assuming we can explain what that means. I doubt that this constraint will be 
sufficient to solve all the tensions, however.

If trading partners want to restrict bundling this way, then 
the parameters in section 3.3 give them the configuration parameters they need 
to do so.  But if partner P1 exchanges 30 types of messages with a trading 
partner P2 (hence 30 PModes), it may be perfectly acceptable to use a single set 
of security/reliability parameters for all 30 message types and to allow all of 
them to be bundled together, and this will increase the benefit on 
bundling. Many current ebMS user communities use configurations based on a 
handful of profiles (or CPA templates). If they exchange messages 
regularly, they immediately benefit from bundling. 
 

 

Tensions in 
MSHDiscretion and OneSharedReliabilityId

 

Here is a case I was struggling with 
describing today. Supposedly the MSH can “just decide” to bundle messages 
together. And if the bundle does not work for some reason (errors come back 
about message parts) it could decide to “back off” and send messages separately. 
In the case of a signature failure, I think we agree that it might be reasonable 
to see whether each part could be sent individually. Presumably this makes sense 
because no part “should” be delivered, given that we have SingleSignature. 

This is certainly an option, but in practice I suspect some 
system manager will want to inspect this message and see what went wrong, 
instead of having a policy to automatically sending the units unbundled.  
We could define a PMode parameter to do this, or leave it as an implementation 
choice. 

If the error is due to e.g. the use of an untrusted 
certificate, then the primary unit will also be rejected when sent 
unbundled.  There could be a benefit in sending a secondary unit whose PMode does 
not specify signing (or references another -trusted- certificate) unbundled, 
since they would not suffer from the certificate errors of the primary unit's 
PMode.  But this doesn't help if, as you and Sander want, 
only units that have the same PMode are bundled:  then you are certain that 
they will all fail when sent unbundled, since the PMode specifies the 
certificate and it is the same for all. So then, when a bundle fails, all 
units in it fail permanently. 

If there is an error in calculating the hash for some cid: 
MIME attachments and not for other, this looks like a product error that would 
probably also occur with standalone units. Resending the units is unlikely to 
solve anything .

Sending each part individually would at least be preferable 
to sending the same bundle again. This is because the message processing failed 
before the bundle was acked, and so retrying the same bundle with its security 
defect could go into an endless loop of retries as long as acks might not be 
sent.

Yes, just as you presumably do not resend a failed unbundled 
message unit.

 

Now suppose 
for a different bundle that its security checks out, and reliability is 
processed, and we arrive at the separate message parts. Resending the whole 
bundle would probably just be acked as a duplicate, so that will not be an 
option for a transparent recovery. So if we do get error messages, should we 
trust that the parts for which we did not receive an error message were ok and 
delivered? What happens when 
you send an unbundled user message unit, you get an RM acknowledgement and no 
error.  Why would you not question whether it was delivered in that case, 
whereas you have doubts about it being delivered if it is sent in a 
bundle?  You can specify that you want an NRR receipt.  Those are 
specified per message unit, so you would get separate eb:Receipts for each 
user message unit that didn't fail. What if (some of ) the error 
messages get lost? What if an 
error for an unbundled message is lost?  With WS-ReliableMessaging, an 
acknowledgment does not mean that no error occurred, even when using InOrder 
delivery. There is no difference when bundling applies, so why is this raised as 
an issue with bundling?  Not sure how to specify it in PModes, but an MSH 
could send errors back over a reliable sequence, to make sure they don't 
get lost.Or to be on the safe side should we send all the parts 
again, individually, adhering to MSHDiscretion and the hope that a more 
determinate outcome of errors and/or acks comes back? If our received error 
messages were infallible indicators of exactly what went wrong, then we might 
try to send each payload with an error and see whether that remedies the error 
message. Otherwise, we retry all the body parts separately. The problem here is 
that if some of the parts had been ok and had been delivered optimistically, 
then retrying individually will get duplicates delivered. Yes, that's why I think 
resending a message unit that you did not get an error for separately, 
if you sent them before in a bundle that you got an acknowledgment for, is a 
questionable idea.  You have to build a second level of duplicate 
elimination at the ebMS level that you otherwise wouldn't need. 
And that violates our modularity assumption because when each part 
has its own reliability id value, duplicates would be eliminated. It could be 
that we just give up and do nothing about these ebMS level errors, just logging 
them. I think the 
receiving MSH should generate errors for the units that have errors and no 
errors (or receipts, if NRR is requested) for the units that are fine at 
ebMS level. We could mandata the errors for units are always bundled, so 
that the sending MSH receives them together, providing strong evidence that some 
units failed but others not (since that bundle would contain explicit errors for 
those if they were).   But that would violate 
modularity too, because in that case, we would not know that any of our parts 
got delivered successfully as we could with an acked, errorless single message 
sent individually.  Not true, with WS-ReliableMessaging, 
an acknowledgment does not mean that no error occurred, even when using InOrder 
delivery, also with unbundled messages. 
Being part of the bundle has made our state alignment a bit foggier in 
the case of some errors at the ebMS header level. Or so it seems to me.   Have a look at 3.4 and 3.9.3.

 

Let me stop 
with these two issues, and ask for simplifications that would diminish the 
interactions threatening Modularity. I think that Sander had a good suggestion 
to disallow P-modes that were “different” but I would like a cleaner way of 
stating this requirement. I personally would not mind dropping the single 
signature idea because that definitely makes sending a message in a bundle 
different from sending it on its own.  I think that if signatures are per unit, can use 
different certificates or algorithms, that some units use encryption and others 
not we are loosing the biggest efficiency benefit for bundling.  In that 
case, I would not use bundling at all.  Or define a batching protocol 
(multiple SOAP envelopes in distinct MIME parts) if saving a separate 
HTTP connection set-up is worth the effort. With batching the various SOAP 
envelopes could also have separate reliabilty sequence 
numbers.  

(One security header can have many 
separate signatures within it, although I need to check WSI BSP to see that they 
have not profiled that away.)  

 

As far as 
ebMS level errors go, either we should state that nothing is to be done in 
response to an error by the sender as part of our specified protocol. Or we 
might state that each part is sent again individually (but I think this is bad 
because the receiver will have to support a second layer of duplicate detection 
by ebMS message id). Or we state that a receiver MAY retry a message with an 
error received when it was sent in a bundle. My preference is now headed toward 
saying that the specified protocol does not say what to do when an ebMS error is 
received for one part of a bundled message. Maybe we even say the same thing 
about the failed-signature error message when received by the sender.  Please review 3.4 and 
3.9.3. 

 

I do not 
think that going into the dependency graph direction is going to make this 
situation simpler or better. I think that we should say that when choosing 
bundling, that is the mode of transmission, and when it fails, each message 
payload must be resubmitted by operations (but with a do-not-bundle flag set! 
)  Please review 3.4 and 
3.9.3.