Re: [virtio-dev] Re: [virtio-comment] Problems with VIRTIO-4 and writeback only disks

[Rusty Russell <rusty@au1.ibm.com>]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Tue, Sep 17, 2013 at 03:03:47PM +0930, Rusty Russell wrote:
>> James Bottomley <jbottomley@parallels.com> writes:
>> > On Mon, 2013-09-16 at 16:50 +0930, Rusty Russell wrote:
>> >> I think we should do the reverse:
>> >> 1) If you negotiate VIRTIO_BLK_F_WCE, you need to flush (ie. it's
>> >>    writeback).  Otherwise it's write-through.
>> >> 2) If (and only if) you negotiate VIRTIO_BLK_F_WCE, you may also negotiate
>> >>    VIRTIO_BLK_F_CONFIG_WCE, in which case you can toggle it.
>> >> 
>> >> But I'm not convinced that #2 should exist at all.  Does someone have a
>> >> real use case?
>> >
>> > There is no use case in Linux.  We provide knobs to toggle the cache
>> > type in the SCSI disk class, but the kernel doesn't do it by default, it
>> > just runs with the cache type because the toggle can be unreliable.
>> 
>> OK.  How about this as a strawman:
>> 
>> 1) Kill VIRTIO_BLK_F_CONFIG_WCE as a bad idea.  You either negotiate
>>    WCE or not during device setup; no toggling.
>
> It's not really useful as guest-controlled, I agree.
> But I have this vague idea:
> - host wants to enable WCE on backend
> - host notifies guest
> - guest enables WCE
> - host can now enable WCE on backend

That seems convoluted to me.  This kind of complexity seems best in
virtio-scsi.

>> 2) If device offers VIRTIO_BLK_F_WCE, and driver negotiates it:
>>    - Completed writes should be persistent if guest crashes.
>>    - No ordering guarantees are given except where flush command sent.
>>    - Flush SHOULD push all preceeding writes to permanent storage.
>>      (in the RFC SHOULD sense, which is weaker than MUST).
>
> We can also document that it's legal for device to
> make the feature required (using the new handshake).

I dislike compulsory features.  Most features are a
complexity/performance tradeoff.  Making a feature compulsory means that
only the device gets to decide the tradeoff, not the driver.

>> 3) If device does not offer VIRTIO_BLK_F_WCE, or driver doesn't
>>    negotiate it:
>>    - Completed writes should be persistent if guest crashes.
>>    - No flush commands are supported.
>>    - No guarantee about writes hitting permanent storage.
>> 
>> This pretty neatly divides it into complex and simple cases.  If you
>> want more fine-grained, you know where to find virtio-scsi...

Subnote:
        This means that quisceing a device non-WCE device forms a write
        barrier.  This is not quite the same as strictly ordered, as
        in-flight requests can be reordered.

        Quiescing a WCE device doesn't do anything (though a
        straightforward aio implementation would cause a write barrier
        too).

My main question remains: is this non-WCE vs WCE split sane?  Non-WCE
means no host persistence, WCE gives you a hope.

Cheers,
Rusty.