virtio — archive

[PATCH requirements v4 0/7] virtio net requirements for 1.4

From: Heng Qi <[email protected]>
To: [email protected]
Date: Thu, 17 Aug 2023 05:14:00 -0000

? 2023/8/17 ??12:57, Parav Pandit ??: > >> From: Heng Qi <[email protected]> >> Sent: Wednesday, August 16, 2023 6:07 PM >> >> >> ? 2023/8/16 ??6:46, Parav Pandit ??: >>>> From: Heng Qi <[email protected]> >>>> Sent: Wednesday, August 16, 2023 2:01 PM >>>> >>>> ? 2023/8/15 ??3:45, Parav Pandit ??: >>>>> Add virtio net device notification coalescing improvements requirements. >>>>> >>>>> Signed-off-by: Parav Pandit <[email protected]> >>>>> Acked-by: David Edmondson <[email protected]> >>>>> >>>>> --- >>>>> changelog: >>>>> v3->v4: >>>>> - no change >>>>> >>>>> v1->v2: >>>>> - addressed comments from Stefan >>>>> - redrafted the requirements to use rearm term and avoid queue enable >>>>> confusion >>>>> v0->v1: >>>>> - updated the description >>>>> --- >>>>> net-workstream/features-1.4.md | 11 +++++++++++ >>>>> 1 file changed, 11 insertions(+) >>>>> >>>>> diff --git a/net-workstream/features-1.4.md >>>>> b/net-workstream/features-1.4.md index 72d04bd..cb72442 100644 >>>>> --- a/net-workstream/features-1.4.md >>>>> +++ b/net-workstream/features-1.4.md >>>>> @@ -8,6 +8,7 @@ together is desired while updating the virtio net >> interface. >>>>> # 2. Summary >>>>> 1. Device counters visible to the driver >>>>> 2. Low latency tx and rx virtqueues for PCI transport >>>>> +3. Virtqueue notification coalescing re-arming support >>>>> >>>>> # 3. Requirements >>>>> ## 3.1 Device counters >>>>> @@ -172,3 +173,13 @@ struct vnet_rx_completion { >>>>> which can be recycled by the driver when the packets from the >> completed >>>>> page is fully consumed. >>>>> 8. The device should be able to consume multiple pages for a >>>>> receive GSO >>>> stream. >>>>> + >>>>> +## 3.3 Virtqueue notification coalescing re-arming support 0. >>>>> +Design >>>>> +goal: >>>>> + a. Avoid constant notifications from the device even in conditions when >>>>> + the driver may not have acted on the previous pending notification. >>>>> +1. When Tx and Rx virtqueue notification coalescing is enabled, and >>>>> +when >>>> such >>>>> + a notification is reported by the device, the device stops sending further >>>>> + notifications until the driver rearms the notifications of the virtqueue. >>>>> +2. When the driver rearms the notification of the virtqueue, the device >>>>> + to notify again if notification coalescing conditions are met. >>>> I'm wondering how this relates to the existing notification >>>> coalesing[1] and notification suppression[2]: >>>> >>>> [1] >>>> The device sends a used buffer notification once the notification >>>> conditions are met and if the notifications are not suppressed as >>>> explained in \ref{sec:Basic Facilities of a Virtio Device / >>>> Virtqueues / Used Buffer Notification Supppression}. >>>> >>>> [2] >>>> If the VIRTIO_F_EVENT_IDX feature bit is not negotiated: >>>> \begin{itemize} >>>> \item The driver MUST ignore the \field{avail_event} value. >>>> \item After the driver writes a descriptor index into the available ring: >>>> \begin{itemize} >>>> \item If \field{flags} is 1, the driver SHOULD NOT send a notification. >>>> \item If \field{flags} is 0, the driver MUST send a notification. >>>> \end{itemize} >>>> \end{itemize} >>>> >>>> Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated: >>>> \begin{itemize} >>>> \item The driver MUST ignore the lower bit of \field{flags}. >>>> \item After the driver writes a descriptor index into the available ring: >>>> \begin{itemize} >>>> \item If the \field{idx} field in the available ring (which determined >>>> where that descriptor index was placed) was equal to >>>> \field{avail_event}, the driver MUST send a notification. >>>> \item Otherwise the driver SHOULD NOT send a notification. >>>> \end{itemize} >>>> \end{itemize} >>>> >>>> Regarding notification suppression: >>>> 1.When there is VIRTIO_NET_F_EVENT_IDX, even if the notification >>>> coalesing condition is met, we need to wait for the used_event >>>> notification condition to be met(the driver does not rearms the >>>> notification of the virtqueue now and the avail ring is set >> VRING_AVAIL_F_NO_INTERRUPT in flag). >>>> 2.When there is no VIRTIO_NET_F_EVENT_IDX, if the driver turns off >>>> the notification, even if the notidication condition is met, the >>>> device cannot send the notification. >>>> >>>> Therefore, if I'm not wrong, a device can issue a notification only >>>> if the device is not suppressed from notifying the driver. >>>> [1][2] seems to have met this condition. >>> Notification suppression using _EVENT_IDX for non-memory transport is just >> sub-optimal for two reasons. >>> 1. It requires device to poll on the used event to learn about when to >>> un-suppress. (arm) 2. this bit also controls driver notifications yet >>> again demand device to arbitrarily poll on new descriptors posting >>> >>> Hence, an efficient scheme is needed and device notifications to be detached >> from driver notification. >>> And now that VQ level notification coalescing is in place, which suppresses >> the device notifications, it is logical to combine it with VQ device notifications. >> Let me summarize: >> 1. When used idx notification is satisfied, but coalescing is not satisfied, the >> driver continues to suppress device notifications. > Ack. > >> 2. When used idx notification is not satisfied, even if coalescing is satisfied, the >> device still cannot notify the driver. > Ack. > >> I think that's what coalescing does, and the description below has satisfied this >> behavior: >> "The device sends a used buffer notification once the notification conditions >> are met and if the notifications are not suppressed as explained in \ref{sec:Basic >> Facilities of a Virtio Device / Virtqueues / Used Buffer Notification >> Supppression}." >> > Ack. > the proposal here is to not use EVENT_IDX scheme, instead driver to enable/disable notification coalescing in different way, even when notification coalescing parameters are configured. > And this to be done in fairly fast way (not like a cvq) command. For example like driver notifications. > >> Or we want to say that it has nothing to do with the used idx notification. > When >> the coalescing is satisfied and the driver rearms the notification of the >> virtqueue, the device now send a notification. >> > Right. > F_NOTIFION_ARM is mutually exclusive with F_EVENT_IDX. OK, I think I get your point, F_NOTIFION_ARM is mutually exclusive with VIRTQ_AVAIL_F_NO_INTERRUPT\used idx\VIRTIO_F_NOTIFY_ON_EMPT, and it seems that F_NOTIFION_ARM has the highest priority, and it needs a new feature bit. Am I right :)? Thanks! > (Like packed vq is mutually exclusive with split q.)