virtio-net: MQ documentation

This documents MQ support for virtio-net

Signed-off-by: Michael S. Tsirkin <>

---

--- virtio-spec.txt	2013-11-27 20:29:49.117191134 +0200
+++ virtio-spec-mq.txt	2013-11-27 20:29:27.883191885 +0200
@@ -1321,7 +1321,10 @@ features.
 
   Subsystem Device ID 1
 
-  Virtqueues 0:receiveq. 1:transmitq. 2:controlq[footnote:
+  Virtqueues 0:receiveq0. 1:transmitq0. .... 2N[footnote:
+N=0 if VIRTIO_NET_F_MQ is not negotiated, otherwise N is derived 
+from max_virtqueue_pairs control field. 
+]: receivqN. 2N+1: transmitqN. 2N+2:controlq[footnote:
 Only if VIRTIO_NET_F_CTRL_VQ set
 ]
 
@@ -1372,12 +1375,22 @@ were required.
   VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous 
     packets.
 
-  Device configuration layout Two configuration fields are 
+  VIRTIO_NET_F_MQ(22) Device supports multiqueue with automatic 
+    receive steering.
+
+  Device configuration layout Three configuration fields are 
   currently defined. The mac address field always exists (though 
   is only valid if VIRTIO_NET_F_MAC is set), and the status field 
   only exists if VIRTIO_NET_F_STATUS is set. Two read-only bits 
   are currently defined for the status field: 
-  VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE. #define VIRTIO_NET_S_LINK_UP	1
+  VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE. The following 
+  read-only field, max_virtqueue_pairs only exists if 
+  VIRTIO_NET_F_MQ is set. This field specifies the maximum number 
+  of each of transmit and receive virtqueues 
+  (receiveq0..receiveqN and transmitq0..transmitqN respectively; 
+  N=max_virtqueue_pairs - 1) that can be configured once 
+  VIRTIO_NET_F_MQ is negotiated. Legal values for this field are 
+  1 to 0x8000. #define VIRTIO_NET_S_LINK_UP	1
 
 #define VIRTIO_NET_S_ANNOUNCE	2
 
@@ -1389,12 +1402,16 @@ struct virtio_net_config {
 
     u16 status;
 
+    u16 max_virtqueue_pairs;
+
 };
 
   Device Initialization
 
 1. The initialization routine should identify the receive and 
-  transmission virtqueues.
+  transmission virtqueues, up to N+1 of each kind. If 
+  VIRTIO_NET_F_MQ feature bit is negotiated, 
+  N=max_virtqueue_pairs-1, otherwise identify N=0.
 
 2. If the VIRTIO_NET_F_MAC feature bit is set, the configuration 
   space “mac” entry indicates the “physical” address of the the 
@@ -1409,15 +1426,24 @@ struct virtio_net_config {
   status can be read from the bottom bit of the “status” config 
   field. Otherwise, the link should be assumed active.
 
-5. The receive virtqueue should be filled with receive buffers. 
+5. Only receiveq0, transmitq0 and controlq are used by default. 
+  To use more queues driver must negotiate the VIRTIO_NET_F_MQ 
+  feature; initialize up to max_virtqueue_pairs of each of 
+  transmit and receive queues; 
+  execute_VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command specifying the 
+  number of the transmit and receive queues that is going to be 
+  used and wait until the device consumes the controlq buffer and 
+  acks this command.
+
+6. The receive virtqueues should be filled with receive buffers. 
   This is described in detail below in “Setting Up Receive 
   Buffers”.
 
-6. A driver can indicate that it will generate checksumless 
+7. A driver can indicate that it will generate checksumless 
   packets by negotating the VIRTIO_NET_F_CSUM feature. This “
   checksum offload” is a common feature on modern network cards.
 
-7. If that feature is negotiated[footnote:
+8. If that feature is negotiated[footnote:
 ie. VIRTIO_NET_F_HOST_TSO* and VIRTIO_NET_F_HOST_UFO are 
 dependent on VIRTIO_NET_F_CSUM; a dvice which offers the offload 
 features must offer the checksum feature, and a driver which 
@@ -1434,7 +1460,7 @@ depending on VIRTIO_NET_F_GUEST_CSUM.
 This is a common restriction in real, older network cards.
 ]
 
-8. The converse features are also available: a driver can save 
+9. The converse features are also available: a driver can save 
   the virtual device some work by negotiating these features.[footnote:
 For example, a network packet transported between two guests on 
 the same system may not require checksumming at all, nor 
@@ -1448,9 +1474,10 @@ segmentation, if both guests are amenabl
 
   Device Operation
 
-Packets are transmitted by placing them in the transmitq, and 
-buffers for incoming packets are placed in the receiveq. In each 
-case, the packet itself is preceeded by a header:
+Packets are transmitted by placing them in the 
+transmitq0..transmitqN, and buffers for incoming packets are 
+placed in the receiveq0..receiveqN. In each case, the packet 
+itself is preceeded by a header:
 
 struct virtio_net_hdr {
 
@@ -1583,6 +1610,9 @@ elements.
 If VIRTIO_NET_F_MRG_RXBUF is negotiated, each buffer must be at 
 least the size of the struct virtio_net_hdr.
 
+If VIRTIO_NET_F_MQ is negotiated, each of receiveq0...receiveqN 
+that will be used should be populated with receive buffers.
+
   Packet Receive Interrupt
 
 When a packet is copied into a buffer in the receiveq, the 
@@ -1745,7 +1775,48 @@ Processing this notification involves:
 2. Sending VIRTIO_NET_CTRL_ANNOUNCE_ACK command through control 
   vq. 
 
-3. . 
+  Automatic receive steering in multiqueue mode
+
+If the driver negotiates the VIRTIO_NET_F_MQ feature bit (depends 
+on VIRTIO_NET_F_CTRL_VQ), it can transmit outgoing packets on one 
+of the multiple transmitq0..transmitqN and ask the device to 
+queue incoming packets into one the multiple receiveq0..receiveqN 
+depending on the packet flow.
+
+	struct virtio_net_ctrl_mq {
+		u16 virtqueue_pairs;
+	};
+
+	#define VIRTIO_NET_CTRL_MQ    1
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET        0 
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN        1 
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX        0x8000
+
+Multiqueue is disabled by default. Driver enables multiqueue by 
+executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying 
+the number of the transmit and receive queues that will be used; 
+thus transmitq0..transmitqn and receiveq0..receiveqn where 
+n=virtqueue_pairs-1 will be used. All these virtqueues must have 
+been pre-configured in advance. The range of legal values for the 
+virtqueue_pairs field is between 1 and max_virtqueue_pairs.
+
+When multiqueue is enabled, device uses automatic receivesteering 
+based on packet flow.Programming of the receive steering 
+classificator is implicit. Transmitting a packet of a specific 
+flow on transmitqX will cause incoming packets for this flow to 
+be steered to receiveqX. For uni-directional protocols, or where 
+no packets have been transmitted yet, device will steer a packet 
+to a random queue out of the specified receiveq0..receiveqn.
+
+Multiqueue is disabled by setting virtqueue_pairs = 1 (this is 
+the default). After the command is consumed by the device, the 
+device will not steer new packets on virtqueues 
+receveq1..receiveqN (i.e. other than receiveq0) nor read from 
+transmitq1..transmitqN (i.e. other than transmitq0); accordingly, 
+driver should not transmit new packets on virtqueues other than 
+transmitq0.
+
+ 
 
 Appendix D: Block Device