virtio-net: MQ documentation

This documents MQ support for virtio-net  Signed-off-by: Michael S. Tsirkin <>  ---  --- virtio-spec.txt	2013-11-27 20:29:49.117191134 +0200 +++ virtio-spec-mq.txt	2013-11-27 20:29:27.883191885 +0200 @@ -1321,7 +1321,10 @@ features.
Subsystem Device ID 1
-  Virtqueues 0:receiveq. 1:transmitq. 2:controlq[footnote: +  Virtqueues 0:receiveq0. 1:transmitq0. .... 2N[footnote: +N=0 if VIRTIO_NET_F_MQ is not negotiated, otherwise N is derived  +from max_virtqueue_pairs control field.  +]: receivqN. 2N+1: transmitqN. 2N+2:controlq[footnote:  Only if VIRTIO_NET_F_CTRL_VQ set  ]
@@ -1372,12 +1375,22 @@ were required.
VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous
packets.
-  Device configuration layout Two configuration fields are  +  VIRTIO_NET_F_MQ(22) Device supports multiqueue with automatic  +
receive steering. + +  Device configuration layout Three configuration fields are
currently defined. The mac address field always exists (though
is only valid if VIRTIO_NET_F_MAC is set), and the status field
only exists if VIRTIO_NET_F_STATUS is set. Two read-only bits
are currently defined for the status field:  -  VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE. #define VIRTIO_NET_S_LINK_UP	1 +  VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE. The following  +  read-only field, max_virtqueue_pairs only exists if  +  VIRTIO_NET_F_MQ is set. This field specifies the maximum number  +  of each of transmit and receive virtqueues  +  (receiveq0..receiveqN and transmitq0..transmitqN respectively;  +  N=max_virtqueue_pairs - 1) that can be configured once  +  VIRTIO_NET_F_MQ is negotiated. Legal values for this field are  +  1 to 0x8000. #define VIRTIO_NET_S_LINK_UP	1
#define VIRTIO_NET_S_ANNOUNCE	2
@@ -1389,12 +1402,16 @@ struct virtio_net_config {
u16 status;
+
u16 max_virtqueue_pairs; +  };
Device Initialization
1. The initialization routine should identify the receive and  -  transmission virtqueues. +  transmission virtqueues, up to N+1 of each kind. If  +  VIRTIO_NET_F_MQ feature bit is negotiated,  +  N=max_virtqueue_pairs-1, otherwise identify N=0.
2. If the VIRTIO_NET_F_MAC feature bit is set, the configuration
space “mac” entry indicates the “physical” address of the the  @@ -1409,15 +1426,24 @@ struct virtio_net_config {
status can be read from the bottom bit of the “status” config
field. Otherwise, the link should be assumed active.
-5. The receive virtqueue should be filled with receive buffers.  +5. Only receiveq0, transmitq0 and controlq are used by default.  +  To use more queues driver must negotiate the VIRTIO_NET_F_MQ  +  feature; initialize up to max_virtqueue_pairs of each of  +  transmit and receive queues;  +  execute_VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command specifying the  +  number of the transmit and receive queues that is going to be  +  used and wait until the device consumes the controlq buffer and  +  acks this command. + +6. The receive virtqueues should be filled with receive buffers.
This is described in detail below in “Setting Up Receive
Buffers”.
-6. A driver can indicate that it will generate checksumless  +7. A driver can indicate that it will generate checksumless
packets by negotating the VIRTIO_NET_F_CSUM feature. This “
checksum offload” is a common feature on modern network cards.
-7. If that feature is negotiated[footnote: +8. If that feature is negotiated[footnote:  ie. VIRTIO_NET_F_HOST_TSO* and VIRTIO_NET_F_HOST_UFO are
dependent on VIRTIO_NET_F_CSUM; a dvice which offers the offload
features must offer the checksum feature, and a driver which  @@ -1434,7 +1460,7 @@ depending on VIRTIO_NET_F_GUEST_CSUM.  This is a common restriction in real, older network cards.  ]
-8. The converse features are also available: a driver can save  +9. The converse features are also available: a driver can save
the virtual device some work by negotiating these features.[footnote:  For example, a network packet transported between two guests on
the same system may not require checksumming at all, nor  @@ -1448,9 +1474,10 @@ segmentation, if both guests are amenabl
Device Operation
-Packets are transmitted by placing them in the transmitq, and  -buffers for incoming packets are placed in the receiveq. In each  -case, the packet itself is preceeded by a header: +Packets are transmitted by placing them in the  +transmitq0..transmitqN, and buffers for incoming packets are  +placed in the receiveq0..receiveqN. In each case, the packet  +itself is preceeded by a header:
struct virtio_net_hdr {
@@ -1583,6 +1610,9 @@ elements.  If VIRTIO_NET_F_MRG_RXBUF is negotiated, each buffer must be at
least the size of the struct virtio_net_hdr.
+If VIRTIO_NET_F_MQ is negotiated, each of receiveq0...receiveqN  +that will be used should be populated with receive buffers. +
Packet Receive Interrupt
When a packet is copied into a buffer in the receiveq, the  @@ -1745,7 +1775,48 @@ Processing this notification involves:  2. Sending VIRTIO_NET_CTRL_ANNOUNCE_ACK command through control
vq.
-3. .  +  Automatic receive steering in multiqueue mode + +If the driver negotiates the VIRTIO_NET_F_MQ feature bit (depends  +on VIRTIO_NET_F_CTRL_VQ), it can transmit outgoing packets on one  +of the multiple transmitq0..transmitqN and ask the device to  +queue incoming packets into one the multiple receiveq0..receiveqN  +depending on the packet flow. + +	struct virtio_net_ctrl_mq { +		u16 virtqueue_pairs; +	}; + +	#define VIRTIO_NET_CTRL_MQ
1 +	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET
0  +	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN
1  +	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX
0x8000 + +Multiqueue is disabled by default. Driver enables multiqueue by  +executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying  +the number of the transmit and receive queues that will be used;  +thus transmitq0..transmitqn and receiveq0..receiveqn where  +n=virtqueue_pairs-1 will be used. All these virtqueues must have  +been pre-configured in advance. The range of legal values for the  +virtqueue_pairs field is between 1 and max_virtqueue_pairs. + +When multiqueue is enabled, device uses automatic receivesteering  +based on packet flow.Programming of the receive steering  +classificator is implicit. Transmitting a packet of a specific  +flow on transmitqX will cause incoming packets for this flow to  +be steered to receiveqX. For uni-directional protocols, or where  +no packets have been transmitted yet, device will steer a packet  +to a random queue out of the specified receiveq0..receiveqn. + +Multiqueue is disabled by setting virtqueue_pairs = 1 (this is  +the default). After the command is consumed by the device, the  +device will not steer new packets on virtqueues  +receveq1..receiveqN (i.e. other than receiveq0) nor read from  +transmitq1..transmitqN (i.e. other than transmitq0); accordingly,  +driver should not transmit new packets on virtqueues other than  +transmitq0. + +
Appendix D: Block Device