RE: [EXT] [virtio] [PATCH requirements 5/7] net-features: Add n-tuple receive flow filters requirements

[Satananda Burla <sburla@marvell.com>]

Hi Parav

> -----Original Message-----
> From: virtio@lists.oasis-open.org <virtio@lists.oasis-open.org> On
> Behalf Of Parav Pandit
> Sent: Sunday, July 23, 2023 8:34 PM
> To: virtio-comment@lists.oasis-open.org
> Cc: shahafs@nvidia.com; hengqi@linux.alibaba.com; virtio@lists.oasis-
> open.org; Parav Pandit <parav@nvidia.com>
> Subject: [EXT] [virtio] [PATCH requirements 5/7] net-features: Add n-
> tuple receive flow filters requirements
> 
> External Email
> 
> ----------------------------------------------------------------------
> Add virtio net device requirements for receive flow filters.
> 
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> ---
> changelog:
> v1->v2:
> - split setup and operations requirements
> - added design goal
> - worded requirements more precisely
> v0->v1:
> - fixed comments from Heng Li
> - renamed receive flow steering to receive flow filters
> - clarified byte offset in match criteria
> ---
>  net-workstream/features-1.4.md | 105 +++++++++++++++++++++++++++++++++
>  1 file changed, 105 insertions(+)
> 
> diff --git a/net-workstream/features-1.4.md b/net-workstream/features-
> 1.4.md
> index 27a7886..d228462 100644
> --- a/net-workstream/features-1.4.md
> +++ b/net-workstream/features-1.4.md
> @@ -9,6 +9,7 @@ together is desired while updating the virtio net
> interface.
>  1. Device counters visible to the driver
>  2. Low latency tx and rx virtqueues for PCI transport
>  3. Virtqueue notification coalescing re-arming support
> +4  Virtqueue receive flow filters (RFF)
> 
>  # 3. Requirements
>  ## 3.1 Device counters
> @@ -175,3 +176,107 @@ struct vnet_rx_completion {
>     notifications until the driver rearms the notifications of the
> virtqueue.
>  2. When the driver rearms the notification of the virtqueue, the device
>     to notify again if notification coalescing conditions are met.
> +
> +## 3.4 Virtqueue receive flow filters (RFF)
> +0. Design goal:
> +   To filter and/or to steer packet based on specific pattern match to
> a
> +   specific context to support application/networking stack driven
> receive
> +   processing.
> +1. Two use cases are: to support Linux netdev set_rxnfc() for
> ETHTOOL_SRXCLSRLINS
> +   and to support netdev feature NETIF_F_NTUPLE aka ARFS.
> +
> +### 3.4.1 control path
> +1. The number of flow filter operations/sec can range from 100k/sec to
> 1M/sec
> +   or even more. Hence flow filter operations must be done over a
> queueing
> +   interface using one or more queues.
> +2. The device should be able to expose one or more supported flow
> filter queue
If there is more than 1 queue, I assume there is no ordering requirement 
between operations submitted on multiple queues.
> +   count and its start vq index to the driver.
> +3. As each device may be operating for different performance
> characteristic,
> +   start vq index and count may be different for each device. Secondly,
> it is
> +   inefficient for device to provide flow filters capabilities via a
> config space
> +   region. Hence, the device should be able to share these attributes
> using
> +   dma interface, instead of transport registers.
> +4. Since flow filters are enabled much later in the driver life cycle,
> driver
> +   will likely create these queues when flow filters are enabled.
> +5. Flow filter operations are often accelerated by device in a
> hardware. Ability
> +   to handle them on a queue other than control vq is desired. This
> achieves near
> +   zero modifications to existing implementations to add new operations
> on new
> +   purpose built queues (similar to transmit and receive queue).
> +6. The filter masks are optional; the device should be able to expose
> if it
> +   support filter masks.
> +7. The driver may want to have priority among group of flow entries; to
> facilitate
> +   the device support grouping flow filter entries by a notion of a
> group. Each
> +   group defines priority in processing flow.
> +8. The driver and group owner driver should be able to query supported
> device
> +   limits for the flow filter entries.
> +
> +### 3.4.2 flow operations path
> +1. The driver should be able to define a receive packet match criteria,
> an
> +   action and a destination for a packet. For example, an ipv4 packet
> with a
> +   multicast address to be steered to the receive vq 0. The second
> example is
> +   ipv4, tcp packet matching a specified IP address and tcp port tuple
> to
> +   be steered to receive vq 10.
> +2. The match criteria should include exact tuple fields well-defined
> such as mac
> +   address, IP addresses, tcp/udp ports, etc.
> +3. The match criteria should also optionally include the field mask.
> +4. The match criteria may optionally also include specific packet byte
> offset
> +   pattern, match length, mask instead of RFC defined fields.
> +   length, and matching pattern, which may not be defined in the
> standard RFC.
> +5. Action includes (a) dropping or (b) forwarding the packet.
> +6. Destination is a receive virtqueue index.
> +7. The device should process packet receive filters programmed via
> control vq
> +   commands first in the processing chain.
> +7. The device should process RFF entries before RSS configuration,
> i.e.,
> +   when there is a miss on the RFF entry, RSS configuration applies if
> it exists.
> +8. To summarize the processing chain on a rx packet is:
> +   {mac,vlan,promisc rx filters} -> {receive flow filters} -> {rss/hash
> config}.
Shouldn't this be
                                                         |-match-> {RFF processing}
{mac,vlan,promisc rx filters} -> {receive flow filters} -|
                                                         |-no match-> {rss/hash config}.
The above looks like RSS processing will always happen.
> +9. If multiple entries are programmed which has overlapping attributes
> for a
> +   received packet, the driver to define the location/priority of the
> entry.
If driver does not provide group or provides same group with 2 rules that
have a same match part, does the new entry overwrite the old one for exact
matches(no mask) ? and for rules with masks, does the rule with longest match take 
precedence? or the latest added rule takes precedence? 
> +10. The filter entries are usually short in size of few tens of bytes,
> +   for example IPv6 + TCP tuple would be 36 bytes, and ops/sec rate is
> +   high, hence supplying fields inside the queue descriptor is
> preferred for
> +   up to a certain fixed size, say 56 bytes.
> +11. A flow filter entry consists of (a) match criteria, (b) action,
> +    (c) destination and (d) a unique 32 bit flow id, all supplied by
> the
> +    driver.
> +12. The driver should be able to query and delete flow filter entry by
> the
> +    the device by the flow id.
The flowid here seems to be used for rule index. Can this be returned by
the device instead of being sent by driver? A 32 bit value to store might
impose undue restrictions on devices that have lesser capacity. Or could
there be a restriction that flowid cannot exceed the value returned by 
device as the capacity. 
> +
> +### 3.4.3 interface example
> +
> +Flow filter capabilities to query using a DMA interface:
> +
> +```
> +struct flow_filter_capabilities {
> +	u8 flow_groups;
> +	u16 num_flow_filter_vqs;
> +	u16 start_vq_index;
> +	u32 max_flow_filters_per_group;
> +	u32 max_flow_filters;
> +	u64 supported_packet_field_mask_bmap[4];
> +};
> +
> +
> +```
> +
> +1. Flow filter entry add/modify, delete:
> +
> +struct virtio_net_rff_add_modify {
> +	u8 flow_op;
> +	u8 group_id;
> +	u8 padding[2];
> +	le32 flow_id;
> +	struct match_criteria mc;
> +	struct destination dest;
> +	struct action action;
> +
> +	struct match_criteria mask;	/* optional */
> +};
> +
> +2. Flow filter entry delete:
> +struct virtio_net_rff_delete {
> +	u8 flow_op;
> +	u8 group_id;
> +	u8 padding[2];
> +	le32 flow_id;
> +};
> --
> 2.26.2
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oasis-
> 2Dopen.org_apps_org_workgroup_portal_my-
> 5Fworkgroups.php&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=NHDPsfcAYlN2z-
> NXHHG4WB09qqS0voo_nf6_kGS625A&m=s4DkD0yyl9BvPMHr6DMuYpiRlBf_vmBXRWinZK_m
> NRho9Fue9HuTbs35i1zY6NeO&s=VS8X60QpRfg49zJm_fZGhwms1_R4J-
> MDPPhlu0nMG2w&e=