guest-csum-offload.pdf

Transcript

1. QEMU and ovs-dpdk Guest L4 CSUM Offload Yusho Yamaguchi
2. With OVS 3.1: offload is not advertised Hypervisor ovs-dpdk · OVS 3.1 does not advertise L4 CSUM offload QEMU offload NOT available virtual NIC device Guest VM offload NOT available guest driver 1 ovs → QEMU 2 QEMU device → guest driver ovs-dpdk tells QEMU that L4 CSUM offload is not supported. The QEMU device tells the guest driver that L4 CSUM offload is not supported.
3. With OVS 3.2+: offload is advertised Hypervisor ovs-dpdk · OVS 3.2 advertises L4 CSUM offload QEMU offload AVAILABLE virtual NIC device Guest VM offload AVAILABLE guest driver 1 ovs → QEMU 2 QEMU device → guest driver ovs-dpdk tells QEMU that L4 CSUM offload is supported. The QEMU device tells the guest driver that L4 CSUM offload is supported.
4. VMs booted on OVS 3.2+ Hosts cannot live-migrate to OVS 3.1 Hosts Hypervisor A (source) Hypervisor B (live-migration destination) ovs-dpdk · OVS 3.2+ ovs-dpdk · OVS 3.1 advertises L4 CSUM offload clears the L4 CSUM feature bits QEMU QEMU feature negotiation QEMU has the L4 CSUM offload features Guest VM QEMU ✕ lacks the L4 CSUM offload features Live migration (LM) running (created on OVS 3.2+) Guest VM (incoming) migration fails Result: migration fails Cause: OVS breaks backward compatibility The destination QEMU lacks the required features (OVS 3.1 clears the feature bits), so feature negotiation between the two QEMUs fails. OVS 3.2+ advertises L4 CSUM offload while OVS 3.1 clears the feature bits, so the two versions are not compatible.
5. Guest behavior changes after the VM moves to OVS 3.2+ Hypervisor A (still on OVS 3.1) Hypervisor B (now on OVS 3.2+) ovs-dpdk · OVS 3.1 ovs-dpdk · OVS 3.2+ does not advertise L4 CSUM offload advertises L4 CSUM offload Cold migration (incl. evacuate) QEMU Resize QEMU L4 CSUM offload not supported Shelve / unshelve L4 CSUM offload supported Guest VM created while the hypervisor ran OVS 3.1 Guest VM (re-created) VM is re-created on Hypervisor B L4 CSUM offload now advertised Result: guest behavior changes Cause: OVS breaks backward compatibility After the move, the guest driver is told that L4 CSUM offload is supported, so guest behavior changes. OVS 3.2+ advertises L4 CSUM offload, which OVS 3.1 did not, so the same VM sees different features on the new hypervisor.
6. TCP/UDP communication fails with Mellanox NIC Hypervisor Guest VM guest driver QEMU ovs-dpdk · OVS 3.2–3.6 virtual NIC device offload enabled ✕ Mellanox NIC hardware offload Guest TCP/UDP communication fails Result: TCP/UDP traffic fails Cause: OVS bug (fixed in 3.6) The guest VM cannot send or receive TCP/UDP packets. A bug in OVS 3.2 to 3.6 breaks hardware offload on the Mellanox NIC; it is fixed in OVS 3.6. Workaround exists The problem can be avoided with a workaround.