Release Notes

Changes in CSIT-2110

  1. VPP PERFORMANCE TESTS

    • Intel Xeon Ice Lake: Performance test data for these platforms continues to be provided by external Intel benchmarking labs executing CSIT-2110 tests. For details about the physical setup see Performance Physical Testbeds.

    • AWS EC2 c5n: Performance test data for AWS c5n instances has been added to report for ip4, ip6 and ipsec tests.

    • Reduction of tests: Removed certain test variations executed iteratively for the report (as well as in daily and weekly trending) due to physical testbeds overload.

  2. TEST FRAMEWORK

    • CSIT test environment version has been updated to ver. 8, see Environment Versioning.

    • CSIT in AWS environment: Completed CSIT automation support for AWS c5n instances environment, including weekly performance trending tests execution.

    • CSIT PAPI support: Due to issues with PAPI performance, and deprecation of VAT, CLI is used in CSIT for many VPP scale tests. See known issues below.

    • General Code Housekeeping: Ongoing code optimizations and bug fixes.

  3. PRESENTATION AND ANALYTICS LAYER

    • Graphs improvements: Updated Packet Latency graphs, see Packet Latency.

Known Issues

Fixed

Issues reported in previous releases which were fixed in this release:

#

JiraID

Issue Description

1

CSIT-1789

AVF driver does not perform RSS in a deterministic way. VPP now uses the same RSS key with AVF driver as with DPDK driver.

2

CSIT-1786

IP4 and IP6 scale tests failing with no traffic forwarded. CSIT replaced the old single VAT command by file full of “exec” CLI commands executed by VAT.

Old

Issues reported in previous releases which still affect the current results.

#

JiraID

Issue Description

3

CSIT-1782

Multicore AVF tests are failing when trying to create interface. Frequency is reduced by s CSIT workaround, but occasional failures do still happen.

4

CSIT-1671

All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much longer to program VPP). This needs to be addressed on the PAPI side. Currently, the time critical code uses VAT running large files with exec statements and CLI commands. Still, we needed to reduce the number of scale tests run to keep overall duration reasonable. More improvements needed to achieve sufficient configuration speed.

VPP-1763

5

5

CSIT-1789

IPSEC SW async scheduler tests show bad behavior. VPP code is not behaving correctly when crypto workers are the bottleneck.

VPP-1998

6

CSIT-1785

NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue.

VPP-1972

7

CSIT-1791

Performance regression in RDMA tests, due to CSIT environment changes. Two symptoms: 1. 10-20% regression across most tests. 2. DUT performance cap just below 38 Mpps.

New

No new issues appeared in CSIT-2110 for VPP performance tests, except for the performance changes listed below.

Root Cause Analysis for Performance Changes

List of RCAs in CSIT-2110 for VPP performance changes:

#

JiraID

Issue Description

0

To be updated when descriptions in Jira tickets are ready.