Test Environment¶
Environment Versioning¶
CSIT test environment versioning has been introduced to track modifications of the test environment.
Any benchmark anomalies (progressions, regressions) between releases of a DUT application (e.g. VPP, DPDK), are determined by testing it in the same test environment, to avoid test environment changes clouding the picture. To beter distinguish impact of test environment changes, we also execute tests without any SUT (just with TRex TG sending packets over a link looping back to TG).
A mirror approach is introduced to determine benchmarking anomalies due to the test environment change. This is achieved by testing the same DUT application version between releases of CSIT test system. This works under the assumption that the behaviour of the DUT is deterministic under the test conditions.
CSIT test environment versioning scheme ensures integrity of all the test system components, including their HW revisions, compiled SW code versions and SW source code, within a specific CSIT version. Components included in the CSIT environment versioning include:
HW Server hardware firmware and BIOS (motherboard, processsor, NIC(s), accelerator card(s)), tracked in CSIT branch in
./docs/lab/<server_platform_name>_hw_bios_cfg.md, e.g. Xeon Skylake servers.Linux Server Linux OS version and configuration, tracked in CSIT Reports in SUT Settings and Pre-Test Server Calibration.
TRex TRex Traffic Generator version, drivers and configuration tracked in TG Settings.
CSIT CSIT framework code tracked in CSIT release branches.
Following is the list of CSIT versions to date:
Ver. 1 associated with CSIT rls1908 branch (HW, Linux, TRex, CSIT).
Ver. 2 associated with CSIT rls2001 branch (HW, Linux, TRex, CSIT).
Ver. 4 associated with CSIT rls2005 branch (HW, Linux, TRex, CSIT).
Ver. 5 associated with CSIT rls2009 branch (HW, Linux, TRex, CSIT).
The main change is TRex data-plane core resource adjustments: increase from 7 to 8 cores and pinning cores to interfaces for better TRex performance with symmetric traffic profiles.
Ver. 6 associated with CSIT rls2101 branch (HW, Linux, TRex, CSIT).
The main change is TRex version upgrade: increase from 2.82 to 2.86.
Ver. 7 associated with CSIT rls2106 branch (HW, Linux, TRex, CSIT).
TRex version upgrade: increase from 2.86 to 2.88.
Ubuntu upgrade: upgrade from 18.04 LTS to 20.04.2 LTS.
Ver. 8 associated with CSIT rls2110 branch (HW, Linux, TRex, CSIT).
Intel NIC 700/800 series firmware upgrade based on DPDK compatibility matrix.
Ver. 9 associated with CSIT rls2202 branch (HW, Linux, TRex, CSIT).
Intel NIC 700/800 series firmware upgrade based on DPDK compatibility matrix.
Ver. 10 associated with CSIT rls2206 branch (HW, Linux, TRex, CSIT).
Intel NIC 700/800 series firmware upgrade based on DPDK compatibility matrix.
Mellanox 556A series firmware upgrade based on DPDK compatibility matrix.
Intel IceLake all core turbo frequency turned off. Current base frequency is 2.6GHz.
SUT Settings - Linux¶
System provisioning is done by combination of PXE boot unattented install and Ansible described in CSIT Testbed Setup.
Linux Boot Parameters¶
isolcpus=<cpu number>-<cpu number> used for all cpu cores apart from first core of each socket used for running VPP worker threads and Qemu/LXC processes https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt
intel_pstate=disable - [X86] Do not enable intel_pstate as the default scaling driver for the supported processors. Intel P-State driver decide what P-state (CPU core power state) to use based on requesting policy from the cpufreq core. [X86 - Either 32-bit or 64-bit x86] https://www.kernel.org/doc/Documentation/cpu-freq/intel-pstate.txt
nohz_full=<cpu number>-<cpu number> - [KNL,BOOT] In kernels built with CONFIG_NO_HZ_FULL=y, set the specified list of CPUs whose tick will be stopped whenever possible. The boot CPU will be forced outside the range to maintain the timekeeping. The CPUs in this range must also be included in the rcu_nocbs= set. Specifies the adaptive-ticks CPU cores, causing kernel to avoid sending scheduling-clock interrupts to listed cores as long as they have a single runnable task. [KNL - Is a kernel start-up parameter, SMP - The kernel is an SMP kernel]. https://www.kernel.org/doc/Documentation/timers/NO_HZ.txt
rcu_nocbs - [KNL] In kernels built with CONFIG_RCU_NOCB_CPU=y, set the specified list of CPUs to be no-callback CPUs, that never queue RCU callbacks (read-copy update). https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt
numa_balancing=disable - [KNL,X86] Disable automatic NUMA balancing.
intel_iommu=enable - [DMAR] Enable Intel IOMMU driver (DMAR) option.
iommu=on, iommu=pt - [x86, IA-64] Disable IOMMU bypass, using IOMMU for PCI devices.
nmi_watchdog=0 - [KNL,BUGS=X86] Debugging features for SMP kernels. Turn hardlockup detector in nmi_watchdog off.
nosoftlockup - [KNL] Disable the soft-lockup detector.
tsc=reliable - Disable clocksource stability checks for TSC. [x86] reliable: mark tsc clocksource as reliable, this disables clocksource verification at runtime, as well as the stability checks done at bootup. Used to enable high-resolution timer mode on older hardware, and in virtualized environment.
hpet=disable - [X86-32,HPET] Disable HPET and use PIT instead.
Hugepages Configuration¶
Huge pages are managed via sysctl configuration located in /etc/sysctl.d/90-csit.conf on each testbed. Default huge page size is 2M. The exact amount of huge pages depends on testbed. All the values are defined in Ansible inventory - hosts files.
DUT Settings - DPDK¶
DPDK Version¶
DPDK-22.03
DPDK Compile Parameters¶
make install T=<arch>-<machine>-linuxapp-gcc -j
Testpmd Startup Configuration¶
Testpmd startup configuration changes per test case with different settings for $$INT, $$CORES, $$RXQ, $$RXD and max-pkt-len parameter if test is sending jumbo frames. Startup command template:
testpmd -v -l $$CORE_LIST -w $$INT1 -w $$INT2 --master-lcore 0 --in-memory -- --forward-mode=io --burst=64 --txd=$$TXD --rxd=$$RXD --txq=$$TXQ --rxq=$$RXQ --tx-offloads=0x0 --numa --auto-start --total-num-mbufs=16384 --nb-ports=2 --portmask=0x3 --disable-link-check --max-pkt-len=$$PKT_LEN [--mbuf-size=16384] --nb-cores=$$CORES
L3FWD Startup Configuration¶
L3FWD startup configuration changes per test case with different settings for $$INT, $$CORES and enable-jumbo parameter if test is sending jumbo frames. Startup command template:
l3fwd -v -l $$CORE_LIST -w $$INT1 -w $$INT2 --master-lcore 0 --in-memory -- --parse-ptype --eth-dest="0,${adj_mac0}" --eth-dest="1,${adj_mac1}" --config="${port_config}" [--enable-jumbo] -P -L -p 0x3
TG Settings - TRex¶
TG Version¶
TRex v2.97
DPDK Version¶
DPDK v21.02
TG Installation¶
T-Rex installation is managed via Ansible role.
TG Startup Configuration¶
$ sudo -E -S sh -c 'cat << EOF > /etc/trex_cfg.yaml
- version: 2
c: 8
limit_memory: 8192
interfaces: ["${pci1}","${pci2}"]
port_info:
- dest_mac: [${dest_mac1}]
src_mac: [${src_mac1}]
- dest_mac: [${dest_mac2}]
src_mac: [${src_mac2}]
platform :
master_thread_id: 0
latency_thread_id: 9
dual_if:
- socket: 0
threads: [1, 2, 3, 4, 5, 6, 7, 8]
EOF'
TG Startup Command (Stateless Mode)¶
$ sudo -E -S sh -c "cd '${trex_install_dir}/scripts/' && \
nohup ./t-rex-64 -i --prefix $(hostname) --hdrh --no-scapy-server \
--mbuf-factor 32 > /tmp/trex.log 2>&1 &" > /dev/null
Also, Python client is now starting traffic with:
core_mask=STLClient.CORE_MASK_PIN
TG Startup Command (Stateful Mode)¶
$ sudo -E -S sh -c "cd '${trex_install_dir}/scripts/' && \
nohup ./t-rex-64 -i --prefix $(hostname) --astf --hdrh --no-scapy-server \
--mbuf-factor 32 > /tmp/trex.log 2>&1 &" > /dev/null
TG API Driver¶
Pre-Test Server Calibration¶
Number of SUT server sub-system runtime parameters have been identified as impacting data plane performance tests. Calibrating those parameters is part of FD.io CSIT pre-test activities, and includes measuring and reporting following:
System level core jitter - measure duration of core interrupts by Linux in clock cycles and how often interrupts happen. Using CPU core jitter tool.
Memory bandwidth - measure bandwidth with Intel MLC tool.
Memory latency - measure memory latency with Intel MLC tool.
Cache latency at all levels (L1, L2, and Last Level Cache) - measure cache latency with Intel MLC tool.
Measured values of listed parameters are especially important for repeatable zero packet loss throughput measurements across multiple system instances. Generally they come useful as a background data for comparing data plane performance results across disparate servers.
Following sections include measured calibration data for testbeds.
Ice Lake¶
Following sections include sample calibration data measured on s71-t212-sut1 server running in one of the Intel Xeon Ice Lake testbeds as specified in `FD.io CSIT testbeds - Xeon Ice Lake`_.
Calibration data obtained from all other servers in Ice Lake testbeds shows the same or similar values.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=3250758a-9bb6-48c8-9c36-ecb6a269223f ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-31,33-63,65-95,97-127 mce=off nmi_watchdog=0 nohz_full=1-31,33-63,65-95,97-127 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-31,33-63,65-95,97-127 tsc=reliable console=ttyS0,115200n8 quiet
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
System-level Core Jitter¶
$ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 30
Linux Jitter testing program version 1.9
Iterations=20
The pragram will execute a dummy function 80000 times
Display is updated every 20000 displayUpdate intervals
Thread affinity will be set to core_id:7
Timings are in CPU Core cycles
Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
last_Exec: The Excution time of last iteration just before the display update
Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
tmp: Cumulative value calcualted by the dummy function
Interval: Time interval between the display updates in Core Cycles
Sample No: Sample number
Inst_Min,Inst_Max,Inst_jitter,last_Exec,Abs_min,Abs_max,tmp,Interval,Sample No
126082,133950,7868,126094,126082,133950,3829268480,2524167454,1
126082,134696,8614,126094,126082,134696,1778253824,2524273022,2
126082,136092,10010,126094,126082,136092,4022206464,2524203296,3
126082,135094,9012,126094,126082,136092,1971191808,2524274302,4
126082,136482,10400,126094,126082,136482,4215144448,2524318496,5
126082,134990,8908,126094,126082,136482,2164129792,2524155038,6
126082,134710,8628,126092,126082,136482,113115136,2524215228,7
126082,135080,8998,126092,126082,136482,2357067776,2524168906,8
126082,134470,8388,126094,126082,136482,306053120,2524163312,9
126082,135246,9164,126092,126082,136482,2550005760,2524394986,10
126082,132662,6580,126094,126082,136482,498991104,2524163156,11
126082,132954,6872,126094,126082,136482,2742943744,2524154386,12
126082,135340,9258,126092,126082,136482,691929088,2524222386,13
126082,133036,6954,126094,126082,136482,2935881728,2524150132,14
126082,137776,11694,126094,126082,137776,884867072,2524239346,15
126082,137850,11768,126094,126082,137850,3128819712,2524342944,16
126082,133000,6918,126094,126082,137850,1077805056,2524160062,17
126082,133332,7250,126094,126082,137850,3321757696,2524158804,18
126082,133234,7152,126092,126082,137850,1270743040,2524174400,19
126082,152552,26470,126094,126082,152552,3514695680,2524857280,20
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several speculative execution CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: YES
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: YES
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: YES
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): YES
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): YES
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): YES
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): YES
* TSX_CTRL MSR indicates TSX RTM is disabled: YES
* TSX_CTRL MSR indicates TSX CPUID bit is cleared: YES
* CPU supports Transactional Synchronization Extensions (TSX): NO
* CPU supports Software Guard Extensions (SGX): YES
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x6a stepping 0x6 ucode 0xd000280 cpuid 0x606a6)
* CPU microcode is the latest known available version: NO (latest version is 0xd0002a0 dated 2021/04/25 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): YES
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
> STATUS: UNKNOWN (/sys vulnerability interface use forced, but its not available!)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling)
> STATUS: VULNERABLE (IBRS+IBPB or retpoline+IBPB is needed to mitigate the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (Not affected)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
> STATUS: NOT VULNERABLE (your kernel reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (Not affected)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:?? CVE-2017-5715:KO CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Skylake¶
Following sections include sample calibration data measured on s11-t31-sut1 server running in one of the Intel Xeon Skylake testbeds as specified in FD.io CSIT testbeds - Xeon Skylake, Arm, Atom.
Calibration data obtained from all other servers in Skylake testbeds shows the same or similar values.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=55d44abd-94d6-4b26-9d93-5877a8658016 ro audit=0 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-27,29-55,57-83,85-111 mce=off nmi_watchdog=0 nohz_full=1-27,29-55,57-83,85-111 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-27,29-55,57-83,85-111 tsc=reliable console=ttyS0,115200n8 quiet
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
System-level Core Jitter¶
$ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 20
Linux Jitter testing program version 1.8
Iterations=20
The pragram will execute a dummy function 80000 times
Display is updated every 20000 displayUpdate intervals
Timings are in CPU Core cycles
Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
last_Exec: The Excution time of last iteration just before the display update
Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
tmp: Cumulative value calcualted by the dummy function
Interval: Time interval between the display updates in Core Cycles
Sample No: Sample number
Inst_Min Inst_Max Inst_jitter last_Exec Abs_min Abs_max tmp Interval Sample No
160022 171330 11308 160022 160022 171330 2538733568 3204142750 1
160022 167294 7272 160026 160022 171330 328335360 3203873548 2
160022 167560 7538 160026 160022 171330 2412904448 3203878736 3
160022 169000 8978 160024 160022 171330 202506240 3203864588 4
160022 166572 6550 160026 160022 171330 2287075328 3203866224 5
160022 167460 7438 160026 160022 171330 76677120 3203854632 6
160022 168134 8112 160024 160022 171330 2161246208 3203874674 7
160022 169094 9072 160022 160022 171330 4245815296 3203878798 8
160022 172460 12438 160024 160022 172460 2035417088 3204112010 9
160022 167862 7840 160030 160022 172460 4119986176 3203856800 10
160022 168398 8376 160024 160022 172460 1909587968 3203854192 11
160022 167548 7526 160024 160022 172460 3994157056 3203847442 12
160022 167562 7540 160026 160022 172460 1783758848 3203862936 13
160022 167604 7582 160024 160022 172460 3868327936 3203859346 14
160022 168262 8240 160024 160022 172460 1657929728 3203851120 15
160022 169700 9678 160024 160022 172460 3742498816 3203877690 16
160022 170476 10454 160026 160022 172460 1532100608 3204088480 17
160022 167798 7776 160024 160022 172460 3616669696 3203862072 18
160022 166540 6518 160024 160022 172460 1406271488 3203836904 19
160022 167516 7494 160024 160022 172460 3490840576 3203848120 20
Memory Bandwidth¶
$ sudo /home/testuser/mlc --bandwidth_matrix
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --bandwidth_matrix
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Numa node
Numa node 0 1
0 107947.7 50951.5
1 50834.6 108183.4
$ sudo /home/testuser/mlc --peak_injection_bandwidth
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --peak_injection_bandwidth
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 215733.9
3:1 Reads-Writes : 182141.9
2:1 Reads-Writes : 178615.7
1:1 Reads-Writes : 149911.3
Stream-triad like: 159533.6
$ sudo /home/testuser/mlc --max_bandwidth
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --max_bandwidth
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Maximum Memory Bandwidths for the system
Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 216875.73
3:1 Reads-Writes : 182615.14
2:1 Reads-Writes : 178745.67
1:1 Reads-Writes : 149485.27
Stream-triad like: 180057.87
Memory Latency¶
$ sudo /home/testuser/mlc --latency_matrix
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --latency_matrix
Using buffer size of 2000.000MB
Measuring idle latencies (in ns)...
Numa node
Numa node 0 1
0 81.4 131.1
1 131.1 81.3
$ sudo /home/testuser/mlc --idle_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --idle_latency
Using buffer size of 2000.000MB
Each iteration took 202.0 core clocks ( 80.8 ns)
$ sudo /home/testuser/mlc --loaded_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --loaded_latency
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject Latency Bandwidth
Delay (ns) MB/sec
==========================
00000 282.66 215712.8
00002 282.14 215757.4
00008 280.21 215868.1
00015 279.20 216313.2
00050 275.25 216643.0
00100 227.05 215075.0
00200 121.92 160242.9
00300 101.21 111587.4
00400 95.48 85019.7
00500 94.46 68717.3
00700 92.27 49742.2
01000 91.03 35264.8
01300 90.11 27396.3
01700 89.34 21178.7
02500 90.15 14672.8
03500 89.00 10715.7
05000 82.00 7788.2
09000 81.46 4684.0
20000 81.40 2541.9
L1/L2/LLC Latency¶
$ sudo /home/testuser/mlc --c2c_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --c2c_latency
Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT latency 53.7
Local Socket L2->L2 HITM latency 53.7
Remote Socket L2->L2 HITM latency (data address homed in writer socket)
Reader Numa Node
Writer Numa Node 0 1
0 - 113.9
1 113.9 -
Remote Socket L2->L2 HITM latency (data address homed in reader socket)
Reader Numa Node
Writer Numa Node 0 1
0 - 177.9
1 177.6 -
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several “speculative execution” CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: NO
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): NO
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: NO
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): NO
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): NO
* CPU supports Transactional Synchronization Extensions (TSX): YES (RTM feature bit)
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x55 stepping 0x4 ucode 0x2000065 cpuid 0x50654)
* CPU microcode is the latest known available version: NO (latest version is 0x2006b06 dated 2021/03/08 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): YES
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): YES
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): YES
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): YES
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): YES
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): YES
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
> STATUS: NOT VULNERABLE (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for firmware code only)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
* Kernel supports RSB filling: YES
> STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports INVPCID, performance impact of PTI will be greatly reduced)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: YES (per-thread through prctl)
* SSB mitigation currently active for selected processes: YES (boltd fwupd irqbalance systemd-journald systemd-logind systemd-networkd systemd-resolved systemd-timesyncd systemd-udevd)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
* Kernel supports PTE inversion: YES (found in kernel image)
* PTE inversion enabled and active: YES
> STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: NO
* Mitigation 2
* L1D flush is supported by kernel: YES (found flush_l1d in /proc/cpuinfo)
* L1D flush enabled: YES (conditional flushes)
* Hardware-backed L1D flush supported: YES (performance impact of the mitigation will be greatly reduced)
* Hyper-Threading (SMT) is enabled: YES
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
> STATUS: NOT VULNERABLE (Mitigation: Clear CPU buffers; SMT vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (KVM: Mitigation: Split huge pages)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: YES (KVM: Mitigation: Split huge pages)
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: YES (found SRBDS implementation evidence in kernel image. Your kernel is up to date for SRBDS mitigation)
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Cascade Lake¶
Following sections include sample calibration data measured on s32-t27-sut1 server running in one of the Intel Xeon Skylake testbeds as specified in FD.io CSIT testbeds - Xeon Cascade Lake.
Calibration data obtained from all other servers in Cascade Lake testbeds shows the same or similar values.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=b1f0dc29-1d4f-4777-b37d-a5e26e233d55 ro audit=0 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-27,29-55,57-83,85-111 mce=off nmi_watchdog=0 nohz_full=1-27,29-55,57-83,85-111 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-27,29-55,57-83,85-111 tsc=reliable console=ttyS0,115200n8 quiet
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
System-level Core Jitter¶
$ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 30
Linux Jitter testing program version 1.9
Iterations=30
The pragram will execute a dummy function 80000 times
Display is updated every 20000 displayUpdate intervals
Thread affinity will be set to core_id:7
Timings are in CPU Core cycles
Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
last_Exec: The Excution time of last iteration just before the display update
Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
tmp: Cumulative value calcualted by the dummy function
Interval: Time interval between the display updates in Core Cycles
Sample No: Sample number
Inst_Min,Inst_Max,Inst_jitter,last_Exec,Abs_min,Abs_max,tmp,Interval,Sample No
160022,167590,7568,160026,160022,167590,2057568256,3203711852,1
160022,170628,10606,160024,160022,170628,4079222784,3204010824,2
160022,169824,9802,160024,160022,170628,1805910016,3203812064,3
160022,168832,8810,160030,160022,170628,3827564544,3203792594,4
160022,168248,8226,160026,160022,170628,1554251776,3203765920,5
160022,167834,7812,160028,160022,170628,3575906304,3203761114,6
160022,167442,7420,160024,160022,170628,1302593536,3203769250,7
160022,169120,9098,160028,160022,170628,3324248064,3203853340,8
160022,170710,10688,160024,160022,170710,1050935296,3203985878,9
160022,167952,7930,160024,160022,170710,3072589824,3203733756,10
160022,168314,8292,160030,160022,170710,799277056,3203741152,11
160022,169672,9650,160024,160022,170710,2820931584,3203739910,12
160022,168684,8662,160024,160022,170710,547618816,3203727336,13
160022,168246,8224,160024,160022,170710,2569273344,3203739052,14
160022,168134,8112,160030,160022,170710,295960576,3203735874,15
160022,170230,10208,160024,160022,170710,2317615104,3203996356,16
160022,167190,7168,160024,160022,170710,44302336,3203713628,17
160022,167304,7282,160024,160022,170710,2065956864,3203717954,18
160022,167500,7478,160024,160022,170710,4087611392,3203706674,19
160022,167302,7280,160024,160022,170710,1814298624,3203726452,20
160022,167266,7244,160024,160022,170710,3835953152,3203702804,21
160022,167820,7798,160022,160022,170710,1562640384,3203719138,22
160022,168100,8078,160024,160022,170710,3584294912,3203716636,23
160022,170408,10386,160024,160022,170710,1310982144,3203946958,24
160022,167276,7254,160024,160022,170710,3332636672,3203706236,25
160022,167052,7030,160024,160022,170710,1059323904,3203696444,26
160022,170322,10300,160024,160022,170710,3080978432,3203747514,27
160022,167332,7310,160024,160022,170710,807665664,3203716210,28
160022,167426,7404,160026,160022,170710,2829320192,3203700630,29
160022,168840,8818,160024,160022,170710,556007424,3203727658,30
Memory Bandwidth¶
$ sudo /home/testuser/mlc --bandwidth_matrix
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --bandwidth_matrix
Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Numa node
Numa node 0 1
0 122097.7 51327.9
1 51309.2 122005.5
$ sudo /home/testuser/mlc --peak_injection_bandwidth
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --peak_injection_bandwidth
Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 243159.4
3:1 Reads-Writes : 219132.5
2:1 Reads-Writes : 216603.1
1:1 Reads-Writes : 203713.0
Stream-triad like: 193790.8
$ sudo /home/testuser/mlc --max_bandwidth
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --max_bandwidth
Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
Measuring Maximum Memory Bandwidths for the system
Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 244114.27
3:1 Reads-Writes : 219441.97
2:1 Reads-Writes : 216603.72
1:1 Reads-Writes : 203679.09
Stream-triad like: 214902.80
Memory Latency¶
$ sudo /home/testuser/mlc --latency_matrix
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --latency_matrix
Using buffer size of 2000.000MiB
Measuring idle latencies (in ns)...
Numa node
Numa node 0 1
0 81.2 130.2
1 130.2 81.1
$ sudo /home/testuser/mlc --idle_latency
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --idle_latency
Using buffer size of 2000.000MiB
Each iteration took 186.1 core clocks ( 80.9 ns)
$ sudo /home/testuser/mlc --loaded_latency
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --loaded_latency
Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject Latency Bandwidth
Delay (ns) MB/sec
==========================
00000 233.86 243421.9
00002 230.61 243544.1
00008 232.56 243394.5
00015 229.52 244076.6
00050 225.82 244290.6
00100 161.65 236744.8
00200 100.63 133844.0
00300 96.84 90548.2
00400 95.71 68504.3
00500 95.68 55139.0
00700 88.77 39798.4
01000 84.74 28200.1
01300 83.08 21915.5
01700 82.27 16969.3
02500 81.66 11810.6
03500 81.98 8662.9
05000 81.48 6306.8
09000 81.17 3857.8
20000 80.19 2179.9
L1/L2/LLC Latency¶
$ sudo /home/testuser/mlc --c2c_latency
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --c2c_latency
Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT latency 55.5
Local Socket L2->L2 HITM latency 55.6
Remote Socket L2->L2 HITM latency (data address homed in writer socket)
Reader Numa Node
Writer Numa Node 0 1
0 - 115.6
1 115.6 -
Remote Socket L2->L2 HITM latency (data address homed in reader socket)
Reader Numa Node
Writer Numa Node 0 1
0 - 178.2
1 178.4 -
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several speculative execution CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: YES
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: YES
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: YES
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): YES
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): YES
* TSX_CTRL MSR indicates TSX RTM is disabled: YES
* TSX_CTRL MSR indicates TSX CPUID bit is cleared: YES
* CPU supports Transactional Synchronization Extensions (TSX): NO
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x55 stepping 0x7 ucode 0x500002c cpuid 0x50657)
* CPU microcode is the latest known available version: NO (latest version is 0x5003102 dated 2021/03/08 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
> STATUS: UNKNOWN (/sys vulnerability interface use forced, but it s not available!)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling)
> STATUS: VULNERABLE (IBRS+IBPB or retpoline+IBPB+RSB filling, is needed to mitigate the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (Not affected)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
> STATUS: NOT VULNERABLE (your kernel reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Mitigation: TSX disabled)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (KVM: Mitigation: Split huge pages)
> STATUS: NOT VULNERABLE (KVM: Mitigation: Split huge pages)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:?? CVE-2017-5715:KO CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: YES
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: YES
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: YES
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): YES
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): YES
* TSX_CTRL MSR indicates TSX RTM is disabled: YES
* TSX_CTRL MSR indicates TSX CPUID bit is cleared: YES
* CPU supports Transactional Synchronization Extensions (TSX): NO
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x55 stepping 0x7 ucode 0x500002c cpuid 0x50657)
* CPU microcode is the latest known available version: NO (latest version is 0x5003102 dated 2021/03/08 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
> STATUS: UNKNOWN (/sys vulnerability interface use forced, but its not available!)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling)
> STATUS: VULNERABLE (IBRS+IBPB or retpoline+IBPB+RSB filling, is needed to mitigate the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (Not affected)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
> STATUS: NOT VULNERABLE (your kernel reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Mitigation: TSX disabled)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (KVM: Mitigation: Split huge pages)
> STATUS: NOT VULNERABLE (KVM: Mitigation: Split huge pages)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:?? CVE-2017-5715:KO CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Denverton¶
Following sections include sample calibration data measured on Denverton server at Intel SH labs.
A 2-Node Atom Denverton testing took place at Intel Corporation carefully adhering to FD.io CSIT best practices.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=26ca7b0f-904a-462d-a1c6-98c420c29515 ro audit=0 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-5 mce=off nmi_watchdog=0 nohz_full=1-5 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-5 tsc=reliable console=tty0 console=ttyS0,115200n8
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
System-level Core Jitter¶
$ sudo taskset -c 2 /home/testuser/pma_tools/jitter/jitter -c 2 -i 20
Linux Jitter testing program version 1.9
Iterations=20
The pragram will execute a dummy function 80000 times
Display is updated every 20000 displayUpdate intervals
Thread affinity will be set to core_id:2
Timings are in CPU Core cycles
Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
last_Exec: The Excution time of last iteration just before the display update
Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
tmp: Cumulative value calcualted by the dummy function
Interval: Time interval between the display updates in Core Cycles
Sample No: Sample number
Inst_Min Inst_Max Inst_jitter last_Exec Abs_min Abs_max tmp Interval Sample No
177530 196100 18570 177530 177530 196100 4156751872 3556820054 1
177530 200784 23254 177530 177530 200784 321060864 3556897644 2
177530 196346 18816 177530 177530 200784 780337152 3556918674 3
177530 195962 18432 177530 177530 200784 1239613440 3556847928 4
177530 195960 18430 177530 177530 200784 1698889728 3556860214 5
177530 198824 21294 177530 177530 200784 2158166016 3556854934 6
177530 198522 20992 177530 177530 200784 2617442304 3556862410 7
177530 196362 18832 177530 177530 200784 3076718592 3556851636 8
177530 199114 21584 177530 177530 200784 3535994880 3556870846 9
177530 197194 19664 177530 177530 200784 3995271168 3556933584 10
177530 198272 20742 177536 177530 200784 159580160 3556869044 11
177530 197586 20056 177530 177530 200784 618856448 3556903482 12
177530 196072 18542 177530 177530 200784 1078132736 3556825540 13
177530 196354 18824 177530 177530 200784 1537409024 3556881664 14
177530 195906 18376 177530 177530 200784 1996685312 3556839924 15
177530 199066 21536 177530 177530 200784 2455961600 3556860220 16
177530 196968 19438 177530 177530 200784 2915237888 3556871890 17
177530 195896 18366 177530 177530 200784 3374514176 3556855338 18
177530 196020 18490 177530 177530 200784 3833790464 3556839820 19
177530 196030 18500 177530 177530 200784 4293066752 3556889196 20
Memory Bandwidth¶
$ sudo /home/testuser/mlc --bandwidth_matrix
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --bandwidth_matrix
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Memory node
Socket 0
0 28157.2
$ sudo /home/testuser/mlc --peak_injection_bandwidth
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --peak_injection_bandwidth
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 28150.0
3:1 Reads-Writes : 27425.0
2:1 Reads-Writes : 27565.4
1:1 Reads-Writes : 27489.3
Stream-triad like: 26878.2
$ sudo /home/testuser/mlc --max_bandwidth
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --max_bandwidth
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Maximum Memory Bandwidths for the system
Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads : 30032.40
3:1 Reads-Writes : 27450.88
2:1 Reads-Writes : 27567.46
1:1 Reads-Writes : 27501.90
Stream-triad like: 27124.82
Memory Latency¶
$ sudo /home/testuser/mlc --latency_matrix
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --latency_matrix
Using buffer size of 2000.000MB
Intel(R) Memory Latency Checker - v3.5
Measuring idle latencies (in ns)...
Memory node
Socket 0
0 93.1
$ sudo /home/testuser/mlc --idle_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --idle_latency
Using buffer size of 200.000MB
Each iteration took 186.7 core clocks ( 93.4 ns)
$ sudo /home/testuser/mlc --loaded_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --loaded_latency
Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject Latency Bandwidth
Delay (ns) MB/sec
==========================
00000 135.35 27186.0
00002 135.47 27176.9
00008 134.97 27063.3
00015 134.41 26825.6
00050 139.83 28419.1
00100 124.28 22616.4
00200 109.40 14139.8
00300 104.56 10275.1
00400 102.02 8120.0
00500 100.38 6751.4
00700 98.30 5124.9
01000 96.56 3852.7
01300 95.65 3149.0
01700 95.06 2585.4
02500 94.43 1988.8
03500 94.16 1621.1
05000 93.95 1343.1
09000 93.65 1052.6
20000 93.43 851.7
L1/L2/LLC Latency¶
$ sudo /home/testuser/mlc --c2c_latency
Intel(R) Memory Latency Checker - v3.5
Command line parameters: --c2c_latency
Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT latency 8.8
Local Socket L2->L2 HITM latency 8.8
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several “speculative execution” CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: NO
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): NO
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: NO
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): NO
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): NO
* CPU supports Transactional Synchronization Extensions (TSX): YES (RTM feature bit)
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x55 stepping 0x4 ucode 0x2000065 cpuid 0x50654)
* CPU microcode is the latest known available version: NO (latest version is 0x2006b06 dated 2021/03/08 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): YES
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): YES
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): YES
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): YES
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): YES
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): YES
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
> STATUS: NOT VULNERABLE (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for firmware code only)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
* Kernel supports RSB filling: YES
> STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports INVPCID, performance impact of PTI will be greatly reduced)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: YES (per-thread through prctl)
* SSB mitigation currently active for selected processes: YES (boltd fwupd irqbalance systemd-journald systemd-logind systemd-networkd systemd-resolved systemd-timesyncd systemd-udevd)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
* Kernel supports PTE inversion: YES (found in kernel image)
* PTE inversion enabled and active: YES
> STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: NO
* Mitigation 2
* L1D flush is supported by kernel: YES (found flush_l1d in /proc/cpuinfo)
* L1D flush enabled: YES (conditional flushes)
* Hardware-backed L1D flush supported: YES (performance impact of the mitigation will be greatly reduced)
* Hyper-Threading (SMT) is enabled: YES
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
> STATUS: NOT VULNERABLE (Mitigation: Clear CPU buffers; SMT vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (KVM: Mitigation: Split huge pages)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: YES (KVM: Mitigation: Split huge pages)
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: YES (found SRBDS implementation evidence in kernel image. Your kernel is up to date for SRBDS mitigation)
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Atom(TM) CPU C3858 @ 2.00GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: NO
* L1 data cache invalidation
* FLUSH_CMD MSR is available: NO
* CPU indicates L1D flush capability: NO
* Microarchitectural Data Sampling
* VERW instruction is available: NO
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: YES
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: NO
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): NO
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): NO
* CPU supports Transactional Synchronization Extensions (TSX): NO
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x5f stepping 0x1 ucode 0x20 cpuid 0x506f1)
* CPU microcode is the latest known available version: NO (latest version is 0x34 dated 2020/10/23 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): NO
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
> STATUS: NOT VULNERABLE (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for firmware code only)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
> STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: NO
* Reduced performance impact of PTI: NO (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: NO
> STATUS: VULNERABLE (an up-to-date CPU microcode is needed to mitigate this vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: NO
> STATUS: VULNERABLE (Your CPU doesnt support SSBD)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports PTE inversion: YES (found in kernel image)
* PTE inversion enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: NO
* Mitigation 2
* L1D flush is supported by kernel: YES (found flush_l1d in kernel image)
* L1D flush enabled: NO
* Hardware-backed L1D flush supported: NO (flush will be done in software, this is slower)
* Hyper-Threading (SMT) is enabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: YES (found md_clear implementation evidence in kernel image)
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: YES (found md_clear implementation evidence in kernel image)
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: YES (found md_clear implementation evidence in kernel image)
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: YES (found md_clear implementation evidence in kernel image)
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: YES (found SRBDS implementation evidence in kernel image. Your kernel is up to date for SRBDS mitigation)
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:KO CVE-2018-3639:KO CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Altra¶
Following sections include sample calibration data measured on s62-t34-sut1 server running in one of the Altra testbeds.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.4.0-65-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 iommu.passthrough=1 isolcpus=1-40,81-120 nmi_watchdog=0 nohz_full=1-40,81-120 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-40,81-120
Linux uname¶
$ uname -a
Linux s62-t34-sut1 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:27:25 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several “speculative execution” CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.45
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:27:25 UTC 2021 aarch64
CPU is ARM v8 model 0xd0c
Hardware check
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): NO
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): NO
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): NO
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
> STATUS: UNKNOWN (/sys vulnerability interface use forced, but it's not available!)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not affected)
> SUMMARY: CVE-2017-5753:?? CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
TaiShan¶
Following sections include sample calibration data measured on s17-t33-sut1 server running in one of the Cortex-A72 testbeds.
Calibration data obtained from all other servers in TaiShan testbeds shows the same or similar values.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=7d1d0e77-4df0-43df-9619-a99db29ffb83 ro audit=0 intel_iommu=on isolcpus=1-27,29-55 nmi_watchdog=0 nohz_full=1-27,29-55 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-27,29-55 console=ttyAMA0,115200n8 quiet
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
System-level Core Jitter¶
$ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 20
Linux Jitter testing program version 1.9
Iterations=30
The pragram will execute a dummy function 80000 times
Display is updated every 20000 displayUpdate intervals
Thread affinity will be set to core_id:7
Timings are in CPU Core cycles
Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
last_Exec: The Excution time of last iteration just before the display update
Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
tmp: Cumulative value calcualted by the dummy function
Interval: Time interval between the display updates in Core Cycles
Sample No: Sample number
Inst_Min Inst_Max Inst_jitter last_Exec Abs_min Abs_max tmp Interval Sample No
160022 172254 12232 160042 160022 172254 1903230976 3204401362 1
160022 173148 13126 160044 160022 173148 814809088 3204619316 2
160022 169460 9438 160044 160022 173148 4021354496 3204391306 3
160024 170270 10246 160044 160022 173148 2932932608 3204385830 4
160022 169660 9638 160044 160022 173148 1844510720 3204387290 5
160022 169410 9388 160040 160022 173148 756088832 3204375832 6
160022 169012 8990 160042 160022 173148 3962634240 3204378924 7
160022 169556 9534 160044 160022 173148 2874212352 3204374882 8
160022 171684 11662 160042 160022 173148 1785790464 3204394596 9
160022 171546 11524 160024 160022 173148 697368576 3204602774 10
160022 169248 9226 160042 160022 173148 3903913984 3204401676 11
160022 168458 8436 160042 160022 173148 2815492096 3204256350 12
160022 169574 9552 160044 160022 173148 1727070208 3204278116 13
160022 169352 9330 160044 160022 173148 638648320 3204327234 14
160022 169100 9078 160044 160022 173148 3845193728 3204388132 15
160022 169338 9316 160042 160022 173148 2756771840 3204380724 16
160022 170828 10806 160046 160022 173148 1668349952 3204430452 17
160022 173162 13140 160026 160022 173162 579928064 3204611318 18
160022 170482 10460 160042 160022 173162 3786473472 3204389896 19
160024 170704 10680 160044 160022 173162 2698051584 3204422126 20
160024 169302 9278 160044 160022 173162 1609629696 3204397334 21
160022 171848 11826 160044 160022 173162 521207808 3204389818 22
160022 169438 9416 160042 160022 173162 3727753216 3204395382 23
160022 169312 9290 160042 160022 173162 2639331328 3204371202 24
160022 171368 11346 160044 160022 173162 1550909440 3204440464 25
160022 171998 11976 160042 160022 173162 462487552 3204609440 26
160022 169740 9718 160046 160022 173162 3669032960 3204405826 27
160022 169610 9588 160044 160022 173162 2580611072 3204390608 28
160022 169254 9232 160044 160022 173162 1492189184 3204399760 29
160022 169386 9364 160046 160022 173162 403767296 3204417762 30
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several “speculative execution” CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64
CPU is Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
Hardware check
* Hardware support (CPU microcode) for mitigation techniques
* Indirect Branch Restricted Speculation (IBRS)
* SPEC_CTRL MSR is available: YES
* CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
* Indirect Branch Prediction Barrier (IBPB)
* PRED_CMD MSR is available: YES
* CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
* Single Thread Indirect Branch Predictors (STIBP)
* SPEC_CTRL MSR is available: YES
* CPU indicates STIBP capability: YES (Intel STIBP feature bit)
* Speculative Store Bypass Disable (SSBD)
* CPU indicates SSBD capability: YES (Intel SSBD)
* L1 data cache invalidation
* FLUSH_CMD MSR is available: YES
* CPU indicates L1D flush capability: YES (L1D flush feature bit)
* Microarchitectural Data Sampling
* VERW instruction is available: YES (MD_CLEAR feature bit)
* Enhanced IBRS (IBRS_ALL)
* CPU indicates ARCH_CAPABILITIES MSR availability: NO
* ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO
* CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): NO
* CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
* CPU/Hypervisor indicates L1D flushing is not necessary on this system: NO
* Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
* CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): NO
* CPU explicitly indicates not being vulnerable to TSX Asynchronous Abort (TAA_NO): NO
* CPU explicitly indicates not being vulnerable to iTLB Multihit (PSCHANGE_MSC_NO): NO
* CPU explicitly indicates having MSR for TSX control (TSX_CTRL_MSR): NO
* CPU supports Transactional Synchronization Extensions (TSX): YES (RTM feature bit)
* CPU supports Software Guard Extensions (SGX): NO
* CPU supports Special Register Buffer Data Sampling (SRBDS): NO
* CPU microcode is known to cause stability problems: NO (family 0x6 model 0x55 stepping 0x4 ucode 0x2000065 cpuid 0x50654)
* CPU microcode is the latest known available version: NO (latest version is 0x2006b06 dated 2021/03/08 according to builtin firmwares DB v191+i20210217)
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): YES
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): YES
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): YES
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): YES
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): YES
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): YES
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): YES
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): YES
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): YES
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
> STATUS: NOT VULNERABLE (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for firmware code only)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
* Kernel supports RSB filling: YES
> STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports INVPCID, performance impact of PTI will be greatly reduced)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: YES
> STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: YES (per-thread through prctl)
* SSB mitigation currently active for selected processes: YES (boltd fwupd irqbalance systemd-journald systemd-logind systemd-networkd systemd-resolved systemd-timesyncd systemd-udevd)
> STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
* Kernel supports PTE inversion: YES (found in kernel image)
* PTE inversion enabled and active: YES
> STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: NO
* Mitigation 2
* L1D flush is supported by kernel: YES (found flush_l1d in /proc/cpuinfo)
* L1D flush enabled: YES (conditional flushes)
* Hardware-backed L1D flush supported: YES (performance impact of the mitigation will be greatly reduced)
* Hyper-Threading (SMT) is enabled: YES
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
* Kernel mitigation is enabled and active: YES
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (Your microcode and kernel are both up to date for this mitigation, and mitigation is enabled)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: YES (Mitigation: Clear CPU buffers; SMT vulnerable)
> STATUS: NOT VULNERABLE (Mitigation: Clear CPU buffers; SMT vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (KVM: Mitigation: Split huge pages)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: YES (KVM: Mitigation: Split huge pages)
> STATUS: NOT VULNERABLE (this system is not running a hypervisor)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: YES (found SRBDS implementation evidence in kernel image. Your kernel is up to date for SRBDS mitigation)
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:27:25 UTC 2021 aarch64
CPU is ARM v8 model 0xd08
Hardware check
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): NO
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: NO
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
* Checking count of LFENCE instructions following a jump in kernel... NO (only 0 jump-then-lfence instructions found, should be >= 30 (heuristic))
> STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: NO (Vulnerable)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: NO
* Kernel is compiled with IBPB support: NO
* IBPB enabled and active: NO
* Mitigation 2
* Kernel has branch predictor hardening (arm): YES
* Kernel compiled with retpoline option: NO
> STATUS: NOT VULNERABLE (Branch predictor hardening mitigates the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: UNKNOWN (dmesg truncated, please reboot and relaunch this script)
* Reduced performance impact of PTI: NO (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: NO
> STATUS: VULNERABLE (an up-to-date CPU microcode is needed to mitigate this vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: NO
> STATUS: VULNERABLE (Your CPU doesnt support SSBD)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports PTE inversion: NO
* PTE inversion enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: N/A (the kvm_intel module is not loaded)
* Mitigation 2
* L1D flush is supported by kernel: NO
* L1D flush enabled: NO
* Hardware-backed L1D flush supported: NO (flush will be done in software, this is slower)
* Hyper-Threading (SMT) is enabled: UNKNOWN
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: NO
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:KO CVE-2018-3639:KO CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
Need more detailed information about mitigation options? Use --explain
A false sense of security is worse than no security at all, see --disclaimer
ok: [10.30.51.37] =>
spectre_meltdown_poll_results.stdout_lines:
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:27:25 UTC 2021 aarch64
CPU is ARM v8 model 0xd08
Hardware check
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): YES
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): NO
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: NO
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
* Checking count of LFENCE instructions following a jump in kernel... NO (only 0 jump-then-lfence instructions found, should be >= 30 (heuristic))
> STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: NO (Vulnerable)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: NO
* Kernel is compiled with IBPB support: NO
* IBPB enabled and active: NO
* Mitigation 2
* Kernel has branch predictor hardening (arm): YES
* Kernel compiled with retpoline option: NO
> STATUS: NOT VULNERABLE (Branch predictor hardening mitigates the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: UNKNOWN (dmesg truncated, please reboot and relaunch this script)
* Reduced performance impact of PTI: NO (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: NO
> STATUS: VULNERABLE (an up-to-date CPU microcode is needed to mitigate this vulnerability)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: NO
> STATUS: VULNERABLE (Your CPU doesnt support SSBD)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports PTE inversion: NO
* PTE inversion enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: N/A (the kvm_intel module is not loaded)
* Mitigation 2
* L1D flush is supported by kernel: NO
* L1D flush enabled: NO
* Hardware-backed L1D flush supported: NO (flush will be done in software, this is slower)
* Hyper-Threading (SMT) is enabled: UNKNOWN
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: NO
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:KO CVE-2018-3639:KO CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK
ThunderX2¶
Following sections include sample calibration data measured on s27-t211-sut1 server running in one of the ThunderX2 testbeds.
Linux cmdline¶
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=7d1d0e77-4df0-43df-9619-a99db29ffb83 ro audit=0 intel_iommu=on isolcpus=1-27,29-55 nmi_watchdog=0 nohz_full=1-27,29-55 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-27,29-55 console=ttyAMA0,115200n8 quiet
Linux uname¶
$ uname -a
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Spectre and Meltdown Checks¶
Following section displays the output of a running shell script to tell if system is vulnerable against the several “speculative execution” CVEs that were made public in 2018. Script is available on Spectre & Meltdown Checker Github.
Spectre and Meltdown mitigation detection tool v0.44+
Checking for vulnerabilities on current system
Kernel is Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:27:25 UTC 2021 aarch64
CPU is
Hardware check
* CPU vulnerability to the speculative execution attack variants
* Affected by CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
* Affected by CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
* Affected by CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
* Affected by CVE-2018-3640 (Variant 3a, rogue system register read): NO
* Affected by CVE-2018-3639 (Variant 4, speculative store bypass): YES
* Affected by CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
* Affected by CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
* Affected by CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
* Affected by CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
* Affected by CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
* Affected by CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
* Affected by CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
* Affected by CVE-2019-11135 (ZombieLoad V2, TSX Asynchronous Abort (TAA)): NO
* Affected by CVE-2018-12207 (No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)): NO
* Affected by CVE-2020-0543 (Special Register Buffer Data Sampling (SRBDS)): NO
CVE-2017-5753 aka Spectre Variant 1, bounds check bypass
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: NO
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
* Kernel has array_index_nospec (arm64): NO
* Checking count of LFENCE instructions following a jump in kernel... NO (only 0 jump-then-lfence instructions found, should be >= 30 (heuristic))
> STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 aka Spectre Variant 2, branch target injection
* Mitigated according to the /sys interface: NO (Vulnerable)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: NO
* Kernel is compiled with IBPB support: NO
* IBPB enabled and active: NO
* Mitigation 2
* Kernel has branch predictor hardening (arm): YES
* Kernel compiled with retpoline option: NO
> STATUS: NOT VULNERABLE (Branch predictor hardening mitigates the vulnerability)
CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: UNKNOWN (dmesg truncated, please reboot and relaunch this script)
* Reduced performance impact of PTI: NO (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3640 aka Variant 3a, rogue system register read
* CPU microcode mitigates the vulnerability: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3639 aka Variant 4, speculative store bypass
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
* SSB mitigation is enabled and active: NO
> STATUS: VULNERABLE (Your CPU doesnt support SSBD)
CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault
* CPU microcode mitigates the vulnerability: N/A
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports PTE inversion: NO
* PTE inversion enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault
* Information from the /sys interface: Not affected
* This system is a host running a hypervisor: NO
* Mitigation 1 (KVM)
* EPT is disabled: N/A (the kvm_intel module is not loaded)
* Mitigation 2
* L1D flush is supported by kernel: NO
* L1D flush enabled: NO
* Hardware-backed L1D flush supported: NO (flush will be done in software, this is slower)
* Hyper-Threading (SMT) is enabled: UNKNOWN
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)
* Mitigated according to the /sys interface: YES (Not affected)
* Kernel supports using MD_CLEAR mitigation: NO
* Kernel mitigation is enabled and active: NO
* SMT is either mitigated or disabled: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2019-11135 aka ZombieLoad V2, TSX Asynchronous Abort (TAA)
* Mitigated according to the /sys interface: YES (Not affected)
* TAA mitigation is supported by kernel: YES (found tsx_async_abort in kernel image)
* TAA mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2018-12207 aka No eXcuses, iTLB Multihit, machine check exception on page size changes (MCEPSC)
* Mitigated according to the /sys interface: YES (Not affected)
* This system is a host running a hypervisor: NO
* iTLB Multihit mitigation is supported by kernel: YES (found itlb_multihit in kernel image)
* iTLB Multihit mitigation enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
CVE-2020-0543 aka Special Register Buffer Data Sampling (SRBDS)
* Mitigated according to the /sys interface: YES (Not affected)
* SRBDS mitigation control is supported by the kernel: NO
* SRBDS mitigation control is enabled and active: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:KO CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK CVE-2019-11135:OK CVE-2018-12207:OK CVE-2020-0543:OK