.. _traces: .. toctree:: Using the trace command ======================== Skills to be Learned --------------------- #. Setup a 'trace' #. View a 'trace' #. Clear a 'trace' #. Verify using ping from host #. Ping from vpp #. Examine ARP table (neighbors) #. Examine ip fib Basic Trace Commands --------------------- Show trace buffer [max COUNT]. .. code-block:: console vpp# show trace Clear trace buffer and free memory. .. code-block:: console vpp# clear trace filter trace output - include NODE COUNT | exclude NODE COUNT | none. .. code-block:: console vpp# trace filter Add Trace ---------- .. code-block:: console vpp# trace add af-packet-input 10 The following list of nodes are supported under trace add: * af-packet-input * avf-input * bond-process * dpdk-crypto-input * dpdk-input * handoff-trace * ixge-input * memif-input * mrvl-pp2-input * netmap-input * p2p-ethernet-input * pg-input * punt-socket-rx * rdma-input * session-queue * tuntap-rx * vhost-user-input * virtio-input * vmxnet3-input Ping from Host to VPP ---------------------- .. code-block:: console vpp# q $ ping -c 1 10.10.1.2 PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data. 64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=0.283 ms --- 10.10.1.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.283/0.283/0.283/0.000 ms Examine Trace of ping from host to VPP ---------------------------------------- .. code-block:: console $ sudo vppctl -s /run/vpp/cli-vpp1.sock vpp# show trace ------------------- Start of thread 0 vpp_main ------------------- Packet 1 00:17:04:099260: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e370 nsec 0x3af2736f vlan 0 vlan_tpid 0 00:17:04:099269: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:17:04:099285: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099290: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099296: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099300: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099301: ip4-icmp-echo-request ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099303: ip4-load-balance fib 0 dpo-idx 13 flow hash: 0x00000000 ICMP: 10.10.1.2 -> 10.10.1.1 tos 0x00, ttl 64, length 84, checksum 0x4437 fragment id 0xe05b, flags DONT_FRAGMENT ICMP echo_reply checksum 0xc843 00:17:04:099305: ip4-rewrite tx_sw_if_index 1 dpo-idx 1 : ipv4 via 10.10.1.1 host-vpp1out: mtu:9000 e20f1e59ecf702fed975d5b40800 flow hash: 0x00000000 00000000: e20f1e59ecf702fed975d5b4080045000054e05b4000400144370a0a01020a0a 00000020: 01010000c8437c92000170e3605b000000001c170f00000000001011 00:17:04:099307: host-vpp1out-output host-vpp1out IP4: 02:fe:d9:75:d5:b4 -> e2:0f:1e:59:ec:f7 ICMP: 10.10.1.2 -> 10.10.1.1 tos 0x00, ttl 64, length 84, checksum 0x4437 fragment id 0xe05b, flags DONT_FRAGMENT ICMP echo_reply checksum 0xc843 Clear trace buffer ------------------- .. code-block:: console vpp# clear trace Ping from VPP to Host ----------------------- .. code-block:: console vpp# ping 10.10.1.1 64 bytes from 10.10.1.1: icmp_seq=1 ttl=64 time=.0789 ms 64 bytes from 10.10.1.1: icmp_seq=2 ttl=64 time=.0619 ms 64 bytes from 10.10.1.1: icmp_seq=3 ttl=64 time=.0519 ms 64 bytes from 10.10.1.1: icmp_seq=4 ttl=64 time=.0514 ms 64 bytes from 10.10.1.1: icmp_seq=5 ttl=64 time=.0526 ms Statistics: 5 sent, 5 received, 0% packet loss Examine Trace of ping from VPP to host --------------------------------------- The output will demonstrate FD.io VPP's trace of ping for all packets. .. code-block:: console vpp# show trace ------------------- Start of thread 0 vpp_main ------------------- Packet 1 00:17:04:099260: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e370 nsec 0x3af2736f vlan 0 vlan_tpid 0 00:17:04:099269: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:17:04:099285: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099290: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099296: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099300: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099301: ip4-icmp-echo-request ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f7c fragment id 0xe516, flags DONT_FRAGMENT ICMP echo_request checksum 0xc043 00:17:04:099303: ip4-load-balance fib 0 dpo-idx 13 flow hash: 0x00000000 ICMP: 10.10.1.2 -> 10.10.1.1 tos 0x00, ttl 64, length 84, checksum 0x4437 fragment id 0xe05b, flags DONT_FRAGMENT ICMP echo_reply checksum 0xc843 00:17:04:099305: ip4-rewrite tx_sw_if_index 1 dpo-idx 1 : ipv4 via 10.10.1.1 host-vpp1out: mtu:9000 e20f1e59ecf702fed975d5b40800 flow hash: 0x00000000 00000000: e20f1e59ecf702fed975d5b4080045000054e05b4000400144370a0a01020a0a 00000020: 01010000c8437c92000170e3605b000000001c170f00000000001011 00:17:04:099307: host-vpp1out-output host-vpp1out IP4: 02:fe:d9:75:d5:b4 -> e2:0f:1e:59:ec:f7 ICMP: 10.10.1.2 -> 10.10.1.1 tos 0x00, ttl 64, length 84, checksum 0x4437 fragment id 0xe05b, flags DONT_FRAGMENT ICMP echo_reply checksum 0xc843 Packet 2 00:17:09:113964: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 42 snaplen 42 mac 66 net 80 sec 0x5b60e375 nsec 0x3b3bd57d vlan 0 vlan_tpid 0 00:17:09:113974: ethernet-input ARP: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:17:09:113986: arp-input request, type ethernet/IP4, address size 6/4 e2:0f:1e:59:ec:f7/10.10.1.1 -> 00:00:00:00:00:00/10.10.1.2 00:17:09:114003: host-vpp1out-output host-vpp1out ARP: 02:fe:d9:75:d5:b4 -> e2:0f:1e:59:ec:f7 reply, type ethernet/IP4, address size 6/4 02:fe:d9:75:d5:b4/10.10.1.2 -> e2:0f:1e:59:ec:f7/10.10.1.1 Packet 3 00:18:16:407079: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e3b9 nsec 0x90b7566 vlan 0 vlan_tpid 0 00:18:16:407085: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:16:407090: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3fe8 fragment id 0x24ab ICMP echo_reply checksum 0x37eb 00:18:16:407094: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3fe8 fragment id 0x24ab ICMP echo_reply checksum 0x37eb 00:18:16:407097: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3fe8 fragment id 0x24ab ICMP echo_reply checksum 0x37eb 00:18:16:407101: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3fe8 fragment id 0x24ab ICMP echo_reply checksum 0x37eb 00:18:16:407104: ip4-icmp-echo-reply ICMP echo id 7531 seq 1 00:18:16:407108: ip4-drop ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3fe8 fragment id 0x24ab ICMP echo_reply checksum 0x37eb 00:18:16:407111: error-drop ip4-icmp-input: unknown type Packet 4 00:18:17:409084: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e3ba nsec 0x90b539f vlan 0 vlan_tpid 0 00:18:17:409088: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:17:409092: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f40 fragment id 0x2553 ICMP echo_reply checksum 0xcc6d 00:18:17:409095: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f40 fragment id 0x2553 ICMP echo_reply checksum 0xcc6d 00:18:17:409097: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f40 fragment id 0x2553 ICMP echo_reply checksum 0xcc6d 00:18:17:409099: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f40 fragment id 0x2553 ICMP echo_reply checksum 0xcc6d 00:18:17:409101: ip4-icmp-echo-reply ICMP echo id 7531 seq 2 00:18:17:409104: ip4-drop ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3f40 fragment id 0x2553 ICMP echo_reply checksum 0xcc6d 00:18:17:409104: error-drop ip4-icmp-input: unknown type Packet 5 00:18:18:409082: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e3bb nsec 0x8ecad24 vlan 0 vlan_tpid 0 00:18:18:409087: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:18:409091: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e66 fragment id 0x262d ICMP echo_reply checksum 0x8e59 00:18:18:409093: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e66 fragment id 0x262d ICMP echo_reply checksum 0x8e59 00:18:18:409096: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e66 fragment id 0x262d ICMP echo_reply checksum 0x8e59 00:18:18:409098: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e66 fragment id 0x262d ICMP echo_reply checksum 0x8e59 00:18:18:409099: ip4-icmp-echo-reply ICMP echo id 7531 seq 3 00:18:18:409102: ip4-drop ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e66 fragment id 0x262d ICMP echo_reply checksum 0x8e59 00:18:18:409102: error-drop ip4-icmp-input: unknown type Packet 6 00:18:19:414750: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e3bc nsec 0x92450f2 vlan 0 vlan_tpid 0 00:18:19:414754: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:19:414757: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e52 fragment id 0x2641 ICMP echo_reply checksum 0x9888 00:18:19:414760: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e52 fragment id 0x2641 ICMP echo_reply checksum 0x9888 00:18:19:414762: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e52 fragment id 0x2641 ICMP echo_reply checksum 0x9888 00:18:19:414764: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e52 fragment id 0x2641 ICMP echo_reply checksum 0x9888 00:18:19:414765: ip4-icmp-echo-reply ICMP echo id 7531 seq 4 00:18:19:414768: ip4-drop ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e52 fragment id 0x2641 ICMP echo_reply checksum 0x9888 00:18:19:414769: error-drop ip4-icmp-input: unknown type Packet 7 00:18:20:418038: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 98 snaplen 98 mac 66 net 80 sec 0x5b60e3bd nsec 0x937bcc2 vlan 0 vlan_tpid 0 00:18:20:418042: ethernet-input IP4: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:20:418045: ip4-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e47 fragment id 0x264c ICMP echo_reply checksum 0xc0e8 00:18:20:418048: ip4-lookup fib 0 dpo-idx 5 flow hash: 0x00000000 ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e47 fragment id 0x264c ICMP echo_reply checksum 0xc0e8 00:18:20:418049: ip4-local ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e47 fragment id 0x264c ICMP echo_reply checksum 0xc0e8 00:18:20:418054: ip4-icmp-input ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e47 fragment id 0x264c ICMP echo_reply checksum 0xc0e8 00:18:20:418054: ip4-icmp-echo-reply ICMP echo id 7531 seq 5 00:18:20:418057: ip4-drop ICMP: 10.10.1.1 -> 10.10.1.2 tos 0x00, ttl 64, length 84, checksum 0x3e47 fragment id 0x264c ICMP echo_reply checksum 0xc0e8 00:18:20:418058: error-drop ip4-icmp-input: unknown type Packet 8 00:18:21:419208: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x20000001 len 42 snaplen 42 mac 66 net 80 sec 0x5b60e3be nsec 0x92a9429 vlan 0 vlan_tpid 0 00:18:21:419876: ethernet-input ARP: e2:0f:1e:59:ec:f7 -> 02:fe:d9:75:d5:b4 00:18:21:419881: arp-input request, type ethernet/IP4, address size 6/4 e2:0f:1e:59:ec:f7/10.10.1.1 -> 00:00:00:00:00:00/10.10.1.2 00:18:21:419896: host-vpp1out-output host-vpp1out ARP: 02:fe:d9:75:d5:b4 -> e2:0f:1e:59:ec:f7 reply, type ethernet/IP4, address size 6/4 02:fe:d9:75:d5:b4/10.10.1.2 -> e2:0f:1e:59:ec:f7/10.10.1.1 After examining the trace, clear it again using vpp# clear trace. Examine ARP table (neighbors) ----------------------------- .. code-block:: console vpp# show ip neighbors Time IP4 Flags Ethernet Interface 1101.5636 10.10.1.1 D e2:0f:1e:59:ec:f7 host-vpp1out Examine routing tables ----------------------- .. code-block:: console vpp# show ip fib ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto ] locks:[src:plugin-hi:2, src:adjacency:1, src:default-route:1, ] 0.0.0.0/0 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:1 buckets:1 uRPF:0 to:[0:0]] [0] [@0]: dpo-drop ip4 0.0.0.0/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:2 buckets:1 uRPF:1 to:[0:0]] [0] [@0]: dpo-drop ip4 10.10.1.0/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:9 to:[0:0]] [0] [@0]: dpo-drop ip4 10.10.1.1/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:13 buckets:1 uRPF:12 to:[5:420] via:[2:168]] [0] [@5]: ipv4 via 10.10.1.1 host-vpp1out: mtu:9000 e20f1e59ecf702fed975d5b40800 10.10.1.0/24 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:8 to:[0:0]] [0] [@4]: ipv4-glean: host-vpp1out: mtu:9000 ffffffffffff02fed975d5b40806 10.10.1.2/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:12 buckets:1 uRPF:13 to:[7:588]] [0] [@2]: dpo-receive: 10.10.1.2 on host-vpp1out 10.10.1.255/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:11 buckets:1 uRPF:11 to:[0:0]] [0] [@0]: dpo-drop ip4 224.0.0.0/4 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:4 buckets:1 uRPF:3 to:[0:0]] [0] [@0]: dpo-drop ip4 240.0.0.0/4 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:3 buckets:1 uRPF:2 to:[0:0]] [0] [@0]: dpo-drop ip4 255.255.255.255/32 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:5 buckets:1 uRPF:4 to:[0:0]] [0] [@0]: dpo-drop ip4