This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/66AK2E05: The problem of the transmission rate through 10 GbE

Part Number: 66AK2E05
Other Parts Discussed in Thread: DS100DF410,

Tool/software: Linux

Hi!

We use custom board with Keystone 2 66AK2E05 <--> DS100DF410 <--> SFP_Module_10Gb. 

We are testing a 10GbE link on line 0 between two our boards. We use utilities iperf3 and netperf under Linux (last SDK). We can not get a speed higher than 1.7 Gbit / s. Processor utilization does not exceed 25%.

SFP modules and cable have been tested on another system. They are capable of running at 10 Gbit / s.

Here is a discussion about the settings of the retimer.  

What is our problem?

Part of DST file:

xgbe_subsys: subsys@2f00000 {
//status = "disabled";
compatible = "syscon";
reg = <0x02f00000 0x100>;
};

xgbe_pcsr: pcsr@2f00600 {
//status = "disabled";
compatible = "syscon";
reg = <0x02f00600 0x100>;
};

xgbe_serdes: phy@231e000 {
#phy-cells = <0>;
compatible = "ti,keystone-serdes-xgbe";
reg = <0x0231e000 0x2000>;
//status = "disabled";
link-rate-kbps = <10312500>;
num-lanes = <2>;
syscon-peripheral = <&xgbe_subsys>;
syscon-link = <&xgbe_pcsr>;
#address-cells = <1>;
#size-cells = <0>;

xserdes_lane0: lane@0 {
#phy-cells = <0>;
reg = <0>;
status = "ok";
control-rate = <0>;
rx-start = <7 5>;
rx-force = <1 1>;
tx-coeff = <2 0 0 12 4>;
};
xserdes_lane1: lane@1 {
#phy-cells = <0>;
reg = <1>;
status = "disabled";
control-rate = <0>;
rx-start = <7 5>;
rx-force = <1 1>;
tx-coeff = <2 0 0 12 4>;
};
};

netcpx: netcp@2f00000 {
//status = "disabled";
compatible = "ti,netcp-1.0";
#address-cells = <1>;
#size-cells = <1>;
ranges;

clocks = <&clkxge>, <&chipclk12>;
clock-names = "xge_clk", "cpts";
dma-coherent;

ti,navigator-dmas = <&dma_xgbe 0>,
<&dma_xgbe 8>,
<&dma_xgbe 0>;
ti,navigator-dma-names = "xnetrx0", "xnetrx1", "xnettx";

netcp-devices {
#address-cells = <1>;
#size-cells = <1>;
ranges;
xgbe@2f00000 {
label = "netcp-xgbe";
compatible = "ti,netcp-xgbe";
syscon-subsys = <&xgbe_subsys>;
syscon-pcsr = <&xgbe_pcsr>;
reg = <0x02f00100 0x200>, <0x02f01000 0xb00>;
tx-queue = <692>;
tx-channel = "xnettx";

interfaces {
xgbe0: interface-0 {
phys = <&xserdes_lane0>;
slave-port = <0>;
link-interface = <11>;
};
/* xgbe1: interface-1 {
phys = <&xserdes_lane1>;
slave-port = <1>;
link-interface = <11>;
};*/
};
};
};

netcp-interfaces {
interface-0 {
rx-channel = "xnetrx0";
rx-pool = <2048 12>;
tx-pool = <1024 12>;
rx-queue-depth = <1024 1024 0 0>;
rx-buffer-size = <1536 4096 0 0>;
rx-queue = <532>;
tx-completion-queue = <534>;
efuse-mac = <0>;
// local-mac-address = [02 18 31 7e 3e 00];
netcp-xgbe = <&xgbe0>;

};
/* interface-1 {
rx-channel = "xnetrx1";
rx-pool = <2048 12>;
tx-pool = <1024 12>;
rx-queue-depth = <1024 1024 0 0>;
rx-buffer-size = <1536 4096 0 0>;
rx-queue = <533>;
tx-completion-queue = <535>;
efuse-mac = <0>;
// local-mac-address = [02 18 31 7e 3e 00];
netcp-xgbe = <&xgbe1>;
};*/
};
};

  • Hi,

    I've notified the team. Their feedback will be posted here.

    Best Regards,
    Yordan
  • Hi, Eduard,

    Could you retry your benchmarking with jumbo packet enabled? we were able to get about 6Gbps in one of the earlier threads, e2e.ti.com/.../1534785

    Rex
  • Rex,

    with MTU = 9000 we have a kernel panic on server iperf3.

    Log attached.

    3527.7268.log.txt
    [  5] local 10.0.0.2 port 5201 connected to 10.0.0.1 port 46232
    [  634.458008] Internal error: Oops: 206 [#1] PREEMPT SMP ARM
    [  634.463487] Modules linked in: keystone_remoteproc uio sch_fq_codel rpmsg_proto virtio_rpmsg_bus remoteproc rpmsg_core davinci_wdt keystone_dsp_mem
    [  634.476744] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.9.28-geed43d1050 #25
    [  634.483783] Hardware name: Keystone
    [  634.487263] task: c1006500 task.stack: c1000000
    [  634.491789] PC is at csum_partial+0x2c/0x130
    [  634.496055] LR is at csum_partial_ext+0x10/0x14
    [  634.500578] pc : [<c04b7788>]    lr : [<c076cc00>]    psr: 00000113
    [  634.500578] sp : c1001a20  ip : 00001000  fp : c1001a3c
    [  634.512043] r10: 00000000  r9 : c1001a98  r8 : 00001000
    [  634.517259] r7 : 000005cc  r6 : 000015cc  r5 : 00001994  r4 : 91d40000
    [  634.523777] r3 : c076cbf0  r2 : 00000000  r1 : 00001000  r0 : 91d40000
    [  634.530296] Flags: nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
    [  634.537421] Control: 30c5387d  Table: 2e9562c0  DAC: fffffffd
    [  634.543158] Process swapper/0 (pid: 0, stack limit = 0xc1000210)
    [  634.549155] Stack: (0xc1001a20 to 0xc1002000)
    [  634.553505] 1a20: 91d40000 00001994 91d40000 c076cc00 c1001a8c c1001a40 c076fc14 c076cbfc
    [  634.561674] 1a40: 00000000 00000000 25d10ee7 eead8900 ee876340 000005cc 9fc60b9b 000005cc
    [  634.569842] 1a60: 00010152 00000000 eead8900 ee875d64 00000020 cfe58640 cfe58640 00000001
    [  634.578011] 1a80: c1001ab4 c1001a90 c076fd94 c076fb30 c1001a98 cfe58000 c076cbf0 c076cbc8
    [  634.586177] 1aa0: c1001abc cfe58640 c1001ad4 c1001ab8 c0774f84 c076fd64 cfe58640 eead8900
    [  634.594346] 1ac0: ee875d64 00000020 c1001b0c c1001ad8 c07d6bb8 c0774f70 c0799ff4 c0277910
    [  634.602514] 1ae0: c076fd94 0076fb30 c1001af0 eead8900 cfe58640 ee421200 ee875d50 cfe58640
    [  634.610681] 1b00: c1001b2c c1001b10 c07e0058 c07d689c eead8900 c1036780 cfe586b0 ee875d50
    [  634.618849] 1b20: c1001b8c c1001b30 c07e24c8 c07dff18 ef2ebe00 00000001 0200000a 0100000a
    [  634.627017] 1b40: 00000000 eead8cc0 0200000a ef3aa800 ee986880 00000000 c1001bfc c1001b68
    [  634.635185] 1b60: c07b8438 eead8900 c0a75bb0 00000000 c1004e7c c1036780 c1036780 ef3aa800
    [  634.643353] 1b80: c1001bb4 c1001b90 c07bb370 c07e18e0 eead8900 ef3aa800 c1036780 ef3aa800
    [  634.651521] 1ba0: eead8900 c1036780 c1001bfc c1001bb8 c07bbbe0 c07bb298 c1036780 ef3aa800
    [  634.659689] 1bc0: c1001bfc c1001bd0 c07e18c8 c07b665c 0200000a 00001451 00000004 c1036780
    [  634.667858] 1be0: ef3aa800 eead8900 ee875d50 c1036780 c1001c2c c1001c00 c07bb870 c07bbba0
    [  634.676026] 1c00: ef3aa800 00000000 00000400 00000050 ee875d00 0029786e 00000000 eead8900
    [  634.684194] 1c20: c1001c84 c1001c30 c07bc068 c07bb5dc 00001000 00000000 00000000 00000000
    [  634.692362] 1c40: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 eead8900
    [  634.700530] 1c60: c1005404 c07bbc88 00000008 ef3aa800 eead89b4 ef3aa800 c1001cfc c1001c88
    [  634.708697] 1c80: c077f748 c07bbc94 00000000 c1003eec 00000000 ef3aa800 ef02ebd0 c100315c
    [  634.716865] 1ca0: ef0f6540 eead8900 80150013 ee813590 00000001 c1001cfc c1001cdc c1001cc8
    [  634.725032] 1cc0: c027306c c04e7a68 ef222f00 00000099 c1001cf4 eead8900 eead8900 ef3aad34
    [  634.733199] 1ce0: 00000998 ee875d00 cf850b00 ef3aad50 c1001d14 c1001d00 c0781e00 c077f09c
    [  634.741367] 1d00: eead8900 eead8900 c1001d44 c1001d18 c0781e84 c0781df4 ee875d00 00000700
    [  634.749535] 1d20: c1001d80 eead8900 ef3aad34 00000998 c1001d54 ef3aad34 c1001d54 c1001d48
    [  634.757703] 1d40: c0781f24 c0781e68 c1001dd4 c1001d58 c06c4874 c0781f20 00000998 00001000
    [  634.765871] 1d60: ef3aad00 00000040 00000000 00000000 c100802c cf850b00 00000030 aebb9000
    [  634.774040] 1d80: eead8900 cf850b20 cf850b30 00000000 c026ecdc c0222b90 c1003438 f0802000
    [  634.782206] 1da0: c1001de8 f0802000 f0803000 ef3aad50 c06c442c 00000040 0000012c 0000829e
    [  634.790374] 1dc0: c1002d00 c1001df8 c1001e34 c1001dd8 c07825e4 c06c4438 c1001e94 c0bf3f30
    [  634.798541] 1de0: c1003eec c1003eec c103d53e 2e858000 c0e504c0 ef6a84c0 c1001df8 c1001df8
    [  634.806710] 1e00: c1001e00 c1001e00 ef008000 00000000 c100208c c1000000 00000101 00000003
    [  634.814878] 1e20: c1002080 40000003 c1001e94 c1001e38 c02227fc c07823e4 c0e4e7f0 00000004
    [  634.823047] 1e40: ef222f00 00200100 c1002d00 0000829d 0000000a c0a01ac0 c1042b40 c0e4b338
    [  634.831214] 1e60: c1002080 c1001e38 c1001e94 c0e4ed88 00000000 00000000 00000001 ef008000
    [  634.839383] 1e80: c1000000 c10030ac c1001ea4 c1001e98 c0222c64 c02226e4 c1001ecc c1001ea8
    [  634.847550] 1ea0: c026ecdc c0222b90 c1003438 f080200c c1001ef8 f0802000 f0803000 c1000000
    [  634.855718] 1ec0: c1001ef4 c1001ed0 c02013e4 c026ec80 c020859c 60000013 ffffffff c1001f2c
    [  634.863886] 1ee0: 00000000 c1000000 c1001f54 c1001ef8 c020bf78 c02013b0 00000001 00000000
    [  634.872054] 1f00: 00000000 c02194c0 c1000000 c100303c c10030a4 00000001 00000000 00000000
    [  634.880222] 1f20: c10030ac c1001f54 c1001f58 c1001f48 c0208598 c020859c 60000013 ffffffff
    [  634.888390] 1f40: c027ba44 00000000 c1001f64 c1001f58 c089d1dc c0208568 c1001f8c c1001f68
    [  634.896558] 1f60: c025e388 c089d1c0 00000002 c101012a c0e4ee68 c1001f68 c0899c04 00000000
    [  634.904726] 1f80: c1001fa4 c1001f90 c0898554 c025e1f0 c104204c 00000001 c1001ff4 c1001fa8
    [  634.912894] 1fa0: c0e00d74 c08984d4 ffffffff ffffffff 00000000 c0e006f8 ffffffff 00000000
    [  634.921062] 1fc0: 00000000 c0e3da30 00000000 c10423d4 c100301c c0e3da2c c1007878 80007000
    [  634.929229] 1fe0: 412fc0f4 00000000 00000000 c1001ff8 80008090 c0e009a4 00000000 00000000
    [  634.937394] Backtrace:
    [  634.939843] [<c076cbf0>] (csum_partial_ext) from [<c076fc14>] (__skb_checksum+0xf0/0x234)
    [  634.948016] [<c076fb24>] (__skb_checksum) from [<c076fd94>] (skb_checksum+0x3c/0x44)
    [  634.955750]  r10:00000001 r9:cfe58640 r8:cfe58640 r7:00000020 r6:ee875d64 r5:eead8900
    [  634.963570]  r4:00000000
    [  634.966099] [<c076fd58>] (skb_checksum) from [<c0774f84>] (__skb_checksum_complete+0x20/0xb8)
    [  634.974612]  r4:cfe58640
    [  634.977142] [<c0774f64>] (__skb_checksum_complete) from [<c07d6bb8>] (tcp_rcv_established+0x328/0x718)
    [  634.986438]  r7:00000020 r6:ee875d64 r5:eead8900 r4:cfe58640
    [  634.992092] [<c07d6890>] (tcp_rcv_established) from [<c07e0058>] (tcp_v4_do_rcv+0x14c/0x1ac)
    [  635.000519]  r8:cfe58640 r7:ee875d50 r6:ee421200 r5:cfe58640 r4:eead8900
    [  635.007213] [<c07dff0c>] (tcp_v4_do_rcv) from [<c07e24c8>] (tcp_v4_rcv+0xbf4/0xc20)
    [  635.014860]  r7:ee875d50 r6:cfe586b0 r5:c1036780 r4:eead8900
    [  635.020512] [<c07e18d4>] (tcp_v4_rcv) from [<c07bb370>] (ip_local_deliver_finish+0xe4/0x344)
    [  635.028941]  r10:ef3aa800 r9:c1036780 r8:c1036780 r7:c1004e7c r6:00000000 r5:c0a75bb0
    [  635.036759]  r4:eead8900
    [  635.039286] [<c07bb28c>] (ip_local_deliver_finish) from [<c07bbbe0>] (ip_local_deliver+0x4c/0xf4)
    [  635.048148]  r9:c1036780 r8:eead8900 r7:ef3aa800 r6:c1036780 r5:ef3aa800 r4:eead8900
    [  635.055882] [<c07bbb94>] (ip_local_deliver) from [<c07bb870>] (ip_rcv_finish+0x2a0/0x494)
    [  635.064048]  r6:c1036780 r5:ee875d50 r4:eead8900
    [  635.068658] [<c07bb5d0>] (ip_rcv_finish) from [<c07bc068>] (ip_rcv+0x3e0/0x59c)
    [  635.075958]  r8:eead8900 r7:00000000 r6:0029786e r5:ee875d00 r4:00000050
    [  635.082653] [<c07bbc88>] (ip_rcv) from [<c077f748>] (__netif_receive_skb_core+0x6b8/0xb68)
    [  635.090908]  r10:ef3aa800 r9:eead89b4 r8:ef3aa800 r7:00000008 r6:c07bbc88 r5:c1005404
    [  635.098726]  r4:eead8900
    [  635.101255] [<c077f090>] (__netif_receive_skb_core) from [<c0781e00>] (__netif_receive_skb+0x18/0x74)
    [  635.110464]  r10:ef3aad50 r9:cf850b00 r8:ee875d00 r7:00000998 r6:ef3aad34 r5:eead8900
    [  635.118282]  r4:eead8900
    [  635.120810] [<c0781de8>] (__netif_receive_skb) from [<c0781e84>] (netif_receive_skb_internal+0x28/0xb8)
    [  635.130191]  r5:eead8900 r4:eead8900
    [  635.133759] [<c0781e5c>] (netif_receive_skb_internal) from [<c0781f24>] (netif_receive_skb+0x10/0x14)
    [  635.142966]  r4:ef3aad34
    [  635.145495] [<c0781f14>] (netif_receive_skb) from [<c06c4874>] (netcp_rx_poll+0x448/0x46c)
    [  635.153750] [<c06c442c>] (netcp_rx_poll) from [<c07825e4>] (net_rx_action+0x20c/0x300)
    [  635.161658]  r10:c1001df8 r9:c1002d00 r8:0000829e r7:0000012c r6:00000040 r5:c06c442c
    [  635.169476]  r4:ef3aad50
    [  635.172006] [<c07823d8>] (net_rx_action) from [<c02227fc>] (__do_softirq+0x124/0x23c)
    [  635.179827]  r10:40000003 r9:c1002080 r8:00000003 r7:00000101 r6:c1000000 r5:c100208c
    [  635.187645]  r4:00000000
    [  635.190173] [<c02226d8>] (__do_softirq) from [<c0222c64>] (irq_exit+0xe0/0x148)
    [  635.197473]  r10:c10030ac r9:c1000000 r8:ef008000 r7:00000001 r6:00000000 r5:00000000
    [  635.205292]  r4:c0e4ed88
    [  635.207820] [<c0222b84>] (irq_exit) from [<c026ecdc>] (__handle_domain_irq+0x68/0xbc)
    [  635.215642] [<c026ec74>] (__handle_domain_irq) from [<c02013e4>] (gic_handle_irq+0x40/0x7c)
    [  635.223983]  r9:c1000000 r8:f0803000 r7:f0802000 r6:c1001ef8 r5:f080200c r4:c1003438
    [  635.231718] [<c02013a4>] (gic_handle_irq) from [<c020bf78>] (__irq_svc+0x58/0x8c)
    [  635.239190] Exception stack(0xc1001ef8 to 0xc1001f40)
    [  635.244232] 1ee0:                                                       00000001 00000000
    [  635.252400] 1f00: 00000000 c02194c0 c1000000 c100303c c10030a4 00000001 00000000 00000000
    [  635.260568] 1f20: c10030ac c1001f54 c1001f58 c1001f48 c0208598 c020859c 60000013 ffffffff
    [  635.268736]  r9:c1000000 r8:00000000 r7:c1001f2c r6:ffffffff r5:60000013 r4:c020859c
    [  635.276475] [<c020855c>] (arch_cpu_idle) from [<c089d1dc>] (default_idle_call+0x28/0x34)
    [  635.284557] [<c089d1b4>] (default_idle_call) from [<c025e388>] (cpu_startup_entry+0x1a4/0x228)
    [  635.293160] [<c025e1e4>] (cpu_startup_entry) from [<c0898554>] (rest_init+0x8c/0x90)
    [  635.300891]  r7:00000000
    [  635.303422] [<c08984c8>] (rest_init) from [<c0e00d74>] (start_kernel+0x3dc/0x3e8)
    [  635.310894]  r5:00000001 r4:c104204c
    [  635.314463] [<c0e00998>] (start_kernel) from [<80008090>] (0x80008090)
    

  • How correctly to swith on the jumbo frame? We use linux command ip link set dev eth2 mtu 9000.
  • We configure mtu size to 9000 after kernel boots up using ifconfig. In iperf we use -b half of the bandwidth, but with 2 processes (-p 2), and length 8000 (--len 8000). I recall BER was brought up in the other thread. BER diag is included in RTOS ProcSDK under PDK package. If you suspect it's signal integrity, you can run the BER on XGE port. I'll have the RTOS engineer to help you with it.
  • Increase receive buffers using setsockopt. We had about the same issue with gigabit on c6670.
  • Please disregard, I thought your are doing that on Linus host, then it matters. We had troubles on Linux PC until buffer was extended. Your case is different, as you run on DSP.
  • Rex,

    How can we do BER diag under Linux ?

  • Eduard,

    BER is only supported under RTOS.

    Rex
  • Eduard,

    BER is under pdk_k2e_4_0_x\packages\ti\diag\serdes_diag in RTOS Proc SDK. User guide is under docs. It can only be run using RTOS

    Rex
  • Rex,

    we run eye and ber tests:

    There are two questions:
    1. What is these artifacts in the eye diagram?
    2. How can I measure the data rate?


  • Hi, Eduard,

    The BER looks good, all green and centered. I am not familiar with EYE diagram and had forwarded to Hardware for more info. I did a google search on ethernet eye diagram, but think you may need more info than that wiki page. I'll post back if I heard back from hardware engineer.

    Rex
  • Eduard,

    How is the DS100DF410 implemented around the 66AK2E05?  Does one exist both in the RX path and the TX path or only in one direction?  Is it physically close to the  66AK2E05?  How close?

    The DS100DF410 has the ability perform RX equalization and then transmit shaping.  It can also interface with SFP+ modules that require interface metrics that the  66AK2E05 cannot meet.

    The EYE tool uses the capabilities of the receiver logic in  66AK2E05 to image the eye diagram *after* RX equalization has been performed.  It then provides an estimate of the eye amplitude and the eye width.  The images provided look very good.

    The EYE tool works in tap delay increments.  It is not aware of the data rate.  If you are not certain that your data rate is correct, I recommend using an oscilloscope to measure the data rate.

    The multiple levels in the eye diagram are the result of inter-symbol interference.  This is normal in multi-gigabit interfaces.  The RX equalization adjusts the receiver logic to maximize the eye opening.

    Tom

  • Tom,

    The retimer is installed on the Rx and Tx lines.

    The screenshot shows the part of the board and the differential lines SFP connector - retimer - Keystone. The length of the lines in the table on the left.

    The Serdes Diag User Guide  shows a drawing of eye diagram:

    But on our diagram there are some strange artifacts. I highlighted them with a red ellipse:

    Why is that?

  • Eduard,

    The routed lengths look reasonable.  Since the links are short, you might want to try adjusting the TX settings in the DS100 so that the eye image at the K2K device is improved.  Over-driving the receiver creates non-linear distortion that the receiver cannot equalize.

    The part of the eye image circled above in red is an artifact of the eye tool.  It is not part of the true image.  Do you see the same thing on every run?  The recovered clock from the CDR appears to have a phase alignment that causes the eye tool to saturate at the edge of the image and the results are distorted.  Regardless of whether you can make this go away, it is not indicative of a transmission problem.

    Tom

  • Hi Tom!

    >> Do you see the same thing on every run?  

    Yes, every run.

    We ran the Board-to-PC speed test. The Board transmits data (client), the computer receives data (server). With MTU=1500 we got the speed 3Gbit/s (and 45% CPU K2E Load). With MTU=9000 we got the speed 5.7Gbit/s (and 99.5% CPU K2E load).

    If the board receives data (the server) and PC transmits data (client), a kernel panic occurs when configuring MTU=9000. We tried the Linux from the last SDK, or custom Linux. We tried different programs: iperf3, netperf.

    Maybe for support MTU=9000 additional configuration of any receiving buffers is necessary?

  • Hi, Eduard.

    Could you try to configure both end with the same MTU size of 9000?

    Rex
  • Hi, Rex.

    Yes, both MTU=9000.
  • We always set the same MTU on the server and on the client. Kernel panic occurs on the Board when MTU=9000 and Board is server (receiver).
  • Sorry for slow response. Have we ever mentioned which software release version you are using?

    I was trying to get the configuration up. I use iperf and don't see the crash, but I am not getting the max throughput either. I am examining the setup to see what could be wrong for not getting the max throughput. I'll also try iperf3 to see if it makes any difference.

    Rex
  • Rex,

    we use iperf 3.1.3 (we download source C++ source and compile use gcc for arm).

  • Eduard,

    I want to confirm with you what I observed that 1500 mtu works fine, but only inbound 9000 mtu crashes the kernel.

    Rex
  • >> I want to confirm with you what I observed that 1500 mtu works fine, but only inbound 9000 mtu crashes the kernel.
    Yes it is !
  • Hi, Eduard,

    I've been trying to see if changing the buffer size and number of descriptors would help, but it seems that it is more to it. We'll investigate the issue further. For now, I would suggest the work around by setting the EVM mtu at 9000, but 1500 on remote system.

    Rex

  • Hi, Eduard,

    You may want to try the change below. It seems to be working for me. This change has not been accepted into TI's development branch yet.

    --- a/drivers/net/ethernet/ti/netcp_core.c
    +++ b/drivers/net/ethernet/ti/netcp_core.c
    @@ -618,7 +618,8 @@ static int netcp_process_one_rx_packet(struct netcp_intf *netcp)
    /* warning!!!! We are retrieving the virtual ptr in the sw_data
    * field as a 32bit value. Will not work on 64bit machines
    */
    - page = (struct page *)KNAV_DMA_GET_SW_DATA0(desc);
    + page = (struct page *)KNAV_DMA_GET_SW_DATA0(ndesc);

    if (likely(dma_buff && buf_len && page)) {
    dma_unmap_page(netcp->dev, dma_buff, PAGE_SIZE,

    Rex

  • Thank you! We will try.

  • By the way, here's tcpdump on Linux host machine pinging K2E with MTU on both ends set to 9000. Ping command: "ping -s 9000 [K2E XGE IP address]"

    guest@guestlocal:~$ sudo tcpdump -i eth3
    [sudo] password for guest:
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
    13:55:48.621881 ARP, Request who-has 192.168.100.22 tell guestlocal.local, length 28
    13:55:48.621997 ARP, Reply 192.168.100.22 is-at 06:33:10:5c:6c:34 (oui Unknown), length 46
    13:55:48.622009 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 1, length 8976
    13:55:48.622016 IP guestlocal.local > 192.168.100.22: icmp
    13:55:48.622196 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 1, length 8976
    13:55:48.622209 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:49.177303 IP6 fe80::21b:21ff:fe60:5bc4.mdns > ff02::fb.mdns: 0 PTR (QM)? 22.100.168.192.in-addr.arpa. (45)
    13:55:54.620071 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 7, length 8976
    13:55:54.620079 IP guestlocal.local > 192.168.100.22: icmp
    13:55:54.620240 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 7, length 8976
    13:55:54.620252 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:55.620053 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 8, length 8976
    13:55:55.620061 IP guestlocal.local > 192.168.100.22: icmp
    13:55:55.620219 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 8, length 8976
    13:55:55.620230 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:56.620028 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 9, length 8976
    13:55:56.620036 IP guestlocal.local > 192.168.100.22: icmp
    13:55:56.620201 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 9, length 8976
    13:55:56.620212 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:57.620065 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 10, length 8976
    13:55:57.620073 IP guestlocal.local > 192.168.100.22: icmp
    13:55:57.620227 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 10, length 8976
    13:55:57.620241 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:58.583823 IP6 fe80::433:10ff:fe5c:6c34 > ip6-allrouters: ICMP6, router solicitation, length 16
    13:55:58.620070 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 11, length 8976
    13:55:58.620075 IP guestlocal.local > 192.168.100.22: icmp
    13:55:58.620236 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 11, length 8976
    13:55:58.620245 IP 192.168.100.22 > guestlocal.local: icmp
    13:55:59.620073 IP guestlocal.local > 192.168.100.22: ICMP echo request, id 4393, seq 12, length 8976
    13:55:59.620080 IP guestlocal.local > 192.168.100.22: icmp
    13:55:59.620242 IP 192.168.100.22 > guestlocal.local: ICMP echo reply, id 4393, seq 12, length 8976
    13:55:59.620254 IP 192.168.100.22 > guestlocal.local: icmp
    ^C
    32 packets captured