Compiler: Secondary ethernet not working in linux on K2H14

Hi, Dario,

If you haven't read the note for KS2 on enabling extra ethernet ports in the NetCP driver, there are info you may find helpful.

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components/Kernel/Kernel_Drivers/Network/NetCP.html

Rex

Hi Rex,

    I have read (too often) the netcp description document:

http://software-dl.ti.com/processor-sdk-linux/esd/docs/05_02_00_10/linux/Foundational_Components_Kernel_Drivers.html#netcp

this link should to be the correct one for my specific linux version environment, but the examples reported here are related at
K2E and the some part is common for both SOCs seems similat at mine.

Sometime your description are partially cut, then the modifications/adapting are a bit difficult.

Eg:

1) In keysyone-k2hk-netcp.dtsi for k2hk not exists pa sections, nor qos

2) the rx-queue = <532> (I think it should to be correct for K2E is wrong for K2HK,
infact in your dtsi we have rx-queue = <8704>;  for interface-0),

3)  tx-pools in our k2hk is not present as vector in interface-1 K2E :

+                        tx-pools = <1024 12 17 17
+                                    64 12 17 17
+                                    64 12 17 17
+                                    64 12 17 17
+                                    64 12 17 17
+                                    64 12 17 17
+                                    64 12 17 17>;

in k2h a single line (I can give you show the variable name has a little differencem in red colour):

tx-pool = <1024 12>;

4) after tx-pools I can see a 10G configuration netcpx mixed in 1G configuration.

Regards,
Dario

Hi Rex,

  I have forgot to give you some informations.

1) All eth interfaces are statically defined in /etc/network/interfaces:

root@k2hk-evm:~# cat /etc/network/interfaces | grep -v \# | uniq
 
auto lo
iface lo inet loopback
   address 127.0.0.1
   netmask 255.0.0.0

auto eth0
iface eth0 inet static
   address 192.168.111.23
   netmask 255.255.255.0

auto eth1
iface eth1 inet static
   address 192.168.112.23
   netmask 255.255.255.0

auto eth2
iface eth2 inet static
   address 192.168.113.23
   netmask 255.255.255.0

auto eth3
iface eth3 inet static
   address 192.168.114.23
   netmask 255.255.255.0

auto eth4
iface eth4 inet static
   address 192.168.115.23
   netmask 255.255.255.0

auto eth5
iface eth5 inet static
   address 192.168.116.23
   netmask 255.255.255.0

2) The MACs address are:
root@k2hk-evm:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 50:33:8B:6B:79:4A  
          inet addr:192.168.111.23  Bcast:192.168.111.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:399 errors:0 dropped:2 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:33798 (33.0 KiB)  TX bytes:4648 (4.5 KiB)

eth1      Link encap:Ethernet  HWaddr 50:33:8B:6B:79:6F  
          inet addr:192.168.112.23  Bcast:192.168.112.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:404 errors:0 dropped:2 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:35458 (34.6 KiB)  TX bytes:4648 (4.5 KiB)

eth2      Link encap:Ethernet  HWaddr 50:33:8B:6B:79:6E  
          inet addr:192.168.113.23  Bcast:192.168.113.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:5188 (5.0 KiB)

eth3      Link encap:Ethernet  HWaddr 50:33:8B:6B:79:6D  
          inet addr:192.168.114.23  Bcast:192.168.114.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:365 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:25708 (25.1 KiB)

3) The routing table:

root@k2hk-evm:~# route -ne
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.111.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0
192.168.112.0   0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.113.0   0.0.0.0         255.255.255.0   U         0 0          0 eth2
192.168.114.0   0.0.0.0         255.255.255.0   U         0 0          0 eth3

root@k2hk-evm:~# arp
? (192.168.113.150) at <incomplete>  on eth2
? (192.168.114.150) at <incomplete>  on eth3

On PC I can see the arp request from custom board and the reply, both MACs and IPs are correct:

root@k2hk-evm:~# ping 192.168.111.150
PING 192.168.111.150 (192.168.111.150): 56 data bytes
64 bytes from 192.168.111.150: seq=0 ttl=64 time=0.335 ms
64 bytes from 192.168.111.150: seq=1 ttl=64 time=0.208 ms
64 bytes from 192.168.111.150: seq=2 ttl=64 time=0.155 ms
^C
--- 192.168.111.150 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.155/0.232/0.335 ms
root@k2hk-evm:~# arp
? (192.168.111.150) at 60:02:92:29:cb:5c [ether]  on eth0
? (192.168.113.150) at <incomplete>  on eth2
? (192.168.114.150) at <incomplete>  on eth3

on PC side:

Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.113.23           ether   50:33:8b:6b:79:6e   C                     eth0
192.168.112.23           ether   50:33:8b:6b:79:6f   C                     eth0
192.168.114.23           ether   50:33:8b:6b:79:6d   C                     eth0
192.168.111.23           ether   50:33:8b:6b:79:4a   C                     eth0

Seems that reply message is not received from linux.

I remember that also uboot 2018 has the same problem for all interfaces and to solve this problem I have used 2019
version (contents in the next environment version), but I don't know if this is a generic bug of 5.2.0.10 version.

Regards,
Dario

Hi, Dario,

The Note in NetCP driver was written based on older ProcSDK and K2E. Your configuration looks good. I don't think I get your (4) about you can see netcpx mixed in 1G configuration. Do you mean in the k2h-netcp.dtsi of 5.2 release or you "can't see" in your k2h-netcp.dtsi?

The logs you showed, there is a broken insertion. before the ping from EVM. The ping 111 subnet from EVM is good. I thought you have issue with eth2, shouldn't you ping 113 subnet? On PC side, all 4 subnet go to eth0 interface. Shouldn't they have different interfaces?

Rex 

Hi Rex,

    In attach you can see the couple of versions of keystone-k2hk-netcp.dtsi.
In our custom board configuration I have added the 10G and eth2/eth3 (connected
at serdes_lane2/serdes_lane3). I have tried to reconfigure in several times
rx-queue/tx-completion-queue to attempt to solve the problem without any results
(regarding ARP reply handling).

During the ping at 111/112 subnets (eth0&eth1 ports) I have give ^C to break the
ping, in eth2 (113 subnet) and eth3 (114 subnet) not have any ping reply after about
10 sec I have removed the cable (on wireshark I have seen several ARP request&reply
about one per second).


? (192.168.111.150) at 60:02:92:29:cb:5c [ether]  on eth0
? (192.168.113.150) at <incomplete>  on eth2
? (192.168.114.150) at <incomplete>  on eth3

Note the status incomplete on eth2 and eth3 because the port not have received the ARP reply.

I have also done a hw check to verify some "error", no problem found.

I don't know what to check to find the problem. Have you any suggestions?

Regards,
Dario

Hi, Dario,

Your zip file can't be accessed. Nonetheless, I re-examined yoour configuration in the original post. Do you purposely switch the netcp-gbe handles in interface-2 and interface-3? That should reflect the hardware layout. It that phandle is correct, then I can't understand why it doesn't work. All your configuration seems to be correct.

        interface-2 {
            rx-channel = "netrx2";
            rx-pool = <1024 12>;
            tx-pool = <1024 12>;
            rx-queue-depth = <128 128 0 0>;
            rx-buffer-size = <1518 4096 0 0>;
            rx-queue = <8706>;
            tx-completion-queue = <8710>;
            efuse-mac = <0>;
            local-mac-address = [50 33 8b 6b 79 6e];
            netcp-gbe = <&gbe3>;                              <== gbe3?
        };

Rex

Hi Rex,

      I'm sorry for delay but I am several problems on this project. Anyway, yes that "incorrect value" was an atempt to solve an HW error, we have inverted eth2&3 connectors on PCB, I have tried to correct it, with any success. Now I have mapped one to one the four eth ports.

For 10G problem we have decided to connect the phy devices on MDIO (1G), I hope to "solve" this other problem.

For 1G problem I have tried to configure my dhcp server, no solution found.

Follow the zip content, in green the differences (related eth2&3 and 10G).

Original keystone-k2hk-netcp.dtsi file:

qmss: qmss@2a40000 {
        compatible = "ti,keystone-navigator-qmss";
        dma-coherent;
        #address-cells = <1>;
        #size-cells = <1>;
        clocks = <&chipclk13>;
        ranges;
        queue-range     = <0 0x4000>;
        linkram0        = <0x100000 0x8000>;
        linkram1        = <0x0 0x10000>;
        qmgrs {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                qmgr0 {
                        managed-queues = <0 0x2000>;
                        reg = <0x2a40000 0x20000>,
                              <0x2a06000 0x400>,
                              <0x2a02000 0x1000>,
                              <0x2a03000 0x1000>,
                              <0x23a80000 0x20000>,
                              <0x2a80000 0x20000>;
                        reg-names = "peek", "status", "config",
                                    "region", "push", "pop";
                };
                qmgr1 {
                        managed-queues = <0x2000 0x2000>;
                        reg = <0x2a60000 0x20000>,
                              <0x2a06400 0x400>,
                              <0x2a04000 0x1000>,
                              <0x2a05000 0x1000>,
                              <0x23aa0000 0x20000>,
                              <0x2aa0000 0x20000>;
                        reg-names = "peek", "status", "config",
                                    "region", "push", "pop";
                };
        };
        queue-pools {
                qpend {
                        qpend-0 {
                                qrange = <658 8>;
                                interrupts =<0 40 0xf04 0 41 0xf04 0 42 0xf04
                                             0 43 0xf04 0 44 0xf04 0 45 0xf04
                                             0 46 0xf04 0 47 0xf04>;
                        };
                        qpend-1 {
                                qrange = <8704 16>;
                                interrupts = <0 48 0xf04 0 49 0xf04 0 50 0xf04
                                              0 51 0xf04 0 52 0xf04 0 53 0xf04
                                              0 54 0xf04 0 55 0xf04 0 56 0xf04
                                              0 57 0xf04 0 58 0xf04 0 59 0xf04
                                              0 60 0xf04 0 61 0xf04 0 62 0xf04
                                              0 63 0xf04>;
                                qalloc-by-id;
                        };
                        qpend-2 {
                                qrange = <8720 16>;
                                interrupts = <0 64 0xf04 0 65 0xf04 0 66 0xf04
                                              0 59 0xf04 0 68 0xf04 0 69 0xf04
                                              0 70 0xf04 0 71 0xf04 0 72 0xf04
                                              0 73 0xf04 0 74 0xf04 0 75 0xf04
                                              0 76 0xf04 0 77 0xf04 0 78 0xf04
                                              0 79 0xf04>;
                        };
                };
                general-purpose {
                        gp-0 {
                                qrange = <4000 64>;
                        };
                        netcp-tx {
                                qrange = <640 9>;
                                qalloc-by-id;
                        };
                        netcpx-tx {
                                qrange = <8752 8>;
                                qalloc-by-id;
                        };
                };
                accumulator {
                        acc-low-0 {
                                qrange = <480 32>;
                                accumulator = <0 47 16 2 50>;
                                interrupts = <0 226 0xf01>;
                                multi-queue;
                                qalloc-by-id;
                        };
                };
        };
        descriptor-regions {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                region-12 {
                        id = <12>;
                        region-spec = <8192 128>;       /* num_desc desc_size */
                        link-index = <0x4000>;
                };
        };
        pdsps {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                pdsp0@0x2a10000 {
                        reg = <0x2a10000 0x1000    /*iram */
                               0x2a0f000 0x100     /*reg*/
                               0x2a0c000 0x3c8     /*intd */
                               0x2a20000 0x4000>;  /*cmd*/
                        id = <0>;
                };
        };
};
knav_dmas: knav_dmas@0 {
        compatible = "ti,keystone-navigator-dma";
        clocks = <&papllclk>;
        #address-cells = <1>;
        #size-cells = <1>;
        ranges;
        ti,navigator-cloud-address = <0x23a80000 0x23a90000
                                   0x23aa0000 0x23ab0000>;
        dma_gbe: dma_gbe@0 {
                reg = <0x2004000 0x100>,
                          <0x2004400 0x120>,
                          <0x2004800 0x300>,
                          <0x2004c00 0x120>,
                          <0x2005000 0x400>;
                reg-names = "global", "txchan", "rxchan",
                                "txsched", "rxflow";
                ti,enable-all;
        };
};
gbe_subsys: subsys@2090000 {
        compatible = "syscon";
        reg = <0x02090000 0x100>;
};
gbe_serdes: phy@232a000 {
        compatible              = "ti,keystone-serdes-gbe";
        reg                     = <0x0232a000 0x2000>;
        status                  = "disabled";
        link-rate-kbps          = <1250000>;
        num-lanes               = <4>;
        #address-cells  = <1>;
        #size-cells     = <0>;

        serdes_lane0: lane@0 {
                #phy-cells      = <0>;
                reg             = <0>;
                status          = "ok";
                control-rate    = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane1: lane@1 {
                #phy-cells      = <0>;
                reg             = <1>;
                status          = "ok";
                control-rate    = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane2: lane@2 {
                #phy-cells      = <0>;
                reg             = <2>;
                status          = "disabled";
                control-rate    = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane3: lane@3 {
                #phy-cells      = <0>;
                reg             = <3>;
                status          = "disabled";
                control-rate    = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
};
netcp: netcp@2000000 {
        reg = <0x2620110 0x8>;
        reg-names = "efuse";
        compatible = "ti,netcp-1.0";
        #address-cells = <1>;
        #size-cells = <1>;

        /* NetCP address range */
        ranges  = <0 0x2000000 0x100000>;
        clocks = <&clkpa>, <&clkcpgmac>, <&chipclk12>;
        clock-names = "pa_clk", "ethss_clk", "cpts";
        dma-coherent;
        ti,navigator-dmas = <&dma_gbe 22>,
                        <&dma_gbe 23>,
                        <&dma_gbe 8>;
        ti,navigator-dma-names = "netrx0", "netrx1", "nettx";
        netcp-devices {
                ranges;
                #address-cells = <1>;
                #size-cells = <1>;
                gbe@90000 { /* ETHSS */
                        #address-cells = <1>;
                        #size-cells = <1>;
                        label = "netcp-gbe";
                        compatible = "ti,netcp-gbe";
                        syscon-subsys = <&gbe_subsys>;
                        reg = <0x90100 0x200>, <0x90400 0x200>, <0x90800 0x700>;
                        tx-queue = <648>;
                        tx-channel = "nettx";
                        interfaces {
                                gbe0: interface-0 {
                                        phys = <&serdes_lane0>;
                                        slave-port = <0>;
                                        link-interface = <1>;
                                        phy-handle = <&ethphy0>;
                                };
                                gbe1: interface-1 {
                                        phys = <&serdes_lane1>;
                                        slave-port = <1>;
                                        link-interface = <1>;
                                        phy-handle = <&ethphy1>;
                                };
                        };
                        secondary-slave-ports {
                                port-2 {
                                        phys = <&serdes_lane2>;
                                        slave-port = <2>;
                                        link-interface  = <2>;
                                };
                                port-3 {
                                        phys = <&serdes_lane3>;
                                        slave-port = <3>;
                                        link-interface  = <2>;
                                };
                        };
                };
        };
        netcp-interfaces {
                interface-0 {
                        rx-channel = "netrx0";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8704>;
                        tx-completion-queue = <8706>;
                        efuse-mac = <1>;
                        netcp-gbe = <&gbe0>;
                };
                interface-1 {
                        rx-channel = "netrx1";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8705>;
                        tx-completion-queue = <8707>;
                        efuse-mac = <0>;
                        local-mac-address = [02 18 31 7e 3e 6f];
                        netcp-gbe = <&gbe1>;
                };
        };
};

sa_subsys: subsys@20c0000 {
        compatible = "simple-bus";
        #address-cells = <1>;
        #size-cells = <1>;
        ranges = <0 0x20c0000 0x40000>;
        dma-coherent;
        dma-ranges;
        sa_config: subsys@0 {
                compatible = "syscon";
                reg = <0x0 0x100>;
        };
        rng@24000 {
                compatible = "ti,keystone-rng";
                reg = <0x24000 0x1000>;
                ti,syscon-sa-cfg = <&sa_config>;
                clocks = <&clksa>;
                clock-names = "fck";
        };
        crypto@0 {
                compatible = "ti,netcp-sa-crypto";
                syscon-subsys = <&sa_config>;
                clocks = <&clksa>;
                clock-names = "fck";
                ti,navigator-dmas = <&dma_gbe 26>, <&dma_gbe 6>;
                ti,navigator-dma-names = "crypto-rx", "crypto-tx";
                ti,rx-channel = "crypto-rx";
                ti,rx-queue-depth = <256 64 0 0>;
                ti,rx-compl-queue = <8712>;
                ti,rx-pool      = <512 12>;
                ti,tx-channel = "crypto-tx";
                ti,tx-queue-depth = <256>;
                ti,tx-completion-queue = <8713>;
                ti,tx-pool = <512 12>;
                ti,tx-submit-queue = <646>;
                ti,sc-id = <0x7000 0x71ff>;
        };
};

Custom keystone-k2hk-netcp.dtsi file:

qmss: qmss@2a40000 {
        compatible = "ti,keystone-navigator-qmss";
        dma-coherent;
        #address-cells = <1>;
        #size-cells = <1>;
        clocks = <&chipclk13>;
        ranges;
        queue-range     = <0 0x4000>;
        linkram0        = <0x100000 0x8000>;
        linkram1        = <0x0 0x10000>;
        qmgrs {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                qmgr0 {
                        managed-queues = <0 0x2000>;
                        reg = <0x2a40000 0x20000>,
                                <0x2a06000 0x400>,
                                <0x2a02000 0x1000>,
                                <0x2a03000 0x1000>,
                                <0x23a80000 0x20000>,
                                <0x2a80000 0x20000>;
                        reg-names = "peek", "status", "config",
                                        "region", "push", "pop";
                };
                qmgr1 {
                        managed-queues = <0x2000 0x2000>;
                        reg = <0x2a60000 0x20000>,
                                <0x2a06400 0x400>,
                                <0x2a04000 0x1000>,
                                <0x2a05000 0x1000>,
                                <0x23aa0000 0x20000>,
                                <0x2aa0000 0x20000>;
                        reg-names = "peek", "status", "config",
                                        "region", "push", "pop";
                };
        };
        queue-pools {
                qpend {
                        qpend-0 {
                                qrange = <658 8>;
                                interrupts =<0 40 0xf04 0 41 0xf04 0 42 0xf04
                                                 0 43 0xf04 0 44 0xf04 0 45 0xf04
                                                 0 46 0xf04 0 47 0xf04>;
                        };
                        qpend-1 { // 1G netcp RX
                                qrange = <8704 16>;
                                interrupts = <0 48 0xf04 0 49 0xf04 0 50 0xf04
                                                  0 51 0xf04 0 52 0xf04 0 53 0xf04
                                                  0 54 0xf04 0 55 0xf04 0 56 0xf04
                                                  0 57 0xf04 0 58 0xf04 0 59 0xf04
                                                  0 60 0xf04 0 61 0xf04 0 62 0xf04
                                                  0 63 0xf04>;
                                qalloc-by-id;
                        };
                        qpend-2 { // 10G xnetcp RX
                                qrange = <8720 16>;
                                interrupts = <0 64 0xf04 0 65 0xf04 0 66 0xf04
                                                  0 59/*67*/ 0xf04 0 68 0xf04 0 69 0xf04
                                                  0 70 0xf04 0 71 0xf04 0 72 0xf04
                                                  0 73 0xf04 0 74 0xf04 0 75 0xf04
                                                  0 76 0xf04 0 77 0xf04 0 78 0xf04
                                                  0 79 0xf04>;
                                qalloc-by-id;
                        };
                };
                general-purpose {
                        gp-0 {
                                qrange = <4000 64>;
                        };
                        netcp-tx { // 1G TX
                                qrange = <640 9>;
                                qalloc-by-id;
                        };
                        netcpx-tx { // 10G TX
                                qrange = <8752 8>;
                                qalloc-by-id;
                        };
                };
                accumulator {
                        acc-low-0 {
                                qrange = <480 32>;
                                accumulator = <0 47 16 2 50>;
                                interrupts = <0 226 0xf01>;
                                multi-queue;
                                qalloc-by-id;
                        };
                };
        };
        descriptor-regions {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                region-12 {
                        id = <12>;
                        region-spec = <16384 128>;      /* num_desc desc_size */
                        link-index = <0x4000>;
                };
        };
        pdsps {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                pdsp0@0x2a10000 {
                        reg = <0x2a10000 0x1000    //iram
                                 0x2a0f000 0x100     //reg
                                 0x2a0c000 0x3c8           //intd
                                 0x2a20000 0x4000>;  //cmd
                        id = <0>;
                };
        };
};
knav_dmas: knav_dmas@0 {
        compatible = "ti,keystone-navigator-dma";
        clocks = <&papllclk>, <&clkxge>;
        #address-cells = <1>;
        #size-cells = <1>;
        ranges;
        ti,navigator-cloud-address = <0x23a80000 0x23a90000
                                                0x23aa0000 0x23ab0000>;
        dma_gbe: dma_gbe@0 {
                reg = <0x2004000 0x100>,
                        <0x2004400 0x120>,
                        <0x2004800 0x300>,
                        <0x2004c00 0x120>,
                        <0x2005000 0x400>;
                reg-names = "global", "txchan", "rxchan",
                                "txsched", "rxflow";
                ti,enable-all;
        };
        dma_xgbe: dma_xgbe@0 {
                reg = <0x2fa1000 0x100>,
                        <0x2fa1400 0x200>,
                        <0x2fa1800 0x200>,
                        <0x2fa1c00 0x200>,
                        <0x2fa2000 0x400>;
                reg-names = "global", "txchan", "rxchan",
                                "txsched", "rxflow";
                ti,enable-all;
        };
};
gbe_subsys: subsys@2090000 {
        compatible = "syscon";
        reg = <0x02090000 0x100>;
};
xgbe_subsys: subsys@2f00000 {
        status = "ok";
        compatible = "syscon";
        reg = <0x02f00000 0x100>;
};
xgbe_pcsr: pcsr@2f00600 {
        status  = "ok";
        compatible      = "syscon";
        reg             = <0x02f00600 0x100>;
};
gbe_serdes: phy@232a000 {
        compatible              = "ti,keystone-serdes-gbe";
        reg                     = <0x0232a000 0x2000>;
        status          = "ok";
        link-rate-kbps  = <1250000>;
        num-lanes               = <4>;
        #address-cells  = <1>;
        #size-cells             = <0>;
        clocks =  <&clkcpgmac>;
        clock-names = "fck";
        serdes_lane0: lane@0 {
                #phy-cells      = <0>;
                reg             = <0>;
                status  = "ok";
                control-rate = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane1: lane@1 {
                #phy-cells      = <0>;
                reg             = <1>;
                status  = "ok";
                control-rate = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane2: lane@2 {
                #phy-cells      = <0>;
                reg             = <2>;
                status  = "ok";
                control-rate = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
        serdes_lane3: lane@3 {
                #phy-cells      = <0>;
                reg             = <3>;
                status  = "ok";
                control-rate = <2>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <0 0 0 12 4>;
        };
};
netcp: netcp@2000000 {
        reg = <0x2620110 0x8>;
        reg-names = "efuse";
        compatible = "ti,netcp-1.0";
        #address-cells = <1>;
        #size-cells = <1>;
        ranges  = <0 0x2000000 0x100000>;
        clocks = <&clkpa>, <&clkcpgmac>, <&chipclk12>;
        clock-names = "pa_clk", "ethss_clk", "cpts";
        dma-coherent;
        ti,navigator-dmas = <&dma_gbe 22>, <&dma_gbe 23>,
                                  <&dma_gbe 24>, <&dma_gbe 25>,
                                  <&dma_gbe 8>, <&dma_gbe 0>;
        ti,navigator-dma-names = "netrx0", "netrx1",
                                         "netrx2", "netrx3",
                                         "nettx", "netrx0-pa";
        netcp-devices {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                gbe@90000 {
                        #address-cells = <1>;
                        #size-cells = <1>;
                        label = "netcp-gbe";
                        compatible = "ti,netcp-gbe";
                        syscon-subsys = <&gbe_subsys>;
                        reg = <0x90100 0x200>, <0x90400 0x200>, <0x90800 0x700>;
                        tx-queue = <648>;
                        tx-channel = "nettx";
                        interfaces {
                                gbe0: interface-0 {
                                        phys           = <&serdes_lane0>;
                                        slave-port     = <0>;
                                        link-interface = <1>;
                                        phy-handle     = <&ethphy0>;
                                };
                                gbe1: interface-1 {
                                        phys           = <&serdes_lane1>;
                                        slave-port     = <1>;
                                        link-interface = <1>;
                                        phy-handle     = <&ethphy1>;
                                };
                                gbe2: interface-2 {
                                        phys           = <&serdes_lane2>;
                                        slave-port     = <2>;
                                        link-interface = <1>;
                                        phy-handle     = <&ethphy2>;
                                };
                                gbe3: interface-3 {
                                        phys           = <&serdes_lane3>;
                                        slave-port     = <3>;
                                        link-interface = <1>;
                                        phy-handle     = <&ethphy3>;
                                };
                        };
                };
        };
        netcp-interfaces {
                interface-0 {
                        rx-channel = "netrx0";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8704>;
                        tx-completion-queue = <8708>;
                        efuse-mac = <1>;
                        netcp-gbe = <&gbe0>;
                };
                interface-1 {
                        rx-channel = "netrx1";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8705>;
                        tx-completion-queue = <8709>;
                        efuse-mac = <0>;
                        local-mac-address = [50 33 8b 6b 79 6f];
                        netcp-gbe = <&gbe1>;
                };
                interface-2 {
                        rx-channel = "netrx3";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8707>;
                        tx-completion-queue = <8711>;
                        efuse-mac = <0>;
                        local-mac-address = [50 33 8b 6b 79 6e];
                        netcp-gbe = <&gbe3>;
                };
                interface-3 {
                        rx-channel = "netrx2";
                        rx-pool = <1024 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <128 128 0 0>;
                        rx-buffer-size = <1518 4096 0 0>;
                        rx-queue = <8706>;
                        tx-completion-queue = <8710>;
                        efuse-mac = <0>;
                        local-mac-address = [50 33 8b 6b 79 6d];
                        netcp-gbe = <&gbe2>;
                };
        };
};
/*
sa_subsys: subsys@20c0000 {
        compatible = "simple-bus";
        #address-cells = <1>;
        #size-cells = <1>;
        ranges = <0 0x20c0000 0x40000>;
        dma-coherent;
        dma-ranges;
status = "disabled";
        sa_config: subsys@0 {
                compatible = "syscon";
                reg = <0x0 0x100>;
        };
        rng@24000 {
                compatible = "ti,keystone-rng";
                reg = <0x24000 0x1000>;
                ti,syscon-sa-cfg = <&sa_config>;
                clocks = <&clksa>;
                clock-names = "fck";
        };
        crypto@0 {
                compatible = "ti,netcp-sa-crypto";
                syscon-subsys = <&sa_config>;
                clocks = <&clksa>;
                clock-names = "fck";
                ti,navigator-dmas = <&dma_gbe 26>, <&dma_gbe 6>;
                ti,navigator-dma-names = "crypto-rx", "crypto-tx";
                ti,rx-channel = "crypto-rx";
                ti,rx-queue-depth = <256 64 0 0>;
                ti,rx-compl-queue = <8712>;
                ti,rx-pool      = <512 12>;
                ti,tx-channel = "crypto-tx";
                ti,tx-queue-depth = <256>;
                ti,tx-completion-queue = <8713>;
                ti,tx-pool = <512 12>;
                ti,tx-submit-queue = <646>;
                ti,sc-id = <0x7000 0x71ff>;
        };
};
*/
xgbe_serdes: phy@231e000 {
        status          = "okay";
        compatible              = "ti,keystone-serdes-xgbe";
        reg                     = <0x0231e000 0x2000>;
        link-rate-kbps  = <10312500>;
        num-lanes               = <2>;
        syscon-peripheral       = <&xgbe_subsys>;
        syscon-link             = <&xgbe_pcsr>;
        #address-cells  = <1>;
        #size-cells             = <0>;
        clocks = <&clkxge>;
        clock-names = "fck";
        xserdes_lane0: lane@0 {
                status  = "okay";
                #phy-cells      = <0>;
                reg             = <0>;
                control-rate = <0>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <2 0 0 12 4>;/* c1 c2 cm att vreg*/
        };
        xserdes_lane1: lane@1 {
                status  = "okay";
                #phy-cells      = <0>;
                reg             = <1>;
                control-rate = <0>;
                rx-start        = <7 5>;
                rx-force        = <1 1>;
                tx-coeff        = <2 0 0 12 4>;/* c1 c2 cm att vreg */
        };
};
netcpx: netcpx@2f00000 {
        status  = "okay";
        compatible      = "ti,netcp-1.0";
        #address-cells  = <1>;
        #size-cells             = <1>;
        reg = <0x02f00000 0xFFFFF>;
        clocks  = <&clkxge>,  <&chipclk12>;
        clock-names     = "clk_xge", "cpts";
        dma-coherent;
        ti,navigator-dmas = <&dma_xgbe 0>, <&dma_xgbe 8>, <&dma_xgbe 0>;
        ti,navigator-dma-names = "xnetrx0", "xnetrx1", "xnettx";
        netcp-devices {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges;
                xgbe@2f00000 {
                        label = "netcp-xgbe";
                        compatible = "ti,netcp-xgbe";
                        syscon-subsys = <&xgbe_subsys>;
                        syscon-pcsr = <&xgbe_pcsr>;
                       reg = <0x02f00100 0x200>, <0x02f01000 0xFEFFF>;
                        tx-queue = <8752>;
                        tx-channel = "xnettx";
                        status = "okay";
                        cpts {
                                status = "ok";
                                rftclk_sel = <0x0>;
                                ts_comp_length = <0x3>;
                                ts_comp_polarity = <0x1>;
                        };
                        interfaces {
                                status = "okay";
                                xgbe0: interface-0 {
                                        phys = <&xserdes_lane0>;
                                        status ="okay";
                                        slave-port = <0>;
                                        link-interface  = <10>; /* XGMII_LINK_MAC_PHY*/
                                        phy-handle = <&xethphy0>;
                                };
                                xgbe1: interface-1 {
                                        phys = <&xserdes_lane1>;
                                        status = "okay";
                                        slave-port = <1>;
                                        link-interface  = <10>; /* XGMII_LINK_MAC_PHY*/
                                        phy-handle = <&xethphy1>;
                                };
                        };
                };
        };
        netcp-interfaces {
                interface-0 {
                        rx-channel = "xnetrx0";
                        rx-pool = <2048 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <1024 1024 0 0>;
                        rx-buffer-size = <1536 4096 0 0>;
                        rx-queue = <8720>;
                        tx-completion-queue = <8722>;
                        efuse-mac = <0>;
                        local-mac-address = [02 18 31 7e 3e 50];
                        netcp-xgbe = <&xgbe0>;
                };
                interface-1 {
                        rx-channel = "xnetrx1";
                        rx-pool = <2048 12>;
                        tx-pool = <1024 12>;
                        rx-queue-depth = <1024 1024 0 0>;
                        rx-buffer-size = <1536 4096 0 0>;
                        rx-queue = <8721>;
                        tx-completion-queue = <8723>;
                        efuse-mac = <0>;
                        local-mac-address = [02 18 31 7e 3e 51];
                        netcp-xgbe = <&xgbe1>;
                };
        };
};

Regards,

Dario

Hi, Dario,

There are a few places your gbe configuration aren't the same as TI's. I don't think you can change the order of nettx in navigator-dmas. Also, you have interface flip back but the rx-queue isn't. 

For XGbE, the last working version is PLSDK 3.3 and has dropped XGbE support afterward. Besides, TI used retimer, not phy. We don't have experience with PHY and don't know what are involved using PHY for XGbE. 

Rex

Hi Rex,

   I have found the real PROBLEM in uboot (almost), on linux I can check again, it was a code's problem. As I have supposed (some time ago) the DMA is not enabled on forth ethernet port.

The file impatted is keystone_net.c, the easy solution follow:

static int ks2_eth_parse_slave_interface(int netcp, int slave,
                     struct ks2_eth_priv *priv,
                     struct eth_pdata *pdata)
{
    const void *fdt = gd->fdt_blob;
    int mdio;
    int phy;
    int dma_count;
    u32 dma_channel[8];
    const char *phy_mode;

    priv->slave_port = fdtdec_get_int(fdt, slave, "slave-port", -1);
    priv->net_rx_buffs.rx_flow = priv->slave_port * 8;

    /* U-Boot slave port number starts with 1 instead of 0 */
//  priv->slave_port += 1; // <<<<<<<<<<<< this incremet is done too early

    dma_count = fdtdec_get_int_array_count(fdt, netcp,
                           "ti,navigator-dmas",
                           dma_channel, 8);

    if (dma_count > (2 * priv->slave_port)) {
        int dma_idx;

//      dma_idx = priv->slave_port * 2 - 1; // <<<<<<<<<<<< replaced line wit follows
        dma_idx = priv->slave_port * 2 + 1;
        priv->net_rx_buffs.rx_flow = dma_channel[dma_idx];
    }

    priv->slave_port += 1;   // <<<<< moved here
.....

I hope this suggestion could to be used from other users.

Regards,
Dario

Hi, Dario,

Thanks for finding the issue in the code. I'll feed it back to our development team. I'll consider your issue is resolved. Please submit new threads you have other issues.

Rex

Hi, Dario,

I took a look further on your suggested changes. TI U-boot doesn't expect the secondary ports enabled when u-boot comes up, and there isn't use case for the secondary ethernet ports at boot time. In linux, it seems to me that the dma should be configured in dts as you did. That's a bit odd to me.

Rex