• State Resolved
  • Locked Locked
  • Replies 15 replies
  • Subscribers 99 subscribers
  • Views 1131 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM6422: U-Boot ICSSG0 Ethernet Port Not Working on EVM Based Design Board

Daryl Jarman
Prodigy 60 points
Part Number: AM6422

Tool/software:

I am porting u-boot to a board design based on the AM64x EVM.  The board design does not use the CPSW gigabit switch, but has 4 ethernet ports, 2 port on each of the ICSSG PRU's.  For the sake of simplicity, I am only focused on the 2 ethernet ports on the ICSSG0 PRU.  I am using the u-boot code "ti-u-boot-2023.04+gitAUTOINC+8366064208" that is part of the SDK download.

My last adventure in u-boot was over 10 years ago (back when Freescale was still Freescale), so I am a bit new to the whole device tree concept.  However I am used to diving into code to figure out what is going on.  Hence the log will have some additional logging in it that I have put in.

For the device tree, in the k3-am64-main.dtsi I am referencing the "icssg0: icssg@30000000" already established in this file.  The "k3-am642-evm.dts" source already had a reference to "iccsg1_eth" so I added the following to this source, and made changes for icssg0, and came up with the following.  I am a bit unsure on the "dmas" portion, but it looked follow what others had done.

------------------------------------

icssg0_eth: icssg0-eth {
    compatible = "ti,am642-icssg-prueth";
    pinctrl-names = "default";
    pinctrl-0 = <&mypruicssg0rgmii1_pins_default
                &mypruicssg0rgmii2_pins_default>;
    sram = <&oc_sram>;
    ti,prus = <&pru0_0>, <&rtu0_0>, <&tx_pru0_0>, <&pru0_1>, <&rtu0_1>, <&tx_pru0_1>;
    firmware-name = "ti-pruss/am64x-sr2-pru0-prueth-fw.elf",
            "ti-pruss/am64x-sr2-rtu0-prueth-fw.elf",
            "ti-pruss/am64x-sr2-txpru0-prueth-fw.elf",
            "ti-pruss/am64x-sr2-pru1-prueth-fw.elf",
            "ti-pruss/am64x-sr2-rtu1-prueth-fw.elf",
            "ti-pruss/am64x-sr2-txpru1-prueth-fw.elf";

    ti,pruss-gp-mux-sel = <2>, /* MII mode */
                    <2>,
                    <2>,
                    <2>, /* MII mode */
                    <2>,
                    <2>;
    ti,mii-g-rt = <&icssg0_mii_g_rt>;
    ti,mii-rt = <&icssg0_mii_rt>;
    ti,iep = <&icssg0_iep0>, <&icssg0_iep1>;
    interrupt-parent = <&icssg0_intc>;
    interrupts = <24 0 2>, <25 1 3>;
    interrupt-names = "tx_ts0", "tx_ts1";
    dmas = <&main_pktdma 0xc100 15>, /* egress slice 0 */
            <&main_pktdma 0xc101 15>, /* egress slice 0 */
            <&main_pktdma 0xc102 15>, /* egress slice 0 */
            <&main_pktdma 0xc103 15>, /* egress slice 0 */
            <&main_pktdma 0xc104 15>, /* egress slice 1 */
            <&main_pktdma 0xc105 15>, /* egress slice 1 */
            <&main_pktdma 0xc106 15>, /* egress slice 1 */
            <&main_pktdma 0xc107 15>, /* egress slice 1 */
            <&main_pktdma 0x4100 15>, /* ingress slice 0 */
            <&main_pktdma 0x4101 15>; /* ingress slice 1 */
    dma-names = "tx0-0", "tx0-1", "tx0-2", "tx0-3",
            "tx1-0", "tx1-1", "tx1-2", "tx1-3",
            "rx0", "rx1";

    ethernet-ports {
        #address-cells = <1>;
        #size-cells = <0>;
        icssg0_emac0: port@0 {
            reg = <0>;
            phy-handle = <&icssg0_phy1>;
            phy-mode = "rgmii-id";
            /* Filled in by bootloader */
            local-mac-address = [00 00 00 00 00 00];
        };
        icssg0_emac1: port@1 {
            reg = <1>;
            phy-handle = <&icssg0_phy2>;
            phy-mode = "rgmii-id";
            /* Filled in by bootloader */
            local-mac-address = [00 00 00 00 00 00];
        };
    };
};

----------------------------------------------

Later in the source file, outside of the root node, I have the followi

-------------------------------------------

&icssg0_mdio {
    status = "okay";
    pinctrl-names = "default";
    pinctrl-0 = <&mypruicssg0mdio1_pins_default>;

    icssg0_phy1: ethernet-phy@1 {
        reg = <0x1>;
        tx-internal-delay-ps = <250>;
        rx-internal-delay-ps = <2000>;
    };

    icssg0_phy2: ethernet-phy@2 {
        reg = <0x2>;
        tx-internal-delay-ps = <250>;
        rx-internal-delay-ps = <2000>;
    };
};

------------------------------------------

The failure is occurring when enabling "clocks"  The "icssg0_cfg" references the "k3_clks" node found in the "dmcs" node in the k3-am64-main.dtsi source.  This has the driver compatibility for "ti,k2g-sci-clk", which I found in the "clk-sci.c driver source.  

From the log, I can see the prueth_probe trying to start the MDIO for port0 and is trying to enable the clock, and sending a message to a "mailbox" found in the "k3-sec-proxy.c" sources.  This is where I get more than a bit lost.  I can see the following data being sent to the mailbox (found in "ti_sci.c" source, function ti_sci_do_xfer()): "00 01 0c f5 02", however it receives back the following data: "00 01 0c f5 00".  The code will validate the acknowledged message via bit 2 in the 5th byte (seemingly the "flags" field).  Since something cleared this bit (or did not set the bit), the function returns -ENODEV error.  This message is placed in "IOMEM" type of memory.  The log is shown below (I am not sure how far back I should go in this log) :

------------------------------------------

drivers/net/ti/icssg_prueth.c: icssg_ofdata_parse_phy: Begin
Using port0
prueth_probe: sram: addr 70000000 size 28000
prueth_probe: MDIO_NODE
ti_sci_clk_of_xlate(clk=00000000fdea6c68, args_count=2)
prueth_probe: Try and enable clock
drivers/clk/clk-uclass.c: clk_enable(clk=00000000fdea6c68)
clk_enable: Name :clock-controller
drivers/clk/ti/clk-sci.c: ti_sci_clk_enable(clk=00000000fdea6c68)
drivers/firmware/ti_sci.c: ti_sci_cmd_put_clock: BEGIN
drivers/firmware/ti_sci.c: ti_sci_set_clock_state: BEGIN
ti_sci_do_xfer: BEGIN: system-controller@44043000
ti_sci_do_xfer: mbox_send
k3MsgDump: sending: 14
00 01 0c f5 02 00 00 00 3e 00 00 00 03 01
mbox_send(chan=00000000fdea10c0, data=00000000fdea1108: To ADDR: 00000000fff359d0)
drivers/mailbox/k3-sec-proxy.c: k3_sec_proxy_send(chan=00000000fdea10c0, data=00000000fdea1108)
k3_sec_proxy_verify_thread: Begin
k3_sec_proxy_verify_thread: Reading message
k3_sec_proxy_verify_thread: Return 0
k3_sec_proxy_send: Message successfully sent on thread 13
ti_sci_do_xfer: mbox_send__end: rxLen 8
ti_sci_get_response: BEGIN
mbox_recv(chan=00000000fdea10d8, data=00000000fdea1108, timeout_us=10000000)
drivers/mailbox/k3-sec-proxy.c: k3_sec_proxy_recv(chan=00000000fdea10d8, data=00000000fdea1108)
k3_sec_proxy_verify_thread: Begin
k3_sec_proxy_verify_thread: Reading message
k3_sec_proxy_verify_thread: Return 0
k3_sec_proxy_recv: Message successfully received from thread 12
drivers/firmware/ti_sci.c: ti_sci_get_response: MBOXRecv: system-controller@44043000
drivers/firmware/ti_sci.c: ti_sci_get_response: end: ret 0
k3MsgDump: received: 8
00 01 0c f5 00 00 00 00
ti_sci system-controller@44043000: Message not acknowledged
Message not acknowledged
drivers/firmware/ti_sci.c: ti_sci_do_xfer: END: -19
ti_sci_set_clock_state: RET: -19
ti-sci-clk clock-controller: ti_sci_clk_enable: put_clock failed (-19)
Enable clock-controller failed
prueth icssg0-eth: clk_enable failed -19
drivers/net/ti/icssg_prueth.c: prueth_probe: Begin
drivers/firmware/ti_sci.c: ti_sci_set_device_state: BEGIN
ti_sci_do_xfer: BEGIN: system-controller@44043000
ti_sci_do_xfer: mbox_send
k3MsgDump: sending: 17
00 02 0c 0a 02 04 00 00 52 00 00 00 00 00 00 00 02
mbox_send(chan=00000000fdea10c0, data=00000000fdea1108: To ADDR: 00000000fff359d0)
drivers/mailbox/k3-sec-proxy.c: k3_sec_proxy_send(chan=00000000fdea10c0, data=00000000fdea1108)
k3_sec_proxy_verify_thread: Begin
k3_sec_proxy_verify_thread: Reading message
k3_sec_proxy_verify_thread: Return 0
k3_sec_proxy_send: Message successfully sent on thread 13
ti_sci_do_xfer: mbox_send__end: rxLen 8
ti_sci_get_response: BEGIN
mbox_recv(chan=00000000fdea10d8, data=00000000fdea1108, timeout_us=10000000)
drivers/mailbox/k3-sec-proxy.c: k3_sec_proxy_recv(chan=00000000fdea10d8, data=00000000fdea1108)
k3_sec_proxy_verify_thread: Begin
k3_sec_proxy_verify_thread: Reading message
k3_sec_proxy_verify_thread: Return 0
k3_sec_proxy_recv: Message successfully received from thread 12
drivers/firmware/ti_sci.c: ti_sci_get_response: MBOXRecv: system-controller@44043000
drivers/firmware/ti_sci.c: ti_sci_get_response: end: ret 0
k3MsgDump: received: 8
00 02 0c 0a 02 00 00 00
drivers/firmware/ti_sci.c: ti_sci_do_xfer: END: 0

-----------------------------------------

Being new to the whole mailbox/security proxy concept, my question revolves around who it receiving the mailbox message?  Also, who is sending the message back?  I am thinking that "the other side" of the mailbox is a system that is communicating to the hardware (mdio to the phy perhaps?) and is not getting the correct response, so did not se the acknowledge bit accordingly.  Does this seem reasonable on what is going on?  Is this a communication with the pru core, and it is not happy with something?

Many thanks with your help.  If you need further information, please let me know.

Daryl

  • 0
    TI__Genius 16005 points

    Hello Daryl,

    May I ask if your end goal is to be able to use all 4 ethernet ports in the U-boot environment? Or will you eventually need the ethernet ports functional on an OS such as Linux?

    -Daolin

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    Initial testing and validation for 2 ethernet ports in u-boot, then further testing and validation in Linux.  Board will ultimately use WolfBoot for booter and Green Hills OS for product.

    Many thanks!!  

  • 0 in reply to Daryl Jarman
    Prodigy 60 points

    I should also clarify, u-boot will also be utilized in production for initial board staging, setting MAC addresses, transfer and storage of security certificates, all of that good stuff.

    I hope this makes sense.

  • 0 in reply to Daryl Jarman
    TI__Mastermind 40080 points

    Hi,

    We will need to verify but that version of the u-boot source tree may not support PRU ICSSG as there may not be support to load the firmware necessary for the PRU-ISSG to function. It may be possible in an upcoming SDK. 

    Best Regards,

    Schuyler

  • 0 in reply to Schuyler Patton
    Prodigy 60 points

    Many thanks for your help, please let me know.

  • 0 in reply to Daryl Jarman
    TI__Genius 16005 points

    Hello Daryl,

    Apologies for the delay in response. Currently I'm trying to check with the internal team on the following:

    1. Does U-boot support running PRU ICSSG ethernet

    2. What exactly are those mailbox messages you are seeing and why the message is not acknowledged

    I hope to get an answer/update by next week.

    -Daolin

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    Hi Daolin,  please don't stress, and your help is greatly appreciated.

  • 0 in reply to Daryl Jarman
    TI__Genius 16005 points

    Daryl,

    Some additional questions I forgot to ask in my last response:

    1. You stated initial testing of 2 ethernet ports in U-boot, are you targeting test the two ports from ICSSG1 or ICSSG0, or one from each? If it does not matter, I would suggest testing with just the ICSSG1 since we know ICSSG1 is already configured by default in the dts. 

    Additionally, do you plan to test each of the two ethernet interfaces separately in U-boot, one at a time, or both at the same time? You might already know this, but in U-boot only one ethernet interface can be active at a time due to it being single-threaded.

    2. Are you able to use an ICSSG1 ethernet port in U-boot without issue?

    3. I found that https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1325243/faq-am6548-how-to-get-pru-ethernet-working-in-u-boot-on-am65x-linux-sdk-9-1/5041846?tisearch=e2e-sitesearch&keymatch=ethact#5041846 indicates PRU ICSSG is supported in U-boot for AM65x, I'm still trying to verify if the same is applicable for AM64x.

    -Daolin

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    Hi Daolin, 

    For question 1:  final target will have 2 ports on ICSSG0 and ICSSG1.  Four ports in total.  Initial test is for 2 ports on ICSSG0.  The main reason for this is the custom backplane the the board will reside in.  The two ports for the ICSSG1 are signaled for backplane use only and have no external connections.  The 2 ports for ICSSG0 have external connections so testing can be done with normal network switches/computers that sort of thing.

    For question 2: Honestly, I don't think I ever tried it, for reasons in #1 above.

    For #3:  Thanks for the info!  I am using "U-Boot 2023.04" (from the version command), so I would think the patches should work.  I'll start diving into it and see what I can find out.  When you have further information, please let me know.

    Again, many thanks for your help.

    Daryl

  • 0 in reply to Daryl Jarman
    TI__Genius 16005 points

    Hi Daryl,

    I checked the u-boot commit log and see changes that have been added to support PRU ICSSG ethernet in U-boot on AM64x. If you have the latest ti-u-boot-2023.04 branch, you might not need those patches from the FAQ I shared earlier since those were meant to be applied on the U-boot that came with SDK 9.1

    https://git.ti.com/cgit/ti-u-boot/ti-u-boot/log/?h=ti-u-boot-2023.04&qt=grep&q=am64x 

    * configs: am64x_evm_a53: Enable ICSSG Driver MD Danish Anwar 2024-02-15 2 -0/+7
    * arm: dts: k3-am642-evm: Add ICSSG1 Ethernet support MD Danish Anwar 2024-02-15 2 -0/+144
    * arm: dts: k3-am64-main: Add ICSSG IEP nodes MD Danish Anwar 2024-02-15 1 -0/+24
    * soc: ti: pruss: Add support for AM64x MD Danish Anwar 2024-02-15 1 -0/+1

    I haven't received information back yet on what those mailbox messages are and why they might be failing when enabling the ICSSG0 clock. As the commit log above states, it appears only ICSSG1 Ethernet support was added and tested, and we have not extensively tested ICSSG0 (both in U-boot and Linux), mostly because our AM64x EVM only breaks out the ICSSG1 to physical ethernet ports. 

    Just to clarify, what were the exact steps you took to initially see that ICSSG0 clock is not enabled? Did you have to set the ICSSG0 ethernet port active in u-boot with "setenv ethact"? 

    P.S. Personally, for a project I've been working on, I have been trying to enable ICSSG0 in Linux with a daughterboard attached to the AM64x EVM but had also faced problems with MDIO clock, so I have great interest in the issue you are facing!

    -Daolin

  • 0
    TI__Genius 16005 points
    Daryl Jarman said:
    I am using the u-boot code "ti-u-boot-2023.04+gitAUTOINC+8366064208" that is part of the SDK download

    I also forgot to clarify with you, what SDK version is this u-boot associated with?

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    Hi Daolin, The failure I am seeing is from the icssg_prueth.c driver, with the following (line #727)

    ret = clk_enable(&prueth->mdiofck);
        if (ret) {
            dev_err(dev, "clk_enable failed %d\n", ret);
            return ret;
    }

    I was able to trace this to the "clk-sci.c" clock sources, into the mailbox code (mbox_send() and mbox_recv()).  From the initial inspection, the error seems to occur when attempting to enable an "mdio" clock.

    This appears to occur when walking through the device tree.

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    This is from the following SDK binary: 

    ti-processor-sdk-linux-am64xx-evm-09.02.01.10-Linux-x86-Install.bin.

    I hope this makes sense.

  • 0 in reply to Daolin Qiu
    Prodigy 60 points

    Hi Daolin,

    Many thanks for your help.  We have decide to continue this develop in Linux vs u-boot.  Other developers are writing their validation tests in Linux already.  Go ahead and cancel the mailbox message request, I have some understanding on what is going on there.  As I get into this, I might well startup another question thread on this topic from the Linux point of view.

    I hope this makes sense.  Again, many thanks for your help, if you are interested, I'll keep you appraised on how things go.

    Daryl

  • 0 in reply to Daryl Jarman
    TI__Genius 16005 points

    Hi Daryl,

    I appreciate your update on how things are going and letting me know you will be moving towards developing in Linux. Please keep me updated on any issues you may encounter when enabling the ethernet interfaces in Linux - on a separate ticket of course. I have particular interest in how your team will enable ICSSG0 in Linux (as I stated within TI we have only tested ICSSG1 before)!

    -Daolin