This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5716: PRU-ETH unavailable, Can not Send/Recv any packet.

Part Number: AM5716

Tool/software:

Hi TI expert,

I am running SYS/BIOS on DSP, using PRU-ETH for sending/receiving packets, and using IPC for communication with the A15 core(host running linux in a15). Under certain circumstances, the issue arises where PRU-ETH becomes unavailable: it can no longer send or receive any packets. However, when I called the function

int32_t ICSS_EmacTxPacket(const ICSS_EmacTxArgument *txArg, void* userArg)
 in the DSP program, I received a return value of 0. The function is located at "processor_sdk_rtos_am57xx_09_02_00_00/pdk_am57xx_1_0_21/packages/ti/drv/icss_emac/src/icss_emacDrv.c". Powering off and restarting the board does not resolve the issue.

One confirmed behavior that triggers the issue is executing a command while the host program and DSP program are running normally:

echo 40800000.dsp > /sys/bus/platform/drivers/omap-rproc/unbind
echo 40800000.dsp > /sys/bus/platform/drivers/omap-rproc/bind

The “BUG” is triggered.

I conducted the following tests:

```bash
root@SunGrow:~# ifconfig eth3 down
[ 4695.832319] prueth pruss2_eth eth3: Link is Down
[ 4695.863972] pruss 4b280000.pruss: unconfigured system_events[63-0] = 00600000,08a00000
[ 4695.879840] pruss 4b280000.pruss: unconfigured host_intr = 0x000002aa
[ 4695.892873] remoteproc remoteproc3: stopped remote processor 4b2b8000.pru
[ 4695.908040] net eth3: stopped
root@SunGrow:~# ifconfig eth3 up
[ 4708.931205] remoteproc remoteproc3: powering up 4b2b8000.pru
[ 4710.965959] remoteproc remoteproc3: loading /lib/firmware/ti-pruss/am57xx-pru1-prueth-fw.elf successfully!    drivers/base/firmware_loader/main.c L349
[ 4710.993076] remoteproc remoteproc3: Booting fw image ti-pruss/am57xx-pru1-prueth-fw.elf, size 6952
[ 4711.012099] pruss 4b280000.pruss: configured system_events[63-0] = 00600000,08a00000
[ 4711.028863] pruss 4b280000.pruss: configured intr_channels = 0x0000032a host_intr = 0x000002aa
[ 4711.048084] remoteproc remoteproc3: remote processor 4b2b8000.pru is now up
[ 4711.076954] net eth3: started
[ 4711.080389] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
[ 4711.089853] prueth pruss2_eth eth3: drivers/net/phy/phy.c L74 Link is Up-- - 100Mbps/Full - flow control off
[ 4711.114863] 8021q: adding VLAN 0 to HW filter on device eth3
[ 4711.126955] IPv6: ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
root@SunGrow:~# ethtool -S eth3
NIC statistics:
     txBcast: 0
     txMcast: 0
     txUcast: 0
     txOctets: 0
     rxBcast: 0
     rxMcast: 0
     rxUcast: 0
     rxOctets: 0
     tx64byte: 0
     tx65_127byte: 0
     tx128_255byte: 0
     tx256_511byte: 0
     tx512_1023byte: 0
     tx1024byte: 0
     rx64byte: 0
     rx65_127byte: 0
     rx128_255byte: 0
     rx256_511byte: 0
     rx512_1023byte: 0
     rx1024byte: 0
     lateColl: 0
     singleColl: 0
     multiColl: 0
     excessColl: 0
     rxMisAlignmentFrames: 0
     stormPrevCounterBC: 0
     stormPrevCounterMC: 0
     stormPrevCounterUC: 0
     macRxError: 1
     SFDError: 0
     defTx: 0
     macTxError: 0
     rxOverSizedFrames: 0
     rxUnderSizedFrames: 0
     rxCRCFrames: 0
     droppedPackets: 0
     txHWQOverFlow: 0
     txHWQUnderFlow: 0
     emacMulticastDropped: 0
     emacVlanDropped: 0
```

Then I connected eth3 to the laptop's Ethernet port and used a tool to send packets to eth3. The Wireshark capture is as follows:

However, when using ethtool -S eth3 again, I got the same result.

  • Hi,

    PRU-Eth support is descoped from SDK 9.2 onwards. Please see the release notes for further referance.

    Regards,
    Tanmay

  • Hi, Tanmay

    Sorry for forgetting to provide the SDK version information. 

    SDK-LINUX_version=06.03.00.106 

    SDK-RTOS_version=09.02.00.00(I have also tested other versions: 08.01.00.09)

    The PRU firmware provided by the 08.01.00.09 version of the SDK_linux file system is used to address the issue mentioned in theAM5746: RX_ERROR_OFFSET conditions.

    I have read the release notes but couldn't find any information that explains the cause of the issue I encountered.

  • Hi,

    The setup is not very clear to me. From my understanding, this is what is happening:

    • PRU are running the PRU ETH binaries
    • A15 is connecting to the PRU cores and creating the eth3 interface
    • A15 is also running an IPC application forwarding traffic to DSP
    • DSP is running IPC application receiving the PRU traffic

    Can you confirm this is what is happening.

    Regards,
    Tanmay

  • I apologize for any confusion caused by my vague description.

    The following is the configuration when my application is running normally:

    • PRU are running the PRU ETH binaries
    • DSP is receiving/sending traffic using  ICSS EMAC low level driver.
    • DSP is runing ethercat stack, but I believe the issue I encountered is unrelated to the EtherCAT protocol stack.
    • DSP is running IPC application.
    • A15 is also running an IPC application.
    • During the Linux startup of A15, eth3 is disabled: ifconfig eth3 down.
    • IPC between A15 and DSP is used to transmit application data as well as instructions.

    In specific situations (the exact cause is uncertain), PRU-ETH may be unavailable, preventing it from sending or receiving packets.

    The following are the steps I took while testing the availability of PRU-ETH:

    • A15 is connecting to the PRU cores and creating the eth3 interface.
    • Connect eth3 to the laptop's Ethernet port.
    • Sending packets on a laptop
    • Use ethtool to check the statistics for eth3.

  • Hello zhijie 

         what's the status of this issue?

    Regards

       Semon

  • The issue is still troubling me, but when I switch to another PRU-ETH network interface, I almost never encounter this problem again.

  • Hi,

    So this is only seen on multiple hardwares?

    Regards,
    Tanmay

  • Hi, Tanmay

    I didn’t quite understand what you meant by “multi-hardware,” but this issue still troubles me because I need to use two PRU-ETH interfaces simultaneously.

    Regards,

    Zhijie

  • Today, I made some new discoveries that might better describe the issue I am encountering. When using PRU-ETH to send packets on the DSP, the function ICSS_EmacTxPacketEnqueue in the source code file icss_emacdrv.c reported an error with the return value: ICSS_EMAC_ERR_TX_OUT_OF_BD. The comment provided is as follows:

    #define ICSS_EMAC_ERR_TX_OUT_OF_BD                     (ICSS_EMAC_SWITCH_ERROR_MAJOR + 8u)
    /**< Tx ran out of Buffer descriptors to use.                               */

    At the same time, I added debugging logs, and the specific function is as follows:

    /*Check if queue is full and there is an wrap around */
    #if defined(ICSS_EMAC_DEBUG) && (ICSS_EMAC_DEBUG > 2)
        printf_INFO("P+%d,HWREG(0x%x) = 0x%x,wr_ptr=0x%x,size=0x%x,rd_ptr=0x%x, desc_off=0x%x\n",portNumber, temp_addr, temp, queue_wr_ptr, size, queue_rd_ptr, txQueue->buffer_desc_offset);
    #endif
        if (((queue_wr_ptr + 4U) % size) == 0U)
        {
            if (queue_rd_ptr == txQueue->buffer_desc_offset) /* Since queue is not starting from 0. */
            {
                txQueue->qStat.errCount++;
                if (emacMode == 0U)
                { /*Switch Mode*/
                    temp_addr = ((((ICSS_EmacHwAttrs *)icssEmacHandle->hwAttrs)->emacBaseAddrCfg)->dataRam1BaseAddr + txQueue->queue_desc_offset + 4U);
                    HWREGB(temp_addr) = 0;
                }
                else
                {
                    temp_addr = (pruSharedMem + txQueue->queue_desc_offset + 4U);
                    HWREGB(temp_addr) = 0U;
                }
                hostStatPtr->txDroppedPackets++;
                printf_INFO("--ICSS_EMAC_ERR_TX_OUT_OF_BD(-1)=0x%x, M:0x%x + 8 \n", ICSS_EMAC_ERR_TX_OUT_OF_BD, ICSS_EMAC_SWITCH_ERROR_MAJOR);
                return ((int32_t)ICSS_EMAC_ERR_TX_OUT_OF_BD); /* No space in queue */
            }
        }

    The debug logs at the time of the error are as follows:

    P+1,HWREG(0x4b281ed8) = 0x162c14ac,wr_ptr=0x162c,size=0x1630,rd_ptr=0x14ac, desc_off=0x14ac

  • The issue is still troubling me, but when I switch to another PRU-ETH network interface, I almost never encounter this problem again.

    Hello Zhijie

        is there any difference for the two network interface? why one is good, and another can't work

    Regards

       Semon

  • As far as I know, the two network interface are not different.
    The information I provided earlier spans a long period and, due to my limited understanding of the issue at the time, may not accurately describe the problem. 

    I have started a new thread to explain the issue. see PRU-ETH unavailable in DSP core, Can not Send/Recv any packet


  • As far as I know, the two network interface are not different.
    The information I provided earlier spans a long period and, due to my limited understanding of the issue at the time, may not accurately describe the problem. 

    I have started a new thread to explain the issue. see PRU-ETH unavailable in DSP core, Can not Send/Recv any packet

    Hello Zhijie

        so we can discuss in the new case, this case will be closed

    Regards

       Semon