• State
  • Locked Locked
  • Replies 30 replies
  • Subscribers 99 subscribers
  • Views 2038 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3359: ICSS_EMAC on AM3359

Gaurav Kumar Singh
Prodigy 10 points

Part Number: AM3359
Other Parts Discussed in Thread: TMDSICE3359,

Hi All,


We are working on creating an User layer application where we want to use the PRU with Time Triggered sent functionality.

For this we are using the TMDSICE3359 with Linux User Space ICSS_EMAC sample application as the reference.

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_PRU-ICSS_PRU_ICSSG.html#icss-emac

 

This application provides the API reference and libraries to initialize the PRU with the ICSS_EMAC firmware and use the UIO driver to communicate with the firmware to send and receive frames.

As a first step we used the sample application that comes pre-compiled in the TI-SDK images for each Processor SDK which works fine. Now I compile this test application with some modifications where I face following problems with the application

  1. First problem is with the correct initialization of the device
    1. With the cross-compiled icss_emac test application, the PRU does not seem to be initialized correctly and I don’t see any communication on the network at all though the application does not report any errors
    2. If I use the pre-compiled application from the TI-SDK image first and then run the compiled application after the pre-compiled application, then the compiled application also works fine and packets are sent correctly.
  2. The second problem is with the Ethernet Link Status interrupt
    1. The test application provided by the TI waits for Link interrupt before starting any communication
    2. I have noticed that the link status interrupt only happens if we plug-out and plug-in the wire after the application has started
    3. This is not ideal for the type of application I am working as we can’t be there always to plug-out and plug-in during the operation or after a system restart
    4. To overcome this in test application, I modify the test application to use the link status register instead to check the link status instead of waiting for the interrupt
      1. Checking link status like this does not allow sending frames which are higher than 60 bytes
      2. If I send a packet >60 bytes, sometimes there are no frames in the network and sometimes the first frame keeps repeating forever with no interval between frames
      3. If I stop the application and start it again, the packets transmission stops but I see that corrupt packets are sent into the network
      4. I assume this happens because the MAC buffers are disoriented somehow.

     Any information and support would be really appreciated.

    The links to the Processor SDK and TI PDK I use is given below.

    Processor SDK Linux RT -> http://software-dl.ti.com/processor-sdk-linux-rt/esd/AM335X/latest/index_FDS.html

    am335x-evm-linux-rt-06.03.00.106.img - http://software-dl.ti.com/processor-sdk-linux-rt/esd/AM335X/latest/exports/am335x-evm-linux-rt-06.03.00.106.img.zip

    TI PDK -> https://git.ti.com/cgit/processor-sdk/pdk/

    In this case I use the branch - DEV.PROCESSOR-SDK_06.03.00.106


    Regards,

    Gaurav

    • 0
      TI__Intellectual 1805 points

      Hello Gaurav,

      Gaurav Kumar Singh said:
      If I use the pre-compiled application from the TI-SDK image first and then run the compiled application after the pre-compiled application, then the compiled application also works fine and packets are sent correctly.

      >> This seems to be issue with Driver/FW initialization.

      I am looking into it, Will revert back to you with more information by Monday.

      Regards,

      Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi Mohan,

      Thanks, Do you have any information for me already?

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Hello Gaurav,

      We apologize for the delayed response. To confirm on 1):

      Are you making any changes to the icss_emac driver? Or are you building icss_emac through Yocto recipes as per http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_PRU-ICSS_PRU_ICSSG.html#icss-emac , but the unmodified driver is not initializing the PRU as expected?

      Please note that the git location is slightly out of date in the documentation. The version of icss_emac that was used in processor SDK 6.3 is here: https://git.ti.com/cgit/processor-sdk/pdk/tree/packages/ti/drv/icss_emac?h=DEV.PROCESSOR-SDK_06.03.00.106 

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi Nick,

      Thanks for the response.

      No I have not made any changes in the first case. Although I do not use the yocto recipes. I am cross-compiling the the driver using the PROCESSOR-SDK-LINUX-RT-AM335X 06_03_00_106

      http://software-dl.ti.com/processor-sdk-linux-rt/esd/AM335X/latest/exports/ti-processor-sdk-linux-rt-am335x-evm-06.03.00.106-Linux-x86-Install.bin

      I have used the https://git.ti.com/cgit/processor-sdk/pdk/tree/packages/ti/drv/icss_emac?h=DEV.PROCESSOR-SDK_06.03.00.106 for the icss_emac source

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Hello Gaurav,

      Got it. Just to make sure we are not making any obvious mistakes, please try the yocto build as documented in section "Rebuild ICSS-EMAC in Linux User Space" of ICSS-EMAC userspace docs:
      See Building the SDK for setting up build environment, and Recipes for guidance on how to do a bitbake.

      Once we get that known good method working, you can start working from there.

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi Nick,

      The yocto method is time consuming which is the reason I used the cross-compilation. 
      Another reason is that we do not want to integrate the target application build as part of the yocto.

      I can spend some time to get the yocto build ready but I would assume that the we would come to the same point again since we require the application to be cross-compiled using the PROCESSOR-SDK-LINUX-RT-AM335X 06_03_00_106 environment.

      I have used this method for building the same application for AM574x and it worked there. 

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Doing a full yotco build of the entire SDK takes hours. But the steps in our documentation above will be much faster, since it should only build the icss-emac software. Please try the documented method first, since that should be a "known good" state. If the documented method does not work, please post the exact steps you took so we can replicate your observations.

      Regards,

      Nick

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      Apologies for the delayed response.

      Can you please try to get the memory details of Port TX Queue Descriptors which are available in DRAM0 for Port1 and DRAM1 for port2 Queue descriptors.

      Offset to the Queue descriptor(PORT_QUEUE_DESC_OFFSET) 0x1EC0 from PRU-ICSS DRAM0/1 base address. 

      There are 4 Queue descriptors per port and each queue descriptor will be 8 bytes so you need to dump 4 queue descriptors for port1 and 4 queue descriptors for port2 totally 64 bytes. 

      Once we get the Queue descriptors we can confirm the state of PRU FW weather the pointer are corrupted or not and will proceed with further debugging,

      Can you please get the above memory details before and after the issue so that it will be more efficient to comment on FW initialisation.

      Regards

      Mohan

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi Nick,

      Sorry for a delayed response, I got caught with other work in meantime.

      I did manage to test it using the yocto build and the reference application works fine. Can you please advise how can I go forward with building the custom application?

      I would guess the best way would be to create a yocto recipe but is there any alternative to it?

      Regards,

      Gaurav

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi Mohan,

      Sorry for the late response. I would try to perform the tests this week and share with you the information you requested. 

      Regards,

      Gaurav

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi ,

      The descriptors you asked is to verify the initialization problem or the Link status problem?

      Regards

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      Gaurav Kumar Singh said:
      I assume this happens because the MAC buffers are disoriented somehow.

      We can confirm weather the queue descriptors are properly initialized before the issue and if you get the Queue descriptors after the issue will check weather the queue descriptors are in a bad state or good state.

      Regards

      Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi ,

      Thanks for the clarification. I did some tests to capture the portQueueDescOffset and I do see some differences

      Let me first explain the setup. I dump the values for each queue from the 0x1EC0 using the constant ICSS_EMAC_FW_PORT_QUEUE_DESC_OFFSET which is assigned to the pDynamicMMap->portQueueDescOffset variable.

      I print the values only from DRAM0 (dataRAM0BaseAddr)

      First case when I wait for the Link interrupt, I get following values before sending a packet

      Queue Id 0 Offset 7872
      4128 4128 0 0
      Queue Id 1 Offset 7880
      4516 4516 0 0
      Queue Id 2 Offset 7888
      4904 4904 0 0
      Queue Id 3 Offset 7896
      5292 5292 0 0
      and following values after 
      Queue Id 0 Offset 7872
      4128 4128 0 0
      Queue Id 1 Offset 7880
      4516 4516 0 0
      Queue Id 2 Offset 7888
      4904 4904 0 0
      Queue Id 3 Offset 7896
      5328 5328 0 0

      The second case where I don't wait for the Link interrupt instead wait only for the ICSS_EMAC_testHandle->object)->linkStatus[0] to become 1 I get following values before sending

      Queue Id 0 Offset 7872
      4128 4128 0 0
      Queue Id 1 Offset 7880
      4516 4516 0 0
      Queue Id 2 Offset 7888
      4904 4904 0 0
      Queue Id 3 Offset 7896
      5292 5292 0 0

      and after

      Queue Id 0 Offset 7872
      4128 4128 0 0
      Queue Id 1 Offset 7880
      4516 4516 0 0
      Queue Id 2 Offset 7888
      4904 4904 0 0
      Queue Id 3 Offset 7896
      5292 5676 0 0

      If I compare these values, I see that in both cases the values of the descriptors before sending the packets is same for each queue.

      After sending the packets the values for queue 3 changes as I use that. In case of Link interrupt it is 5328 5328 0 0 but in case of waiting for link status it is 5292 5676 0 0 which looks odd to me. Can you please help me understanding these and share your feedback to what might be going wrong?

      Regards,

      Gaurav

       

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      If read pointer and write pointer are equal then it is expected that the TX task has transmitted all the packets you pushed into the queue.

      If both are not equal means PRU TX task is not transmitted the packets. As the write pointer is moved properly with in range, wanted to check following things.

      1) Can you please check PRU statistics. You will find the statistics in DRAM0 for PRU0 and  DRAM1 for PRU1. 

           For your reference Statistics will start from DRAM0 + 0x1F00 offset for PRU0 and  DRAM1 + 0x1F00 offset for PRU1. Can you please try       to get below statistics. Each will be 4 bytes.

           a) TX_BC_FRAMES_OFFSET      0x1F00          Number of Transmitted Broadcast Frames

           b) TX_MC_FRAMES_OFFSET     0x1F04           Number of Transmitted Multicast Frames

           c) TX_UC_FRAMES_OFFSET     0x1F08           Number of Transmitted Unicast Frames

      2) Can you please try to get the value at the PORT_STATUS_OFFSET. This is one byte value and used to check weather           port link is up or down in firmware. Can you please try to get this value in both the cases.

           For your reference PORT_STATUS_OFFSET is at DRAM0 + 0x1F98 for PRU0 and  DRAM1 + 0x1F98 for PRU1.

      Regards

       Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi ,

      So following are values:

      In working setup with waiting for link Status interrupt:

      Before starting the transmission

      Port Status 0x1
      txBcast:0x0
      txMcast:0x0
      txUcast:0x0

      After transmission

      Port Status 0x1
      txBcast:0x0
      txMcast:0x64
      txUcast:0x0

      In non-working setup with waiting for linkStatus flag the values for txXXX are 0 before and after the transmission loop:

      Port Status 0x1
      txBcast:0x0
      txMcast:0x0
      txUcast:0x0

      I send 100 packets so txMcast 0x64 is correct. 

      Can you please let me know which part do you suspect as creating this problem?

      I have tried in the past to add delays as well before sending the frames when I check the linkStatus flag but that also does not work.

      Let me know your observations here.

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      Port status has updated and the number of packets you sent to FW seems to be not transmitted. 

      I am suspecting Port link, Port speed and some of the PHY related initializations which are done in ISR callback are not being done and you are trying to send the packages to PRU port.

      Can you try adding code in the test application similar to Link ISR(ICSS_EmacPollLink) when you detect link status is up.

      Can you reverify the code changes you done to the application. If possible can you share the changes you made to test application, So that will review and give feedback.

      Regards,

      Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi

      I have attached the patch file for the changes I did. Please review and let me know.

      /cfs-file/__key/communityserver-components-multipleuploadfilemanager/da77d889_2D00_361e_2D00_430a_2D00_a0ab_2D00_2fa501b530fa-482425-complete/ti_5F00_patch.txt

      I will check the changes you mentioned and let you know.

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      I am unable to open the patch file you have provided and it says page not found. Can you please share the file again. 

      Regards,

      Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Fullscreen ti_patch.txt Download 
      diff --git "a/pdk/packages/ti/drv/icss_emac/test/src/test_common_utils.c" "b/pdk/packages/ti/drv/icss_emac/test/src/test_common_utils.c"
index 5ecfd5d4..6a733ec1 100644
--- "a/pdk/packages/ti/drv/icss_emac/test/src/test_common_utils.c"
+++ "b/pdk/packages/ti/drv/icss_emac/test/src/test_common_utils.c"
@@ -3011,7 +3011,7 @@ Void ICSS_EMAC_testTaskPruss1(UArg a0, UArg a1)
     
 #ifndef SWITCH_EMAC
 #if defined(icev2AM335x) || defined(iceAMIC110)
-        while (!ICSS_EMAC_testLinkIsrPru1Eth0)
+        while (((ICSS_EmacObject*)ICSS_EMAC_testHandle->object)->linkStatus[0] != 1)
         {
             PRINT("ICSS_EMAC_testTaskPruss1: LINK IS DOWN,skipping PRU1 ETH0\n");
         }
@@ -3045,7 +3045,28 @@ Void ICSS_EMAC_testTaskPruss1(UArg a0, UArg a1)
             txArgs.portNumber = 1;
             txArgs.queuePriority = 3;
             txArgs.srcAddress = &ICSS_EMAC_testPkt[0];
-        
+            uint32_t dataRAM0BaseAddr = (((ICSS_EmacHwAttrs const *)ICSS_EMAC_testHandle->hwAttrs)->emacBaseAddrCfg)->dataRam0BaseAddr;
+            uint32_t dataRAM1BaseAddr = (((ICSS_EmacHwAttrs const *)ICSS_EMAC_testHandle->hwAttrs)->emacBaseAddrCfg)->dataRam1BaseAddr;
+            ICSS_EmacFwDynamicMmap *pDynamicMMap = (&((ICSS_EmacObject*)ICSS_EMAC_testHandle->object)->fwDynamicMMap);
+            ICSS_EmacFwStaticMmap *pStaticMMap = (&((ICSS_EmacObject*)ICSS_EMAC_testHandle->object)->fwStaticMMap);
+            uint32_t qCount = 0U;
+            uint32_t temp_addr = 0U;
+
+            volatile uint8_t *portStatusPtr=NULL;
+            temp_addr = ((((ICSS_EmacHwAttrs const *)ICSS_EMAC_testHandle->hwAttrs)->emacBaseAddrCfg)->dataRam0BaseAddr + pStaticMMap->portStatusOffset);
+            portStatusPtr = (uint8_t*)(temp_addr);
+            printf("*****************************BEFORE***********************************\n");
+            for (qCount = 0U; qCount < pDynamicMMap->numQueues; qCount++)
+            {
+                uint16_t    queue_desc_offset = pDynamicMMap->portQueueDescOffset + (qCount * 8U);
+                uint16_t* values = (uint16_t *)(dataRAM0BaseAddr + queue_desc_offset);
+                printf("Queue Id %d Offset %d\n",qCount, queue_desc_offset);
+                printf("%d %d %d %d \n", values[0],values[1],values[2],values[03]);
+            }
+
+            printf("Port Status 0x%x\n", *portStatusPtr);
+            ICSS_EMAC_testGetPruStats(1, ICSS_EMAC_testHandle);
+
             for (count=0;count < ICSS_EMAC_TEST_PKT_TX_COUNT;count++)
             {
                 if(((ICSS_EmacObject*)ICSS_EMAC_testHandle->object)->linkStatus[0] )
@@ -3054,6 +3075,7 @@ Void ICSS_EMAC_testTaskPruss1(UArg a0, UArg a1)
                     {
                         ICSS_EMAC_testPacketRcvdPort0 = 1;
                         ICSS_EmacTxPacket(&txArgs, NULL);
+                        SLEEP(100);
                         while(!ICSS_EMAC_testPacketRcvdPort0)
                         {
                             SLEEP(100);
@@ -3061,6 +3083,17 @@ Void ICSS_EMAC_testTaskPruss1(UArg a0, UArg a1)
                     }
                 }
             }
+            printf("*****************************AFTER***********************************\n");
+            for (qCount = 0U; qCount < pDynamicMMap->numQueues; qCount++)
+            {
+                uint16_t queue_desc_offset = pDynamicMMap->portQueueDescOffset + (qCount * 8U);
+                uint16_t* values = (uint16_t *)(dataRAM0BaseAddr + queue_desc_offset);
+                printf("Queue Id %d Offset %d\n",qCount, queue_desc_offset);
+                printf("%d %d %d %d \n", values[0],values[1],values[2],values[03]);
+            }
+
+            printf("Port Status 0x%x\n", *portStatusPtr);
+            ICSS_EMAC_testGetPruStats(1, ICSS_EMAC_testHandle);
             /* Sending Unicast packets in Non-Promiscuous Mode. The eth should not be able to recieve packets as destination address != eth own address*/
             PRINT("\n\n\n============================================================");
             PRINT("\n ICSS_EMAC_testTaskPruss1: Testing NonPromiscuous Mode ");

    • 0 in reply to Gaurav Kumar Singh
      TI__Intellectual 1805 points

      Hi Gaurav,

      Your code changes looks ok and there is no issue in checking the linkStatus[0].

      The linkStatus[0] is only updated in link ISR and in your case if the linkStatus[0] is becoming 1 means the ISR is called, Can you confirm this weather the link ISR(ICSS_EmacPollLink) is called and returned successfully.

      After servicing the link ISR a call back function will be called and that function will increment the ICSS_EMAC_testLinkIsrPru1Eth0. So can we check weather ICSS_EMAC_testLinkIsrCb() is called in your case and also can we check weather this call back function is registered or not by the time when link interrupt is generated.

      Regards,

      Mohan

    • 0 in reply to mohanreddy
      Prodigy 10 points

      Hi ,

      Thanks for the review. I checked the ICSS_EmacPollLink and ICSS_EMAC_testLinkIsrCb functions and they seems to be called only in case when the ethernet cable is plugged after starting the application.

      This is not our use-case. In general the Ethernet Cable should be connected always and the Application should be able to detect the presence of existing network connection.

      I also did some more test with the original application without my modifications and I see following.

      - After a powercycle of the board, if the cable is already plugged in and I start the application for the first time after boot, the application detects the presence of network cable correctly and sends frames. In this case the ISR are also called.

      - Now if I close the application using ctrl+c and restart the application without removing the cable, the application does not detect the presence of the cable and wait with msg "ICSS_EMAC_testTaskPruss1: LINK IS DOWN,skipping PRU1 ETH0". And the ISR are not called

      - If the cable is unplugged and plugged back while the application is already running with the above msg then the ISR is generated and the link is detected

      Based on the above test I think that some register configuration is not cleaned up during the device initialization and is only cleaned after a device powercycle or unplugging and replugging the device. So if we powercycle the device the registers are clean and the ISR is generated even if the cable is already plugged.

      Can you please do some testing for this on your end and let me know?

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      Prodigy 10 points

      Hi ,

      Can you please let me know if you would be able to test this on your end as well?

      This is quite critical now as the customer who is planning to use AM3359 wants this feature to be working and waiting for a long time to get this resolved.

      Speeding things up would help us a lot.

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      Prodigy 10 points

      Hi ,

      Any updates for me already?

      Regards,

      gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Hello Gaurav,

      Apologies for the delayed response. I will take a look at running tests tomorrow and get back to you.

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      TI__Guru** 110050 points

      Hello Gaurav,

      I recently moved offices and did not bring the right hardware to run tests - next week is the earliest I'll have access to hardware. Let me see if Mohan can do anything in the meantime.

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi ,

      Do you have some update for me

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Hello Gaurav,

      Thank you for the ping through email to ressurect this thread. I'm signing off for the day, putting this response so your thread moves to the top of my TODO list tomorrow.

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi Nick,

      Do you have any update for me?

      The customer is waiting for quite some time now and if we do not provide a solution to this problem soon they would be forced to choose a different chip other than TI.

      If you can support us to resolve this issue then we have more chances that the customer uses TI AM3359 chips for their product.

      Regards,

      Gaurav

    • 0 in reply to Gaurav Kumar Singh
      TI__Guru** 110050 points

      Hello Gaurav,

      I tried to replicate your tests, but ran into hardware issues.

      Is the customer usecase to be constantly reloading the program? I would not be surprised if hitting Ctrl-C during the test application and reloading it was not tested by whoever ported ICSS_EMAC from RTOS to Linux. Support is going to be limited from a "understanding what the Linux drivers are doing in the background" standpoint. Mohan in previous responses supports the RTOS driver, but at this point in time we do not have a Linux developer we can rely on who understands what the code is doing in the background.

      Also, please note that Linux Processor SDK 6.3 is the last AM335x SDK that supports ICSS_EMAC. As of October 6 2021, the AM335x Linux Processor SDK 7.3 still mentions ICSS_EMAC. However, the dev team has told me they are not supporting ICSS_EMAC in SDK 7.3. I expect them to update the documentation sometime in the next couple of weeks.

      Regards,

      Nick

    • 0 in reply to Nick Saulnier
      Prodigy 10 points

      Hi Nick,

      The usecase for the linkstatus is to allow to communicate without plugging out and replugging the wires all the time.

      Does this mean that ICSS_EMAC would not be usable on AM3359 boards in future or are they going to have fixed functionality which the users can't modify?


      Regards,

      Gaurav