CCS: Checking Ethernet port IP on K2E EVM with PA EMAC example

If you look at the NDK example BIOS configuration, you will see that the PA module is integrated underneath the NDK setup.

The PA driver example is not meant for external network connectivity but only tests. The CPSW (EMAC) Single core example demonstrates the use of CPSW CSL APIs to  configure the ethernet switch on target (silicon and simulator). It further demonstrates  the use of CPPI/QMSS/PA LLDs to send/receive data through the ethernet switch using Host descriptors on any given DSP core. The example mostly runs in loopback mode. Internal loopback through SGMII and external loopback though PHY

Please refer to the FAQ for the driver for more details:

https://software-dl.ti.com/processor-sdk-rtos/esd/docs/06_03_00_106/rtos/index_device_drv.html#id130

Regards,

Rahul

Hi Rahul,

   Correct, PA module has been integrated on the NIMU example source code.

   Here we need to adding the MAC, IP & Ports in LUT Layer through we need to check like the PA example code to send/receive packets on Ethernet.

   Attached the NIMU source. We added Pa_addIP & Pa_addPort funcionalities to the NIMU source.

   Changes done only in below attached source .


   Console Log below in NIMU example source

[C66xx_0] QMSS successfully initialized
CPPI successfully initialized
PA successfully initialized

TCP/IP Stack 'Hello World!' Application

StackTest: using localIp
EMAC start ***
Pa_addIP returned error -18
Network Added: If-1:192.168.0.170

    Please verify & guide me to help in any way.

    Issue facing :

    1. For PA EMAC code, There is no network stack so we don't get IP address of our board
        
        Help me to get my IP address of our board
    (or)

    2. For NIMU code, Need to add MAC, IP & Ports on PA LUT layer

        Help me to how to use Pa_addIP, Pa_addPort() in this code.
Regards,
SanthanaKumarS

Hi Rahul,

  In this post, we attached the source file  NIMU example to adding MAC, IP & Ports in LUT table.

Attached file

4578.nimu_eth.c

Console Log:

[C66xx_0] QMSS successfully initialized
CPPI successfully initialized
PA successfully initialized

TCP/IP Stack 'Hello World!' Application

StackTest: using localIp

Add MAC Address success
Pa_addIP returned error -18

Please check this log Here we are facing adding the IP on the LUT is not taking it will returning error

if (hdr->status != PA_TBL_STAT_ACTIVE) {
ret = pa_INVALID_INPUT_HANDLE;

Mac & IP link is not propered, so we get this Please guide me to resolve the issues

Hi Santhana,

I see you have modified nimu_eth.c similar to PA emac examples to add IP and UDP port entries as well.

Fundamentally, we should not be adding a specific IP, UDP port entries in the NIMU layer, as the higher stack is implemented in NDK and if you add specific IP/UDP entries under NIMU.

The Add Mac is done under NIMU, just to filter out packets intended for the board to be reaching to NDK higher layer stack.

Why do you want to Add IP/UDP under NIMU? All the network stack would be implemented by NDK.

The Pa_addIP returned error code of -18, indicates there is some invalid MAC handle passed for ADD IP.

Please check if you are having all input arguments valid for Pa_addIP(). I do not suggest you updating the NIMU layer for IP/UDP.

Thanks.

Dear Aravind,
 
We correctly verify MAC handle on pa_addIP () so we can't go to add the IP & Port entries PA layer.

  Why we are updating NIMU layer means?

    Our requirement for adding MAC, IP & Port in LUT table & We need to send/receive the packets on EVM side.

Why we don't use the PA EMAC code?

    1. Assuming that this directory "CCS/pdk_k2e_4_0_16/packages/ti/transport/ndk/nimu" has the NDK full stack code.

In this NIMU example, the IP address is fixed and we are able to ping this board from the other system. Here we want to add the L3 LUT in PA. Is it possible to add the L3 LUT in this example (we did not find adding L3 LUT in this example). NIMU code able to find the IP address of our board,...

     NIMU code able to find the IP address of our board, that's the reason we moved to NIMU code for adding the IP & Port in L3, L4 layer.

    2. In this PA EMAC code, We can't able to find IP address of our EVM board  but we know the MAC address.

       Can you please support to finding the way of IP address on this board, then only We need to use the ethernet functionality?

Regards,
SanthanaKumarS

Hi Santhana,

1. You should be able to send/receive packets on the EVM without modifying anything in NIMU.

2. /ti/transport/ndk/nimu is *NOT* full stack code. This is just an ethernet hardware interface for NDK (Full NDK stack is implemented under NDK package outside the PDK folder). The NIMU transport layer should provide all the packets meant for the board to higher layer (NDK).

3. You do not need to add any IP/UDP entries for PA to send/receive packets using NDK. 

4. NIMU example, you can modify to get the IP address from the DHCP server if you do not want to use the Static IP for the example. Note that nimu_eth.c does not need any modification, for your need.

Please undo all the changes in nimu_eth.c and run the examples as it is. If your goal is to get the IP address for the board dynamically via DHCP server, please modify "StackTest()" function to have the DHCP for the example. 

Thanks.

Dear Aravind,

  I understood your points.Leave the NDK stack for don't need to add IP & Port entries on NIMU example.

Can you please provide support to help me out of below issues?

  1. Using PA emac example, We need to send/receive packets from Ethernet.

        We didn't saw any IP address on this PA example of our K2E board. Please help us to find the way for finding IP address of our board.

Regards,
SanthanaKumarS

Hi Santhana,

The PA examples are meant to show case how to add an entry to PA LUTs. (Not full network stack).

NDK examples (such as NIMU examples) are there to show case NDK APIs and usage. 

You would get the IP address for the board by running the NDK examples. (Via DHCP or Static - both should be doable via NDK- NIMU examples).

You can use NDK-NIMU examples (Possibly with no change to examples provided and nimu_eth.c) you should be able to send/receive packets from Ethernet.

Can you let me know, if there is an issue for you to use NDK-NIMU examples for sending/receiving packets from Ethernet?

Thanks,

Dear Aravind,

     Our requirment is to send/receive the packets on PA example or NDK example. but we need to add the LUT entries(L2,L3,L4) of MAC, IP & Ports in Packet accelerator.

     We know above your are telling PA example doesn't have network stack & NDK example to getting the IP address.

     In PA example, I know the entry for adding LUT, but we can't find the IP address of the board. Please tell this.

     In NDK example, We are able to getting the IP address of our board but we can't able to add IP & Port LUT entry except MAC is already added.

    Can you Please support to finding the IP address of our board in PA example. If it is not possible, then please help to add the IP & Port LUT entry in NDK example.
Regards,
SanthanaKumarS

Dear Aravind,

     Wishing you & your TI community a "Happy New Year".

     Any updates on this thread?

Regards,
SanthanaKumarS

Hi Santhana

Happy new year to you as well.

If you want to add an entry for IP address in PA LUT, you can add the entry after you get the IP address in the NIMU example.

Note that this is going to be modified NIMU example for your need.

If I understand it, you want to add the Board's IP address to allow the packets intended for the specific board? Wondering, why only MAC addresses is not enough as done in the NIMU example for your needs? (If you want to filter out only the packets meant for this specific EVM)?

Thanks

Dear Aravind,

   After bringing up IP address, then We called IP entry (ie) not able to add IP entry on PA LUT table.

   Getting same error on that PA invalid input handle.Verified input arguments passed through Pa_addMac, Pa_addIP.
 
Console Log:
  [C66xx_0] QMSS successfully initialized
CPPI successfully initialized
PA successfully initialized

TCP/IP Stack 'Hello World!' Application

StackTest: using localIp
Getting SGMII Link for mac port 0
Getting SGMII Link for mac port 0
Getting SGMII AutoNegotiationstatus for mac port 0
Network Added: If-1:192.168.0.170
Add MAC Address success
Pa_addIP returned error -18
ti.sysbios.heaps.HeapMem: line 429: assertion failure: A_invalidFree: Invalid free
xdc.runtime.Error.raise: terminating execution

   Attached code with my changes.Please verify & update your comments.

 3733.nimu_eth.c


   For PA EMAC example, Adding MAC, IP & Port entry on the PA LUT, but we can't find the IP address of our board to communicate with EVM board.

   So we choose the NIMU example, because We know the IP address , Only MAC entry is available not IP & Ports So we are trying to adding the LUT entry & proceed with our requirement.

   Please help me to getting IP address in PA EMAC example (or) Adding IP & Port LUT entry on the NIMU example?

Note:
   Again I am mentioning my requirements:
   Why we need to add the IP address because We need to add our MAC, IP & Port entry in PA LUT table to sent/receive packets from Ethernet side.

Regards,
SanthanaKumarS
 

Hi Santhana,

>> Getting same error on that PA invalid input handle.

- You would need to check if your system has any memory corruption or something else happening. You can single step into Pa_addIP function and trace the root of the error code (invalid handle = -18).

>>Again I am mentioning my requirements:
   >>Why we need to add the IP address because We need to add our MAC, IP & Port entry in PA LUT table to sent/receive packets from Ethernet side.

I am repeating my response here. You do not need to add your IP address to PA LUT to send and receive the packets from Ethernet side. The Out of box NIMU examples should be good for you to send/receive packets with an assigned IP address for the board. I still do not understand your need for this.

Again, note that PA LUT entries is meant for Receive side of the packets from ethernet and has nothing to do with the sending ethernet packets to the network.

Thanks

Dear Aravind,

  Getting error on PA invalid Input handle

    We are verified the Add_MACaddress(), Add_IPaddress & Add_Port() API to comparing with NIMU example & PA emac example.

    I don't think so why it's failing, Please verfiy my code & give your comments.

 "The Out of box NIMU examples should be good for you to send/receive packets with an assigned IP address for the board"

     Aravind, We are able to send/receive packets with an assigned IP address, but we need to adding the MAC, IP & Port in PA LUT to receiving the packets on ethernet side. NIMU example contains only MAC address but our requirement adding MAC, IP & PORT in PA LUT & receiving the packets on ethernet side.


    Above we are working parallel to adding IP & Port entry in PA LUT.

Aravind,
  If you think, it's not the right way to adding IP address in PA LUT on NIMU example, then Please help me to getting (EMAC_PORT0 & EMAC_PORT1) IP address of our board in PA emac example.

Regards,
SanthanaKumarS

Hi Santhana,

Again, like I mentioned before, PA examples are not full network stack examples. You would not be able to get an IP address for the board in PA emac example. NIMU examples (NDK) are the ones that integrates the board with NDK for network stack development. 

I mentioned this before, repeating the same:

On the PA LLD error, you need to single step in your application and identify why is the wrong handle seen by the driver. Please check if there are any memory corruptions in the system?

To debug the PA err:

Please identify the condition in Pa_addIp2() function, which is causing the return value to be "pa_INVALID_INPUT_HANDLE". Note that there are many paths to get to this error and you need to identify which one is resulting this.

Thanks

Dear Aravind,

    Can you please tell me the way, We need to receive packets from Ethernet side with adding MAC, IP & Port in PA LUT table.

We are stuck on both example, to get above requirement. So please guide me to go in a correct way to explore my requirement.

    Now we are debugging the issue on Pa_addIp() API to let me know the status to you.

Regards,
SanthanaKumarS

Hi Santhana,

Like I mentioned to you before, you do not need to add IP and Port entries to PA LUT to send and receive the packets. The NIMU layer already has the MAC entry added. 

The NDK higher layer does the network stack protocol processing. 

The NDK today does not support full capabilities of PA to be utilized. (There is no further plan to support it as well on NDK).

By adding further filters, I think you may be blocking packets to the NDK - I do not know what all operations may get impacted. Hence I suggested no change to NIMU layer for sending and receiving the packets. 

I do not get the reason for you to add the IP and UDP port entries to NIMU layer.

If you still want to add the IP and Port LUTs from NIMU, like you are doing now, it is up to you to proceed. 

As far as the PA Add IP API is concerned, you can single step and debug what is causing the invalid handle to be seen by the LLD.

If there is nothing more, please close the thread.

Thanks.

Dear Aravind,
 
   Debugged the issue, We are giving our inputs Please verfiy & add your comments.

   Using PROCESSOR-SDK-RTOS-K2E  (v06.03) in K2E development board.

   Flashing the NIMU example with prebuilt images

Source path: nimu/example/helloWorld/k2e/c66/bios/helloWorld.cfg

Testing:
   helloWorld.cfg ->

    Changes done in NIMU example pre-built code:
    /* Load the PA package */
    var Pa                             =     xdc.useModule('ti.drv.pa.Settings')
    Pa.deviceType = devType; (devType is K2E) // Aravind this line is added my me.

Console log:
[C66xx_0] QMSS successfully initialized
CPPI successfully initialized
PA successfully initialized

TCP/IP Stack 'Hello World!' Application

StackTest: using localIp
Network Added: If-1:192.168.0.170
Add MAC Address success
Add IPAddress success
Error obtaining a Tx free descriptor   ========> Facing this error log.
Add Port failed
Setup_PASS Failed

Aravind, Error log is coming depends on the nimu_internal.h ->

#define NIMU_MAX_NUM_TX_CMD_DESC        2u  /**< Maximum number of TX Command descriptors used by NIMU */

If you change to 1u, it's successfully adds MAC but not IP & PORTs.

if you change to 2u, its successfully adds MAC & IP but failing to adding Port entries.

if you change to 3u, After emac driver starts, it's hanging on. = > Aravind, Please verify the code & give your comments

Result:
   This code is passed by Linking with library ti.drv.pa:./lib/c66/ti.drv.pa2.ae66

So this code is accessed in pa/src/v1/pa.c
  
    Aravind, If we add the devType= K2E, library linking in PA2(Linking with library ti.drv.pa:./lib/c66/ti.drv.pa2.ae66) & its passed adding IP address  in LUT entry.

"I do not get the reason for you to add the IP and UDP port entries to NIMU layer."

   Aravind, Why we need to add the IP & UDP port entries in NIMU layer, because our requirement is used to receive the packets on ether-net side to how PA example receiving the packets on adding MAC, IP & Port entries in PA subsystem.

  Exactly We need to receive the packets on NIMU example like PA emac example receiving packets on the Ethernet side.

  Aravind, PA emac example does not have network stack So we chooses the NIMU example to adding MAC, IP & ports entries in PA subsystem to receiving packets from external PC to EVM.

Regards,
SanthanaKumarS

Hi Santhana,

1. For K2E, you should be using pa2 lib. 

2. It appears to me that the command descriptor that you sent to PA did not return back.. It appears to me that either you did not download the PA firmware or PASS is hung for some reason. Note that a bad CPPI descriptor can make the system hang. You should be checking if the PA ADD MAC (and any of your modifications) is successful, in the sense you should be getting the PA response (command response queue) and push it back to the Command Free queue and use Pa_forwardResult() API to check if the commands were successful. 

This sequence is available in the nimu_eth.c for Add Mac. Please verify them at your side.

You are running out of command descriptors and hence noticing the above behavior. 

3. I am not sure how to convey to you, you do not need to add IP and PORT entries for in NIMU layer. NDK is going to take care of it in software. The out of box NIMU library and NIMU examples are good for you to send/receive packets over ethernet.

Please note that I can't help you to review/debug your modifications. I can suggest to you the areas to look for, if needed.

In your case, I do not understand what are you trying to do and why you need to do something which is not needed.

Thanks

Hi,

"I am not sure how to convey to you, you do not need to add IP and PORT entries for in NIMU layer. NDK is going to take care of it in software. The out of box NIMU library and NIMU examples are good for you to send/receive packets over ethernet."

Please check my response below,

  Aravind, Why we need to add the IP & UDP port entries in NIMU layer, because our requirement is used to receive the packets on ether-net side to how PA example receiving the packets on adding MAC, IP & Port entries in PA subsystem.

  Exactly We need to receive the packets on NIMU example like PA emac example receiving packets on the Ethernet side.

  Aravind, PA emac example does not have network stack So we chooses the NIMU example to adding MAC, IP & ports entries in PA subsystem to receiving packets from external PC to EVM.

Hi Santhana,

In spite of this, if you are looking to update nimu_eth.c with IP and UDP Port LUT entries as well, Based on the previous information, you are not getting the command response back from PA sub system. 

Please note by adding (filtering specific IP and UDP port filters) to the Rx path, you would be blocking other packets to NDK stack. I think, it may hinder NDK operations.

Can you check/make sure below?

1. After Pa_addMac(), the Tx completion happens (you get the descriptor back to gTxCmdReturnQHnd queue)

2. And you get the response back from PA SS in gPaCfgCmdRespQHnd queue

3. Also, you forward the command response packet to PA SS using Pa_forwardResult() and there are no errors returned back from this API.

You need to repeat steps 1, 2 and 3 for Add IP and Add UDP as well - If the entries are not successful, you would have descriptors getting lost.

Please un plug the network cable while you do above, just to make sure there is no traffic while you are setting up PA and the descriptors involved are only the ones that are used to configure PA SS.

Thanks.

Dear Aravind,

    Thanks for understanding my needs in this post.

    Correct, We are not getting the command response back from PA sub system only in Adding port entries. but not in adding MAC & IP entry.

    Above 3 points followed correctly, but we are not getting response only on PA adding Port entries. Below Attached console log.

    Console log:
[C66xx_0] QMSS successfully initialized
CPPI successfully initialized
PA successfully initialized

TCP/IP Stack 'Hello World!' Application

StackTest: using localIp
Setup_Tx free Queue
Add_MACAddress Verifying count from Tx completion Q
Add_MACAddress Response from PA
Add_IPAddress Verifying count from Tx completion Q
Add_IPAddress Response from PA
Add IPAddress success
Adding Port  in PA LUT
Added Port entry in PA LUT
Send the command to the PA
Add_Port Verifying count from Tx completion Q
Timeout waiting for reply from PA to Pa_addMac command
Add Port failed

    Below attached my file, so please review the code & give your feedback.

7140.nimu_eth.c


Regards,
SanthanaKumarS

Hi Santhana,

I see you have modified a lot under nimu_eth.c and your modifications have errors/issues.

1. You have changed the PA configuration command response recycle to gRxFreeQHnd, instead of gTxCmdFreeQHnd as in the original release. You should do the correct recycle of the descriptor, otherwise, you will be running out of descriptors for sending the PA commands
2. I see under Add Mac you have added the command reply flow ID to be rxFlow Id -- this is not in the original nimu_eth.c - please verify at your side.
3. you have removed "goto return_fail;" statements, for Add_mac (also for other functions that you have added) which would create issues. Please compare your changes with original working code and debug your modifications.
4. I think you might have corrupted the host descriptor that you are sending to PA SS for the Add Port command. Please compare the descriptor with the working case (PA emac example) and check if you are setting any unexpected values. Note that a corrupted descriptor can hang the entire system.

I think, I have provided all the information and details for you to debug your changes. If there is nothing else to discuss, can you please close this thread? 

Thanks,

Hi,

  Thanks for your comments on my codes.
 
  We compared original nimu_eth.c & PA emac example (pa_mgmt.c) according to the API changes, We modified.
 
  Aravind, So above you mentioned first 3 out of 4 comments, We verified it's correctly added on our modified code.


  Aravind,

    Above last comments also verified in PA emac example , It's same on both side, So please give your more detailed tracing scenario to debug the issues?

    Error log is shown as "Timeout waiting for reply from PA to Pa_addMac command" in Add_Port()

   if (j == 1000) // -> If you change j == 100 its shown as a error in my code.
    {
        printf ("Timeout waiting for reply from PA to Pa_addMac command\n");
                ret_val = -1;
    }

   Aravind, Original nimu_eth.c file is j==1000 Issue is solved by changing j from 100 to 1000.

   1. I m not able to get response from PA command . Not entering into this function => if (Qmss_getQueueEntryCount (gPaCfgCmdRespQHnd) > 0)

   2. pinging board IP address from system is not working. Please give your comments.


Regards,
SanthanaKumarS

Hi Santhana,

If you have stopped the network traffic and have only these (PA commands) send to PA SS.. It is easier to debug. I am sure you have a corrupted descriptor sent to PA SS.

As suggested, earlier - have you done below?

Also, please make sure the descriptor needs to be write back to memory - please make sure there are no cache issues.

You can dump the host descriptor (every descriptor that is send to PA) and check if any values are unexpected. You can also compare the descriptor from nimu modifications that you have with PA examples which are working for Add MAC/IP/PORT.

As far as PA is concerned, it does not matter if it comes from EMAC example or NIMU - All that matters is descriptor that has information for PA.

"pinging board IP address from system is not working. Please give your comments."

This was exactly my suggestion earlier, NDK may not work if you add only specific IP and PORT packets to be sent to NDK. During Ping, there is no IP address and the packets from NIMU would not reach NDK, if your entry is only specific MAC/IP/UDP match in PA.

Also, it appears like you you had removed Broad cast MAC entry removed, which will hinder ping operations. Also, you should not remove the entries that were added by default by nimu_eth.c, 

Still, I am not clear on what you are doing and why you are adding IP/PORT, entries, which you don't need to do for NIMU. The default nimu is good enough to send and receive packets, Only tweaks we may need to do is to increase the number of Tx/Rx descriptors if there are high packet traffic.

Thanks.

Dear Aravind,

 Please give your comments to solve the issue on this thread. Then I am able to close both the threads on my side including this thread.

  => Please verify & put your inputs.

Regards,

SanthanaKumarS

Dear Aravind,

   Thanks for your support. You can close this thread & related thread also. My issue is solved. We able to get packets on linux side.