Receiving Packets from NETCP Packet accelerator for EVM using PA LLD API's

Hi Santhana,

                                                    1             /* Input EMAC port = dont care */ ----> Here we changed 0 -> 1 Why reason 0-> Host port 0 1-> Ethernet port1

It appears to me that you need to go through pa.h for EMAC port numbers. Note, that EMAC_PORT number '0' does not mean PC. When you change the PORT number to 1, it means only the packets from EMAC_PORT_0 would be matching the lookups. I have provided the pa.h snippet 

You can also check if the GbE switch statistics and make sure it is not dropping the packets due to some errors in the ingress packets that you are sending from your UDP packet generator.

As an intermediate step (debug), you can always have the match rule for the packet to route to a Host Queue, to make sure that stage of PA rules that you have added is good and is working as intended. 

Suggest to go step by step for adding the PA MAC, IP and UDP rules.

Snippet from pa.h file:

/**

* @defgroup paEmacPort Ethernet MAC port
* @ingroup palld_api_constants
* @{
*
* @name Ethernet MAC port
*
* @brief The module user specifies the Ethernet MAC port of the ingress and egress packets.
*
* @details In the from-network direction, the module user can specify the input port as one of classification parameters.
* In the to-network direction, the module user can force the egress packets to be sent over the specified
* destination Ethernet MAC port of the switch regardless of its states or configurations.
*/
/** @ingroup customType */
/* @{ */
/**
* @def pa_EMAC_PORT_NOT_SPECIFIED
* From-Netwprk: Don't care
* To-Network: Use standard switch forwarding
*/
#define pa_EMAC_PORT_NOT_SPECIFIED 0

/* @def pa_EMAC_PORT_0
* Use EMAC Port 0
*/
#define pa_EMAC_PORT_0 1

/* @def pa_EMAC_PORT_1
* Use EMAC Port 1
*/
#define pa_EMAC_PORT_1 2

Dear Aravinth,

    You are correct for this reason for selecting input EMAC port for matching entries on LUT engines.

    Previously we tested sending packets from EVM to PC is working
    
    now my testing is PC to EVM using packeth application to sending packets on ethernet.

    1. Able to capture packets on wireshark application.

    2. Not working on matching LUT engines & it's not receiving packets on receivePacket() API's

    3. I'm thought of Setup_Rx() we need to give a correct destination queue to receive packets on ethernet.

Regards,
SanthanaKumarS

   

Please check my response & let me know the receiving part of evm.

Dear Aravinth,

Any update on this thread.Please let me know your available.

Hi Santhana,

Did you check if the packets are reaching to PA? I had asked to check the GbE switch statistics to make sure they are not dropped outside or make sure packets are forwarded to host port.

Please let me know.

-Best

Aravind

We checked  ping from evm to pc & pc to evm is like dropped the packet after we running the  example application & it checking pa lutable. Ping from evm to other network  ia working

Hi Santhana,

I am not sure what you are trying to do. .Let me know if my understanding is not correct on the activity.

Appears like you are expecting to get the ping response out of PA Example/PA Unit test for PC's ping request? If yes, it is not possible since PA example does not have full network stack and does not understand PINC packets. (Or any other packets).

Linux Kernel supports the full network stack and ping from PC to EVM should work. Did you try it after power cycle (Reboot the board)? 

PA examples/Unit Test are meant to verify the PA functionality only - Also, please note the full Linux functionality may be broken after you run the examples/Unit Test. (Since PA examples directly updates the LUT table entries from the user space - may corrupt/interfere with the original Linux LUT entries that were added)

Is your intent is to check/understand if the MAC/IP/UDP rule you added is good for PA? OR expecting the PA example to respond back to the PC for the PING?

AFAIK, you had already verified this with PA Emac loopback example right? Where the packet from SendPacket() is routed to PA Input queues and hence there is there is a receive packet from PA examples matching to the LUT rules that has been added.

Summary:

PA Examples are not meant to be run with full network stack.

Hope it clarifies your question.

Thanks,
Aravind

Here what my expectation is using SendPacket api used to push the packet on ethernet wire ia working fine, now using same example code receivePacket api is used to get the UDP packet on wire contains source & destination MAC IP & ports should compare with pa lut table

HI Santhana,

I hope it is clear now with you on the PA example <--> PING questions. You can't check PING with PA Out of box Example.

Now, coming to receive packet for PA, what I mentioned to you earlier is you already have run the out of box PA example for the loopback, which is working.

Can you please confirm, on the out of box loopback example, just by changing the information in the packet for MAC/IP/UDP (and update in the cooked up packet in the example) for the expected MAC/IP/UDP, did the example work? It should work given it had worked for you with out of box mac/ip/udp information.

If this is not working, it means there is some mismatch between the actual packet that PA is seeing and the entries you have added.

So, suggestion is to get the example working with the loopback before attempt to receive the packet on wire. and make sure it works for the updated packet in the loopback example.

After example (updated as above) works, then you can send the same packet via Wire to the EVM and see if the packets are received. Note that when you are expecting the same packets from wire, you can comment the push to PA input queue test code in cppi_qmss_mgmt.c file (  Do not need this code, since there is loopback test and packets are expected to be arriving via the Eth interface.  Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_INPUT], pCppiDesc, SIZE_HOST_DESC);)

If updated loopback works and receive from wire did not work, the only possibility is, PA sub system did not see this packet- For PA it does not matter how the packet arrives. (whether through a queue push to PA Input queue or via the Ethernet to PA Input queue). This is what I suggested you to check. As far as the PA LUT entries are concerned, the LUT rules are the same... it does not matter, how the packets arrives at the PA input queue (via direct queue push or via ethernet).

Hope this is clear.

-Best,

Aravind

Dear Aravind,

"Can you please confirm, on the out of box loopback example, just by changing the information in the packet for MAC/IP/UDP (and update in the cooked up packet in the example) for the expected MAC/IP/UDP, did the example work? It should work given it had worked for you with out of box mac/ip/udp information"

Response:

Example application works correctly on the expected MAC/IP/UDP with loopback example & sending packets on wire also captured in wireshark application.

"After example (updated as above) works, then you can send the same packet via Wire to the EVM and see if the packets are received. Note that when you are expecting the same packets from wire, you can comment the push to PA input queue test code in cppi_qmss_mgmt.c file ( Do not need this code, since there is loopback test and packets are expected to be arriving via the Eth interface. Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_INPUT], pCppiDesc, SIZE_HOST_DESC);)"

Response

Here I commented sendPacket() Api & we sending packets from UDP packet generator like packeth application we need to try to give correct MAC, IP & PORT with UDP packets & we are trying to get udp packets on match with PA LUT entries

Please check above respone & our trying inputs are correctly or not?

"If updated loopback works and receive from wire did not work, the only possibility is, PA sub system did not see this packet- For PA it does not matter how the packet arrives. (whether through a queue push to PA Input queue or via the Ethernet to PA Input queue). This is what I suggested you to check. As far as the PA LUT entries are concerned, the LUT rules are the same... it does not matter, how the packets arrives at the PA input queue (via direct queue push or via ethernet)."

Response:

I checked correctly sending with MAC, IP & Ports on matched correctly with PA LUT entries but we are not getting packets on ReceivePacket() api but we received packets correctly on Wireshark application.

Here we need to get send/receive packets on SendPacket() & receivePacket() api

SendPacket() API - sending packets on ethernet on API's with pktmatchbuff ----------- Successfully captured with pktmatchbuff & captured packets on Wireshark application

Source code changes :

commented ReceivePacket()  using to receivePacket on Wireshark application

ReceivePacket() API - Not Receiving packets on ethernet on API's -------- Successfully captured with Wireshark application but not on receivePacket() API.

Source code changes:

commented SendPacket()  using to send packet on UDP packet generator like packeth 

Hi Santhana,

If the cooked up packet with the same MAC, IP and UDP are working with PA subsystem, then the rules you have added is good/correct.

Are you sure the packets from ethernet are going to PA or is Linux processing it differently? Are there any settings in the Linux side not using the PA Subsystem? Can you please double check? 

PA would not be able to do any look ups if the packet does not reach the PA sub system.

Thanks,
Aravind

Hi,

"If updated loopback works and receive from wire did not work, the only possibility is, PA sub system did not see this packet- For PA it does not matter how the packet arrives. (whether through a queue push to PA Input queue or via the Ethernet to PA Input queue). This is what I suggested you to check. As far as the PA LUT entries are concerned, the LUT rules are the same... it does not matter, how the packets arrives at the PA input queue (via direct queue push or via ethernet)."

Response:

Updated Loopback example application code correctly working.

Receive from Ethernet wire is not working

Commented this below line on code:

Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_INPUT], pCppiDesc, SIZE_HOST_DESC);

Hi,

Packets is going to PA & matching the MAC, IP & Port number but we are not captured the packets in ReceivePacket() API.

Regards,

SanthanaKumarS

Hi Santhana,

How are you sure that packets are going to PA and matching the MAC, IP and UDP entries? 

I understand your motivation to comment out "Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_INPUT], pCppiDesc, SIZE_HOST_DESC);" code since the packet is coming from ethernet instead of local SendPacket(). (I also assume you have commented out SendPacket() in this experiment).

If packets are going to PA and Matching MAC, IP and PORT lookup rule, the routeInfo for the match at the UDP port as shown below, should kick in and route the packet to gRxQueue.(Just like the case, where you got the loopback example working (out of box)).

Pick from ADD_Port() for the route info at port match stage.

/* Setup the Rx queue as destination for the packets */
routeInfo.queue = Qmss_getQIDFromHandle(gRxQHnd);
routeInfo.flowId = (uint8_t)Cppi_getFlowId(gRxFlowHnd);

The receive packet is simply waiting on packets on the above queue, in the example.

int32_t ReceivePacket (void)
{
Cppi_Desc *hd;
Int j;
int32_t status=0;

/* Wait for a data packet from PA */
for (j = 0; j < 100; j++)
{
CycleDelay (1000);
if (Qmss_getQueueEntryCount (gRxQHnd) > 0)
{
hd = (Cppi_Desc *)(((uint32_t)Qmss_queuePop (gRxQHnd)) & ~0xf);
if(VerifyPacket(hd) != 0)
status=-1;
}
}

return (status);
}

One suggestion for you to check:

How different is the external packet from that of the locally created packet used for loopback example? Have you compared the headers and have you matched it? 

Have you checked if the packets are reaching to PA by checking the mail box registers? Can you please share the mailbox dump before you send the packet and after you send the packet? This can help to check if the packet is seen by PA sub system.

By the way, what is the board you are using? Is this K2H OR K2K OR K2E OR K2L?

Thanks,

Aravind

Hi,
 
   "How are you sure that packets are going to PA and matching the MAC, IP and UDP entries?"

    Response:

    Here we ping <ip> from what we added IP in example applicaition code.
    
    ping from EVM & ping from PC    < Here you will confuse, If I am pinging <IP> from EVM to PC & PC to EVM is discarding the packets.>

   "If packets are going to PA and Matching MAC, IP and PORT lookup rule, the routeInfo for the match at the UDP port as shown below, should kick in and route the packet to gRxQueue.(Just like the case, where you got the loopback example working (out of box))."

    Response:

    Can you please tell gRxQHnd (QueueOpen Handler function) to handle the destination queue to ReceivePacket() API on  if (Qmss_getQueueEntryCount (gRxQHnd) > 0) from ethernet wire.

Please check below attached source code & check my queue configurations.

pa.zip

"Have you checked if the packets are reaching to PA by checking the mail box registers? Can you please share the mailbox dump before you send the packet and after you send the packet? This can help to check if the packet is seen by PA sub system."

    Response:

    Please tell how to check mail box registers?

 
Regards,
SanthanaKumarS

For K2E:

Please dump

4 32-bit word values (Mailbox registers) from below base addresses: (Note the addresses are provided in Hex)

You can do this before and after you send the packet from external source. Note that it would be helpful to study the data, if you have controlled network environment. (Like no other packet source to pump in the packets to the system).

You can experiment the same with the out of box PA example, (for out of box example, you do not need the Eth cable (RJ45) plugged to the board, as the packet is internally generated and pushed to PA).

You can study the mailbox before the SendPacket() and after the SendPacket().

Please send the data in below format, for the external packets that you are sending towards PA.

Addr:

0x24000000: val01 val02 val03 val04

0x24000010: val05 val06 val07 val08

...

...

...

0x240000E0: val19 val20 val21 val22

Thanks

Aravind

Hi,

Please response below inputs & for mail box registers we need to dump, you have any user space application or any commands to dump the packets from mail box registers

"Can you please tell gRxQHnd (QueueOpen Handler function) to handle the destination queue to ReceivePacket() API on  if (Qmss_getQueueEntryCount (gRxQHnd) > 0) from ethernet wire."

Hi,

  Clearly I will tell my points of testing & need of this experiment.

  Without modifications, We run the PA LLD example codes some loopback testing sent/receive packets from Core to PA & PA to Core.


Captured Results:

    Sent packets on PA LLD example code to transmission on Ethernet wire & based on PA rules.
           
            1. Captured Packets on Wireshark application.

    Received Packets on PA LLD example code from EVM ethernet wire & based on PA rules.

            1. Captured Packets on Wireshark application.

PA Rules testing:

    We are using laptop, PC & EVM board.

    Here we added PA MAC, IP & ports on PA LLD example application code.

    Only we added laptop & EVM mac, IP & port in PA rules (PA LLD API's) not added PC information.


    Using Packeth application to sent packets:

    Before running PA LLD code.
    ==========================
    
    Sent packets    ---------   Laptop to EVM   ---------- SUCESS (Note: Laptop is added as a Source MAC on PA rules using Pa_addMAC )

    Sent packets    ---------   PC to EVM       ---------- SUCESS (Note: PC information is not added as a Source MAC on PA rules using Pa_addMAC)

    After running PA LLD code.
    ==========================
    
    Sent packets    ---------   Laptop to EVM   ---------- Failure -> not capturing Packets (Note: Laptop is added as a Source MAC on PA rules using Pa_addMAC )

    Sent packets    ---------   PC to EVM       ---------- SUCESS -> Capturing Packets (Note: PC information is not added as a Source MAC on PA rules using Pa_addMAC)

Please guide me to using PA rules to compare MAC, IP & Port with LUT table.

Hi Aravinth,

    Any updates?

Regards,
SanthanaKumarS

Santhana,

Please find my response inline as below.

Hi,

Clearly I will tell my points of testing & need of this experiment.

Without modifications, We run the PA LLD example codes some loopback testing sent/receive packets from Core to PA & PA to Core.

Captured Results:

Sent packets on PA LLD example code to transmission on Ethernet wire & based on PA rules.

1. Captured Packets on Wireshark application.

[Aravind]: AFAIK, you are using "Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_EMAC], pCppiDesc, SIZE_HOST_DESC); " to send the packets on wire. Please note that this does not involve PA sub system. You are directly sending the packet to ethernet. No PA rules matter here. PA Rules (Add MAC, Add IP, Add port etc) are meant for the packets that are in ingress path (Receive direction).

Received Packets on PA LLD example code from EVM ethernet wire & based on PA rules.

1. Captured Packets on Wireshark application.
[Aravind]: Sorry - I do not understand your statement here. Can you please elaborate here? The PA LLD example sends the packet to PA Input queue in Linux User space application to have the PA MAC/IP/UDP rules verified. There is no packet on the ethernet from PA example. Have you made any changes to the application?

I suggest you to go through the details on PA driver, if not done previous: git.ti.com/.../pa.h

PA Rules testing:

We are using laptop, PC & EVM board.

Here we added PA MAC, IP & ports on PA LLD example application code.

Only we added laptop & EVM mac, IP & port in PA rules (PA LLD API's) not added PC information.

[Aravind]: I do not understand, what you mean by added laptop and EVM MAC, IP and PORT in PA Rules? From your earlier post, I have below information: Is there a change since then? Did you fix the Input EMAC PORT value as suggested?

paEthInfo_t ethInfo = {{ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }, /* Src mac = dont care */ ---> Here we changed PC
{ 0xe0, 0xd5, 0x5e, 0x37, 0xbc, 0xec }, /* Dest mac */ ---->Here we changed EVM
0, /* vlan = dont care */
0x0800, /* ether type = IPv4 */
0, /* MPLS tag = dont care */
1 /* Input EMAC port = dont care */ ----> Here we changed 0 -> 1 Why reason 0-> Host port 0 1-> Ethernet port1
};

Please note that I had suggested you to change the Input Port to 0 (=pa_EMAC_PORT_NOT_SPECIFIED) to have the packet from either of the ports. Not sure if you changed it as per suggestion. Let me know.

Using Packeth application to sent packets:

Before running PA LLD code.
==========================

Sent packets --------- Laptop to EVM ---------- SUCESS (Note: Laptop is added as a Source MAC on PA rules using Pa_addMAC )

Sent packets --------- PC to EVM ---------- SUCESS (Note: PC information is not added as a Source MAC on PA rules using Pa_addMAC)

After running PA LLD code.
==========================

Sent packets --------- Laptop to EVM ---------- Failure -> not capturing Packets (Note: Laptop is added as a Source MAC on PA rules using Pa_addMAC )

Sent packets --------- PC to EVM ---------- SUCESS -> Capturing Packets (Note: PC information is not added as a Source MAC on PA rules using Pa_addMAC)

Please guide me to using PA rules to compare MAC, IP & Port with LUT table.

[Aravind]: I could not understand what is before and after scenarios and what you mean by success here. I am assuming you got the PA LUT matches and packet was received at the intended queue at the host. Please clarify. Not sure, what are you trying to do here. When you mention "Sent Packets", is it via external packet source or using the PA example SendPacket() function? Like I mentioned before, the Pa Example's SendPacket() function simply pushes the packet to PA input queue in the linux user space application. The PA rules for Add MAC, Add IP and Add Port is for ingress (Receive direction) traffic lookup.

Are you using the SendPacket() function or tweaked SendPacket() function or is there an external send packet (Packet generator pumping the packets into the system?)

I had previously asked for the mailbox register dump and provided you the addresses for K2E evm. Without knowing what you are trying to do on PA User mode example, it may not be possible to help. Can you describe your goal/intention on how do you want to use PA in the user mode example?

Can you please explain what are you trying to achieve with PA user mode example and how do you want to use PA?

Also, please go through PA documentation to understand the PA sub system and functionality it offers. It would help you to design for your needs and help me to address any gaps you may be facing (if it can be addressed) to meet your goals.

-Best regards & Thanks,
Aravind

Hi Santhana,

I see yours and my responses are going under below for this post.

e2e.ti.com/.../3527769

Would like to get everything back to this main post.

Basically, I am repeating the question here. 

What are you trying to do and how are you visualizing (/expecting) PA subsystem to be used for your use case?

I have below data from your previous responses. Please correct/update the data accordingly, if it is changed recently. Please provide details as much as possible in the diagram form on what you are trying to do? I am getting a feel that you are adding a MAC, IP and UDP port match rule in PA, to match a packet received from an ethernet port and route to host queue.

I assume you have all these devices connected via Ethernet Switch.

1. Is the Laptop or PC capable of sending packets to EVM, using some utility or software?

2. Note that PA examples are not full network stack capable, and hence any Ping or ARP packets send to the board and expected to be processed by the PA examples would not have a response back from PA Example running on K2E EVM.

3. Let me know your experiment in detail and PA rules that you would like to add for MAC, IP and UDP.

Thanks,

Aravind

Dear Aravinth,
    
   Sorry for my response in this thread, I thought you are able to receive my response correctly but not, I will update below my needs & Project Flow need by PA.

   Using PA LLD example codes, We are able to sent the packets from EVM to any other network correctly & captured in Wireshark application.

Needed:
   From Receiving side of EVM, We need to use PA rules to compare the headers extraction from LUT table what we added.

"I am getting a feel that you are adding a MAC, IP and UDP port match rule in PA, to match a packet received from an ethernet port and route to host queue"  -> Exacyly We need this.

TestResults with PA:

   Device             MAC                                                      IP                                  UDP PORT
   Laptop     DATA NOT AVAILABLE                        DATA NOT AVAILABLE              DATA NOT AVAILABLE
   PC            0xe0, 0xd5, 0x5e, 0x37, 0xbc, 0xec     0xc0, 0xa8, 0x00, 0x71,            0x631
   K2E EVM     0x02, 0x18, 0x31, 0x7e, 0x3e, 0x00     0xc0, 0xa8, 0x00, 0x7f         0x5355

Response:

   Above laptop information we are not added means, that is unwanted packets coming to EVM. We need to avoid that one.

   All devices are connected to EVM only with using hub & below test results are here please verify.

   1. Not receiving the packets from EVM, (ie) We added PA rules with PC as a source, using Packeth(UDP packet generator) to sending packets from PC to EVM.

   2. Receiving the packets from EVM, (ie) We are not added PA rules with Laptop as a source, using Packeth(UDP packet generator) to sending packets from Laptop to EVM.


Above my TestResults is done on EVM board

Question:
   
   1. Why sending packet is discarding to with added PA rules of our MAC, IP & ports

   2. Why sending packet is forwarding to EVM board with not added MAC, IP & port in PA rules.

Question:

From loopback test, We are able to receive & verify the packets, sending the packets on Input queues using Qmss_queuePushDescSize (gPaTxQHnd[TF_PA_Q_INPUT], pCppiDesc,SIZE_HOST_DESC);

int32_t ReceivePacket (void)
{
    Cppi_Desc     *hd;
    Int            j;
    int32_t          status=0;
 
    /* Wait for a data packet from PA */
    for (j = 0; j < 100; j++)
    {
      CycleDelay (1000);
      if (Qmss_getQueueEntryCount (gRxQHnd) > 0)    ============================ > Please tell Here I need to receive the packets on EVM ethernet port
      {
        hd = (Cppi_Desc *)(((uint32_t)Qmss_queuePop (gRxQHnd)) & ~0xf);
        if(VerifyPacket(hd) != 0)
            status=-1;
      }
    }
 
    return (status);
}

Our need is sending UDP packets from PC to EVM, need to receive on above ReceivePacket() API & need to verify the packet also like below code

/* Verify the packet matches what we had sent */
    pDataBuffer = (uint8_t *) pHostDesc->buffPtr;
    for (i = 0; i < PACKET_SIZE; i++)
    {
        if (pktMatch[i] != pDataBuffer[i])
        {
            System_printf ("VerifyPacket: Byte %d expected 0x%02x, found 0x%02x\n", i, pktMatch[i], pDataBuffer[i]);
            System_flush();
 
            /* Free the packet back to the Rx FDQ */
            pHostDesc->buffLen = pHostDesc->origBufferLen;
            SYS_CACHE_WB (pHostDesc, SIZE_HOST_DESC, CACHE_FENCE_WAIT);
            Qmss_queuePush (gRxFreeQHnd, (Ptr)pHostDesc, pHostDesc->buffLen, SIZE_HOST_DESC, Qmss_Location_TAIL);
            return -1;
        }

Our project requirment using PA with Qos queues.

  Below I attached flowchart Please check & let me know.



Thanks& Regards,
SanthanaKumarS

Hi Santhana,

I had requested for your updated data that you did in the PA Example for below. Please share. Otherwise, I would not be able to help/comment on this. I may request more data as well to provide further inputs.

1. ethInfo, nextFailInfo and routeInfo data that you added in Add_MACAddress()

2. ipInfo, nextFailInfo and routeInfo in Add_IPAddress()

3. Port Num, routeInfo in Add_Port() 

Also, please make sure PA did accept all of your commands and you do not see responses like "PA sub-system rejected..." kind of messages anywhere in the log.

By the way, can you please confirm if you went through PA overview and how do you use it? What is your intent/how do you plan on using PA sub system? I have not heard that details from you yet.

Note that Packet sent out from PA has nothing to do with the LUT entries you add. The LUT is meant for *ingress* packet look up.

Thanks,
Aravind

Dear Aravinth,

Step 1:

We attached the PA LLD example source code with my PA Rules modifications for receiving packets from EVM side (eg: You asked about Add_MACAddress(). Add_IPAddress()).
 Note:
   In my example code, Some modifications for Sending packets on EVM to PC === > Please avoid that.

Laptop, PC & EVM are connected to same network using HUB.

After adding my PC & EVM MAC, IP & port, below table we are getting the test results Please verify.
 
   Device             MAC                                         IP                         UDP PORT            Captured packets in EVM     
   ______________________________________________________________________________________________________________________________________________          
   Laptop     0x10, 0x1f, 0x74, 0x17, 0x25, 0x4f       0xc0, 0xa8, 0x00, 0x78                                       No
                                                                                                    

   PC         0xe0, 0xd5, 0x5e, 0x38, 0x66, 0x88       0xc0, 0xa8, 0x00, 0x80,                                      Yes                

   K2E EVM    0xB4, 0x99, 0x4C, 0x0A, 0x8F, 0x6F       0xc0, 0xa8, 0x00, 0x68,                

   Above if you are confusing Please verify the attached EVM test results.

root@AM5K2E04:~# ifconfig
eth0      Link encap:Ethernet  HWaddr B4:99:4C:0A:08:6F  
          inet addr:192.168.0.104  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::b699:4cff:fe0a:86f%132688/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:40 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12324 (12.0 KiB)  TX bytes:2710 (2.6 KiB)

eth1      Link encap:Ethernet  HWaddr 02:18:31:7E:3E:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
           
                             /*********** Dear Aravinth, here not running PA lld example application **************/

Capturing Packets from EVM on Laptop
========================================================================================================================================================================
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.120
[  263.524927] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:23:50.452160 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.549773 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.734045 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:51.014296 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
^C
4 packets captur[  268.112777] device eth0 left promiscuous mode
ed
4 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# 

Capturing Packets from EVM on Laptop
========================================================================================================================================================================

root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16



		/************* Dear Aravinth,After running PA LLD example application. it adds PA rules.*************/
========================================================================================================================================================================

root@AM5K2E04:~# ./paMCExample.out 
No command line given.  Forking off 4 tasks
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
Waiting for global config (core 1) ...


Initialized RM_Client0

************************************************
*** PA Multi Core Example Started on Core 2 ***
************************************************
Waiting for global config (core 2) ...
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 3) ...
Initializing Free Descriptors. 
QMSS successfully initialized 
Unable to open below cppi tx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5 
 CPPI successfully initialized 
PASS successfully initialized 
Tx setup successfully done 
Rx setup successfully done 
Add_MAC successful for corenum:0
Add_IP successful for corenum:0
Destination for the packet Rx queue: 1027 (Core 0) 
Add_Port successful for corenum:0
PASS setup successfully done 
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ... 


Initialized RM_Client1

QMSS Local successfully initialized 


Initialized RM_Client2

CPPI Local successfully initialized 


Initialized RM_Client3

QMSS Local successfully initialized 
PA Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
CPPI Local successfully initialized 
PA Local successfully initialized 
PA Local successfully initialized 
Rx setup successfully done 
Rx setup successfully done 
Destination for the packet Rx queue: 1031 (Core 1) 
Add_Port successful for corenum:1
PASS setup successfully done 
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Destination for the packet Rx queue: 1032 (Core 2) 
Add_Port successful for corenum:2
PASS setup successfully done 
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ... 
Destination for the packet Rx queue: 1033 (Core 3) 
Add_Port successful for corenum:3
PASS setup successfully done 
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ... 


Packet Transmission Start ... core id: 3 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3


Packet Transmission Start ... core id: 0 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0


Packet Transmission Start ... core id: 1 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1


Packet Transmission Start ... core id: 2 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 2
[  711.947350] sched: RT throttling activated



^C
root@AM5K2E04:~# 
root@AM5K2E04:~# 
root@AM5K2E04:~# 
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.120
[  722.367287] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes



/************** Dear aravinth, Packet is not reaching in EVM from Laptop (Laptop information is added on PA rules)************************/

^C
0 packets captur[  731.550881] device eth0 left promiscuous mode
ed
0 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.794627 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:32:46.210981 IP 192.168.0.128.57914 > 239.255.255.250.1900: UDP, length 172


/************** Dear aravinth, Packet is reaching in EVM from PC (PC information is not added on PA rules)************************/

Screenshot from Laptop to EVM

Step3:

    Packet is forwarding, if you are not added in PA rules.

    Packet is discarding, if you are added in PA rules.

Project requirement:

    Below I attached flowchart Please check & let me know

4478.GPPNETCPFLOW.pptx
Source code Attached

3386.multicoreExample.zip

Question:

1. ReceivePacket() API used to receive the packet.

    Setup_RX() => gRxQHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, QMSS_PARAM_NOT_SPECIFIED, &isAllocated)

     Using RX handler to receive the packets is correct or not . Please verify according to our project requirement flow.

Regards,
SanthanaKumarS

Hi Santhana,

Observations:

• I see you modified SendPacket() for below changes
  • Push to EMAC directly - Note that this is not involving any PA functionalities.
  • PsFlags: I see you wanted to use pa_EMAC_PORT_0 (instead of PORT_1 as done in the example).
  • However, Noticed that you commented out SendPacket() from the example, so I am discarding all the changes you did for SendPacket() as that code is not used.
• Multicore_example.c change
  • I see you commented SendPacket() function and waiting forever for packets to be received in the Rx
  • You also modified local_variable.c file where there is a cooked up packet, which you changed .. This is used, when you are sending the packets via SendPacket() function for the example. Since it is not used, I am not looking into details there, as they are not used in your modified test, that has SendPacket() commented.
• pa_mgmt.c
  • I see, you have modified the Eth and IP Info as shown in below diagram.

    Summary of changes I see from your code:

MAC Lookup

SRC MAC: 0xe0, 0xd5, 0x5e, 0x37, 0xbc, 0xec

DST MAC: 0xB4, 0x99, 0x4C, 0x0A, 0x08, 0x6F

INPUT EMAC PORT: pa_EMAC_PORT_0

Next FAIL RULE: DISCARD PACKET

IP LookUp

SRC IP: 192.168. 0. 120

DST IP: 192.168.0.104
TOS: Don’t care

Next FAIL RULE: DISCARD PACKET

UDP Lookup

DST PORT: 0x5678

MATCH: Route to Host Queue as specified by gRxQHnd

From the diagram of the packet generator you shared,
I see the entries you added do not match to the packet either from Laptop or from PC, which can explain why you are not seeing the packets in the Rx queue.

MAC Lookup:
1. The MAC address of PC does not match to the source mac address you have added in the MAC LUT lookup, hence packets would not be forwarded to next stage from PA, from MAC
If your intent is to match the packets from PC or Laptop, you need to have the exact matching parameters OR have wild card (don't care) fields to match wide range.

2. I do not see a specific entry added for Laptop MAC ID added as the source mac ID in the Add_Mac(), hence none of the packets from this MAC source would be matching and hence they would not be forwarded to next stage (IP lookup).

IP Lookup:
1. I see you added IP packet source = 192.168.0.120 designated to 192.168.0.104, which are originating from the MAC source and designated to the EVM MAC ID as shown in the above diagram.

As per your test, I do not see any packets matching this MAC/IP combination, so you would not see any packets from IP Lookup to proceed to LUT2 look up.

UDP Lookup (LUT2):
Please check if the packets you are sending towards the EVM are having the UDP port number 0x5678 as the destination UDP port as that is the Add_Port() rule you have added for the test. Again, since no packets match previous stages, none would arrive at LUT2 for UDP port lookup.

Please note that, for the LUT entries you have added/modified, the exact matching rule is:

Packet should be coming from pa_EMAC_PORT_0 and should have below footprints:

SRC MAC: 0xe0, 0xd5, 0x5e, 0x37, 0xbc, 0xec
DST MAC: 0xB4, 0x99, 0x4C, 0x0A, 0x08, 0x6F

SRC IP: 192.168.0.120
DST IP: 192.168.0.104
(Linked to previous Mac handle)

DST UDP PORT: 0x5678
(Lined to previous IP handle)

I see PA sub system is behaving as expected. Please update/modify the PA LUT rules as per your need.

If you follow original PA example, you can understand how a packet is sent and made as receive from ethernet port to PA input queue with the matching parameters (MAC/IP/UDP match) and received at the Rx queue.

On Step 3 that you mentioned:

Step3:

    Packet is forwarding, if you are not added in PA rules. ==> I do not understand what you mean by this...Without adding any LUT rules, packets can't be forwarded via the PA sub system. You need to have Pa forward rules for the packet to be forwarded via PA sub system. Can you provide details on what you mean by packets are forwarded? which queue are you seeing them? I assume, you are not running the PA example if you have not added the PA rules from user mode example side.

    Packet is discarding, if you are added in PA rules.==> I explained this earlier, on why this is the case.

Thanks,

Aravind

Hi,

"Please note that, for the LUT entries you have added/modified, the exact matching rule is:

 Packet should be coming from pa_EMAC_PORT_0 and should have below footprints"

      SRC MAC: 0xe0, 0xd5, 0x5e, 0x37, 0xbc, 0xec
      DST MAC: 0xB4, 0x99, 0x4C, 0x0A, 0x08, 0x6F

    SRC IP: 192.168.0.120
    DST IP: 192.168.0.104
    (Linked to previous Mac handle)

    DST UDP PORT: 0x5678
    (Lined to previous IP handle)"

Step2 : (Response)
    
    Above you are mentioned the footprints to match the packeth application, We are giving the same matching MAC, IP & PORTS  in PA LLD example codes, but we given different PC network information..


Step3: (Response)

    "Packet is forwarding, if you are not added in PA rules. ==> I do not understand what you mean by this...Without adding any LUT rules, packets can't be forwarded via the PA sub system. You need to have Pa forward rules for the packet to be forwarded via PA sub system. Can you provide details on what you mean by packets are forwarded? which queue are you seeing them? I assume, you are not running the PA example if you have not added the PA rules from user mode example side"

Pleas note below lines captured from EVM, & From previous post, You are telling mismatching on MAC, IP. Please consider that one wrongly tested results. But I am surely whatever below outputs only we tested.

                            /*********** Dear Aravinth, here not running PA lld example application **************/

Capturing Packets from EVM on Laptop
========================================================================================================================================================================
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.134
[  263.524927] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:23:50.452160 IP 192.168.0.134.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.549773 IP 192.168.0.134.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.734045 IP 192.168.0.134.1234 > 192.168.0.104.5678: UDP, length 16
10:23:51.014296 IP 192.168.0.134.1234 > 192.168.0.104.5678: UDP, length 16
^C
4 packets captur[  268.112777] device eth0 left promiscuous mode
ed
4 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~#

Capturing Packets from EVM on Laptop
========================================================================================================================================================================

root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16



/************* Dear Aravinth,After running PA LLD example application. it adds PA rules.*************/
========================================================================================================================================================================

root@AM5K2E04:~# ./paMCExample.out
No command line given.  Forking off 4 tasks
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
Waiting for global config (core 1) ...


Initialized RM_Client0

************************************************
*** PA Multi Core Example Started on Core 2 ***
************************************************
Waiting for global config (core 2) ...
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 3) ...
Initializing Free Descriptors.
QMSS successfully initialized
Unable to open below cppi tx channels...presuming linux has already enabled it
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5
 CPPI successfully initialized
PASS successfully initialized
Tx setup successfully done
Rx setup successfully done
Add_MAC successful for corenum:0
Add_IP successful for corenum:0
Destination for the packet Rx queue: 1027 (Core 0)
Add_Port successful for corenum:0
PASS setup successfully done
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ...


Initialized RM_Client1

QMSS Local successfully initialized


Initialized RM_Client2

CPPI Local successfully initialized


Initialized RM_Client3

QMSS Local successfully initialized
PA Local successfully initialized
QMSS Local successfully initialized
CPPI Local successfully initialized
CPPI Local successfully initialized
PA Local successfully initialized
PA Local successfully initialized
Rx setup successfully done
Rx setup successfully done
Destination for the packet Rx queue: 1031 (Core 1)
Add_Port successful for corenum:1
PASS setup successfully done
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ...
Rx setup successfully done
Destination for the packet Rx queue: 1032 (Core 2)
Add_Port successful for corenum:2
PASS setup successfully done
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ...
Destination for the packet Rx queue: 1033 (Core 3)
Add_Port successful for corenum:3
PASS setup successfully done
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ...


Packet Transmission Start ... core id: 3
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3


Packet Transmission Start ... core id: 0
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0


Packet Transmission Start ... core id: 1
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1


Packet Transmission Start ... core id: 2
Packet Transmission Done.
Wait for all packets to be Received ... core num: 2
[  711.947350] sched: RT throttling activated



^C
root@AM5K2E04:~#
root@AM5K2E04:~#
root@AM5K2E04:~#

Screenshots attached from laptop to EVM

root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.134 ============================================>  Added in PA rules
[  722.367287] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes



/************** Dear aravinth, Packet is not reaching in EVM from PC (PC information is added on PA rules)************************/

^C
0 packets captur[  731.550881] device eth0 left promiscuous mode
ed
0 packets received by filter
0 packets dropped by kernel

Screenshots attached from PC to EVM

root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128 ====================================>  Not added in PA rules
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.794627 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:32:46.210981 IP 192.168.0.128.57914 > 239.255.255.250.1900: UDP, length 172


/************** Dear aravinth, Packet is reaching in EVM from Laptop (Laptop information is not added on PA rules)************************/


Note:

MAC, IP & port lookup is correct, but PA subsystem is discarding my packets. Without adding PA rules, It is receiving the packets from EVM.

Below attached my  source code & project requirement based on the receiving packets need to send on Qos queues

1004.multicoreExample.zip

2768.GPPNETCPFLOW.pptx

Regards,
SanthanaKumarS

Hi Santhana,

I noticed your IP address for the PC is different than the previous one that you shared. Please do not keep changing the IP addresses and any other information

from post to post. 

I am still finding discrepancies in the LUT entries you are showing in the example to the actual packet you are sending towards the EVM as I explained before.

*Please make sure the PA LUT entries and match parameters (care fields) should match to the packets that are received by PA).

So, suggest you to try the example

without external packet generator  approach, please try the out of box PA example first with the Packet information updated for your interest. After the loopback example works for you, you can move to external packet generator. Please do not change the packet footprints between multiple tries/trying with internal loopback/external packet generator for packets towards EVM.

Please note that PA example is meant to be for reference design on how to add MAC/IP/UDP entries in the LUT from the user space. Also, after the example is done, it removes the 

entries from PA LUT.  So, please do not expect the entries to stay in PA subsystem after the examples are done.

Also, Linux functionalities are not guaranteed to be preserved after you run the out of box PA user mode example.

Did you try loopback example with the SendPacket() sending the cooked up packet with the MAC/IP/UDP information of your interest? (PC or Laptop) Please make sure 

the example works with your updates.

This would provide you an understanding on how to use PA sub system.

After you understand the functionality and details on implications on LUT rules and how they work, it becomes easy for you to migrate the changes to your user space

application.

Note that if your goal is not user space application development and needed to add entries to PA LUTs, from Kernel Space, the RTOS PA driver are *NOT* the APIs. 

Please let me know, if you are developing PA user mode application or it is for something else.

For User mode development, I think below may help you.

1. Understand PA examples by modifying the MAC/IP/UDP entries to match the packet of your interest

2. Get the example functional (with the way it is now - Using SendPacket() function)

After above is done, 

then you can comment out the Send packet function from the example and use your packet send tool to make sure PA example receives the packet from outside. This would

pipe clean any issues you may be facing to receive the packet via PA to host queue.

After the external packet send/receive works, you have a good reference code to take it to your user space application that you are developing.

Let me know if this staged (step by step incremental) approach works for your need.

-Best regards & Thanks,

Aravind

Hi,
 

"Please do not change the packet footprints between multiple tries/trying with internal loopback/external packet generator for packets towards EVM"

   Response:
    Sorry my network is disconnected, so my network information is updated.

  

"Did you try loopback example with the SendPacket() sending the cooked up packet with the MAC/IP/UDP information of your interest? (PC or Laptop) Please make sure"

   Response:
     Yes it's working fine.

 

   Response: (Below Results)

    Step 1 :
    We added match of network information with the EVM & PC

    Tested with loop back examples is working fine with this modifications. (Able to receive the packets)

    paEthInfo_t                 ethInfo     =   {   { 0xe0, 0xd5, 0x5e, 0x38, 0x66, 0x88 },     /* Src mac = dont care */   ^M
                                                    { 0x02, 0x18, 0x31, 0x7E, 0x3E, 0x00 },     /* Dest mac */^M
                                                     0,                                          /* vlan = dont care */      
                                                     0x0800,                                            /* ether type = IPv4 */     
                                                     0,                                          /* MPLS tag = dont care */
                                                     0                                          /* Input EMAC port = dont care */  

./paMCExample.out 
No command line given.  Forking off 4 tasks
************************************************
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
************************************************
Waiting for global config (core 1) ...


Initialized RM_Client0

************************************************
************************************************
*** PA Multi Core Example Started on Core 2 ***
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 2) ...
Waiting for global config (core 3) ...
Initializing Free Descriptors. 
QMSS successfully initialized 
Unable to open below cppi tx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5 
 CPPI successfully initialized 
PASS successfully initialized 
Tx setup successfully done 
Rx setup successfully done 
Add_MAC successful for corenum:0
Add_IP successful for corenum:0
Destination for the packet Rx queue: 1027 (Core 0) 
Add_Port successful for corenum:0
PASS setup successfully done 
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ... 
Initialized RM_Client1
QMSS Local successfully initialized 
CPPI Local successfully initialized 
Initialized RM_Client2
Initialized RM_Client3
PA Local successfully initialized 
QMSS Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
CPPI Local successfully initialized 
PA Local successfully initialized 
PA Local successfully initialized 
Rx setup successfully done 
Destination for the packet Rx queue: 1031 (Core 1) 
Add_Port successful for corenum:1
PASS setup successfully done 
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Rx setup successfully done 
Destination for the packet Rx queue: 1032 (Core 2) 
Add_Port successful for corenum:2
PASS setup successfully done 
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ... 
Destination for the packet Rx queue: 1033 (Core 3) 
Add_Port successful for corenum:3
PASS setup successfully done 
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ... 


Packet Transmission Start ... core id: 3 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3
Core 3: Packets Sent            =       10 
Core 3: Packets Received        =       10 
--- PA (3) User-defined STATS --- 
number of Rx bytes(index = 3):    1220
number of Rx packets (index = 67): 10
Core 3 : CPPI CPDMA closed successfully
Core 3 : CPPI exit successful
Core 3: exit QMSS
Updating the Test Status core: 3, status:TEST_PASSED
 Done...
**********************************************
*** PA Multi Core Example Ended on Core 3 ***
**********************************************


Packet Transmission Start ... core id: 0 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0
Core 0: Packets Sent            =       10 
Core 0: Packets Received        =       10 
Wait for all packets to be Received in all cores... 
Updating the Test Status core: 0, status:TEST_PASSED


Packet Transmission Start ... core id: 1 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1
Core 1: Packets Sent            =       10 
Core 1: Packets Received        =       10 
--- PA (1) User-defined STATS --- 
number of Rx bytes(index = 1):    1220
number of Rx packets (index = 65): 10


Packet Transmission Start ... core id: 2 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 2
Core 2: Packets Sent            =       10 
Core 2: Packets Received        =       10 
--- PA (2) User-defined STATS --- 
number of Rx bytes(index = 2):    1220
number of Rx packets (index = 66): 10
Core 1 : CPPI CPDMA closed successfully
Core 1 : CPPI exit successful
Core 1: exit QMSS
Updating the Test Status core: 1, status:TEST_PASSED
 Done...
**********************************************
*** PA Multi Core Example Ended on Core 1 ***
**********************************************
Core 2 : CPPI CPDMA closed successfully
Core 2 : CPPI exit successful
Core 2: exit QMSS
Updating the Test Status core: 2, status:TEST_PASSED
 Done...
**********************************************
*** PA Multi Core Example Ended on Core 2 ***
**********************************************
All tests have passed!
--- PA (0) User-defined STATS --- 
number of Rx bytes(index = 0):    1220
number of Rx packets (index = 64): 10
Core 0 : CPPI CPDMA closed successfully
Core 0 : CPPI exit successful
Core 0: exit QMSS
 Done...
**********************************************
*** PA Multi Core Example Ended on Core 0 ***
**********************************************
Waiting for children to exit
All children passed
root@AM5K2E04:~#

root@AM5K2E04:~# ./paMCExample.out 
No command line given.  Forking off 4 tasks
************************************************
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
************************************************
Waiting for global config (core 1) ...


Initialized RM_Client0

************************************************
*** PA Multi Core Example Started on Core 2 ***
************************************************
Waiting for global config (core 2) ...
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 3) ...
Initializing Free Descriptors. 
QMSS successfully initialized 
Unable to open below cppi tx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5 
 CPPI successfully initialized 
PASS successfully initialized 
Tx setup successfully done 
Rx setup successfully done 
Add_MAC successful for corenum:0
Add_IP successful for corenum:0
Destination for the packet Rx queue: 1027 (Core 0) 
Add_Port successful for corenum:0
PASS setup successfully done 
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ... 


Initialized RM_Client1

QMSS Local successfully initialized 


Initialized RM_Client2

CPPI Local successfully initialized 


Initialized RM_Client3

QMSS Local successfully initialized 
PA Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
CPPI Local successfully initialized 
PA Local successfully initialized 
PA Local successfully initialized 
Rx setup successfully done 
Rx setup successfully done 
Destination for the packet Rx queue: 1031 (Core 1) 
Add_Port successful for corenum:1
PASS setup successfully done 
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Destination for the packet Rx queue: 1032 (Core 2) 
Add_Port successful for corenum:2
PASS setup successfully done 
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ... 
Destination for the packet Rx queue: 1033 (Core 3) 
Add_Port successful for corenum:3
PASS setup successfully done 
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ... 


Packet Transmission Start ... core id: 3 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3


Packet Transmission Start ... core id: 0 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0


Packet Transmission Start ... core id: 1 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1


Packet Transmission Start ... core id: 2 
Packet Transmission Done.

   Response: (Below Results)
    Step 2:
    commented SendPacket() in PA lld code with loop back example modification & we are sending Packeth (UDP packet generator) packets to EVM with matching MAC. IP information is not working.

    I am not able to receive packets from PA application, but able to receive packets on Wireshark application.

     Not able to capture packets in PA Input queue.


Questions:
   1. "pipe clean any issues you may be facing to receive the packet via PA to host queue" ===> Please explained clearly.  

   2.  Are you review my project flowchart & Let me know how to use PA subsystem to capture packets from Ethernet connection.

        3201.GPPNETCPFLOW.pptx

   3.  SendPacket() using EMAC to sending packets is working fine. For ReceivePacket() API Can you please check my destination queue is correct or not to receive packets on Ethernet?

   4. "Packet is forwarding, if you are not added in PA rules. ==> I do not understand what you mean by this...Without adding any LUT rules, packets can't be forwarded via the PA sub system. You need to have Pa forward rules for the packet to be forwarded via PA sub system. Can you provide details on what you mean by packets are forwarded? which queue are you seeing them? I assume, you are not running the PA example if you have not added the PA rules from user mode example side."

Response

          Without adding any LUT rules, packets can't be forwarded via the PA sub system => Correct you are telling.

          You need to have Pa forward rules for the packet to be forwarded via PA sub system => If the packet is forward to PA subsystem only by matching  the certain packets &  drop those packets instead of forwarding them to HOST processor. Below attached doc with 4th Page, Read PA functionalities

       Below attached document there mentioned for Packet Accelerator with QoS features for packet discarding & priority on QoS queues(check on 4th page )

       QualityOfService(QOS).pdf

 

Regards,
SanthanaKumarS

Hi Santhana,

On your below response:

Tested with loop back examples is working fine with this modifications. (Able to receive the packets)

paEthInfo_t ethInfo = { { 0xe0, 0xd5, 0x5e, 0x38, 0x66, 0x88 }, /* Src mac = dont care */ ^M
{ 0x02, 0x18, 0x31, 0x7E, 0x3E, 0x00 }, /* Dest mac */^M
0, /* vlan = dont care */
0x0800, /* ether type = IPv4 */
0, /* MPLS tag = dont care */
0 /* Input EMAC port = dont care */

Tested with loop back examples is not working fine with this modifications. ( Not able to receive the packets)

paEthInfo_t ethInfo = { { 0xe0, 0xd5, 0x5e, 0x38, 0x66, 0x88 }, /* Src mac = dont care */ ^M
{ 0x02, 0x18, 0x31, 0x7E, 0x3E, 0x00 }, /* Dest mac */^M
0, /* vlan = dont care */
0x0800, /* ether type = IPv4 */
0, /* MPLS tag = dont care */
2 /* Input EMAC port = dont care */

Aravind: Note that in first ethInfo, Input EMAC PORT is set to '0', which means PA LUT does not match the input port for the ingress eth traffic. In the second case, you provided, info to match the packets from a specific eth port. The packets from a specific eth port have port information, which PA can see and if that is not matching to specified rules, it would discard the packet.

Answers to your questions:
===========================

Questions:
1. "pipe clean any issues you may be facing to receive the packet via PA to host queue" ===> Please explained clearly.
[Aravind]: I want to make sure you have correct understanding on the PA LUT rules, forwarding and discard capabilities. I just provided details on why one case you see packets forwarded and another case, it is not. The only change as per your inputs, is the ethInfo EMAC Port number is also considered for the match and if the packets did not match this, simply indicate that the ingress traffic that PA is seeing is not having the matching Eth Port Info for it to match and continue parse to next stage.

Basically, what I mean here is, add all expected MAC/IP/UDP rules and verify it by sending a cooked up packet (like in the example, not through the ethernet port). If you receive the packets via this way, it proves PA
is ready to route the packets to your intended queues, if they arrive to PA queue. After this is done, you can comment out the internal SendPacket() and after adding the LUT rules, wait for the packets to arrive. Then
you can use your packet generator outside to pump in the packets to receive them in your configured host queue. This enables you to proceed step by step progress.

2. Are you review my project flowchart & Let me know how to use PA subsystem to capture packets from Ethernet connection.

3201.GPPNETCPFLOW.pptx
Aravind: Yes.. You are looking for priority routing based on DSCP, and Port value. Please refer to my comments below on prority based routing.

3. SendPacket() using EMAC to sending packets is working fine. For ReceivePacket() API Can you please check my destination queue is correct or not to receive packets on Ethernet?

Aravind:I do not know if you are asking for PA example or something different (Your need case for receiving packets based on port/DSCP value). If you are referring to the Example, the receive queue is correct.. I do not see you are modifying anything for the host queue. I have not got confirmation from you are looking for user space application development. Assuming, you are developing user space application, if you are looking for priority based routing, you need to write your own user space application similar to PA example.

4. Priority Routing:
Aravind: On the QoS specific routing, from page #4 of e2e.ti.com/.../QualityOfService_2800_QOS_2900_.pdf :

Please note that, PA example do not show case priority routing details. The older API Pa_addMac() which is used in PA Example, can't be used for this. You need to refer to the other API such as Pa_addMac2(), as shown below and provide the ethInfo as per paEthInfo2_t parameters for the QoS routing.

paReturn_t Pa_addMac2 ( Pa_Handle iHandle,
paEthInfo2_t *ethInfo, /**< Value @ref paEthInfo2_t */
paParamDesc *params,
paLnkHandle_t *retHandle, /**< Pointer to the returned L2 handle */
paCmd_t cmd,
uint16_t *cmdSize,
paCmdReply_t *reply,
int *cmdDest);

There are some other PA Unit Tests that you can refer on how to use Pa_addMac2(). You can refer to test2.c under pa\test\paunittest\src\tests\test2.c which demontrates the pa L2Routing using Priority type.
After you understand the details, you need to write your own examples for your specific needs, referring to PA example and Unit Tests for QoS routing of your need.

Hope it is clear for you now.

-Best regards & Thanks,
Aravind

1. "Note that in first ethInfo, Input EMAC PORT is set to '0', which means PA LUT does not match the input port for the ingress eth traffic".

Response:

From Loop back example codes, Above Input EMAC PORT is set to '0' & receive the packet from Host queue,VerifyPacket() API is used to compared the SendPacket() API having pktMatchBuf[] & PA inputs,

if it matches, Host queue able to receive the packets,

but it not matches, Host queue not receiveing the packets.

Below Why it is discarding & receiving

pDataBuffer = (uint8_t *) pHostDesc->buffPtr;
    for (i = 0; i < PACKET_SIZE; i++)
    {
        if (pktMatch[i] != pDataBuffer[i])
        {
            System_printf ("VerifyPacket: Byte %d expected 0x%02x, found 0x%02x\n", i, pktMatch[i], pDataBuffer[i]);
            System_flush();
 
            /* Free the packet back to the Rx FDQ */
            pHostDesc->buffLen = pHostDesc->origBufferLen;
            SYS_CACHE_WB (pHostDesc, SIZE_HOST_DESC, CACHE_FENCE_WAIT);
            Qmss_queuePush (gRxFreeQHnd, (Ptr)pHostDesc, pHostDesc->buffLen, SIZE_HOST_DESC, Qmss_Location_TAIL);
            return -1;
        }
    }

2. "In the second case, you provided, info to match the packets from a specific eth port. The packets from a specific eth port have port information, which PA can see and if that is not matching to specified rules, it would discard the packet."

Response:

My observation

   Matches the specific eth port to receiving the packets from EVM & PA could see that the packets, but it matches the MAC, IP & PORT.  Then it would discard the packets.

   Above if you give Input EMAC port '0' =>  ReceivePacket() API successfully receiving the packets on PA Input queues.

  Above if you give Input EMAC port '2' =>  ReceivePacket() API  not able to  receiving the packets on PA Input queues. ===> Sending packets from SendPacket()

Above if you give Input EMAC port '2' =>  ReceivePacket() API  not able to  receiving the packets on PA Input queues. ===> Sending packets from Packeth UDP packet generator application Note: commented SendPacket()

3. "Basically, what I mean here is, add all expected MAC/IP/UDP rules and verify it by sending a cooked up packet (like in the example, not through the ethernet port). If you receive the packets via this way, it proves PA
is ready to route the packets to your intended queues, if they arrive to PA queue. After this is done, you can comment out the internal SendPacket() and after adding the LUT rules, wait for the packets to arrive. Then
you can use your packet generator outside to pump in the packets to receive them in your configured host queue. This enables you to proceed step by step progress."

 Response:

  Dear Aravind, Understanding your points, but We are not able to receive the packets using packet generator to pump the packets to receive in my host queue

  We are able to receive the packets on host queue  by using SendPacket() API.


"I do not know if you are asking for PA example or something different (Your need case for receiving packets based on port/DSCP value"

  Response:

   Just for my confirmation my packets is receiveing on host queue. * You can guide me to recieve the packets on Qos queue.


  Qos routing observations:

  Why I am selecting my base queue 6400 because i am using Input EMAC port 2.
  qosinputs1: qos-inputs-1 {
                                qrange = <6400 192>;   ============================> 6400 is my Qos base queue
                                pdsp-id                 = <7>;
                                ticks-per-sec           = <10000>;
                                qos-cfg                 = <4 50 1 1 0xf00 0 0>;

                                sched-port-configs      = <0 20 0x1c8>;
                                drop-out-profiles       = <0 36 0xc>;
                                drop-cfg-profiles       = <0 8 0x14>;
                                drop-queue-configs      = <0 80 0x4>;

                                drop-policies           = <&droppolicies>;
                                qos-tree                = <&qostree1>;
                                statistics-profiles     = <0 48>;
                                qalloc-by-id;
                        };

  Based on my project flow, Need to receive on 4 QoS Queue

  So we selecting the queues depends on Egress clusters.

  Below Link you can refer : software-dl.ti.com/.../NetCP.html

  4.  "When egress shaper is enabled, all packets will be sent to the QoS firmware for shaping via a set of the queues starting from the Q0S base queue which is 8000 by default. DSCP value in the IP header(outer IP incase of IPSec tunnels) or VLAN pbits (if VLAN interface) are used to determine the QoS queue to which the packet is sent. E.g., if the base queue is 8000, if the DSCP value is 46, the packet will be sent to queue number 8046. i.e., base queue number + DSCP value Incase of VLAN interfaces, if the pbit is 7, the packet will be sent to queue number 8071. i.e., base queue number + skip 64 queues used for DSCP + pbit value."
   
    qos@0 {
                        label = "netcp-qos";
                        compatible = "ti,netcp-qos";
                        tx-channel = "pa-cl6";

                        interfaces {
                                qos0: interface-0 {
                                        tx-queues = <910 8072 8073 8074
                                                     8075 8076 8077>;
                                };
                                qos1: interface-1 {
                                        tx-queues = <910 6472 6473 6474
                                                     6475 6476 6477>;
                                };
                        };
                };
 
   Please guide me to receive the packets on Qos queue on above mentioned in qos1 inputs.

Regards,
SanthanaKumarS

Dear Aravind,

   Any updates on this thread.

Regards
SanthanaKumarS

Hi Santhana,

From your previous explanation, since you are getting the packets to the host queue, PA LUT entries are good, when you have SendPacket() in place. You need to understand why pushing packet from external is not working. I had mentioned to dump mailbox, GbE switch statistics to help debug. I have not got any data from you yet on this.

I am not sure if you got everything that I had been mentioning earlier. I have not got details from you yet. Still waiting on them to comment further.  (Especially 3 and 4 below)

Please note that I am repeating it again here:

"   Results
        Sending (pktmatchbuff) packets on wire based on added MAC, IP on (pa_addMAc, pa_addIP)APIs."

Sending the packets over wire, has nothing to do with the LUT entries added for MAC, IP, UDP. The LUT is only for receiving packets.

Summary:

1. You had confirmed that the packet via SendPacket() function that have the MAC, IP and UDP information is matching the PA and you are seeing the packets in the expected host queue.

This confirms that the any packet having this signature either from SendPacket() or via ethernet port would match the LUT entries and should arrive in the same queue.

2. You had also mentioned you are basically looking for  QoS example, which you need to build based on the EMAC example. EMAC example is a basic example, that shows how to use PA APIs.

3. I had also asked you to check if the PA Mailbox registers (you can refer to earlier thread for the addresses and dump format), when you are pumping in the packets from external source. Please make sure there are no other packets in the system, which can help to understand if the packets are reaching to PA.

PA mailbox Registers:

Dump from PA Single Core Example: (After sending all the packets). Please send your dump as below, which can help me to comment if the packets are reaching to PA or not. If you can send multiple dumps of this memory block, it would be further helpful . (Like before sending any packet, after sending one packet, sending other packet etc).

0x24000000    0000000B    00000000    00000001    00000000
0x24000010    0000000B    00000000    00000000    00000000
0x24000020    0000000B    00000000    00000001    00000000
0x24000030    0000000B    00000000    00000000    00000000
0x24000040    00000000    00000000    00000000    00000000
0x24000050    00000000    00000000    00000000    00000000
0x24000060    0000000B    00000000    00000001    00000000
0x24000070    0000000B    00000000    00000001    00000000
0x24000080    00000000    00000000    00000000    00000000
0x24000090    00000000    00000000    00000000    00000000
0x240000A0    0000000A    00000000    00000000    00000000
0x240000B0    0000000A    00000000    00000000    00000000
0x240000C0    0000000A    00000000    00000000    00000000

4. If Packets are not reaching to PA, then you need to identify where is it getting dropped. You can refer to KeyStone GbE switch user guide to dump the Switch statistics.

You need to find the CPSW statistics register dumps  for all the MAC ports. (e.g, 0x2423A000 for EMAC PORT0, 0x2423A200 for EMAC PORT 1 etc). Since there are 9 ports available, you can dump data for all ports.

0x2423A000+n*0x200 (where n=0, 1,2, 3,4, 5,6,7,8).

Once you understand PA, You can debug your code, with these dumps. Later on you can write your user space application for QoS.

Thanks,

Aravind

Hi Aravind,

    We tried to dump the PDSP mailbox registers for all EMAC ports Below we tried to dump the registers using linux commands & cpsw docs.

    software-dl.ti.com/.../CPSW.html

    Method-I :
    root@AM5K2E04:/usr/bin# switch-config --dump
    Cannot get driver information: Operation not supported

    Method-II
    root@AM5K2E04:/usr/bin# ethtool -d eth1
    Cannot get register dump: Operation not supported

    Gbe Switch status registers:

    root@AM5K2E04:~# cat /sys/devices/platform/soc/2620110.netcp/gbe_sw/stats/
    0  1  2  3  4  5  6  7  8  
    root@AM5K2E04:~# cat /sys/devices/platform/soc/2620110.netcp/gbe_sw/stats/0
    cat: can't open '/sys/devices/platform/soc/2620110.netcp/gbe_sw/stats/0': Permission denied

    Please tell any other way to dump PDSP mailbox registers.

    One more information from my side.

    We are able to receive the packets on PA subsystem. PA able to see the packets & matching the packet information to discarding the packets not matched.

    Attached file Please check clearly tcpdump to passing my source & capture the packets before running PA LLD appplication & After running application.

root@AM5K2E04:~# ifconfig
eth0      Link encap:Ethernet  HWaddr B4:99:4C:0A:08:6F  
          inet addr:192.168.0.104  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::b699:4cff:fe0a:86f%132688/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:40 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12324 (12.0 KiB)  TX bytes:2710 (2.6 KiB)

eth1      Link encap:Ethernet  HWaddr 02:18:31:7E:3E:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
           
                             /*********** Dear Aravinth, here not running PA lld example application **************/

Capturing Packets from EVM on Laptop
========================================================================================================================================================================
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.120
[  263.524927] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:23:50.452160 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.549773 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:50.734045 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
10:23:51.014296 IP 192.168.0.120.1234 > 192.168.0.104.5678: UDP, length 16
^C
4 packets captur[  268.112777] device eth0 left promiscuous mode
ed
4 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# 

Capturing Packets from EVM on Laptop
========================================================================================================================================================================

root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16



		/************* Dear Aravinth,After running PA LLD example application. it adds PA rules.*************/
========================================================================================================================================================================

root@AM5K2E04:~# ./paMCExample.out 
No command line given.  Forking off 4 tasks
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
Waiting for global config (core 1) ...


Initialized RM_Client0

************************************************
*** PA Multi Core Example Started on Core 2 ***
************************************************
Waiting for global config (core 2) ...
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 3) ...
Initializing Free Descriptors. 
QMSS successfully initialized 
Unable to open below cppi tx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5 
 CPPI successfully initialized 
PASS successfully initialized 
Tx setup successfully done 
Rx setup successfully done 
Add_MAC successful for corenum:0
Add_IP successful for corenum:0
Destination for the packet Rx queue: 1027 (Core 0) 
Add_Port successful for corenum:0
PASS setup successfully done 
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ... 


Initialized RM_Client1

QMSS Local successfully initialized 


Initialized RM_Client2

CPPI Local successfully initialized 


Initialized RM_Client3

QMSS Local successfully initialized 
PA Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
CPPI Local successfully initialized 
PA Local successfully initialized 
PA Local successfully initialized 
Rx setup successfully done 
Rx setup successfully done 
Destination for the packet Rx queue: 1031 (Core 1) 
Add_Port successful for corenum:1
PASS setup successfully done 
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Destination for the packet Rx queue: 1032 (Core 2) 
Add_Port successful for corenum:2
PASS setup successfully done 
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ... 
Destination for the packet Rx queue: 1033 (Core 3) 
Add_Port successful for corenum:3
PASS setup successfully done 
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ... 


Packet Transmission Start ... core id: 3 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3


Packet Transmission Start ... core id: 0 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0


Packet Transmission Start ... core id: 1 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1


Packet Transmission Start ... core id: 2 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 2
[  711.947350] sched: RT throttling activated



^C
root@AM5K2E04:~# 
root@AM5K2E04:~# 
root@AM5K2E04:~# 
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.120
[  722.367287] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes



/************** Dear aravinth, Packet is not reaching in EVM from Laptop (Laptop information is added on PA rules)************************/

^C
0 packets captur[  731.550881] device eth0 left promiscuous mode
ed
0 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# tcpdump -i eth0 -c500 -nn src 192.168.0.128
[  736.887196] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:42.436042 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:44.884396 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:45.987640 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.202750 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.402552 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.595093 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:31:46.794627 IP 192.168.0.128.1234 > 192.168.0.104.5678: UDP, length 16
10:32:46.210981 IP 192.168.0.128.57914 > 239.255.255.250.1900: UDP, length 172


/************** Dear aravinth, Packet is reaching in EVM from PC (PC information is not added on PA rules)************************/

Hi Santhana,

Like I had mentioned before, the LUT entries in the examples are removed after the example is complete. I had also suggested you to have PA example as a reference for your code. 

It appears to me you are trying to add the entries to PA LUT tables with the way PA Example is doing for your QoS routing needs. I am repeating the suggestion here that it is not correct. You need to use a different PA API that I had mentioned before. 

I am not a Linux expert for commenting on the CPSW dump issues you are seeing. Checking with local Linux experts if they have a say on your CPSW Method 1 and 2.

Alternatively, I think you can use CCS (With JTAG) connect to C66x Core and view the memory locations for debug from C66X core.

Also, note that PA examples delete the entries after the example is run. You can try commenting the delete entries part in the example to see if the packets are still received after running the test. Please note that Out of Box, PA examples are not meant to retain the full Linux capability as they update the PA LUT entries to demonstrate the PA capabilities and may interfere with native Linux operations.

I'm still not sure from you on if you are developing the user mode application or something else? The changes you are doing are all related to user mode application. If your end goal is not user mode application, you probably need to proceed differently.

Let me know your debug dumps of those memory locations either via CCS or some other ways (like devmem2 etc)

Thanks,
Aravind

Hi,

   "Also, note that PA examples delete the entries after the example is run. You can try commenting the delete entries part in the example to see if the packets are still received after running the test."

   Respone:
   I am confirmed that you PA LUT entries are not removed running code already have Correct Del_Mac(), Del_IP() & Del_Port() is removed.

  "I'm still not sure from you on if you are developing the user mode application or something else?"

   Response:
    Need to develop user mode application with using Pa_addMAC(), Pa_addIP(), PA_addPort().

  "Alternatively, I think you can use CCS (With JTAG) connect to C66x Core and view the memory locations for debug from C66X core"

   Response:
    This part I will try to let you know. but I will try to give help me on linux side.
   

   One more updates:

   Changed my queue to Qos Queue => 6472 & I am able to receiving my packet on loopback example code. but Sending from PC to EVM is not working. Able to help alone this part.

   Please help to recieve the packet on EVM, Can you please confirm my receivePacket() API to receive the packet on particular queue.

   Verify my code & let me know any changes required for receiving packet from PC to EVM.

   Dumping the memory, I will try & let me know the memory part

Regards,
SanthanaKumarS

Hi Santhana,

...."but Sending from PC to EVM is not working"

Please note, I had mentioned in earlier thread to add additional, separate entries for PC as well, if there is no general rule matching this. Had you confirmed the rule that you added, which is receiving the packets from Laptop to EVM is good for receiving the packets from PC as well?

Basically, did you add entries (separate entries for packets having PC's signatures for MAC, IP, UDP etc? or looking for same entry to serve for both laptop and pc?

I will wait for your logs to comment further.

Thanks,
Aravid

Hi,

  You are asking the logs for both laptop & PC entries are added, giving the logs for your further clarification.

   Please check this file to clarify your points & comment your lines to helping for receiving the packet on queues for using to sending from UDP packet generator like packeth application

   Below attached source code & screenshots from testing points, Previously you told, don't to change, but some disconnected network connection cause issue to changing my IP Address.

   TestResults (Please open in Notepad to open the file & highlighted some points to you )

********************************************************** EVM ifconfig **********************************************************   

eth1      Link encap:Ethernet  HWaddr 02:18:31:7E:3E:00  
          inet addr:192.168.0.101  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::18:31ff:fe7e:3e00%132688/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:22 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3500 (3.4 KiB)  TX bytes:1758 (1.7 KiB)

********************************************************** Laptop ifconfig **********************************************************   
zumi@zumi-HP-Pavilion-dv6-Notebook-PC:~$ ifconfig
eno1      Link encap:Ethernet  HWaddr 10:1f:74:17:25:4f  
          inet addr:192.168.0.139  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::e3d7:8e50:6e93:e015/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:90414 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43456 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:92168837 (92.1 MB)  TX bytes:4843157 (4.8 MB)

********************************************************** PC ifconfig **********************************************************   
enp2s0    Link encap:Ethernet  HWaddr e0:d5:5e:38:66:88  
          inet addr:192.168.0.137  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::46d5:a151:ac8d:6758/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:117500 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56940 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:83752445 (83.7 MB)  TX bytes:7655915 (7.6 M)

############### Dear Aravind, Before running PA example application, We able to capture packets on EVM #################

root@AM5K2E04:~# tcpdump -i eth1 -c500 -nn src 192.168.0.139
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
05:49:46.511251 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:46.675711 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:46.860345 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:46.996085 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:47.210452 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:47.479990 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:47.659450 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:47.809057 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:47.979311 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:48.173019 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:48.352578 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
05:49:48.632373 IP 192.168.0.139.1234 > 192.168.0.101.5678: UDP, length 16
^C
12 packets captu[   68.293816] device eth1 left promiscuous mode
red
12 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# 
root@AM5K2E04:~# 
root@AM5K2E04:~# tcpdump -i eth1 -c500 -nn src 192.168.0.137
[   70.774952] device eth1 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
05:49:59.963605 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
05:50:05.051432 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
05:50:07.651410 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
05:50:09.755292 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
05:50:09.795410 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
05:50:10.307306 IP 192.168.0.137.1234 > 192.168.0.101.5678: UDP, length 15
^C
12 packets captu[   93.086096] device eth1 left promiscuous mode
red
12 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# 

############### Dear Aravind, Now we running PA example application with Laptop & EVM rules #################

root@AM5K2E04:~# rmServer.out /usr/bin/device/k2e/global-resource-list.dtb /usr/bin/device/k2e/policy_dsp_arm.dtb
Starting rmServer.out
Daemon returned 0 as return value.
root@AM5K2E04:~# ./paMCExample.out 
No command line given.  Forking off 4 tasks
************************************************
************************************************
*** PA Multi Core Example Started on Core 0 ***
*** PA Multi Core Example Started on Core 1 ***
************************************************
************************************************
Waiting for global config (core 1) ...
Initialized RM_Client0
************************************************
*** PA Multi Core Example Started on Core 3 ***
************************************************
Waiting for global config (core 3) ...
************************************************
*** PA Multi Core Example Started on Core 2 ***
Initializing Free Descriptors. 
************************************************
QMSS successfully initialized 
Waiting for global config (core 2) ...
Unable to open below cppi tx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it 
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5 
 CPPI successfully initialized 
PASS successfully initialized 
Tx setup successfully done 
Qmss_getqueueNumber queue[0] 1 
Qmss_queueOpening queue 6472 
rxQinfo[0] = 1
Rx setup successfully done 
Add_MAC successful for Laptop:0
Add_MAC2 successful for PC:0
Add_IP successful for LAPTOP:0
Add_IP successful for PC:0
Core number 0
Add_Port RxQueue[0] = 6472
Destination for the packet Rx queue: 6472 (Core 0) 
Add_Port successful for corenum:0
PASS setup successfully done 
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ... 
Initialized RM_Client1
QMSS Local successfully initialized 
CPPI Local successfully initialized 
Initialized RM_Client3
PA Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
Initialized RM_Client2
PA Local successfully initialized 
QMSS Local successfully initialized 
CPPI Local successfully initialized 
PA Local successfully initialized 
Qmss_getqueueNumber queue[0] 1 
Qmss_queueOpening queue 6472 
rxQinfo[0] = 1
Rx setup successfully done 
Qmss_getqueueNumber queue[0] 1 
Qmss_queueOpening queue 6472 
rxQinfo[0] = 1
Core number 1
Add_Port RxQueue[0] = 6472
Destination for the packet Rx queue: 6472 (Core 1) 
Add_Port successful for corenum:1
PASS setup successfully done 
Publishing local config done from core num: 1...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Qmss_getqueueNumber queue[0] 1 
Qmss_queueOpening queue 6472 
rxQinfo[0] = 1
Core number 3
Add_Port RxQueue[0] = 6472
Destination for the packet Rx queue: 6472 (Core 3) 
Add_Port successful for corenum:3
PASS setup successfully done 
Publishing local config done from core num: 3...
Waiting for all cores to reach the barrier before transmission starts ... 
Rx setup successfully done 
Core number 2
Add_Port RxQueue[0] = 6472
Destination for the packet Rx queue: 6472 (Core 2) 
Add_Port successful for corenum:2
PASS setup successfully done 
Publishing local config done from core num: 2...
Waiting for all cores to reach the barrier before transmission starts ... 
Packet Transmission Start ... core id: 2 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 2
Packet Transmission Start ... core id: 0 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0
Packet Transmission Start ... core id: 1 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 1
Packet Transmission Start ... core id: 3 
Packet Transmission Done.
Wait for all packets to be Received ... core num: 3
[  151.954906] sched: RT throttling activated
^C

############### Dear Aravindh after succesfully added MAC, IP & Ports using PA application, capturing packets on using tcpdump #################

root@AM5K2E04:~# tcpdump -i eth1 -c500 -nn src 192.168.0.139 
[  166.774939] device eth1 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captur[  279.517096] device eth1 left promiscuous mode
ed
0 packets received by filter
0 packets dropped by kernel
root@AM5K2E04:~# tcpdump -i eth1 -c500 -nn src 192.168.0.137
[  282.114935] device eth1 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes

############### Dear Aravindh, Here if we added PA rules with both Laptop & PC, Not able to receive Packets #################

   Screenshots from Laptop to EVM

   Screenshots from PC to EVM

Source code:

7444.multicoreExample.zip

Changes:

1. Added entries for Laptop & PC.

2. changing my queue number to Qos Queue.

Regards

SanthanaKumarS

Hi,

  Please see this above response, & Packet is receiving from PA subsystem I am sure whatver i am adding the PA rules it matches & receiving the packet from PA subsystem & discarding the packet, but my packet is not receiing on Host descriptor buffer & Rx flow queue. 

Please verify my source code & Please verify my Setup_Rx() API, rxFlowCfg & receivePacket() API uses to receive the packet on queue entry.

Please tell your availability on tomorrow, I will response on tomorrow. My issue is going on long .. Please support on your quick updates if it is possible .

Regards,

SanthanaKumarS

Hi Santhana,

Sorry you are still not successful.

I think, there are issues in the way you are checking the packets. Also, by the way, were you able to figure out how to check the logs for mailbox and switch via JTAG? 

I still do not understand what you are doing or trying to achieve. I see you are adding entries from the user mode program and running tcpdump to capture the packets?

You can not add an entry in the PA rule routing the packets to Host Queue "N" and kill that user mode program and run tcpdump for the capture. 

You should be checking for packets from the user mode example itself (not tcpdump).

My understanding so far from you:

My understanding was you are having external packet generators sending packets towards the EVM from PC and Laptop and you are testing to receive the packets by adding the appropriate entries in the PA. Based on your PA rules, the packets should arrive on the receive queue in the user mode application.

Since you wanted the packets from two different sources, I suggested to add two different entries in PA, which can still route to same queue in the multicore example. 

Note that I had also suggested multicore example is not meant to demonstrate the QOS queue (Interface based or Priority based routing) - You would need to implement it using different APIs.

Spent some time on reviewing your updates and few more inputs on your code changes:

1. You have defined #define NUM_PA_RX_QUEUES 1
2. Why are you using, Queue 6472, as Queue handler? I see you are not going through QMSS LLD API? (Qmss_queueOpen() API)
3. I also do not understand why you had to do this change?
4. You can still use the Qmsss_queueOpen() for opening a specific Queue, if that is your need.
5. why are you defining pktMatchBuf_1 and pktMatch_1, in your code... You wanted to send the packet from the external packet generator.
6. Flow configuration: I see below change from you. Since NUM_PA_RX_QUEUES ==1, Note, this is run only once -- I do not understand your intent of changing this as well. What if you have more receive queues, do you intent to hold the last queue number for the flow configuration?

for(index = 0; index < NUM_PA_RX_QUEUES; index++)
{
System_printf("rxQinfo[%d] = %d\n",index,rxQInfo[index]);
rxFlowCfg.rx_dest_qmgr = rxQInfo[index].qMgr;
rxFlowCfg.rx_dest_qnum = rxQInfo[index].qNum;
}

paEthInfo_t ethInfo = { { 0x10, 0x1f, 0x74, 0x17, 0x25, 0x4f }, /* LAPTOP Src mac = dont care */ ==> Comment is wrong, for SRC MAC to be don't care, all 6 byes for the source should be 0
{ 0x02, 0x18, 0x31, 0x7E, 0x3E, 0x00 }, /* EVM Dest mac */
0, /* vlan = dont care */
0x0800, /* ether type = IPv4 */
0, /* MPLS tag = dont care */
2 /* Input EMAC port = dont care */ ==> For EMAC Port to be don't care, value should be '0' (not 2)
};
paEthInfo_t ethInfo_2 = { { 0xe0, 0xd5, 0x5e, 0x38, 0x66, 0x88 }, /* PC Src mac = dont care */ ==> Comment is wrong, for SRC MAC to be don't care, all 6 byes for the source should be 0
{ 0x02, 0x18, 0x31, 0x7E, 0x3E, 0x00 }, /* EVM Dest mac */
0, /* vlan = dont care */
0x0800, /* ether type = IPv4 */
0, /* MPLS tag = dont care */
2 /* Input EMAC port = dont care */ ==> For EMAC Port to be don't care, value should be '0' (not 2)
};
paIpInfo_t ipInfo = { { 0xc0, 0xa8, 0x00, 0x8B, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* Laptop IP source = dont care */ ==> Comment is wrong, for SRC IP to be don't care, all byes for the source should be 0
{ 0xc0, 0xa8, 0x00, 0x65, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* EVM IP dest */
0, /* SPI = dont care */
0, /* flow = dont care */
pa_IPV4, /* IP type */
0, /* GRE protocol */
0, /* Ip protocol = dont care (TCP or UDP or anything else) */
0, /* TOS */
FALSE, /* TOS = dont care (seperate field since TOS=0 is valid */
0 /* SCTP destination port = dont care */
};
paIpInfo_t ipInfo_2 = { { 0xc0, 0xa8, 0x00, 0x89, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* PC IP source = dont care */ ==> /* Laptop IP source = dont care */ ==> Comment is wrong, for SRC IP to be don't care, all byes for the source should be 0
{ 0xc0, 0xa8, 0x00, 0x65, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* EVM IP dest */
0, /* SPI = dont care */
0, /* flow = dont care */
pa_IPV4, /* IP type */
0, /* GRE protocol */
0, /* Ip protocol = dont care (TCP or UDP or anything else) */
0, /* TOS */
FALSE, /* TOS = dont care (seperate field since TOS=0 is valid */
0 /* SCTP destination port = dont care */
};

Thanks,

Aravind

Dear Aravind,
 
    Correct, I am still not successfull. but I am trying to work on those thanks for keep on supporting.

My Goal :

  1. Using external packet generator to receive the packet on User mode example application.

 2.   Receive the packets on Qos queue 1, 2, 3, 4 etc..
   "I think, there are issues in the way you are checking the packets. Also, by the way, were you able to figure out how to check the logs for mailbox and switch via JTAG?"

    Response:
      Some jtag firmware is corrupted when i updating the firmware to JTAG on EVM board. Still trying to get my results.

    "You should be checking for packets from the user mode example itself"
 
     Anyway we are receiving packet from ethernet port on EVM, for this reason I am checking on tcpdump to capture packets on tcpdump.

    "My understanding was you are having external packet generators sending packets towards the EVM from PC and Laptop and you are testing to receive the packets by adding the appropriate entries in the PA. Based on your PA rules, the packets should arrive on the receive queue in the user mode application."

     Our intent is trying to capture packets on ethernet & route to Qos queue.

     Source code changes Explanation:

     1. Why we changing the queue handler from GENERAL_PURPOSE_QUEUE to Qos queue, because we receiving the packets on Qos1 queue.

     2. "You have defined #define NUM_PA_RX_QUEUES 1" Need to receive the packet from Qos queue(1 to 4)

     3. "You can still use the Qmsss_queueOpen() for opening a specific Queue, if that is your need"
        Response
     
         We need to receive packets from ethernet & routing to Qos queue

     4. why are you defining pktMatchBuf_1 and pktMatch_1, in your code... You wanted to send the packet from the external packet generator.

        Response:
         Using to send packet from external packet generator only.

     5. "Flow configuration: I see below change from you. Since NUM_PA_RX_QUEUES ==1, Note, this is run only once -- I do not understand your intent of changing this as      well. What if you have more receive queues, do you intent to hold the last queue number for the flow configuration?"

    Response:
    
         We need to receive from more queues like Qos queue1, Qos queue2 like that.

     Note:

      Don't confuse my source code changes, because our intent is trying to acheive the receiving packet using external UDP packet generator.

Source code:

1563.multicoreExample.zip

    Logs:

root@AM5K2E04:~# ./paMCExample.out
No command line given.  Forking off 1 tasks
************************************************
*** PA Multi Core Example Started on Core 0 ***
************************************************


Initialized RM_Client0

Initializing Free Descriptors.
QMSS successfully initialized
Unable to open below cppi tx channels...presuming linux has already enabled it
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20  
 Unable to open below cppi Rx channels...presuming linux has already enabled it
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5
 CPPI successfully initialized
PASS successfully initialized
Tx setup successfully done
opening Rx Free descriptor queue 737
Rx Queue info 1027
Rx setup successfully done
Add_MAC successful for Laptop:0
Add_IP successful for LAPTOP:0
Core number 0
Destination for the packet Rx queue: 1027 (Core 0)
Add_Port successful for corenum:0
PASS setup successfully done
Publishing global config Done from SYSINIT Core...
Publishing local config done from core num: 0...
Waiting for all cores to reach the barrier before transmission starts ...


Packet Transmission Start ... core id: 0
Packet Transmission Done.
Wait for all packets to be Received ... core num: 0



Regards,
SanthanaKumarS

Hi Santhana,

"Anyway we are receiving packet from ethernet port on EVM, for this reason I am checking on tcpdump to capture packets on tcpdump."

How would tcpdump receive the packets from the user mode queue that you are programming? 

I think, for your setup (using a different flow from user mode application and routing the packet to host queue at user mode) and expecting the tcpdump to see that packet is not correct.

I am not an expert on Linux Kernel/tcpdump details. I can provide you details on how PA works. Based on the entries and routing information you added,  I see that  after the example is run, you are re-routing the traffic to some queues that are using a different flow than the one used by Linux. Hence tcpdump is not seeing the packets.

To verify if the packets are received at the queue you are programming, you can run the user mode program and wait until you receive the packets at the programmed host queue.

The packet source can be the packet generator (external).

Since you already verified packet routing to the host queue is working with the SendPacket, the LUT entries are good for the signature of the cooked up packet that you are sending from SendPacket().

If you did not receive packet in the user mode application's host queue, from the external packet generator,

the issue could be:

1. Your packet signature is different from external source, hence PA LUT match does not happen

2. OR there is something, preventing the packet reaching to PA. If you need to check if packets are reaching to PA, you need to check the mail box registers at PA to make sure packets are reaching to PA.

I have provided all the details for you to make changes for your needs.

-Thanks,
Aravind

Dear Aravind,

      "Your packet signature is different from external source, hence PA LUT match does not happen"

        Attached my packet generator screenshot with MAC, IP & Ports , I thought my PA LUT is matching only,

       Please check my source code with some API's with my doubts:

       1. Setup_Rx()

       2. rxFlowCfg , Queue number

       3. Receiving packet from HOST descriptor buffer ? 

   Please verify above API's clearly with my source code.

Regards,

SanthanaKumarS

Hi Santhana,

At this time there are many loose ends.

It does not make sense for me to review the code, as I have no idea on what you want to achieve. 

1. You do not know if the packets are reaching PA, so getting the mail box dump working is important.

2. I have no idea on what you are trying to do.. You are adding PA LUT rules in user mode application and using a different packet flow for routing the packets. You are then, killing that application and  later expecting tcpdump to work. 

3. Limiting support to user mode application, I had provided details on what you can inspect if you see packets are reaching to PA, but the match does not happen. I had provided possible debug path. 

At the high level, what do you want to achieve? 

I had provided details on the debug path. I can provide details on if there something not clear. Do you have any questions on PA APIs or PA functionality, which I can help.

-Best,

Aravind

Our main intent is to receive the packets on ethernet in user mode PA LLD application & confirm able to capture packets , If that is achieved, then I routing  the Host queue to QoS routing 

I thought any buffer descriptor issues or Rx Flow configuration?

Hi,

  "You need to find the CPSW statistics register dumps  for all the MAC ports"

    Response:

     I am confirmed below Initialize CPSW is not called on Linux Userspace API for PA lld.

     #ifndef __LINUX_USER_SPACE
 
        if (no_bootMode == TRUE)
        {
            /* Initialize the CPSW switch */
            if (Init_Cpsw () != 0)
            {
                System_printf ("Ethernet subsystem init failed \n");
                System_flush();
                APP_exit (-1);
            }
            else
            {
                System_printf ("Ethernet subsystem successfully initialized \n");
                System_flush();
            }
        }
#endif

"Please send your dump as below, which can help me to comment if the packets are reaching to PA or not. If you can send multiple dumps of this memory block, it would be further helpful"

       Response:

             Tomorrow I will provide the information for PA mailbox registers. waiting for external emulator to provide the memory block.

      My requirement is used to receive the packets on PC from EVM & verify the packets on PA LLD example application code.

Regards,

SanthanaKumarS

Hi Santhana,

Let me know after you update your user mode application as per your needs. Like I mentioned before, the Pa Example does not showcase the interface based or priority based routing, which is needed for QoS.

You would need to write your application following the sequences as done in PA example, which is meant for a simple MAC/IP/UDP match.

Any issue you face would need to be debugged using the approach I suggested before.

Please close the thread if you received all the details needed for your to proceed coding for your needs.

-thanks,

Aravind

Hi Aravind,

   We are trying to using PDK RTOS k2e version in K2E board, reference of PDK RTOS version  

  We tried PA LLD RTOS user application to flashed on board K2E, but our ethernet port not displaying MAC & IP address on CCS.

   Using NIMU application(NDK stack) to displaying the K2E ethernet port. It's working.

  Question:

     1. Need to integrate NDK & PA example application to bringup the ethernet interfaces?

     2. Any other way it's possible to check my IP address on PA emac example?

Regards,

SanthanaKumarS

Hi Santhana,

The new questions that you are posting are not related to this post. Please close this thread and you may start a new thread for your new set of questions.

1. We tried PA LLD RTOS user application to flashed on board K2E, but our ethernet port not displaying MAC & IP address on CCS

[Aravind]: I do not understand your question. Do you mean linux user space application? All these days, you were modifying the user mode LLD example, you were using PLSDK releases (not PRSDK (RTOS) release).

2. PA example and NDK are two different things. For NDK you need to refer to NIMU examples under transport, which is again not in PLSDK release, You would need the RTOS PDK (PRSDK) release.

NDK- NIMU examples demonstrate ethernet bring up examples from RTOS side. They print the IP addresses.

3. Please note that Linux and NDK do not co-exist. 

4. "Any other way it's possible to check my IP address on PA emac example?" 

Like I mentioned before PA examples do not implement full network stack -  NDK is meant for RTOS side Network stack support. Are you looking for just knowing the IP address in the packet, from PA?

-Thanks,
Aravind

Dear Aravind,

   Thanks for replying to the post.

Created new thread for displaying the IP address:e2e.ti.com/.../964283

[Aravind]: I do not understand your question. Do you mean linux user space application? All these days, you were modifying the user mode LLD example, you were using PLSDK releases (not PRSDK (RTOS) release).
 
  Response:
    Previous post You replied :
"I am not a Linux expert for commenting on the CPSW dump issues you are seeing. Checking with local Linux experts if they have a say on your CPSW Method 1 and 2.

Alternatively, I think you can use CCS (With JTAG) connect to C66x Core and view the memory locations for debug from C66X core"

   Why I am switch to PDK RTOS example?

     In  Linux side  PA emac example not receive the external packets & debugging method to receive the packets on PA, It's very compilicated on that side [In Old post, You mentioned on that You will support on CCS side to check the PA side]


   "Like I mentioned before PA examples do not implement full network stack -  NDK is meant for RTOS side Network stack support. Are you looking for just knowing the IP address in the packet, from PA?"

  Response:
    1. Correct PA example does not have network stack.
    
    2. We are able to flash the PA emac example [PDK RTOS] on K2E board. Need to check the IP address of EVM board. because we need to test the packet generator to sending the packet to EVM from PC.

Regards,
SanthanaKumarS

Hi Santhana,

Thanks.

If there is nothing to discuss in this thread, please close this thread.

-Best,
Aravind