PASS setup failure - timesout waiting for response from PA to Pa_addPort

Hi, Aamir:

I am sorry, but I am totally confused. I thought that you are calling Pa_addCustomLut2 for a range of UDP ports. Why did you call API Pa_addPort() as well?
How do you call those APIs? Do you wait for the response between calls?

Can you send us the code and/or the simulator log?

Best regards,

Eric

Eric,

Sorry for the delay in replying, I was out of town. Sorry I confused you but I had used the wrong name of the function. The post should have read as shown again below. I am calling the Pa_addCustomLUT2 and not Pa_addPort. I am waiting for responses for each call. The Add_Port function is similar in nature to the Add_Port function in the emacs example except I have made some changes to do with the calls being switched from Pa_addPort to Pa_addCustomLUT2 and passing in parameters. If you would like to see the code, I can send it to you offline or through some other means if that is okay. I am not running it on the simulator as I am not able to run the same desired setup on the simulator. It is a long story but I was having some issues with setting up the switch so that I can have packets arrive from an outside machine through our network to my PC then through the network card to the shannon simulator and then through the switch to the PA and then onto the corePac simulator.

I have been successful in getting UDP for IPv4, Ipv6 and ICMP for Ipv4 and Ipv6 working with 8 entries each for a range of ports in the UDP message. However, I am not able to get all of the 32 add entries for LUT2 (8 each for IPv4 UDP, IPv6 UDP, ICMP ipv4 and ICMP ipv6) working at the same time, Whenever I try to do that I get a timeout waiting for response from the PA to Pa_addCustomLUT2 command. The Add_Port command is called 8 times each for each of the above 4 types of IP packets, ICMP, UDP etc.

If I make the code use just 2 entries for each type of packet so total 8 Add_port calls, it works fine but then gives me the error when I bump it to 32 as I would like.

Any reasons why you think the PA is not able to respond back in time to the commands to the PA to add the range of ports?''In the Add_Port function, I call the Pa_addCustomLUT2 function is called and returns no error then it is marked as a configuration packet and pushed to the PA and then the host waits for the response from the PA. It is at this stage that it timeouts waiting for the response.

Hi, Aamir:

It is most likely that you run out the CPPI buffers for command response.

Can you show me your CPPI flow configuration and link buffer queue settings?
You should be able to run this portion of configuration code at simulator or you can send me the *.out file.

Best regards,

Eric

I am making use of the cppi_qmss_mgmt.c file from the emacs example directory. I have not made any change in the number of descriptors etc being used by the TI emacs Example in the pdk directory.

Aamir

Eric,

I changed the number of HOST descriptors from 32 to 64 for the example and it worked i.e. NUM_HOST_DESC is set to 64.

I guess I was running out of descriptors. In the Emacs example in the Add_Port function when it calls the Pa_addPort and then pushes the command to the PA. After it receives a reply back it puts back the descriptor to the queue for use by the next Pa_addPort does it not? So I am not sure why I run out of buffers as with every push to the PA of the command packet, the code on receving the PA response puts the descriptor etc back for future use on the TxFreeQ. Am I missing something here.

Aamir

1882.pa_mgmt.zip

Hi, Aamir:

Thank you very much for pointing out another issue at PA example!
PA EMAC example does not serve as a good example for memory buffer usage and there is also some memory leak.
We shall update all PA examples accordingly for the next BIOS-MCSDK release. Attach please find a sample update for your reference only.
Let me summarize the general rules of memory buffer usage at NetCP.

1. The PKTDMA will return the CPPI descriptor to the queue specified at the "return queue" filed by MACRO Cppi_setReturnQueue after it finishes processing the descriptor from any hardware queue.
2. The output PKTDMA of PASS will get (dequeue) the CPPI buffer descriptors from the rx queues specify by the CPPI flow and send the rx packets to the destination queue specified by the route information and/or the command reply information.

PA EMAC example intended buffer usage:

Tx Operation (Send packets to PASS):

• Tx Free queue(gTxFreeQHnd) contains NUM_TX_DESC of CPPI descriptors.
• The return queue of each descriptor is set to the Tx Free queue so that it will recycle to itself.
• The CPPI descriptor does not contain buffer.
• The descriptor will be used for both configuration command packet and transmit data packet.

Rx Operation (Receive packets from PASS):

• Rx Free queue(gRxFreeQHnd) contains NUM_RX_DESC of CPPI descriptors.
• The return queue of each descriptor is set to the Rx Free queue so that it will recycle to itself for hardware queue operation.
• Each CPPI descriptor conatins a 1518-byte packet buffer
• The descriptor will be used for both configuration command response packet and received data packet.
• The Rx Free queue is specified by the CPPI flow configuration (gRxFlowHnd)
• The application should recycle the descriptor back to the Rx Free queue manually.

Could you please click "verify answer" button if I answer your questions?!

Best regards,

Eric

Eric,

Thanks for the updated pa_mgmt code. I see the problem now, the code was putting it back to the TxFreeQ when it should have been going back to the RxFreeQ hence the running out of decriptors. The flow ID addition in the pa_addIp is not necessary though as it is not to dest HOST - Am I correct in that?  I think I was hitting other issues too when trying to implement the ICMP nFailInfo in the pa_addIp call for next stage failure routing when I have optional headers. I think it is related to these issues as I should be able to do what I am trying to do (I think). I was trying to have the swInfo0 set to 0xbbbbbbbb for the nFailInfo routing to distinguish it from the 0xaaaaaaaa which is directed towards a specific core and the 0xbbbbbbbb being broadcast to all cores in my multicore implementation which I am going to start working on next.

My nFailInfo for icmp in the pa_addIp is as follows: dest=host,customType=CUSTOM_LUT2,customIndex=custIndex (passed in), swInfo0=0xbbbbbbbb,mRouteIndex=-1 (for now will need to change when multicore), flowId=get flow ID from gRxFlowHnd, queue=queue number from gRxQHnd. so when it fails my custom LUT2 rules for ICMP it uses the fail route to get to the HOST.

Thanks, Aamir

Hi Eric,

Thanks for solving my problem!

I've been terribly frustrated with this same problem since Monday morning and just stumbled on this thread.  (I'm using multicoreExample).  After 128 PA commands, I get the same timeout.  I changed "Tx" to "Rx" in 3 places and it works!

Thanks for posting pa_mgmt.c because it really clarifies it.  Please let me know if we should additionally change "TxFreeQHnd" to "gRxFreeQHnd" at lines 341, 539, and 722?

Greg

Greg,

I noticed the same additional changes you mentioned. Yes, you should change them as they are dealing with the same Rx Queue.

Thanks, Aamir

Hi, Greg and Aamir:

I am sorry for the confusion due to bugs at the PA examples.
We shall fix all examples at the next BIOS-MCSDK release (2.0.8).

Yes, all receive buffers should be returned to the rx free queue pointed by gRxFreeQHnd.

Best regards,

Eric

Thanks Eric,

I got my optional headers with a different swInfo0 code to work.

I had just one more question in regards to the updated pa_mgmt.c code you sent earlier and that is to do with setting pCmdDataBuffer to paCmdBuf. Is there any problem with using the old code ?

When is the updated mcsdk release out?

Thanks, Aamir

thanks Eric. I thought as much.

Aamir

Eric,

Note though that the declaration of paCmdBuf in the pa_mgmt.c file is wrong as it uses the buffer in the Add_Port and Add_IpAddress functions expecting a buffer size of 1518 while the Add_Mac makes use of 320 as the buffer size.

static UInt8 paCmdBuf[320]; should be paCmdBuf[1518];

I forgot to mention that to you earlier.

Aamir

Aamir:

Thanks for points this out.
I intended to use smaller buffer size since the command packet is small.
In all the addXXX function, the code should be changed as the followings:

    /* Populate the Rx free descriptor with the buffer we just allocated. */
    Cppi_setData (Cppi_DescType_HOST, (Cppi_Desc *)pHostDesc, (UInt8 *)Convert_CoreLocal2GlobalAddr((UInt32)pCmdDataBuffer), 1518);

==>

    /* Populate the Rx free descriptor with the buffer we just allocated. */
    Cppi_setData (Cppi_DescType_HOST, (Cppi_Desc *)pHostDesc, (UInt8 *)Convert_CoreLocal2GlobalAddr((UInt32)pCmdDataBuffer), sizeof(paCmdBuf))

Please note that pCmdDataBuffer can be replaced with paCmdBuf as well.

Please note that you can also prepare the tx CPPI descriptor with pre-allocated buffer in stead of the static local buffer so that you do not need to wait for command completion before you issue another configuration command.

We will clean up all PA examples at the next release.

Best regards,

Eric