DM648 CPSW - HostPend Interrupt - RX_HDP[0] weird address

Hi Krishna,

I'm also working with DM648 and I have a similar application. My application sends a stream of data (UDP packets) to a PC. In parallel to this, the PC sends to DM648 a few commands per second and DM648 responds. 

At the beginning, only streaming of data was implemented and everything worked fine for many days. Then I added an exchange of additional information and DM648 stopped responding after a few hours. Later, I found out that it is enough to ping DM648 continuously or just have multicast messages in the net while DM648 is streaming to invoke the problem.

I made some experiments while investigating the problem and in some cases I also saw 0x2000 in DMASTATUS register, but not in all cases. The reason is still unknown and the investigation is in progress.

The experiments are described in the following thread http://e2e.ti.com/support/embedded/bios/f/355/p/223029/819604.aspx#819604 (the stream was started by another engineer ). Maybe you can find it useful.

Finally, to avoid my mistakes, I took EVM board an an example, provided by TI. I slightly modified the example to make it send UDP messages in an endless loop. I started this example on DM648 and continuous ping on a PC and DM648 stopped responding to ping and sending data in a few hours. I hope TI will be able to reproduce the situation and help me.

Due to the fact that our applications are quite similar, I have a question to you. You have said that in most cases your application worked fine. Have you tested it for a long time in the situation when other traffic exist? Also, could you please explain a little further what does loopback test do?

Best Regards

Victor

I have been trying to get some inputs as to what could contribute to this issue.  So far here is the feedback I have received:

Reading through the description, I can only suspect that either there is cache coherency related issues i.e. when the updated descriptors are handed to the CPPI DMA it is not being properly flushed (assuming the descriptors are in cached region of DDR).

Hi Brad!

Thanks a lot for your attention to our problem! If it can help, I can do any additional check or experiments.

I also saw 0x2000 in DMASTATUS during my tests but in  most cases I just didn't see any RX interrupts without any errors (as you can see in http://e2e.ti.com/support/embedded/bios/f/355/p/223029/835523.aspx#835523).  I can try to reproduce this problem if you say what is the most efficient way to do it.

Maybe we deal with the same problem which shows up in the different ways.

Thanks

Victor

Hi Brad,

I have tried to check your idea about cache.  I know that NDK documentation requires cache to be enabled, but anyway I disabled the cache in my test project and ran it.

The system works significantly slower. The test project tries to send data to PC. It was able to send not more then 30 Mb/sec but at the same time I was not able to create the problem with receiving packets by NDK.

When I overload the system with incoming traffic it stops answering to ping but it recovers as soon as I remove the pressure. In the situation when the incoming traffic is less, it sends data and answers to ping. I understand, that because of the slowness, the conditions in which NDK works, are different. But still the idea with cache seems to be promising. How else can we check the idea?

Thanks!

Victor

Victor,

A couple thoughts come to mind:

1. I don't know how much visibility you would have into this, but I wonder if the various buffers on which the NDK operates are padded to 128 bytes (i.e. L2 line size).  Presumably the NDK will be performing cache operations on the data buffers being passed around.  Therefore it could be important to make sure those buffers are aligned to a 128 byte boundary and padded to a multiple of 128 bytes.  Otherwise if "other data" happens to reside in the same cache line as your ethernet data then the EMAC might inadvertently step on something when performing manual cache operations.
2. Obviously things perform MUCH slower with the cache disabled.  You might potentially have some kind of timing issue somewhere.  Perhaps putting some delays at various places in your code to see if delays make any difference at all in terms of the behavior.  Watch out for the optimizer when doing dummy delay loops, i.e. make sure you declare it volatile so the optimizer doesn't remove your delay entirely.

Best regards,
Brad

Hi Brad,

it happened that I'm discussing my problem in two threads on e2e. The second one (actually the first one) is http://e2e.ti.com/support/embedded/bios/f/355/p/223029/842421.aspx#842421

I don't know about all the buffers which NDK uses but  pBufMem and pHdrMem are padded to 128 bytes. Just as experiment I made a part of the memory uncacheable and placed these buffers in that memory. The system worked quite long (more then 12 hours) and I started to think that it helped but today the problem appeared again.

In this experiment the system worked with its normal speed but sending/receiving data via Ethernet worked slower (about 100 Mb/s). Maybe it worked longer because of lower speed. So, it could indeed be a timing issue but I can't put delays in my code because I use heloWorld example which contains only a few lines of my code. And putting delays in NDK requires better understanding of its work. Stewen Connell has promissed to try to reproduce my problem. I'm waiting for his results now.

Thanks for your help!

Victor

Dear Krishna,

in your first post you mentioned that you did loopback tests. I also tried to do a loopback test but I didn't succeed. Could you please explain how you did it?

Thanks a lot!

Victor

Hi Victor,

I have successfully tried loopback at GMAC, SGMII and PHY level.

The simplest of all is the GMAC loopback, as per the CPSW docs just set the Loopback and GMII_en bit high in the GMAC control and send out a packet. You will receive the same data back.

At what level are you trying your loopback ?

Regards

Krishna