• State
  • Locked Locked
  • Replies 8 replies
  • Subscribers 98 subscribers
  • Views 676 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DSPLINK hanging?

Flamingo
Expert 2290 points

(OMAP_L137, codec engine)

I finally have SPI1 working, including the necessary timing for the SPI ADC and my background tasks (one of which handles data collection, and the other handles data-processing). Both are working. That is, the data are collected successfully and the pointer to the data is mailed to the second task, which processes it and "publishes" it, releasing the pointer. The SPI1 activity starts and ends and restarts as desired.

The part that no longer works is the ARM9 app's ability to talk to the DSP. That is, the app appears to have hung after kicking off the SPI data collection. I have verified that the codec _control function returned _EOK, but I believe that the app side never saw it. That is, I don't think the call on the app side ever returned.

I have verified that the stacks are okay (using less than 10% of the available). I was concerned that the background tasks were so busy that the codec (on the DSP) was no longer getting a time-slice, so I checked priorities, and the codec has priority 5 and the background tasks both have priority 1, so that isn't it.

I can run the same code with a memory-to-memory transfer via the DMA (rather than the SPI to memory transfer), with no problem (no buffer overflow, etc.) I can run the SPI transmit with the receive "disconnected" with no problem.

I know that the -L137 I have is old and has the SPI bug, but I was told that the bug would result in bad data, not hanging. If that isn't true, would someone let me know? Also, if anyone has debugged a similar behavior, please give me some hints. In the past when the app hung, it was because something bad had happened on the DSP side (overrunning buffers, etc). To minimize the chance of that, I've hard-coded the SPI transfer to 10 16-bit elements, even though the buffer is large enough to hold 10000.

I've checked the memory map, and the background task mailboxes hang out in the same region as DSPLINK_shmBaseAddress. Would posting too many messages to the mailboxes overflow the mailbox? Or would it just timeout?

Thanks for any help.

  • 0
    Expert 2290 points

    More info:

    I've continued trying to characterize this failure by disabling the EDMA SPI1 Rx event/PaRAM set and "receiving" data via a SPI Rx interrupt (interrupt 43). When the interrupt triggers, I read the receive buffer, but just discard the data. I count the number of received SPI elements and disable the receive interrupt when I get all that I'm expecting. (I disable the interrupt by writing 0x01000000 to SPIINT0 (0x01e12008).

    Even with the DMA out of the picture and with the SPI data thrown away, the DSPLINK/codec engine is frozen.

  • 0 in reply to Flamingo
    TI__Guru*** 125430 points

    Try to keep backing out your EDMA/SPI stuff until dsplink works again.  Once you know what exactly is killing dsplink perhaps I can help explain why or explain how to get around it.  Right now it's too hard to tell.

  • 0 in reply to Brad Griffis
    Expert 2290 points

    More info:

    I can poll SPI1 receive data without killing DSPLINK. Next on my list to try is rebuilding the Kernel to use a different or no console. That will take a little while, since I'll ahve to separate the boards to get the UART connection hooked up to re-flash the Kernel.

    Summarizing information to date:

    SPI1 boots as Linux console, but I telnet in, so never use the console. The only thing I see go to the console is some messages from loadmodules.sh. When my server code runs, it changes the pinmux to assign UART2 pins to SPI1 pins (4-pin w/ cs).

    Normally, my server would use PaRAM set 18 to receive SPI1 data, but when I tried that, I lost my GT_trace output (on the app side, but both app and server trace statements).

    The app seems to hang when I first enable SPI1 receive via either EDMA events or, skipping EDMA, the Rx interrupt. It does not hang when I poll SPI1. In other words, I think it might be event/interrupt related, but not pinmux related.

    I've tried various things to "back out" my EDMA/SPI stuff, many experiments are not detailed above. The gist of it is, if I enable the event/interrupt, it hangs. That is, I can set everything up without a problem, but if I set DMAREQEN in SPIINT0 or if I enable the receive interrupt AND I initiate transfers, it hangs. If I set DMAREQEN or enable the receive interrupt and don't initiate transfers, everything works. Or if I transmit, but don't receive or if I transmit and poll to receive, everything works.

  • 0 in reply to Flamingo
    Expert 2290 points

    I've tried a Kernel that sets the pinmux to SPI1 right away. I've also changed the boot parameters to use ttys9 (which causes Linux to disable the console, since that device is not connected). Neither helped the cause.

    It was suggested that I use some line other than SPI_ENA to trigger my transmit events, so I had the HW guy swap GPIO 92 and GPIO 65. (65 used to be one of my LEDs)

    That didn't help either.

  • 0 in reply to Flamingo
    TI__Guru*** 125430 points

    Flamingo said:

    I've tried various things to "back out" my EDMA/SPI stuff, many experiments are not detailed above. The gist of it is, if I enable the event/interrupt, it hangs. That is, I can set everything up without a problem, but if I set DMAREQEN in SPIINT0 or if I enable the receive interrupt AND I initiate transfers, it hangs. If I set DMAREQEN or enable the receive interrupt and don't initiate transfers, everything works. Or if I transmit, but don't receive or if I transmit and poll to receive, everything works.

    It sounds like the interrupt is tied into the issue.  Are you perhaps stomping on someone else's interrupt?  As a test to help diagnose the issue, please try the following:

    1. Comment out the calls to your SPI functions such that you're not setting up SPI and any of the associated interrupts or DMA transfers.
    2. Run the application.
    3. Halt while it is in progress.
    4. Tell me the state of the INTMUX1 (0x01800104), INTMUX2 (0x01800108), INTMUX3 (0x0180010C)

    Those registers should tell us if any interrupts have been mapped.  We should then be able to see which interrupts are still free for use.

  • 0 in reply to Brad Griffis
    Expert 2290 points

    Okay, here is what I did. My interrupts are set up in server.tcf. If I disable SPI1 in the code, but leave server.tcf as is

    INTMUX1 = 0x08064305, INTMUX2 = 0x0a2b0941, INTMUX3 = 0x28040d0b

    If I comment out all the interrupt definitions in my server.tcf file

    INTMUX1 = 0x07064305, INTMUX2 = 0xa0a0908, INTMUX3 = 0x28040d0b

    I did not cycle power/reset between the two runs and they were performed in the order listed above. (I don't know if that matters.)

    My server.tcf file maps HWI7 to the DMA transfer complete interrupt, HWI8 to GPIO bank 0 (rising edge of GPIO0_9), HWI10 to SPI INT. During testing, I've also been known to map HWI 9 to GPIO bank 5 (falling edge of GPIO5_12).

    The intent is to enable 10 for SPI errors, although I've used it for testing the SPI receive interrupt (instead of using the receive event via DMA).

    Near as I can tell (using a scope), the SPI is running just fine on the DSP. I've just lost my communications. I've successfully run transmit only from the SPI interrupt withou impacting ARM9/DSP communications.

    What I'm trying to say is that this is weird in the extreme. Somehow appropriate for the Halloween season. Thanks for any insight.

  • 0 in reply to Flamingo
    Expert 2290 points

    UPDATE!

    TI has reproduced the ARM9/DSP loss of communications using a modified version of the randgen codec, the TI DSP SPI driver, SPI0 in loopback.

    They are investigating the failure mode.

  • 0 in reply to Flamingo
    Expert 2290 points

    The first thing we tried was configuring the SPI1 during boot. To accomplish that, we configured TTYS9 for the console. (It doesn't exist, but we were telnetting in to avoid a collision with SPI1 anyway, so we don't need a console for anything except uboot.)

    That did no good, so we disabled SPI support in the kernel. That resulted in no SPI clocks at all. TI explained that a kernel built with no SPI support will leave the module turned off, so we added power configuration to the DSP code (as well as pinmux setup for SPI1.) That resulted in ARM9/DSP communications most of the time (but not all the time.)

    Occasionally, we would catch the ARM9 stuck in an infinite loop polling UART2 LSR for a received character. This was not occurring in the kernel memory space and was not related to my app code. Maybe it was in the codec engine. I'm not sure how to tell what symbol was associated with the address where the loop was.

    Finally, we rebuilt the kernel with no SPI and no UART support. This resulted in restored communications! This is not a long-term solution, but we haven't had time to try editing kernel files to disable just one of each.

    I have moved on for now (trying to figure out why restoring the communications have degraded the SPI timing so much that SPI traffic completely halts.)

    I hope this helps someone else with similar problems.