• State Resolved
  • Locked Locked
  • Replies 8 replies
  • Answers 1 answer
  • Subscribers 82 subscribers
  • Views 814 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

F28M36P63C2: USB TX and RX interrupts stop firing

Todd Atadero
Prodigy 60 points
Part Number: F28M36P63C2
Other Parts Discussed in Thread: C2000WARE

Hello,

I am working with a F28M36 Concerto Control card with an F28M36P63C2 chip (Rev B) on it.  We are running CCS 6.1.1.00022 and using control suite v3.4.9 with the v220 libraries under device_support. Currently on the M3 side I am running as a host trying to gather data from 3 USB devices running through a hub as bulk endpoints with a custom usb driver.  In my main loop I work serially through each device doing a USBHCDPipeSchedule to request data from a particular device on the out pipe.  When this transmit is complete I then do another pipe schedule to read the data in from the in pipe.  When the pipeincallback interrupt happens I read the data off the in pipe using USBHCDPipeReadNonBlocking and store it for processing later in main loop.  This data is then retrieved from the M3 through a TCPIP connection.  I am using LWIP on the M3.  

Currently I can collect data from the USB devices for about an hour before they finally stop responding.  This time can vary from 20 minutes to 2 hours.  When we stop receiving data I believe it is because we no longer are getting TX interrupts (not sure about RX since we no longer are requesting data) in the USB stack.   When our reading stops USB_TXIS and USB_RXIS will always read 0.    Reviewing the different USB_TXCSRL registers both the FIFONE and TXRDY flags are set.  No other flags, such as stall, error, or other potential error flags are set.  in the USB_IS register the USB_IS_SOF flag is always set.  

Before using lwIP we were using the uIP libraries.  When we were using these the USB devices seemed to stay up for much longer.  We did not detect any crashes in reading from them, however uIP was crashing around the 24 hour mark so I cannot guarantee the USB devices would never go down.  After moving to lwIP the ip communications will now stay up but USB goes down.  Because of the added overhead of lwIP I have looked at reducing as much of the processing of data as I can outside of the interrupts to keep the speed up.  This has increased our up time but still having problems.  

  • 0
    TI__Mastermind 32995 points
    Hi,

    By reading the USB_IS register, it clears the SOF flag. Can you check the NVIC registers. There may be some race condition which is preventing interrupts from being received by the CPU. They may be getting blocked along the way.

    Also, can you using a USB analyzer to analyze the communication and find the irregularity?

    Regards,
    sal
  • 0 in reply to Sal Pezzino
    Prodigy 60 points

    I was able to get the usb system to stay up overnight last night.  I did this by changing the SysTickPeriodSet function from being 

    SysCtlClockGet(SYSTEM_CLOCK_SPEED) / 100

    To

    SysCtlClockGet(SYSTEM_CLOCK_SPEED) / 175

    If/When the system goes down again I will take a look at the NVIC registers. I am currently looking into purchasing a USB protocol analyzer at the moment.  Thank you for the help.

    Todd

  • 0 in reply to Todd Atadero
    Prodigy 60 points
    The USB system crashed again this afternoon. This was close to 24 hours of running. I know setting the value in SysTickPeriodSet to 150 has a crash happen in around 3 hours. If I set it to 200 then my USB devices can't be enumerated.

    Here are the NVIC registers. I believe USB is at IRQ 60 and lwIP is at 58. The only changes I am seeing are the PEND1 and UNPEND1 change between 0x00001000 and 0x00001400. The SysTick interrupt method is still being entered into and lwip is still responding.

    NVIC_EN0 0x00000000 IRQ 0 to 31 Set Enable Register [Memory Mapped]
    NVIC_EN1 0x00001400 IRQ 32 to 63 Set Enable Register [Memory Mapped]
    NVIC_EN2 0x00000000 IRQ 64 to 95 Set Enable Register [Memory Mapped]
    NVIC_DIS0 0x00000000 IRQ 0 to 31 Clear Enable Reg [Memory Mapped]
    NVIC_DIS1 0x00001400 IRQ 32 to 63 Clear Enable Reg [Memory Mapped]
    NVIC_DIS2 0x00000000 IRQ 64 to 95 Clear Enable Reg [Memory Mapped]
    NVIC_PEND0 0x00000000 IRQ 0 to 31 Set Pending Register [Memory Mapped]
    NVIC_PEND1 0x00001400 IRQ 32 to 63 Set Pending Reg [Memory Mapped]
    NVIC_PEND2 0x00000000 IRQ 64 to 95 Set Pending Reg [Memory Mapped]
    NVIC_UNPEND0 0x00000000 IRQ 0 to 31 Clear Pending Reg [Memory Mapped]
    NVIC_UNPEND1 0x00001400 IRQ 32 to 63 Clear Pending Reg [Memory Mapped]
    NVIC_UNPEND2 0x00000000 IRQ 64 to 95 Clear Pending Reg [Memory Mapped]
    NVIC_ACTIVE0 0x00000000 IRQ 0 to 31 Active Register [Memory Mapped]
    NVIC_ACTIVE1 0x00000000 IRQ 32 to 63 Active Register [Memory Mapped]
    NVIC_ACTIVE2 0x00000000 IRQ 64 to 95 Active Register [Memory Mapped]


    The USB analyzer should show up monday.

    Best Regards,
    Todd
  • 0 in reply to Todd Atadero
    TI__Mastermind 32995 points
    What are you using the SysTick for? If you are not using it, then you can probably remove it. Can you see what happens if you remove it?

    It appears there may be some race condition that is deadlocking the interrupts. I recommend you start looking for a potential race condition in your code.

    sal
  • 0 in reply to Sal Pezzino
    Prodigy 60 points

    SysTick is being used in the provided lwIP example in control suite.  It was also setup in the OTG example though we are running as host now so it is not needed.  From enet_lwip.c under the F28m36x_examples_Master there is a function setup..

    Looking through this function I would not be surprised if this was causing the race condition. I have removed this and lwIP is still responding to requests. I am running tests right now with hopes this fixes the issue and will probably update on the 2nd.

    Best Regards,
    Todd Atadero

    void
    SysTickIntHandler(void)
{
    //
    // Call the lwIP timer handler.
    //
    lwIPTimer(SYSTICKMS);
}


  • 0 in reply to Todd Atadero
    Prodigy 60 points

    It went down again over the new years.  I have hooked up the packet analyzer to see what we can capture when a crash occurs and will update when I have some of that data.  In the mean time I am looking at places where there might be a race condition in my code.  I have taken almost all actual processing out of my USB driver interrupts and moved them to my main function.  While looking for other potential things to do I did come across a post of yours from 2015 dealing with SOF interrupt issues and code modifications that should be made to the usb libraries to help with this.  Looking at the v220 usb libraries, it looks like this change was not implemented in that code.  

    https://e2e.ti.com/support/microcontrollers/c2000/f/171/p/397570/1458462?pi320098=3

    Should I look into implementing this for the USB libraries?  

    Best Regards,

    Todd Atadero

  • 0 in reply to Todd Atadero
    TI__Mastermind 32995 points
    That issue should only pertain to the USB library running on the C28x. But it would not hurt to try it at this point.

    Please see the latest code in C2000Ware. I believe there were two functions modified. One in usb.c driver (USBIntStatusControl)and another in the library (USBDeviceIntHandlerInternal).

    Let me know if this makes any difference.

    sal
  • 0 in reply to Sal Pezzino
    Prodigy 60 points

    Ok, I believe I have the problem solved, or at a minimum, the issue fully identified. It turns out our usb devices were crashing after a set amount of time and all would crash around the same time due to them all using the memory resources at the same rate. However, when they would crash, their USB stack would still continue to run. They would stop processing data and would send a NAK command back with every data request. The M3 would continue to resend the data since it wasn't accepted. This was why we would no longer get any interrupts. Because the devices were also still responding, just with NAKs, we were not getting stall flags either that would indicate the device had failed.

    I am still losing the first enumerated device, but based on the packet analyzer, I believe it is the device side that is still having the issue. Thank you for the help.

    Best Regards,
    Todd Atadero