L138 USB DRVVBUS Interrupt Not Firing

Hi Brian,

Are you able to reproduce the problem ?

So that I can it try on my end and debug the problem.

I am able to reproduce the problem on my board, but I'm not sure if it would happen on one of the development kits.  All I have to do is unplug the GPS module I have plugged into my board and occasionally the driver gets latched to where I can't connect anything to my board (either as host or client) until I reboot.  As I mentioned, it seems like there is probably a race condition involved because it doesn't happen every time, but that also means that the clock speed my board is running at (100 MHz) or some other similar configuration might explain why I'm seeing this and no one else is. 

Mostly what I'm wondering is how it's possible that I can see the DRVVBUS bit set in the INTMASKEDR and INTSRCR registers, and yet the interrupt is not occurring.  I'm wondering if there is a setting in a register that I don't know about that would prevent the interrupt or if it is possible that if the DRVVBUS change event occurs under specific conditions that the interrupt will not occur.   

Thanks,

Brian

Hi Brian,

Thanks for posting your observation.

Is it happens, whenever you unplug only the GPS module or other devices too? You mentioned here it is occassional... Once in how many times this error is been observed?

Would you please post your test case with steps? So that we will try to reproduce in the development kits.

Regards,

Shankari

At this point I only have the one device that I'm testing.  I've been testing this at my bench and I've seen it tens, up to maybe a hundred times.  At times it happens every single time I unplug the GPS device, and other times I can go for 30 minutes of casually plugging and unplugging the device without a failure. 

My test case is really very simple.  Plug in the device, optionally open /dev/ttyUSB0 to see that the GPS device is sending data, then unplug. As I mentioned before, it may be hard to reproduce the issue since otherwise a lot of other people would have reported the issue.  I'm pretty sure it's a race condition that is exposed due to some timing on my board. 

I've determined why the driver is failing - it's not getting the DRVVBUS interrupt when it should be.  All I really want to know, from someone who knows this processor better than I do, is how it is possible that I can see the DRVVBUS interrupt signalled in the interrupt source register, but no interrupt occurs. 

Thanks,

Brian

I am not that USB expert, but i suggest:

Open drivers/usb/musb/musb_core.c and start to put some traces into

static irqreturn_t musb_stage0_irq(struct musb *musb, u8 int_usb,
				u8 devctl)

First point where int is fired is here, then, if this routine is entered, debug and follow the path, then finally open drivers/usb/musb/da8xx.c and put some traces into

static irqreturn_t da8xx_musb_interrupt(int irq, void *hci)

See where the interrupt is lost or skipped on this way.

It was my understanding that da8xx_musb_interrupt() was installed directly as the interrupt handler since musb->isr is set to da8xx_musb_interrupt() in musb_platform_init, and then we have the line:

if (request_irq(nIrq, musb->isr, 0, dev_name(dev), musb)) {
    ...
}

in musb_init_controller() that should install da8xx_musb_interrupt() as the interrupt handler for the USB interrupt.

If this is correct, I have installed diagnostics in da8xx_musb_interrupt() which is how I know that the interrupt is not firing.  It's definitely strange behavior.

Correct.

Sorry for my previous incorrect answer. The interrupt path starts from:

da8xx_musb_interrupt() 

as a first callback, that in certain cases then calls musb_interrupt(),

and this last routine in certain cases calls  musb_stage0_irq() or other routines.

Good luck.

Hi Brian,

When I experimented with the starterware USB driver code with OMAPL138 LCDK, not able to observe any issues with DVCTL:session bit or INTSRCR - Connect/Disconnect bit while plugging in and plugging out the pen drive on OTG USB port.

Brian said:

I've determined why the driver is failing - it's not getting the DRVVBUS interrupt when it should be.  All I really want to know, from someone who knows this processor better than I do, is how it is possible that I can see the DRVVBUS interrupt signalled in the interrupt source register, but no interrupt occurs.

Please send us the register dump of USB0ARM  during the error condition.

Is your GPS module a self powered device or receiving power from oMAPL138's USB 2.0 Host controller?

And also, please make sure that the pin multiplexing, CFGCHIP2 and PHYclock is good as per the guidelines given in the TRM of OMAPL138 below.

When the USB2.0 controller assumes the role of a host, it is tasked to source the required 5V supply via USB0_VBUS (must be at least≥ 4.75V) pin. The USB2.0 controller makes use of an external charge pump or logic by enabling and disabling the external power logic from the USB2.0 controller core level. It uses the USB0_DRVVBUS for controlling the enable/disable state of the external power logic. In order to achieve this task, the pin multiplexing control registers should be configured accordingly to map the USB0_DRVVBUS pin to be used for USB2.0 purposes. In addition, the source (internal or external) and frequency of the PHY clock should be identified and should be configured by the firmware. This is achieved using CFGCHIP2. Other PHY related fields within CFGCHIP2 should be programmed as: USB0PHYPWDN and USB0OTGPWRDN should be cleared to 0 and USB0DATPOL, USB0SESNDEN, and USB0VBDTCTEN should be set to 1. This will configure the PHY for normal operation as well as also turn on the PHYs VBUS comparator logic. The final task is to turn on the PHY PLL and wait until it locks. You should wait for the PHY clock good status to be set prior to ending the PHY initialization process.

Regards,

Shankari

Please send us the register dump of USB0ARM  during the error condition.

I attached a dump of the USB registers at 0x01e00000.  I dumped the entire 64 KB region of memory into the file.  I copied the memory from /dev/mem after the error had occurred.  As you can see, the DRVVBUS bit in INTMASKEDR is set, and no interrupt occurred, so the bit never got cleared.

5008.usb0_registers.dat

Is your GPS module a self powered device or receiving power from oMAPL138's USB 2.0 Host controller?

It is receiving power from our board.

And also, please make sure that the pin multiplexing, CFGCHIP2 and PHYclock is good as per the guidelines given in the TRM of OMAPL138 below.

Confirmed.

Did the register dump I sent provide any more information?

Brian Niebuhr said:

All I really want to know, from someone who knows this processor better than I do, is how it is possible that I can see the DRVVBUS interrupt signalled in the interrupt source register, but no interrupt occurs. 

The most common scenario is that the interrupt-clearing conditions did not occur. The IFR's bit in the processor core requires an edge before a new interrupt will be recognized; this is the way we avoid false interrupts from long pulses from peripherals.

There can be a long path of logic and/or registers from the interrupt source (DRVVBUS in your case) to the IFR bit. The input to the IFR bit must go low in order for a new interrupt to be recognized. The way this is generally solved is by checking all possible interrupt sources and making sure they are cleared, before leaving the ISR; this may require repeating the ISR's operation if a second interrupt has occurred before the first ISR operation has completed.

If we have sample code for drivers for a peripheral, we always recommend starting with that before writing your own. There can be tough logic conditions to debug, and we should have debugged them already in our code. I do not know if you have that luxury for your situation.

Regards,
RandyP

Hi Brian,

Is it that only with the particular GPS module peripheral on OMAPl138, this behavior is observed or with other peripherals like pendrive ( different make ) , USB sound card, USB data card, also this behaviour is observed? 

By any chance you repeated this test with other peripheral devices ? 

Using linux-3.3-psp03.22.00.06.sdk, I have experimented on OMAPL138 SDI EVM by connecting and disconnecting multiple times with the following peripheral devices and not able to observe any errors while disconnecting and connecting the devices. 

1. Pen drive ( San Disk and other make as well )

2. USB sound card (C-Media Electronics )

3. USB 3G data card ( Huwaei )

To reproduce the problem, I have changed the CPU scaling frequency to 300 MHz , 100MHz and repeated the testing of connecting and disconnecting.

I do observe that there is a small delay in displaying the re-connection status after disconnecting when compared to connecting the device for the first time.

But, Overall, it is always able to re-connect after disconnecting the USB device.

Can you give any particular test case for us to reproduce?

Screenshot for your reference.

Regards,

Shankari

Shankari - 

At this point we've only seen it with this one device.  We've tried some WiFi modules and we did some limited testing a while back with a a flash drive and didn't see this issue, so it's probably going to be hard to reproduce.  I would ordinarily just assume it was a problem with the device, except that I tracked the problem down to the fact that an interrupt bit is being set in the USB controller but the interrupt isn't occuring.  It could be due to a timing issue exhibited only by this one device.  I guess I was hoping that someone could explain how the interrupt source bit could be set without the interrupt firing.  Perhaps if the event occurs while the ISR is running?  If I could understand how that could occur, I could try making changes to the driver to see if I could eliminate the issue.

Thanks,

Brian 

Hi Brian,

Brian says said:

I guess I was hoping that someone could explain how the interrupt source bit could be set without the interrupt firing.

Actually, the USB interrupt sources can also be manually triggered using " USB Interrupt Source Set Register" in software. The flow, actually should be if there is a connection event in the hardware, the INTSRCR will get updated and not via the INTSETR. 

For debugging purpose, the option of manually triggering the interrupt source via INTSETR is given. In your case, the INTSETR is set. Please do check the values of INTSETR and INTSRCR during your working condition.

please do check whether by accidentally or due to memory corruption, the values of INTSETR are getting set due to which the INTSRCR values are getting set!!?? which in turn the CPU acknowledges the interrupt source??

I have looked into the register dump you have sent and observed that the 24th bit USBDRVVBUS is set in the INTSRCR, INTSETR and INTCLRR

To narrow down this issue, you may have to find how the 24th bit is getting set in the INTSETR. 

offset of INTSRCR - 20h, INTSETR - 24h.

Regards,

Shankari 

Shankari - 

The reason that INTSRCR, INTSETR, and INTCLRR all show the DRVVBUS interrupt set is because when you read INTSETR or INTCLRR the device returns the value of INTSRCR.  

The fact that INTSRCR has the DRVVBUS interrupt set was the source of my original question.  That bit is set, and yet the interrupt handler is not being called and therefore the bit is never getting cleared.  That missing interrupt is what's causing all of my problems.  Thus back to my original question:  how is it possible that the DRVVBUS bit can be set in INTSRCR but the interrupt not be called...

Brian

I have some additional information that may help.  I turned on some more debugging in the kernel, and in the case where the USB controller appears to fail is when a CPPI interrupt occurs after the USB disconnect interrupt has occurred.  For example, below is an example of a normal (non-failing) disconnect:

musb-hdrc musb-hdrc: RX DMA3, transparent, maxpkt 64, addr 0xc24efe00, rec'd 0/256
musb-hdrc musb-hdrc: TX DMA1, accelerated, maxpkt 1, 1 PDs, addr 0xc24efa00, len 1
musb-hdrc musb-hdrc: TX PD ffde9f40: buf c24efa00, len 00000001, pkt info 14000018
musb-hdrc musb-hdrc: Allocated DMA Tx channel 1 for EP2
musb-hdrc musb-hdrc: TX DMA1, accelerated, maxpkt 22, 1 PDs, addr 0xc24efb00, len 22
musb-hdrc musb-hdrc: TX PD ffde9f40: buf c24efb00, len 00000016, pkt info 14000018
musb-hdrc musb-hdrc: USB IRQ 00280000
musb-hdrc musb-hdrc: ** IRQ peripheral usb0028 tx0000 rx0000
musb-hdrc musb-hdrc: <== Power=e0, DevCtl=19, int_usb=0x28
musb-hdrc musb-hdrc: START_OF_FRAME
musb-hdrc musb-hdrc: DISCONNECT (a_host) as Host, devctl 19
musb-hdrc musb-hdrc: a_wait_bcon inactive, starting idle timer for 1100 ms
usb 1-1: USB disconnect, device number 3
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 3020 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c3088060) popped from teardown completion queue
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 2020 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c3088080) popped from teardown completion queue
musb-hdrc musb-hdrc: RX DMA3, transparent, maxpkt 64, addr 0xc24efd00, rec'd 0/256
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 2020 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c30880a0) popped from teardown completion queue
pl2303 ttyUSB0: pl2303 converter now disconnected from ttyUSB0
pl2303 1-1:1.0: device disconnected
musb-hdrc musb-hdrc: Poll devctl 19 (a_wait_bcon)
musb-hdrc musb-hdrc: Poll devctl a_wait_bcon 18
musb-hdrc musb-hdrc: USB IRQ 01000000
musb-hdrc musb-hdrc: VBUS off (b_idle), devctl 98
musb-hdrc musb-hdrc: Poll devctl 80 (b_idle)

Note that after the disconnect interrupt, DMA is torn down, and eventually the DRVVBUS interrupt occurs.  This is what is supposed to happen.  Next is the case where disconnect fails and the controller appears to hang:

musb-hdrc musb-hdrc: TX PD ffde9f40: buf c2473100, len 00000001, pkt info 14000018
musb-hdrc musb-hdrc: Rx complete: dma channel(3) ep4 len 1 timeout 1
musb-hdrc musb-hdrc: curr_pd=ffde9fc0, len=1, origlen=64,rxch(alen/len)=1/256
musb-hdrc musb-hdrc: RX DMA3, transparent, maxpkt 64, addr 0xc2473700, rec'd 0/256
musb-hdrc musb-hdrc: Rx complete: dma channel(3) ep4 len 1 timeout 1
musb-hdrc musb-hdrc: curr_pd=ffde9fc0, len=1, origlen=64,rxch(alen/len)=1/256
musb-hdrc musb-hdrc: RX DMA3, transparent, maxpkt 64, addr 0xc2473300, rec'd 0/256
musb-hdrc musb-hdrc: USB IRQ 00281000
musb-hdrc musb-hdrc: ** IRQ peripheral usb0028 tx0000 rx0010
musb-hdrc musb-hdrc: <== Power=e0, DevCtl=19, int_usb=0x28
musb-hdrc musb-hdrc: START_OF_FRAME
musb-hdrc musb-hdrc: DISCONNECT (a_host) as Host, devctl 19
musb-hdrc musb-hdrc: a_wait_bcon inactive, starting idle timer for 1100 ms
usb 1-1: USB disconnect, device number 8
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 3020 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c30881e0) popped from teardown completion queue
musb-hdrc musb-hdrc: Descriptor (c3088200) popped from teardown completion queue
musb-hdrc musb-hdrc: Teardown Desc (c3088200) rcvd
musb-hdrc musb-hdrc: TX DMA1, accelerated, maxpkt 1, 1 PDs, addr 0xc2473f00, len 1
musb-hdrc musb-hdrc: TX PD ffde9f80: buf c2473f00, len 00000001, pkt info 14000018
musb-hdrc musb-hdrc: Descriptor (c3088220) popped from teardown completion queue
musb-hdrc musb-hdrc: Teardown Desc (c3088220) rcvd
musb-hdrc musb-hdrc: Tx-PD(ffde9f80) popped from completion queue
musb-hdrc musb-hdrc: ch(1)epnum(2)len(1)
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 2004 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c3088240) popped from teardown completion queue
musb-hdrc musb-hdrc: RX DMA3, transparent, maxpkt 64, addr 0xc2473700, rec'd 0/256
musb-hdrc musb-hdrc: before rx-teardown: rxcsr 2020 rxcount 0
musb-hdrc musb-hdrc: Descriptor (c3088260) popped from teardown completion queue
pl2303 ttyUSB0: pl2303 converter now disconnected from ttyUSB0
pl2303 1-1:1.0: device disconnected
musb-hdrc musb-hdrc: Poll devctl 19 (a_wait_bcon)
musb-hdrc musb-hdrc: Poll devctl a_wait_bcon 98
musb-hdrc musb-hdrc: Poll devctl 80 (b_idle)
musb-hdrc musb-hdrc: Poll devctl 5d (b_idle)

Note that in this case you see some CPPI TX activity after the disconnect interrupt has occurred.  I believe, from my testing, that an additional CPPI interrupt occurs after the USB disconnect interrupt in the failure case.  So far it has been very consistent that when the extra interrupt occurs I get a failure during disconnect, and when the extra interrupt does not occur everything works fine.  Note that at the end of this failure case, the INTSRCR shows that a DRVVBUS interrupt is pending but the ISR is never called.  It will remain that way until I reboot the board.  

Hopefully this extra information may help reveal what is going on.