Hello,
I am using a target very similar to the beagleboard OMAP-3530 cpu running at 500MHz.
I am running linux kernel 2.6.27, configured with CONFIG_PREEMPT enabled
I have a GPIO line hooked up to an external FPGA, and this generates an interrupt once every 1ms.
I monitor the time when my linux environment generates an interrupt.
Even though the FPGA generates the interrupt at precise 1ms intervals, I notice that there is jitter experienced in the time when my IRQ handler gets executed by the driver code. The jitter is as high as 70microseconds peak to peak (+/- 35microsenconds) at a minimum. The system has very few threads running and only 1 other thread has interrupts going.
I can see this by repeatedly doing "cat /proc/interrupts ".
The thread with interrupts is the "gptimer" interrupt. (other than the one I am observing)
The amount of jitter experienced becomes even higher ( about 250us peak to peak) when I run ethernet traffic using the smsc9500 usb over ethernet driver.
This jitter behaviour is not desirable and I was wandering if anyone out there knows of ways to cut it down by either: kernel configuration or OMAP3530 processor settings.
I am running my code from Flash, JFFS2 file system. (Have tried Ramdisk with no improvement). Any Ideas welcome. (OMAP or Linux)
Thanks
One far shot since nobody else chimed in. Are you sure debouncing isn't enabled on the GPIO pin you use for the interrupt? (Should be disabled by default AFAIR) In case debouncing is enabled the pin will be debounced with the 32kHz clock which will obviously introduce a 1/32ms= ~30us jitter. For more info about this please have a look in the OMAP3 TRM chapter 25 with respect to the GPIO module...
With respect to jitter getting larger when having Ethernet traffic I'm not sure what can be expected of jitter in Linux on an ARM CONFIG_PREEMPT configured kernel, but I tend to agree that 250us seems to be a (very) long time...
Please let us know if/when you find any further info on this - Best regards and thanks in advance Søren
Søren Steen Christensen - SSC Solutions ApS - Web: www.ssc-solutions.dk
De-bouncing for the GPIO lines was turned off.
I turned off various peripherals in the system and was able to narrow down the cause to the gp-timer interrupt.
If I mask this interrupt, the jitter disappears. This is the GPTIMER12 on the OMAP.
When this interrupt runs it shows up as IRQ-95 in the linux environment.
I would like to modify the priority scheme to work in such a manner that my Driver 's GPIO interrupt gets higher priority that this GP-TIMER interrupt.
This means I need to be able to preempt the timer interrupt, and set my other interrupt to a higher priority than the gp-timer (timer-gp) interrupt.
Does anyone have any suggestions on how to do this ? If I use kernel version 2.6.31 with a PREEMPT_RT patch I still get the same behavior. In this case I can control priorities of threaded interrupts but unfortunately the GP-TIMER irq is not a threaded interrupt. It runs in the interrupt context.
Thank you,
P.Mutasingwa
Hi,
This I unfortunately don't have any really great idea about and will let it be up to some of the guys more knowledable in the "upper" Linux layers to answer this. One idea though: You might be able to move you interrupt to the FIQ interrupt line instead ofthe normal IRQ line? I don't know exactly if/how this is supported in Linux, but it might be another valid option for getting you interrupt the highest priority?
Good luck - Glad that you managed to get a bit closer to the problem source :-) Søren
Hi Marc,
I think the "threaded interrupt handling" is your problem, as I think the scheduler in 2.6.32 typical run at 100Hz. It has been quite a while since I last looked into this, but try to do something like descrobed at: http://wiki.gumstix.org/index.php?title=GPIO_Event_Driver#Driver_Sources and I'm rether sure you will be able to achieve much better performance than your current 10ms latency...
Good luck Søren
Hi Søren
I have tried the gpio lib and test app and I can still only get around 10ms reaction.
Is there anything special about the pinmux and interrupt handler that I need to setup. From reading the register documentation the control is in mux block 6 and I have configured the input as an input.
Have you previously configured a GPIO input as a source of an interrupt ? and what sort of duty cycle could be applied to get a reliable interrupt ?
I am just a bit confused as I can get 500us reaction time from a little STM32 ARM ucontroller with a fraction of the clock speed of the DM3730. There has to be a simple explanation to solve my issue. Hopefully you or someone who has delved into pinmux configuration and the Linux kernel will have an idea.
Thanks in advance
Marc
I am also wondering about the setup of the interrupt controller and if I need to activate the input assigned to INTC instead of GPIO.
One thing I notice is that when I can /proc/interrupts :
root@taodemo:~# cat /proc/interrupts CPU0 7: 33322 INTC TWL4030-PIH 11: 0 INTC prcm 12: 669 INTC DMA 24: 0 INTC omap-iommu.0, Omap 3 Camera ISP 25: 2 INTC OMAP DSS 37: 130407 INTC gp timer 56: 1800910 INTC i2c_omap 57: 2 INTC i2c_omap 61: 24 INTC i2c_omap 72: 735 INTC serial idle, serial 73: 3 INTC serial idle 74: 3 INTC serial idle 77: 73 INTC ehci_hcd:usb1 83: 0 INTC mmc0 86: 3684 INTC mmc1 92: 1 INTC musb_hdrc 93: 0 INTC musb_hdrc161: 23481 GPIO eth0296: 0 GPIO ads7846330: 30 GPIO gpio irq handler378: 33322 twl4030 twl4030_usb384: 0 twl4030 mmc0
I have written a simple irq handler and used it to handle the input. Supplying a 2ms duty cycle signal it sees a couple of interrupts.
How can I check the configuration of the pin to see if there are any additional settings, e.g. debounce filter (as that could cause issues) or anything else that may prevent the signal from being passed through ?
Thanks again
Quick replies to you two previous posts:
1) AFAIR there is nothing special you need to setup, but of cause kernel defaults might have changed since I last checked...
2) Yes - I'm using GPIOs being inputs for interrupts all the time. Haven't tested any minimum duty cycle for a long time though (a I haven't seen problems), but AFAIR it's more in the range of 10-100uS compared to 10ms. The GPIOs can be debounced using the 32kHz clock, which of cause will add something like 31us per 32kHz debouncing clock cycle...
3) It's (unfortunately) a big and unpleasant surprise to many that it's actually possible to get more "fast response/real time" performance out of a 1MHz MSP430 than of a full blown 1GHz Cortex-A8 system running Linux (for small tasks like flipping a GPIO, etc). In general I think unloaded Linux interrupt latency typically is around 10-100us of cause highly dependent on HW.. Going to userspace timing might go up to like 10ms as indicated above as well. 4) It's OK to have the interrupt routed to the GPIO. Only internal IP blocks can go directly to INTC => No way of coupling you GPIO pin directly into INTC, as it needs to go through the GPIO IP block on it way to INTC...
5) For further info on complete DM3730 and GPIO module in particular, please check the DM3730 TRM (http://www.ti.com/lit/ug/sprugn4p/sprugn4p.pdf) chapter 25. Maybe in particular "Table 25-47. GPIO_DEBOUNCINGTIME"
I hope this help you forward - Good luck Søren
I have been playing with my simple kernel driver and the only way I can get any kind of speed of reaction out of the irq handler is to have it in level detect rather than edge detect.
With the irq servicing function performing a do_gettimeofday and subtracting last from current to give me a delta then doing a printk I am closer to the 1ms required. The debug is showing ~1.5ms but I could believe that the printk could easily add 3-400 us. My next step is to toggle a GPIO output and scope it to see how accurately it resembles the input.
I think my problems have been due to the edge detect mechanism and the fact that the software is still in the irq servicing routine, or at least exiting and not cleared the irq request before the next one appears.
From what I have seen the IRQ is cleard at "some point" once the IRQ_HANDLED is returned by the handler function and I think this may be the latency issue that I am seeing.
I will run some more tests with the level detect rather than edge detect.
I will let you know my findings and hopefully anyone else reading this and who may have a similar issue.
Thanks for you help and being a sounding board.
Please keep us posted with any further results you get. And just as a short encouraging note I agree to your statements above (printk-time, IRQ-handling and edge/level notes), so I definitely think you are on right track... Glad that you found my comments useful and that you are progressing :-) Søren
"With the irq servicing function performing a do_gettimeofday and subtracting last from current to give me a delta "
Would you not get a more accurate result if you read a hardware hires counter ? Sorry, do not know whether your system includes any !
I have been moving accross onto the 2.6.37 kernel. I now have the system booting via NFS so I will compile the test module to see if there have been any fixes in the irq handling. There were notes in the new kernel release to state that there have been changes.
I will post my findings.
Don't know how power-saving mode is implemented in linux but in case of the Windows CE 6.0 100 usec GPIO interrupt latency was caused by reducing CPU clocks. I used simple console application which just enables periodic SPI transfers on GPIO interrupts so WinCE sees no user-mode activity and slows CPU after each interrupt.
Beware of power management!
Sergey