Problems with OMAP3525, TPS65930 and 32kHz clock

Hmmm, no answer yet - must be a very finicky problem.... ??

Meanwhile I try to find out more about the circumstances:

- Cold boards tend to hang more often than warm ones.
- Trying several different capacitors gives interesting results: Some values cause the board not to boot, some let it run for a short while and some others seem to cure the problem.

But will the cure help under all circumstances and over time???
Is it specific to the board (layout) or to the OMAP chip (production run, silicon version, ...)?

Anybody care/dare to give more information?

Stefan

Stefan,

the 32KHz clock is used for things like detecting frequency of high speed clock, watchdog timers, and in low power sleep mode to clock the wakeup domain.  When you say you narrowed it down to the scheduler not restarting sleeping tasks, are you in some sort of OMAP3 sleep or OFF mode when this happens, or is this just a task that Linux has put to sleep (ie, the chip is still in a normal power and clocking state)?  Are you performing any DVFS or sleep/wake sequences around the time of the hang?  When the hang occurs, is it the first time the kernel is performing some function (certain tasks, getting into a low power mode)?

James   

Hi James,

what we see is this: all processes that are scheduled for a certain event (here: the sleep timer) are hanging at a point. If we kill the hanging process, the rest of the queue will be serviced normally (until the next hangup occurs).  The problem is only occurring when processes use the 'nanosleep' system call. The OMAP itself isn't put in any sleep mode.

The whole thing is looking like a (HW) timer event that is triggering an interrupt that then somehow isn't recognized/handled properly by the software. Since the ISR never resets the interrupting hardware, no new interrupt will be generated and the nanosleep queue stalls.

The "funny" thing (not!) is that a small capacitor in the low pF range with no visible effects to a rather slow clock signal makes the difference between a system that runs or one that sooner or later stalls. It obviously is so, but I'd like to understand it in order to find a cure better than "trying some capacitors here and there".

Stefan

Stefan, here are a couple of ideas to try:

-check the 32KHz clock for noise, glitches, etc., especially around the hang point if possible. 

-try changing the linux config to have the system clock sourced from a GPtimer (using MPU clock) instead of 32KHz clock. 

-try feeding an external 32KHz from a clock generator

-ensure input specs from TPS65930 are met by your 32KHz clock source

James

Try as well to do a FFT of the clock signal with your oscilloscope. Some years ago I had an issue with a 32kHz clock and a FM radio chip (not from TI), which under some circumstances worked just fine and under others would "tick"/"create noise" at random times.  I couldn't see any difference on the 32kHz clock signal in the time domain, but doing a FFT it was pretty clear to see that the error correlated directly to a minor change (caused by another chip going into low power mode) in the harmonics creating the 32kHz square wave...

The frequency domain normally reveal a lot of information which is very difficult (if not impossible) to see in the time-domain [:)]
  Søren 

@James

I did check the signal and there are no glitches etc. to be found. I set the scope to infinitive persistence for hours and there are no perceptible distortions around the trip point. When looking at a slope several periods away from the trigger point there is very few jitter to be seen. On other occasions I'd be glad to have such a stable clock signal.

Switching to a different clock (timer 12) with the latest kernel (2.6.26/29) seems to cure the problem, but a) didn't I have the time to fully test it and b) can't I switch the kernel right in the middle of delivery (the first 44 displays with the old kernel are already mounted in the customer's trains). Maybe we can do a field update of the whole fleet later next year, but at the moment I have to live with what we have.

Feeding an external clock isn't possible since I don't have any generator in the kHz range at hand that fulfills the OMAP's spec. of a rise time of max. 20ns.

@Søren:

I had a look at the FFT of the signal and was indeed a little bit surprised to find an expressed peak at the HFCLK frequency of 26MHz in the spectrum. But at -50db I don't really expect problems from this. Even with maximum magnification of the signal around the trip point of the OMAP input doesn't show distortions that seem in any way 'dangerous'. There is also no visible difference in the spectrum whether a 15pF cap is mounted (system stable) or not (system unstable).

Stefan