[DM3730]: Interrupt missing issue

Hi all,

Please note - If the above post is not clear, I am reposting it. 

We are using our Vendor supplied TI Arago Linux 2.6.32 Kernel with our DM3730 based board..

Our Processor is acting as an SPI master and an AFE board is acting as SPI slave.

SPI slave is configured to generate periodic interrupts at 125 microseconds.

During this time, some input data to be read at the master side. This is done at the master's tasklet.

A GPIO of DM3730 is configured to generate interrupt when the interrupt is reached from SPI slave.

Using a scope, we have ensured the correctness of interrupts from the SPI slave (AFE board).

But we can see interrupts missing at the master side in every 94 milliseconds.

 We have seen this by toggling another GPIO inside interrupt handler. 

Another observation is if the CPU becomes loaded, interrupt missing happens very frequently.

Because of this, our data acquisition becomes wrong and our output result is incorrect.

We are reading our input data from SPI interface inside a tasklet. This data acquisition takes only 

40 microseconds anyway.

We suspect if there is any high priority interrupts being serviced by the Kernel at this 94ms interval.

We have verified the output of cat /proc/interrupts. The other frequent interrupts running simultaneously are 

gp_timer and eth0. We have also tried to increase the priority of our interrupt by writing some registers at our 

IRQ controller. Could anyone please shed some light to this issue? What may be the reason of this interrupt miss / interrupt latency?

The processor is running at 1GHz.. Please suggest solutions/ideas/workarounds to this issue...

Thanks,

Honey S

Hi Honey,

The problem seems very complicated and need debugging but I only could advice you to make the source code in the SPI receiving interrupt shorted because it is possible to appears sometimes overlapping therefore try to start there work queue. In addition I would like to ask you about the SPI master clock rate which you are using to calculate the time necessary for receiving the data.

BR

Tsvetolin Shulev

Hi Tsvetolin Shulev,

Thank you for your response.

Regarding your suggestions, I will be answering towards the end of this mail. But before that, I would like to 

share some of the experiments we have done till now. I have posted the same under 

http://e2e.ti.com/support/embedded/linux/f/354/p/275552/965517.aspx#965517

For your reference, I am posting the same here also.

**************************************************************************************************************************************************************************

• We have tried to increase the SPI clock to 12MHz. Then also, horrible interrupt latency causing 3 or 4 interrupt miss occur at every 94ms.
• We suspect if there is any high priority interrupts being serviced by the Kernel at this 94ms interval. We have verified the output of cat /proc/interrupts. The other frequent interrupts running simultaneously are gp_timer and eth0. We have also tried to increase the priority of our interrupt by writing some registers at our IRQ controller.
• We have also tried to disable the Ethernet support in our kernel to disable the eth0 interrupt and still there is no change in the issue.
• We have also tried to mask the gp_timer interrupt inside our SPI driver init; Then whole system hangs.
• Also tried to decrease the priority of gpi_timer interrupt from inside our SPI driver init. Still there is no change in the issue.
• We  commented the tasklet_schedule which recieves SPI data from our Interrupt handler and enable GPIO Toggling on every interrupts . We registered our  interrupt handler on Linux as IRQ interrupt using “request_irq “ call. We  observed that:- 
  • Average interrupt latency is about 4 micro second.
  • But in some case it goes to 70 micro seconds. In the attached diagram  Point ‘a’ shows that GPIO set to high on an interrupt. Slave will generate another interrupt after 125 micro second. But  due to latency it get reflect only after 200 micro second.

  •  We observed that on every 94  milliseconds, there is an interrupt latency of more than 125 micro seconds. This may cause our SPI data read to be incorrect.  The time period 94 milliseconds is fixed and periodic. Diagram attached.

    

    *********************************************************************************************************************

    Regarding your queries, the following is my answer.

1. We have performed our SPI data read from inside a tasklet. At that time, the SPI clock rate was set as 6MHz.

We observed that with 6MHz SPI clock, we need around 40microseconds to read our data during every interrupt at 125 microseconds.

For experimentation purpose, we have changed our SPI clock rate to maximum that is supported by our SPI slave.

This is 12 MHz. So around 20microseconds needed for SPI data read during every interrupt.

When we performed our SPI data read at tasklet, we observed horrible interrupt latency every 94milliseconds which causes  3 or 4 interrupts 

to miss and several other occurances of interrupt latencies in between which may cause our SPI data read to be incorrect (like during interrupt

processing time, another SPI_DRDY interrupt occurs and input data to be overwritten). So for experimentation purpose, we commented our tasklet

and put SPI data read also from inside interrupt handler. Now also horrible interrupt latency during 94ms exists. But less number of occurances of other

interrupt  latencies which causes incorrupt SPI data read. Not every interrupt miss/interrupt latency solved. But better than when used with tasklet for SPI data read.

So from now on we are doing our experiments with SPI data read from inside interrupt handler itself.

Please give your valuable suggestions to this horrible interrupt latency causing interrupt miss/incorrect data read issue.. We are still stuck here... 

To be frank, may I ask you -> Can Linux be able to handle 125 microseconds interrupt on 1GHz DM3730 processor?

Thanks,

Honey S