Trying to find a solution for this interrupt problem

Hi Cor,

The most straightforward way to attack this (in my opinion) would be with an RTOS.    You could try FreeRTOS if you have an RM4x or TMS570 and are using the HalCoGen program. 

Recipe with FreeRTOS would be something like this:

a) Set the RTOS tick to 1ms

b) Create two semaphores, one for 'fastLoop' and one for 'slowLoop'.

c) Implement a 'tick hook' function that simply calls a post to the semaphore for 'fastLoop' every time there is a tick, and a post to 'slowLoop' semaphore every 10th time.

d) Configure fastLoop to be higher priority than slowLoop  (priority is a parameter in the task create function)

e) fastLoop and slowLoop should first sleep pending their respective semaphores.  then they should do their function, and finally loop back to the pend.

The above recipe should create exactly the function that you want,  and you won't have to worry at all about the details of implementing ISRs  (RTOS will handle this for you).

Best Regards,

Anthony

Thanks Anthony,

I have difficulties with the overhead of the RTOS. In my example I talked about 1 mS and 10 mS. But in fact I want this as fast as possible. I need very low overhead.

I have the feeling that it is not very difficult. But I do not have much expertise on this subject. So I hope someone can tell me if it can be done this way.

I assume the stack is in a state that new interrupts may occure (The FIQ can ocure too). But maybe some IRQ related information is not stored on the stack but in registers. This info would get corrupted when a nested IRQ occurs. But if I know what this info is, it is probably easy to save/restore it before/after the enabling of the nested IRQ.

Hi Cor,

Ok, understand. 

Are you using the TI compiler?  I think it'll do the job for you if you use the interrupt keyword before your ISRs, I gave it a try with the simple test program below.

2500.main.c

http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/766/0825.main.lst

The "LR" is a usual suspect for trouble when it comes to not being saved.   And function calls inside ISRs are also another trouble case.

You can see how the test program compiles;  and if you want more detail we should add someone from our compiler team to this thread.  (the compiler manual doesn't go into a whole lot of detail other than saying interrupt makes the compiler pessimisticallys save registers.

Best Regards,

-Anthony

Hello Anthony,

Yes I use the TI compiler.

I'm not sure what your are trying to tell with the example. Could you please explain a little more?

I found some information in the Technical Refercence Manual. There seem to be some special irq registers, like:

• SPST_irq
• SP_irq
• R13_irq
• R14_irq

But it is not clear to me if these are all seperate registers, or maybe some are different names for the same thing. It is also not clear to me if I need to save/restore these registers for the one level nesting.

If this is the case, how can I save/restore these registers using C-language? Or do I need to write assembly code for this. If so, can you help me out here?

Cor,

You may consider this option if you do not like RTOS.

(1) Generate a RTI interrupt every 1 ms. Increment an global variable (Counter, for example) and do the fast loop processing in the ISR.

(2) Poll global  variable "Counter"  in main loop.When it reaches 10, reset the value to zero and start the slow loop processing.

This approach has minimum CPU overhead. However, you need to do the following analysis to see if it work for you.

(1) What is the latency tolerance for the slow loop processing in your application?

(2) What is the maximum delay caused by other processing or interrupt ISRs in polling the global variable "Counter"?

The approach only works if the value in (1) is larger than the value in (2).

Thanks and regards,

Zhaohong

I did some more investigations.

If I understand correctly, the R13_irq, R14_irq and SPST_irq are banked registers valid during IRQ processing. SP_irq seems another name for R13_irq.

So when I save/restore the R13, R14 and SPST when in IRQ mode, It should be possible to nest one level of IRQ. Right?

So how can I save/restore these registers? Can I just push/pull them on the stack? If so, how? (I have no experience with ARM assembly:-).

Or can I easily write C-code to save/store these registers?

Thanks, Zhaohong,

The problem with that is that both the slowLoop and fastLoop need to be interrupt based. The main loop is already busy doing other things and uses functions that run relatively long times.

Hi Cor, 

Yes, for the most part**, when you use R13 in an instruction, it means R13 of the mode you're in.

Since the IRQ puts you into IRQ mode,  the code that executes in the IRQ routine will be saving R13_irq if it saves R13.

  ** there are some very special instructions that access registers of another mode;  but that's not relevant here.

The example code I posted had several cases:

  - isr written as a 'c' function without interrupt keyword

  - isr with interrupt keyword

  - isr with interrupt keyword, also calling another function inside the isr.

The .lst file has the generated assembly so you can see which registers are saved.

The cases where the interrupt keyword is used clearly show that the compiler is saving registers that it wouldn't ordinarily save for a regular function call.

For example,  LR is getting saved because if there's another IRQ it would overwrite LR_irq.    You would be able to return back to the first IRQ, but not back to the user mode code that was interrupted in the first place,  unless you save LR_irq onto the stack.     A regular user mode function doesn't need to do this because if it is interrupted, the LR used by the interrupting function will be the banked LR_irq not the user mode LR.

To me the compiler examples look fine except I don't see SPSR being saved;  I think that it might be saved as a side-effect of one of the instructions that's listed but trying to dig this up.

-Anthony

Hi Cor,

So the trick I was thinking of might be that if you include R15 in a LDMxx instruction, then this causes SPSR to also be written back to CPSR.

Not finding a trick that works in the other direction;  but it still might be there.   I'll want to discuss this with our compiler team.

In the meantime, I think that if you use the interrupt keyword from the compiler but also insert some assembly instructions to push and pop SPSR from the stack in addition to what the compiler saves then you should be OK.   Also you should probably save the floating point status register if you are using floating point in your interrupt routines.

You can look at the context switch code in the HalCoGen FreeRTOS port to Cortex R4 source code:   <HalCoGen_Instlal>\drivers\FreeRTOS\ports\Cortex-R4.  The file portasm.inc has macros to save and restore context.  You can see SPSR is being saved and restored explicitly  (the mrs instruction moves SPSR to R0 and then the stmfd instruction moves this to the stack.)

Will let you know what we find out re. the compiler.

Thanks Tony,

I will investigate this keyword ussage.

I still would like to be able to read/write the registers SP (=R13), LR (=R14) and SPSR.

I was thinking using the inline assembly feature like this:

uint32_t readRegXXX()
{
     __asm("  ......"); // assembly instruction to move the register to read to register A1 (result register)
}

void writeRegXXX(uint32_t value)
{
     __asm("  ......"); // assembly instruction to move register A1 (argument register) to the register to write
}

Then I only need to know the assembly instructions to move:

• SP -> A1
• LR -> A1
• SPSR -> A1
• A1 -> SP
• A1 -> LR
• A1 -> SPSR

I do not know where to look for the correct assembly instructions (RM4 processor). Tried several document, whithout success.

Could you help me with this?

Hi Cor,

To get details on the instruction set, you want the ARM document DDI0406 (ARM® Architecture Reference Manual ARM®v7-A and ARM®v7-R edition)    You can get that from ARM's website.  Sorry I can't really put a link here directly to it;  link I copied goes to a login page (for this particular document, you need to be registered on their site and logged in).

The instruction to copy the SP into A1 is just "MOV".  It's documented on in section A8.6.97 in the version that I've got access to.

I would probably write your ISR and also your register read/write function completely in assembly or completely in C the more I think about it.

A good example of assembly functions that you call from C is the 'sys_core.asm' file that contains assembly functions called from the 'sys_startup.c' file; both of these are output by HalCoGen.  You can see how the assembly function '_coreInitRegisters_' is called by the C routine c_int00() for example.

As an aside, assuming that you write your interrupt handler in assembly also, I noticed in DDI0406 that there are two instructions added to ARMv7  for saving and restoring both LR and SPSR at the same time.   These would be "SRS" (store return state) and "RFE" (return from exception) respectively.   From the psuedo code, it looks like you would need to subtract 4 from LR before the SRS on IRQ entry;  but I haven't tried this myself so might be wrong.   (for IRQ, the value you return to is always LR - #4 to compensate for the pipeline).

Best Regards,

Anthony

Now I'm completely confused!

I did a simple test:

Just enabling the IRQ (_enable_IRQ();) at the beginning of an IRQ handling function and disabling it at the end (_disable_IRQ();). No register saving/restoring at all.

Then I generated an IRQ interrupt request during this interrupt handling routine. And it works!!!? The IRQ routine is interruptable by a new IRQ request. No crashes. As far as I can see now: correct code execution!

How is this possible? As far as I understand the stack and link register en status registers are not saved during IRQ calls because there are sepperate IRQ bank registers. Or does TI (I use TI compiler and HalCoGen) save these registers on the stack enabling nested IRQ request?

I tried to find out looking at te code, but I can not find the IRQ handling assembly source. Most code is generated by HalCoGen.

Where can I found the low level IRQ handling code?

Hi Cor,

Here is an example IRQ handler from HalCoGen.  I added some code (between the /* USER CODE BEGIN */  and END comments... to test nesting.

/** @fn void rtiCompare0Interrupt(void)
*   @brief RTI1 Compare 0 Interrupt Handler
*
*   RTI1 Compare 0 interrupt handler
*
*/
#pragma CODE_STATE(rtiCompare0Interrupt, 32)
#pragma INTERRUPT(rtiCompare0Interrupt, IRQ)

void rtiCompare0Interrupt(void)
{
/* USER CODE BEGIN (74) */
    static unsigned int x, y;
/* USER CODE END */

    rtiREG1->INTFLAG = 1U;
    rtiNotification(rtiNOTIFICATION_COMPARE0);

/* USER CODE BEGIN (75) */
    _enable_IRQ( );
    for (x = 0; x < 1024*1024*1024; x++) {
        y = x;
    }
/* USER CODE END */
}

I had to manipulate the PC a little to jump around the _enable_IRQ() on the second time I hit the ISR and then set the interrupt disable bit again when returning from the first nest, in order to get back to the main loop.

But what I found confirmed my suspicion,  in this HalCoGen project you enter main() in SVC mode.  But after nesting the RTI interrupt without saving the SPSR register during nesting, when you return to main you are in IRQ mode and not SVC mode.   The original value in CPSR before the interrupt is lost.

I suspect you might find the same on your setup. 

Note that this doesn't mean there is a crash.  In fact I think my simple test case that has just while(1) in the main loop after enabling interrupts would run forever and be just fine.   Yet the main loop would be using the IRQ mode stack and not the SVC stack like it is supposed to.

Since we used the #pragma INTERRUPT()  and declared the RTI handler as an IRQ,  which I believe is functionally equivalent to using the 'interrupt' keyword in the function definition, the LR is being pushed onto the stack and restored on exit.     If the LR were not saved, then an obvious 'crash' would be much more likely.

The effects of corrupting SPSR are harder to see;   running with the wrong stack and possibly wrong status flags will not create a problem unless some other condition happens that makes use of these resources.    A while(1) loop as I've got doesn't make use of the stack and doesn't make use of status flags so nothing bad happens.

But you can't build on this because when you start calling functions and doing arithmetic, the SPSR corruption will soon become evident.

I did open a ticket in our internal compiler support forum regarding the suggestion to support nesting.   I believe all that you need to add to the ISR that has the interrupt keyword is a save & restore of SPSR.   But the best way to accomplish this will probably need some discussion.   I will let you know.   

In the mean-time, I think you can add in inline assembly these instructions:

    SRSFD SP!, #0x12

    ... enableIRQ() ...

    RFEFD SP!

 The "FD" suffix after the SRS and the RFE is the type of stack, compiler uses FD (full descending).

The #0x12 is the M value for IRQ mode, SRS needs to know which banked registers to save.

I haven't tried the above yet myself so there might be a typo... I'll try now and post if this works as expected.

-Anthony

Hope this makes sense  & thanks for your perseverance.

Best Regards,

Anthony

Cor,

I ran into a different issue with the SRSFD instruction inline assembly that I need to run down, assembler just accepted SRSFD SP!   (no 0x12) and the instruction generated had mode bits set to 0.  Looked like nothing happened when I executed this.  

So, next best thing below,  adding these lines in RED saves SPSR and it's restored on exit of the ISR.   I don't like having to use R0 in inline assembly because you really don't know if the compiler is using it for some other purpose, but putting these as the first instruction right after entry of the ISR they come after R0 is stored on the stack.
So for now, I'd try the below.   I confirmed that when the save & restore of SPSR is added, now even after nesting when I return to the main / foreground loop the CPU is back in the correct mode (SVC) with the original CPSR value from before the interrupt.

void rtiCompare0Interrupt(void)
{
/* USER CODE BEGIN (74) */
    static unsigned int x, y;
    asm("   MRS R0, SPSR ");
    asm("   STMFD SP!, {R0} ");
/* USER CODE END */

    rtiREG1->INTFLAG = 1U;
    rtiNotification(rtiNOTIFICATION_COMPARE0);

/* USER CODE BEGIN (75) */
    _enable_IRQ( );
    for (x = 0; x < 1024*1024*1024; x++) {
        y = x;
    }
    asm("   LDMFD SP!, {R0} ");
    asm("   MSR SPSR, R0 ");
/* USER CODE END */
}

note: there's some bugs in the above code in the for loop ... I used the 'move to line' option in CCS to jump around the loop to test,  anyway pls. only copy the assembly statements in RED.

Thanks and Best Regards,

Anthony

Thanks Anthony.

You are right!

I did not much in the main loop at first and the nested IRQ seemed to work. Byt when I start to do more inside the main loop the nested interrupt doesn't work correct anymore.

So I tried your solution, but the IRQ doesn't seem to do its job anymore (while the main loop keeps running). Even when not using the nested interrupts. So somehow the IRQ routine is corrupted.

So I tried this:

uint32_t savedSPSR[10];
uint32_t savedSPSRptr = 0;

uint32_t read_SPSR()
{
    __asm(" MRS A1,SPSR");
}

void write_SPSR(uint32_t value)
{
    __asm(" MSR SPSR,A1");
}

void gioNotification(int bit)
{
    // to make the gioNotification (IRQ) re-entrant, we have to save and restore the SPSR
    savedSPSR[savedSPSRptr++] = read_SPSR();

    ... some interrupt stuff

    _enable_IRQ();

    ... my interruptable interrupt stuff

    _disable_IRQ();


    // restoring the SPSR
    write_SPSR(savedSPSR[--savedSPSRptr]);
}

Now it works! The main loop seems to continue and I use nested IRQs. I know it is a dirty implementation but it was just for testing. I prefer to store the SPSR on the stack.

I will test a bit further to be sure all is ok.

Bad news:

When I use local variables in my interrupt routine, the nesting does not work anymore. Maybe this is also what you experienced when the for loop inside the interrupt routine did not work?

Even when only using one local and set it to zero like here:

void gioNotification(int bit)
{

    uint32_t dummy;

    // to make the gioNotification (IRQ) re-entrant, we have to save and restore the SPSR
    savedSPSR[savedSPSRptr++] = read_SPSR();

    dummy = 0;

    ... some interrupt stuff

    _enable_IRQ();

    ... my interruptable interrupt stuff

    _disable_IRQ();


    // restoring the SPSR
    write_SPSR(savedSPSR[--savedSPSRptr]);
}

How can I enable the compiler to produce .lst listing files? So that I can see what  code is generated?

Are the vectors in the vimRam called directly by hardware? Or is there some dispatching assembly routine inbetween? I can not find any low level IRQ assembly code so I assume these vectors are called by hardware? (I'm using HalCoGen 3.01.01)

Hi Cor,

I'll start with the questions from the end of the last post and work backward, because the ones at the end have simpler answers :)

VIM Ram Question:

The R4 processor actually supports three different vector dispatch mechanisms,  the older ones there in order to make it easier to migrate from the ARM7/9 series.

In HalCoGen, on the "Interrupts Tab" you can see that there are two options for IRQ - "Dispatch Mode" and "Vector Mode".    If you pick Dispatch this is the simplest and you have to write your own routine to read flags and dispatch to the right vector.   This would be the very old style ARM7 type solution where there is just one IRQ vector at address 0x18.

The default for HalCoGen should be the "Vector Mode"  ...  You can see that the IRQ interrupt still lands you at 0x018 but the instruction there is 'ldr pc, [pc, #-0x1b0]'.   So what is that cryptic ldr pc instruction doing?    Well, it's reading the hardware prioritized vector from the VIM and putting this value into the PC.   The IRQ still makes a 'stop' at 0x00018 which is nice because you can put a breakpoint there,  but there's no real dispatch routine per-se except for this single instruction.  This single instruction is sort of like a 'go-to what VIM tells you is the right ISR'..

Now, there's another variant of the Vector mode that also uses VIM.  There is a bit in the CP15 system control coprocessor,  the register is "C1, System Control Register" and the bit is VE.   If this bit is set, then the CPU will skip 0x18 and the odd looking ldr instruction there.   Instead the vector from the VIM is pushed in hardware to the CPU and the CPU goes directly there,  without passing through address 0x18.   this is going to be the fastest way to dispatch to an interrupt.   

Note that the HalCoGen code would still generate the LDR instruction at address 0x18,  and until VE got set any IRQs would stop there.  But once VE is set the IRQs would just skip this step and go directly to the address that is in the VIM table.

Compiler .lst file:

To generate .lst files,  the compiler option '-al'  or '--asm_listing' is what I use.   In CCS, if you select the project from the Project Explorer pane, right click and select "Properties" then this brings up all the options for the project.   The -al option is under the Build -> Arm Compiler -> Advanced Options -> Assembler Options level.

I also have found it useful to set the debugging model to --symdebug:none when I generate a .lst that I just want to read,  then I set it back to full debug before building the program to download and test.   The reason to switch to --symdebug:none is just to make the .lst file easier to read.   If you don't do this, then it will be filled with debug directives and it'll be hard (IMO) to find the actual code.     This option is under the  Build -> Arm Compiler -> Debug Options level.

Assuming your active profile is Debug,  look for the .lst files in the Debug-><project name>->source folder.  It'll be there mixed in with the intermediate .obj files.

Local Variables Question:

First, I noticed that your example is the "gioNotification" function.   This is not actually the ISR for the GIO module.   You can tell because in the vector table in HalCoGen the names are 'gioHighLevelInterrupt' and 'gioLowLevelInterrupt'  (channels 09 and 23).

The gioNotification function is the call-back routine that is actually called by the ISR.    It might be safer to stack the SPSR inside the actual interrupt service routine, right after entry. 

So for example the high level interrupt (channel 09) will dispatch first to this function.  (I believe HalCoGen will not output this function unless you enable the GIO interrupt):

#pragma CODE_STATE(gioHighLevelInterrupt, 32)
#pragma INTERRUPT(gioHighLevelInterrupt, IRQ)

void gioHighLevelInterrupt(void)
{
    int offset = gioREG->OFFSET0 - 1U;

/* USER CODE BEGIN (14) */
/* USER CODE END */

    if (offset >= 0)
    {
        gioNotification(offset);
    }

/* USER CODE BEGIN (15) */
/* USER CODE END */

}

Regarding variables inside the ISR - my example with the loop was really bad this is why I'd mentioned only copying the couple of inline assembly statements.
I used 'static' because I thought this might help ensure the ISR would be writing to RAM and this would slow that loop down.  Only objective was to kill some time with that loop in order to let a 2nd RTI tick hit while still in the 1st RTI interrupt so nesting would occur.

I'm not clear where the "savedSPSR[]" array is intended to live in the example that you attached.   I would go back to the suggestion of using the SP and saving the SPSR onto the IRQ stack rather than somewhere else in memory:  

    asm("   MRS R0, SPSR ");
    asm("   STMFD SP!, {R0} ");

Also, I would suggest starting with just the above inline assembly *directly* in the ISR like this rather than as a function call.  I think if you try to push the SPSR inside a function that is called from the ISR, then calling that function will push things onto the stack and when you exit the function it will unwind, so the SPSR pushed onto the stack during the function call won't really be where you'd want it to be.  [Later you could try inlining the function to avoid this]   I'd start with the just this code below:

#pragma CODE_STATE(gioHighLevelInterrupt, 32)
#pragma INTERRUPT(gioHighLevelInterrupt, IRQ)

void gioHighLevelInterrupt(void)
{
    int offset = gioREG->OFFSET0 - 1U;

/* USER CODE BEGIN (14) */

    asm("   MRS R0, SPSR ");
    asm("   STMFD SP!, {R0} ");

/* USER CODE END */

    if (offset >= 0)
    {
        gioNotification(offset);
    }

/* USER CODE BEGIN (15) */

    asm("   LDMFD SP!, {R0} ");
    asm("   MSR SPSR, R0 ");


/* USER CODE END */

}

And take out any code from the notification routine that tries to save SPSR.

Note that the above is in the actual IRQ handler and not the notification function.    So SPSR is pushed onto the IRQ stack right after all the other context is saved by the compiler,  and before any other registers are pushed onto the stack  (i.e. for another function call...)    Likewise, SPSR is popped off the stack as the last thing right before the other context is restored by the compiler.

Hope this is helpful.  It may also clear up the issue you're having with local variables since they would tend to live on the stack.   If it doesn't then please let me know and we can keep analyzing further.

Thanks and Best Regards,

Anthony

Hi Anthony,

Thanks you very much for the very lengthy answer. I followed your advise and inserted the assembly instructions inside the gioHighLevelInterrupt routine.

HalCoGen did not generate the #pragma CODE_STATE(gioHighLevelInterrupt, 32) line. Should I add it?

Somehow I can not get it to work. Some times it looks like it is working, but when adding more code (especially local variables?) it stops working.

I did all kind of tests to try to find out what causes it to malfunction. But I see no pattern in the test results.

Is it an idea to replace the complete gioHighLevelInterrupt routine by an assembly version that is suitable for nested interrupts? Is there an example of such an

low level IRQ routine (for nested IRQ) written in assembly?

Hi Cor,

I don't think it would hurt to add this pragma.  I actually used an TMS570 where the default state of the project is --code_state 32 (Under Build->ARM Compiler ->Processor Options).   So for me it wouldn't make a difference.

But if you are otherwise compiling to Thumb 2 then it would probably make a difference to have the pragma because these products are configured to always start exceptions in the ARM instruction set mode.   So you woudln't want the compiler to generate Thumb2 code for gioHighLevelInterrupt.

I just finished talking to the compiler engineer who owns the ARM compiler,  I think he may post some suggestions.   This thread was great to stimulate discussion.

The thing that he pointed out to me which might be affecting you is that the inline assembly might be re-ordered by the compiler if optimizations are turned on.
I also tested without optimization, so the code may not work with optimization on.   And even if it did, it may break in the future if we release a compiler with better optimization.  

So for interrupt nesting, I think the recommendation will be to code the interrupt service routine entry and exit in assembly and then if you want to code the body in "C" you can insert a call to the C function after the critical things like stacking the processor state have been taken care of.  

This will make you responsible for pushing the registers that your function is using to the stack,  so you might start just pushing all the registers to be safe;  assuming you can afford the extra cycles and additional stack space.   Then when you're 'done' with your ISR you might go back and try to optimize (or not) and only push the registers that wind up getting used.

Actually, I think you could look at the assembly that is generated by the "C" functions that were compiled with the interrupt keyword as a good starting place.

So for example:  (I'm going to paste lines from my .lst file:)

 .sect   ".text:retain"
  .retain
  .retainrefs
  .armfunc rtiCompare0InterruptNesting
  .state32
  .global rtiCompare0InterruptNesting

rtiCompare0InterruptNesting:

  STMFD     SP!, {A1, A2, A3, A4, V9, LR} ; ** SEE NOTE

  MRS R0, SPSR 

  STMFD SP!, {R0} 

   BL        rtiNotification

   LDMFD SP!, {R0}

   MSR SPSR, R0

   LDMFD     SP!, {A1, A2, A3, A4, V9, LR} ;  ** SEE NOTE

   SUBS      PC, LR, #4  

** Note:  these two lines need to stay in sync, and of course you will need to adjust them as your ISR uses more registers

(including registers used by the notification routine).    For starters, you might want to push all 14 of the working registers 

(everything but the PC) in this list just to be safe.

Not exactly sure what the .retain directives do but I copied them because I assume they make this a 'nice' function for the debugger.

Now, the above keeps it very simple in the assembly routine,  but in the notification routine you would need to add back the code

to clear the interrupt flag in the peripheral:

   rtiREG1->INTFLAG = 1U;

And, when you are ready to re-enable interrupts you can insert the intrinsic for that as well in the notification routine. 

If you're still having trouble with the local variables, then I think we'll need to have you post at least some segments of the code that use the variables and explain a little what the problem is.

Best Regards,
Anthony

Cor,

One additional point I should mention. 

For the above example, the assembly language ISR shell that calls the notification routine is now named  rtiCompare0InterruptNesting instead of 

rtiCompare0Interrupt.   You will also need to edit the VIM RAM table in HalCoGen so that it points to this new function.  

Best Regards,
Anthony

Hi Anthony

I may have something working now. But there is a strange thing I do not understand. This is what I did:

I used the assembly listing of an existing IRQ routine from the compiler as a starting point. This is the routine I started with:

    .sect    ".text:retain"
    .retain
    .retainrefs
    .armfunc _irqDispatch
    .state32
    .global    _irqDispatch
_irqDispatch:
        STMFD     SP!, {A1, A2, A3, A4, V9, LR}
        VMRS      V9,FPSCR
        STMFD     SP!, {V9}
        VMRS      V9,FPEXC
        STMFD     SP!, {V9}
        VSTMDB    SP!, {D0-D7}
        BL        C_irqDispatch
        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {V9}
        VMSR      FPEXC, V9
        LDMFD     SP!, {V9}
        VMSR      FPSCR, V9
        LDMFD     SP!, {A1, A2, A3, A4, V9, LR}
        SUBS      PC, LR, #4
    .global    C_irqDispatch

It does save a lot of register. I assume this is the maximum number of registers ever needed to save during IRQ. Without re-entrant interrupt this works ok.

To make re-entrant IRQ possible I added the save/restore of the SPSR register as below:

 _irqDispatch:
        STMFD     SP!, {A1, A2, A3, A4, V9, LR}
        MRS       R0, SPSR
        STMFD     SP!, {R0}
        VMRS      V9,FPSCR
        STMFD     SP!, {V9}
        VMRS      V9,FPEXC
        STMFD     SP!, {V9}
        VSTMDB    SP!, {D0-D7}
        BL        C_irqDispatch
        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {V9}
        VMSR      FPEXC, V9
        LDMFD     SP!, {V9}
        VMSR      FPSCR, V9
        LDMFD     SP!, {R0}
        MSR       SPSR, R0
        LDMFD     SP!, {A1, A2, A3, A4, V9, LR}
        SUBS      PC, LR, #4

But this does not work! System crashes.

The strange thing is that it seems to work when I comment out the save/restore of the FPSCR and FPEXC registes like this:

 _irqDispatch:
        STMFD     SP!, {A1, A2, A3, A4, V9, LR}
        MRS       R0, SPSR
        STMFD     SP!, {R0}
;        VMRS      V9,FPSCR
;        STMFD     SP!, {V9}
;        VMRS      V9,FPEXC
;        STMFD     SP!, {V9}
        VSTMDB    SP!, {D0-D7}
        BL        C_irqDispatch
        VLDMIA    SP!, {D0-D7}
;        LDMFD     SP!, {V9}
;        VMSR      FPEXC, V9
;        LDMFD     SP!, {V9}
;        VMSR      FPSCR, V9
        LDMFD     SP!, {R0}
        MSR       SPSR, R0
        LDMFD     SP!, {A1, A2, A3, A4, V9, LR}
        SUBS      PC, LR, #4

Those are floating point related register if I'm correctly. I do not use floating point yet, so I can savely remove that.

But why does it work now? It is as if the stack space is limitted and the by removing these FPSCR and FPEXC save/restore there is room for the

SPSR save/restore.

Any idea how this is possible?

Question: Do I also have to store registers R4 to R12?. The original didn't. And when I do, it crashes.

Hi Cor,

I don't see anything obviously wrong with the stacking / unstacking.

I did a little searching for appnotes from ARM and so far best I've found is this example code:

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0273a/Babcgdia.html

Note that their vectored interrupt controller is a little different than VIM, so this is why the dispatch at 0x18 is different.    Also you don't need the two lines of code that are "LDR r1, =VectorAddr" and "STR r0, [r1] ; Acknowledge Vectored IRQ has been serviced"    (but on TMS570, you do need to clear the interrupt flag inside the GIO interrupt routine before you re-enable interrupts.)

There are two things I see in this example that might apply:

  1) There is a switch into system mode before calling the 2nd level handler. 

      I need to think about 'why'.   first thought is that this might help with stack sizes,  or at least it minimizes the size needed on the IRQ stack as most registers

     will be stored on the user/system mode stack.

 2) This one I think is a must:  Interrupts are disabled again before the SPSR and LR are popped off the stack prior to the return from the IRQ interrupt.

    I think I understand the need for this one:  while the SPSR and LR values are not on the stack but are in the IRQ registers, another interrupting IRQ would corrupt them.   So not only do they need to be pushed onto the stack before interrupts are re-enabled, but they need to be restored after interrupts are disabled again.

I think the registers that you are saving look ok,  you've got all the 'A' registers which would not be saved normally by any functions you call from the ISR (so have to be saved before the ISR call).  Now this is assuming that the "C_irqDispatch" routine is compiled and that the compiler is saving R4 to R12 (but only if the compiled code uses them...)   Also it's assuming that your C_irqDispatch routine is compiled for ARM mode not Thumb.

I think you can skip the FP registers IF you never use any FP instructions inside your ISR.  Since you mentioned doing most of the processing inside the ISR though, this might be something you need to put back eventually should you use the floating point unit.   The "D" registers are 64-bit so they will use a lot of stack space.

If the above doesn't help, then will need to know more about the 'crash' and it's signature.   For example, is the crash immediate (1st time through the nesting process?) or it more 'random' which might indicate that there's a critical section of code that doesn't like to be interrupted again ...   Are there any signs that the stack has overflowed? (can you increase the stack sizes to see if this helps?)

Best Regards,

Anthony

Cor,

Everything Anthony has stated is correct and I see no issues with your assembly handler routine. I have filed two Clearquest entries from this discussion.

SDSCM00045250: This is a bug against the assembler to correctly handle the SRS instruction. You can use the instruction today, but the syntax is different than the official ARM syntax. You can write it as SRS <mode>{!}, which is equivalent to the correct syntax SRS SP{!}, <mode>. You should specify the mode as 18, which is the mode for IRQ. This will cause the state to be saved to the IRQ SP. You must decrement LR before the instruction. You should then be able to use RFE. This step should be done first, before saving any other registers to the stack.

SDSCM00045251: This is an enhancement request to support nested interrupts in the compiler. We currently only support code generation for non-nested interrupts.

You can track these issues through SDOWP

Cor,

One of my colleagues suggested this forum post:   http://e2e.ti.com/support/microcontrollers/hercules/f/312/t/164944.aspx

And the associated link on this post to the ARM infocenter doc.

The doc may be specific to ARM's toolchain but it covers the basics of the processor.  Sec. 6.12 covers re-entrant interrupt service routines.

Best Regards,

-Anthony

Given all the examples given by all the people,  I used the following code:

    .sect    ".text:retain"
    .retain
    .retainrefs
    .armfunc _gioInterrupt
    .state32
    .global    _gioInterrupt
_gioInterrupt
        SUB       LR,LR,#4
        SRSDB     #31! ; save LR-irq and SPSR_irq to system mode stack (note that this syntax is used because of assembler issue SDSCM00045250)
        CPS       #31 ; switch to system mode
        PUSH      {R0-R3,R12,LR} ; save AAPCS registers and system mode LR
        VMRS      R0,FPSCR
        STMFD     SP!, {R0}
        VMRS      R0,FPEXC
        STMFD     SP!, {R0}
        VSTMDB    SP!, {D0-D7}
        BL        gioInterrupt
        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {R0}
        VMSR      FPEXC, R0
        LDMFD     SP!, {R0}
        VMSR      FPSCR, R0
        POP       {R0-R3,R12,LR} ; restore AAPCS registers and system mode LR
        RFEIA     SP!
    .global    gioInterrupt

Note that the gioInterrupt C-code (called inside the _gioInterrupt above) will enable the IRQ at the beginning and disable the IRQ at the end. It also adapts the IRQ masks.

It seems to work perfectly.

One question: Do I have to switch to IRQ mode just before the return from interrupt? Or does the RFEIA do this?

I hope this implemenatation turns out to be stable.

Thank you all that contributed to this thread!

If you have any comments about this implementation (improvements, possible issues, etc), please reply!

Hi Cor, 

Great news - glad this is coming together!

The RFEIA instruction does update the PC and CPSR with the values from the stack,  so it might change modes; which I guess is why this is confusing.

However, the RFEIA instruction is undoing the work of the SRSDB instruction, which pushed the LR, SPSR on interrupt entry onto the stack.  This stack was the IRQ mode stack using the IRQ mode stack pointer.     So,  before you can pop these values back into PC and CPSR, you will need to switch back to IRQ mode so that the stack pointer is pointing to the same stack where the values were originally pushed.

[Updated:] Strike the above out.  I didn't catch the #31 specifying the system mode stack for the SRSDB instruction.   In that case, you do need to be in system mode before the RFE so I agree with Cody.

Regarding the code above, the only comment I'd have is that you might want to switch all the stack related instructions to the same suffix  (like "FD") assuming the assembler takes them.   Might just make the code easier to understand when you come back to it after a while.     The other comment I'd have is it would be a good idea to step through the ISR and just confirm that the pushes & pops happen as expected;   or that all the registers are saved and restored.   Maybe you add some test code to corrupt the registers that you are supposed to be saving to just confirm that they really are saved.   

A few notes:

I believe you can enable interrupts with the CPS instruction before saving registers to the stack (CPSIE i, #31). I think this is safe because if another interrupt is encountered, those registers will be saved by that handler before being modified. You can also skip saving VFP state if floating point is not used in your handler routine. You can most likely avoid saving FPEXC, unless you plan on disabling the VFP unit inside of gioInterrupt.  My understanding of RFE is that you do not need to switch to IRQ mode before executing the instruction, since you saved the state the the supervisor stack.

Cody,

One thing that isn't in the ISR dispatch code that Cor posted is the part about confirming the SP is 8-byte aligned before making the function call to BL gioInterrupt.

This seems to be a compiler / ABI requirement based on the advisory:  http://infocenter.arm.com/help/topic/com.arm.doc.ihi0046b/IHI0046B_ABI_Advisory_1.pdf

It would be interesting to know how this requirement would apply to the TI compiler (as it seems the problem they talk about is created by an assumption the application code might make regarding stack alignment.)    However, I might be safest to assume that in the future an object compiled with a different toolchain (e.x. CMSIS DSP library compiled with ARM's toolchain) and decide to follow the advisory so problems don't crop up in the future.   It looks like the 'problem' described would be pretty difficult to isolate and debug in a complex system.

-Anthony

Anthony,

You are correct. The stack should be aligned before the call. I think the chances that this would cause a problem on Cortex-R4 are low. The LDRD/STRD instructions only require 4 byte alignment on a Cortex-R4. The issue relating to varargs is an issue on Cortex-R4, but if they are not used then there is no concern. I'm not sure if the TI compiler relies on 8-byte alignment in our stdarg.h implementation. The safest thing to do is align the stack.

Thanks.

So this would be the final code:

_gioInterrupt
        SUB       LR,LR,#4
        SRSDB     #31! ; save LR-irq and SPSR_irq to system mode stack (note that this syntax is used because of assembler issue SDSCM00045250)
        CPS       #31 ; switch to system mode

        PUSH      {R0-R3,R12,LR} ; save AAPCS registers and system mode LR
        VMRS      R0,FPSCR
        STMFD     SP!, {R0}
        VMRS      R0,FPEXC
        STMFD     SP!, {R0}
        VSTMDB    SP!, {D0-D7}

        ; allign the stack
        AND         R1, sp, #4
        SUB         sp, sp, R1
        PUSH        {R1, lr}

        BL        gioInterrupt

        ; reverse the stack allignment
        POP         {R1,lr}
        ADD         sp, sp, R1

        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {R0}
        VMSR      FPEXC, R0
        LDMFD     SP!, {R0}
        VMSR      FPSCR, R0

        POP       {R0-R3,R12,LR} ; restore AAPCS registers and system mode LR

        RFEIA     SP!

@ Anthony: I do not understand your remark about the "FD" suffix. I'm not that familiar with the assembly syntax.

Aaaaargh, there still seems to be a problem!!

Everything seems ok when running the application from CCS. But when just connecting power to the board (after running once from CCS so that the application is flashed), the application does start, but it crashes very quickly. It seems that as soon as the IRQ is generated, the application crashes!

For this test I did not use the re-entrancy. But the IRQ will be interrupted by FIQ interrupts.

When not using the IRQ, the application works fine this way. Using Debug or Release does not matter.

When using the (none-re-entrant) interrupt code generated by the assembler when using the #pragma INTERRUPT(_gioInterrupt, IRQ), it works fine this way.

So I compared that code with my code. Here is the code generated with the pragma:

_gioInterrupt:
        STMFD     SP!, {A1, A2, A3, A4, V9, LR}
        VMRS      V9,FPSCR
        STMFD     SP!, {V9}
        VMRS      V9,FPEXC
        STMFD     SP!, {V9}
        VSTMDB    SP!, {D0-D7}
        BL        gioInterrupt
        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {V9}
        VMSR      FPEXC, V9
        LDMFD     SP!, {V9}
        VMSR      FPSCR, V9
        LDMFD     SP!, {A1, A2, A3, A4, V9, LR}
        SUBS      PC, LR, #4

Why does this code work and my code not? (remember I'm not usingre-entrancy in this test).

One big difference is the fact that in my case the link register is adjusted at the beginning. While in the code above it is at the end. Could that be the problem?

Cor Jansen said:

@ Anthony: I do not understand your remark about the "FD" suffix. I'm not that familiar with the assembly syntax.

Cor,

There are four possible types of stacks,  (Empty or Full) x (Ascending or Descending).  There's a table that describes this in the DDI0406B architecture manual (Table A4-11)  ..  but the EABI standard requires the Full Descending type,  which means the loads should use the "IA" increment after suffix and the stores should use the "DB" decrement before suffix.

The assembler offers the option to just use a single suffix "FD" as an alias which might be simpler, you just need to remember the stack type and not that loads get IA and stores get DB...     So LDMIA and LDMFD are the same, and STMDB and SDTFD are the same.

I was just noting that the assembly uses a mix of the IA and FD suffixes, and it might be cleaner to use FD everywhere.   But that's a style comment so it shouldn't have any affect on functionality.

Best Regards,

Anthony

Hi Cor,  just wanted to check to see if you've got any updates or news.

Is the code that is *not* working when you run freely the code from the post :  Posted by Cor Jansen on Thu, Sep 6 2012 3:10 AM

If so, I see two things:

  - comparing to the example in the ARM reference,  the LR is pushed twice onto the stack where the ARM example code pushes it once.

    I can't see that this would be a problem, it appears the code in your post is symmetric in pushing and popping.   But it might make sense to step

    through that and see what's happening. **

** I know you say the code doesn't crash while running under the emulator,  but I am still asking because some problems might take a while to accumulate under the debugger control (like a stack 'leak') but when you run free could happen very quickly..

  - Interrupts should be disabled again after returning from the BL gioInterrupt.   I wasn't sure if you were doing this inside the gioInterrupt subroutine or not.

     And I think you're saying also that this issue happens even if you don't disable & re-enable interrupts.   So mainly mentioning this point for when you try to turn on nesting again.

So, by looking at the code,  I'm not seeing anything that would crash.   Would be good to get more details on the type of crash you are seeing.

For example, are you certain that the crash is on the very first IRQ?   How about the error pin on the micro, are you seeing it lit?

And, do you know it is actually a crash or could there be some function that's not behaving as expected, but the flow of code could still be fine...

Also - what is the exact part # of the micro?   On some of the older parts we've got an issue w. input buffers being disabled by default which was disabled when the emulator was attached, so this is a common question whenever someone reports code acting differently with/without JTAG.  

Last one, in CCS Tools->ARM Advanced Features -> Generic Debugger Options   do you have any of the checkboxes set under "Disable Interrupts"?
(scroll to the bottom to see this group of checkboxes)

Best Regards,
Anthony

Hello Anthony, thank you very much for inquiring.

I'm really getting frustrated with this problem. I did a lot of test trying to get closer to the problem. But there seem to be no logic behind it. When adding a few lines here may make the code work and adding a few lines there may result in the bug. It makes no sense. Maybe an uninitialized memory problem? I checked all variables but can not find any problem. Maybe a stack read from a location not written? You say the stack seems symmetric so that may not be the issue.

About your questions:

Yes the the interrupt code used is the one posted Sep 6.

About the LR pushing. The first one pushes the lrq-mode LR and the second one the system-mode LR. I'm not sure it is needed, but I have seen it in some example code.

Yes I disable the IRQ interrupt before returning from the high level interrup routine. And the problem also exists when not enabling/disabling the interrupt (thus not using nesting). So it seems something is wrong with the low level routine (or something completely different?).

The used processor is: RM48L950AZWTT YFA-I5A930W GI (that is what is on the package. Do you need more?)

The Debugger Options: From the Disabled Interrupt the following checkboxes are checked: When assembly stepping and When source stepping

I'm pretty sure that it crashes at the first IRQ. I use the FIQ interrupt for a high speed interrupt and I generate pulses on a GIO pin at each FIQ interrupt. After 10 FIQ interrpts I generate the IRQ interrupt. This interrupt also will pulse some other GIO pin. When it crashes I see the 10 pulses on the pin for the FIQ interrup pulses, but the pin for the IRQ interrupt pulses stays low.

The error LED stays off.

Question:

When I run the code from CCS, it flashes the code to the evaluation board. Should after this the board always startup correctly when powering it up (without CCS)? Or do I have to flash it using nowFlash with some special options?

Hi Cor,

Thanks for the answers.

Regarding the lr,  i noticed it's pushed 3 times & that's what I'm questioning.   But it looks like it's also popped 3 times so

I don't think this is the cause of any problem.

        SUB       LR,LR,#4
        SRSDB     #31! ; save LR-irq and SPSR_irq to system mode stack (note that this syntax is used because of assembler issue SDSCM00045250)
        CPS       #31 ; switch to system mode
        PUSH      {R0-R3,R12,LR} ; save AAPCS registers and system mode LR
        VMRS      R0,FPSCR
        STMFD     SP!, {R0}
        VMRS      R0,FPEXC
        STMFD     SP!, {R0}
        VSTMDB    SP!, {D0-D7}

        ; allign the stack
        AND         R1, sp, #4
        SUB         sp, sp, R1
        PUSH        {R1, lr}

- RM48L950AZWTT This chip doesn't have the input buffer disable function that works differently with & without the emulator attached.

So that should not be the problem.

Debugger options look fine.  I wanted to make sure that the third box wasn't checked, because that might really make the emulator runs different from free runs.

There are some issues on the Rev A silicon that you have that you have and these can affect startup if you are including the code that initializes and runs all of the RAM self tests, but I've always gotten stuck there in the debugger too, (not just with debugger detached).  And you report getting the 10 FIQ pulses so I (hope) you are further along in code than just the very initial stuff before main().

Flashing in CCS and powering up again should work.  Are you enabling ECC on the flash or have you kept that off?

Please do let me know if the fail always happens, or if you can get the code to work sometimes by cycling power and/or hitting the power-on reset button on the kit.

Have you been able to step through an FIQ and watch it return to the IRQ correctly through CCS?

How about the rest of the vector table (say for the abort vectors and swis and the like)  is there any function handler tied to these routines?   Maybe you could toggle another GIO pin from inside these just to see if you ever hit them.

A bit hard to think what to do next.  Is your code example small and non-proprietary enough that you can send it to us so we can check what's going on?

Only other thing I can think of to ask is related to this comment:  "When it crashes I see the 10 pulses on the pin for the FIQ interrup pulses, but the pin for the IRQ interrupt pulses stays low."  --  does this mean the FIQs stop after the first 10 too?  Or do they continue in bursts of 10 and you're just not seeing the IRQ pin toggle.

Best Regards,

Anthony

Hello,

I didn't read the complete thread but the problem here looks like a re-entrant interrupt handler is needed. I posted a source code example in this thread: http://e2e.ti.com/support/microcontrollers/hercules/f/312/t/164944.aspx. This example codes can also just be used for single interrupt functions.

You might also find this documentation from ARM helpful: http://infocenter.arm.com/help/topic/com.arm.doc.dui0471h/Bgbeacfi.html

Best Regards,
Christian

Hello Anthony,

I copied the stack allign code from some other example. I didn't realize it pushed the same LR there. I will make that nicer.

I do not get stuck in the initialization code. The main function is called always.

Once I have a version that does not start correctly at powerup, then it always seems to fail at powerup.

Yes I have have stepped through the code and everything seems ok to me.

I will try your suggestion to toggle a GIO pin for any remaining interrupt. I did that for all notifications but not for unused interrupts.

When it fails afer 10 FIQ pulses, the FIQ pulses stop too! So somehow the processor crashes or goes to some stopped-state.

The stranges thing is that adding some (dummy) code lines influences the problem. So it somehow depends on code, stack or variable locations.

I will try to isolate a small example showing the problem.

@Christian

Hello Christian.

Nested interrupt is exactly what I try to get working here. I have seen your code and the Arm article. I used both as input for the IRQ routine I posted on sept 6. This code seems to work perfectly. I can nest IRQ interrupts. But only when running from CSS. When just powering-up the board I run into problems. Sometimes it work ok (at each startup), but when adding a few lines of code here or there, it can happen that it does not work when start from powerup. And it will fail always at startup in that case.

I'm not sure if it has anything to do with the IRQ routine. But when it fails, it seems to fail as soon as the IRQ is generated.

Cor,

you can try the attached example project, it an advanced version of the one I posted in may.
For me it works with a RM48L950 MCU.

It simply uses a couple of RTI interrupts to demonstate the interrupt nesting.
There are several variables in the file notifications.c to track how often the single RTI interrupts where executed.

I hope this helps.

Best Regards,
Christian

@Christian

Thanks. I will check your version and compare it with my version.

@Anthony

Yes, I'm afraid something completely different is the problem here. Your idea to connect the debugger after powerup seems a very good idea. I checked that I can indeed connect after poweringup and I can see where the program is executing and start debugging. Unfortunately the problem didn't occur for a while. I added some GIO pin manipulation to check if phantominterrupts occur and it did not happen anymore. Even when going back to the old situation. Only some interrupt names in the VIMRAM are still changed (I can not restore the old values). But the functionality is the same. But problem is gone !!??? Weird!

I assume the nested IRQ is ok. When the problem occurs again, I can connect to debugger and check what is going on.

Hi,

My conclusion is that the instability was caused by a stack problem. A bigger supervisor stack did the trick. This is my final nestable interrupt routine (for all people that need something like this):

    .sect    ".text:retain"
    .retain
    .retainrefs
    .armfunc interrupt10
    .state32
    .global    interrupt10
interrupt10
        SUB       LR,LR,#4
        SRSFD     #31! ; save LR-irq and SPSR_irq to system mode stack (note that this syntax is used because of assembler issue SDSCM00045250)
        CPS       #31 ; switch to system mode
        STMFD     SP!, {A1, A2, A3, A4, V9, LR}

        VMRS      V9,FPSCR
        STMFD     SP!, {V9}
        VMRS      V9,FPEXC
        STMFD     SP!, {V9}
        VSTMDB    SP!, {D0-D7}

        ; allign the stack
        AND         R1, sp, #4
        SUB         sp, sp, R1
        PUSH        {R1}

        BL        hetInterrupt

        ; reverse the stack allignment
        POP         {R1}
        ADD         sp, sp, R1

        VLDMIA    SP!, {D0-D7}
        LDMFD     SP!, {V9}
        VMSR      FPEXC, V9
        LDMFD     SP!, {V9}
        VMSR      FPSCR, V9

        LDMFD     SP!, {A1, A2, A3, A4, V9, LR}
        RFEFD     SP!

    .global    hetInterrupt

Note that I enable/disable the IRQ in the high level routine. Also the interrupt masking is handled there. I'm not sure if the stack allignment (plus reverse) is needed.

At the beginning of the routine I save the LR-irq and SPSR_irq to system mode stack. After that I set the mode to system mode so that he high level interrupt routine is run in this mode and the RFE instruction will restore the registers from the correct stack (system mode stack). I think I can also choose IRQ mode in stead of system mode or maybe supervisor mode. I do not know what is best practise.

So my last question probably is: Which mode should I use to save the registers and to run the high level interrupt routine in?