This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TM4C1294NCPDT: SysTick issue

Part Number: TM4C1294NCPDT

Hello.

I've already created a ticket in costumer support center (CS0724274, ticsc.service-now.com/csm), but I was asked to move it here since it's more suitable for this kind of problem.

So I try to copy here all the information from the ticket ...

I have a problem with systick timing. At the beginning of the investigation we found out that there's a problem with periodical sending of Profinet cyclic data. The cyclic frames should be sent each 2 ms, but we measured that sometimes (once in a few minutes) one frame is sent just 1 ms after the previous one. And then continue again with correct 2 ms cycle.

I did a long investigation going deeper to find the source of the issue. As I finally find out and verified the problem is in timing of systick timer, which is set to 1 ms with highest priority. It's very strange but sometimes the systick interrupt  is raised additionally in much shorter time.

I was going through the err data document but I haven't found there such an issue. Don't you know about similar problem which could appear on the controller? Or would you have any explanation for the behavior?

 I checked out the systick registers when the interrupt is raised and it shows that sometimes it's really called too early since the timer value doesn't time out according to the actual timer value gained by MAP_SysTickValueGet function.

I'm adding a few screenshots from my measurements. To see the behavior there are cyclic buffers storing the last 10 values. There can be seen the time since last interrupt, the actual timer value and actual system state register (0xE000ED24).

I have additional information for the issue. I've recorded a few additional registers to see the actual state when the sys tick interrupt handler is called. I attached the screenshots of last 10 values of

- time from last sys tick interrupt (measured by another timer)

- System Handler Control and State (SYSHNDCTRL), offset 0xD24

- SysTick Control and Status Register (STCTRL), offset 0x010

- SysTick Reload Value Register (STRELOAD), offset 0x014

- SysTick Current Value Register (STCURRENT), offset 0x018

- Interrupt Control and State (INTCTRL), offset 0xD04

- System Handler Priority 3 (SYSPRI3), offset 0xD20 As can be seen from the pictures, something strange happened in the cycle 3, see the pictures.

Complete Record 01.zip

I'm sending you the file with the systick handling.

The SysTick configuration ...

	ABC_IRQ_DISABLE(ABC_INT_SCHEDULER);

	/* Set the default priority level to be masked */
	CPUbasepriSet(ABC_INT_PRIORITY_MASKED);

	/* set the priority for the systems interrupts */
	MAP_IntPrioritySet(ABC_INT_SYSTICK,  ABC_INT_PRIORITY_0);
	MAP_IntPrioritySet(ABC_INT_SCHEDULER,ABC_INT_PRIORITY_SCHEDULER);
	MAP_IntPrioritySet(ABC_INT_SVCALL,   ABC_INT_PRIORITY_3);

	ABC_IRQ_ENABLE(ABC_INT_SCHEDULER);
	
	...
	
	// Configure SysTick for a periodic interrupt.
	(void) MAP_SysTickPeriodSet( (uint32_t) cpuinfo_drv_get_frequency() / SYSTICKHZ);
	(void) MAP_SysTickEnable();
	(void) MAP_SysTickIntEnable();

Where ...

	#define ABC_INT_SYSTICK       FAULT_SYSTICK
	#define ABC_INT_SCHEDULER     FAULT_PENDSV
	#define ABC_INT_SVCALL        FAULT_SVCALL

	...

	#define ABC_INT_PRIORITY_0 (0x00 << 5)
	#define ABC_INT_PRIORITY_1 (0x01 << 5)
	#define ABC_INT_PRIORITY_2 (0x02 << 5)
	#define ABC_INT_PRIORITY_3 (0x03 << 5)
	#define ABC_INT_PRIORITY_4 (0x04 << 5)
	#define ABC_INT_PRIORITY_5 (0x05 << 5)
	#define ABC_INT_PRIORITY_6 (0x06 << 5)
	#define ABC_INT_PRIORITY_7 (0x07 << 5)
			
	// The default IRQ priority of scheduler
	#define ABC_INT_PRIORITY_SCHEDULER ABC_INT_PRIORITY_6

	// The IRQ priority which is by default masked
	#define ABC_INT_PRIORITY_MASKED    ABC_INT_PRIORITY_7
	
	...
	
	CPUbasepriSet(uint32_t ui32NewBasepri)
	{
		//
		// Set the BASEPRI register
		//
		__asm("    msr     BASEPRI, r0\n"
			  "    bx      lr\n");
	}
	
	...
	
	// enable/disable all interrupts
	#define ABC_IRQ_ENABLE_ALL            (void) MAP_IntMasterEnable()
	#define ABC_IRQ_DISABLE_ALL           (void) MAP_IntMasterDisable()

	// enable/disable specific interrupts
	#define ABC_IRQ_ENABLE(IRQ_ID)        abc_irq_enable(IRQ_ID)
	#define ABC_IRQ_DISABLE(IRQ_ID)       abc_irq_disable(IRQ_ID)
	
	...
	
	ABC_STATIC_INLINE void abc_irq_enable(unsigned int irq_id)
	{
	  if (irq_id != ABC_INT_SCHEDULER)
	  {
		MAP_IntEnable(irq_id);
	  }
	  else
	  {
		MAP_IntPrioritySet(FAULT_PENDSV, ABC_INT_PRIORITY_SCHEDULER);
	  }
	}

	ABC_STATIC_INLINE void abc_irq_disable(unsigned int irq_id)
	{
	  if (irq_id != ABC_INT_SCHEDULER)
	  {
		MAP_IntDisable(irq_id);
	  }
	  else
	  {
		MAP_IntPrioritySet(FAULT_PENDSV, ABC_INT_PRIORITY_MASKED);
	  }
	}


And the Systick ISR handler ...

/***
 * Process tick of the system timer.
 ***/
void SysTick_Handler(void)
{
  ABC_IRQ_ACK(ABC_INT_SYSTICK);
    
  // call the system tick handler
  abc_sys_timer_tick();
  
  // trigger clocks
  abc_tmr_clock_trigger(CLOCK_REALTIME);

  ABC_IRQ_EOI(ABC_INT_SYSTICK);
}

Where ...

/***
 * Acknowledge interrupt
 **/
#define ABC_IRQ_ACK(irq_nr)   ABC_IRQ_DISABLE_ALL 

/***
 * End of interrupt
 **/
#define ABC_IRQ_EOI(irq_id)  if (0 == abc_spin_irq_count) { ABC_IRQ_ENABLE_ALL; }

It seems to me there's some problem between SysTick interrupt and PendSV interrupt used by the OS scheduler. The scheduler uses following functions ...

// Add scheduler interrupt request
ABC_IRQ_SET_REQUEST(ABC_INT_SCHEDULER);
// Remove scheduler interrupt request
ABC_IRQ_CLR_REQUEST(ABC_INT_SCHEDULER);

// set a specific interrupt request
#define ABC_IRQ_SET_REQUEST(IRQ_ID) MAP_IntPendSet(IRQ_ID)
#define ABC_IRQ_CLR_REQUEST(IRQ_ID) MAP_IntPendClear(IRQ_ID)

// IntPendSet from TivaWare driverlib
void IntPendSet(uint32_t ui32Interrupt)
{
...
//
// Pend the SysTick interrupt.
//
HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PENDSTSET;
...
}

Can't you see there any problem?

  • Hello,

    Thank you for sharing all these details and the code as well. This isn't an issue I've really seen before with SysTick so I don't have a sure fire answer to provide right away. This really seems to be specific to your application currently which makes it rather challenging to help debug.

    I have tried to create this problem on my end using a basic example and doing something similar in terms of an ISR that turns off interrupts and does some other processing and I don't think I've observed this yet, but I want to try another setup on how to track it before I verify that I cannot re-create it. I should have that done this afternoon but wanted to give some form of reply before then.

    I would agree that the initial thought that comes to mind for me would be that IntPendSet is used in a manner to trigger this occurrence. Have you investigated the cases for when IntPendSet is used for SysTick and what occurs?

    Best Regards,

    Ralph Jacobi

  • Hello,

    Just wanted to report I have a code running now with a timer setup to run along side the SysTick and a tight tolerance fail case to trip an LED if the SysTick fires too quickly. I have it setup with disabling interrupts at the start, handling processing for the Timer value checking and a second LED toggling, and then enabling interrupts at the end. It's been running for over 30 minutes without tripping an error. I will let it run overnight to be sure but initial results would indicate that the issue is application related.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    I move forward in the investigation of the issue.

    I reused the IntPendSet function from the Tivaware and did there some changes ...
    The original IntPendSet function is ...

    //*****************************************************************************
    void
    IntPendSet(uint32_t ui32Interrupt)
    {
    //
    // Check the arguments.
    //
    ASSERT(ui32Interrupt < NUM_INTERRUPTS);

    //
    // Determine the interrupt to pend.
    //
    if(ui32Interrupt == FAULT_NMI)
    {
    //
    // Pend the NMI interrupt.
    //
    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_NMI_SET;
    }
    else if(ui32Interrupt == FAULT_PENDSV)
    {
    //
    // Pend the PendSV interrupt.
    //
    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV;
    }
    else if(ui32Interrupt == FAULT_SYSTICK)
    {
    //
    // Pend the SysTick interrupt.
    //
    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PENDSTSET;
    }
    else if(ui32Interrupt >= 16)
    {
    //
    // Pend the general interrupt.
    //
    HWREG(g_pui32PendRegs[(ui32Interrupt - 16) / 32]) =
    1 << ((ui32Interrupt - 16) & 31);
    }
    }

    When I change the setting of pending of PendSV interrupt in the following way ... 

    HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV)  & ~NVIC_INT_CTRL_PENDSTSET;

    so I set the NVIC_INT_CTRL_PENDSTSET of the NVIC_INT_CTRL register to 0 every time I'm setting the PendSV interrupt, the behavior is correct.

    Our operating system uses the calling of IntPendSet function with FAULT_PENDSV parameter for switching between threads.

    My idea of the problematic situation is ...

    - HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV) is not atomic operation

    - it could happen that between reading the NVIC_INT_CTRL register and it change  + writing back, it's interrupted and the NVIC_INT_CTRL_PENDSTSET is set to 0 value. But since the original read value of NVIC_INT_CTRL_PENDSTSET bit is 1, it's rewritten once again and the SysTick interrupt is raised once again. 

    - I was able to simulate this behavior. After some time, when the read value of NVIC_INT_CTRL_PENDSTSET bit was 0, I did following operation ... 

    HWREG(NVIC_INT_CTRL) = HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV | NVIC_INT_CTRL_PENDSTSET;

    which set the NVIC_INT_CTRL_PENDSTSET to 1. This really causes the raising of the SysTick interrupt after too short time with the sign that the SysTick timer didn't timed out.

    Do you agree with my results, does it makes sense to you?

    But how should I fix it? How could I ensure that the HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV) operation is not interrupted?

    Zdenek

  • Hello Zdenek,

    For starters, I am working under the understanding that your IntPendSet is now setup like as follows:

    //*****************************************************************************
    void
    IntPendSet(uint32_t ui32Interrupt)
    {
        //
        // Check the arguments.
        //
        ASSERT(ui32Interrupt < NUM_INTERRUPTS);
    
        //
        // Determine the interrupt to pend.
        //
        if(ui32Interrupt == FAULT_NMI)
        {
            //
            // Pend the NMI interrupt.
            //
            HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_NMI_SET;
        }
        else if(ui32Interrupt == FAULT_PENDSV)
        {
            //
            // Pend the PendSV interrupt.
            //
            HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV)  & ~NVIC_INT_CTRL_PENDSTSET;
        }
        else if(ui32Interrupt == FAULT_SYSTICK)
        {
            //
            // Pend the SysTick interrupt.
            //
            HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PENDSTSET;
        }
        else if(ui32Interrupt >= 16)
        {
            //
            // Pend the general interrupt.
            //
            HWREG(g_pui32PendRegs[(ui32Interrupt - 16) / 32]) =
                    1 << ((ui32Interrupt - 16) & 31);
        }
    }
    

    If that is wrong then please clarify because the following is based on that understanding.

    So is the intent here that when PendSV is being set for pending, you want to clear any pends on the SysTick interrupt at the same time?

    If so, can you give some sort of background for why then?

    And if so, would it be simply possible to just disable global interrupts before and after the operation which would be the most straight forward fix? Would that negatively impact other elements of the RTOS?

    How could I ensure that the HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV) operation is not interrupted?

    Is that the full operation you are trying to avoid interrupts for or is it this?

    HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV)  & ~NVIC_INT_CTRL_PENDSTSET;

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    Yes, you are right, the only change in IntPendSet function is this line ... 
    HWREG(NVIC_INT_CTRL) = (HWREG(NVIC_INT_CTRL) | NVIC_INT_CTRL_PEND_SV)  & ~NVIC_INT_CTRL_PENDSTSET;
    But he intent is to set PendSV for pending and avoid to set to set SysTick interrupt for pending by mistake, not to clear any pends on the SysTick interrupt at the same time. There's another bit in the register intended for clearing, NVIC_INT_CTRL_PENDSTCLR. As I found out I haven't lost any SysTick interrupt this way, the timing is still correct.

    Let me describe it in more details ...

    • as I understand the SysTick behavior, there's running the SysTick timer and when it times out (ticks to 0 value), there's set the NVIC_INT_CTRL_PENDSTSET bit in NVIC_INT_CTRL register to 1. 
    • Change of this bit from 0 to 1 causes that the SysTick interrupt to be raised.
    • But I don't know who and when sets the NVIC_INT_CTRL_PENDSTSET back to 0.
    • Then the IntPendSet(FAULT_PENDSV) is called. It reads the actual state of NVIC_INT_CTRL register, where NVIC_INT_CTRL_PENDSTSET is still set to 1.
    • But before it adds the NVIC_INT_CTRL_PEND_SV and writes the register back, it's interrupted and the NVIC_INT_CTRL_PENDSTSET is set to 0.
    • Then the processing of IntPendSet continues, it writes back the new value of NVIC_INT_CTRL register but it contains the original NVIC_INT_CTRL_PENDSTSET value of 1 which causes the SysTick interrupt to be raised by mistake.

    I don't know if my description is completely right since I don't know exactly the SysTick processing, but I understand it this way from my investigation.

    I had the same idea with disabling the interrupts to avoid the interruption during setting of NVIC_INT_CTRL_PEND_SV to NVIC_INT_CTRL register, but it stops our RTOS from processing. 
    When I tried following change ...
    SysTickIntDisable();
    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV;
    SysTickIntEnable();
    the result was that it stopped in the RTOS idle thread trying to enable interrupts

    Disabling of all the interrupts by IntMasterDisable causes similar problem somewhere else in the RTOS threads processing.

    To be honest I don't understand it since I don't have deep knowledge of the RTOS but I would assume that disabling of interrupts in this point just causes the registers operation to be finished and then when the interrupts are enabled again, it continues in normal processing.

    Best regards,

    Zdenek Krejsa

  • Hello Ralph.

    During the investigation we've noticed today that all the bits of the NVIC_INT_CTRL register are read only, write only or reserved. And the write only bits have some effect just on writing of 1 value, 0 has no effect. So I changed the setting of PendSV interrupt in following way ... HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV and it works correctly. So I assume it's correct to do it this way, isn't it? I can set write there value with just the one bit which I want to apply, the rest of the bits in the written value can be 0?! Could this be the fix of our issue?

    Regards,

    Zdenek

  • Hi Zdenek,

    During the investigation we've noticed today that all the bits of the NVIC_INT_CTRL register are read only, write only or reserved

    That is not true. NMISET, PENDSV, and PENDSTSET are all labelled 'RW' and thus are Read/Write.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    Sorry for late reply, I was on holiday on Thursday and Friday.

    You are right, I overlooked it, some bits are RW. But it doesn't influence the rest of my question.

    All the bits enabled for writing (WO or RW) have some effect just on writing of 1 value, 0 has no effect according to the data sheet.

    So I changed the setting of PendSV interrupt in the IntPendSet function in following way ... HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV and it works correctly. The systick issue disappeared and all the rest interrupts works fine. So I assume it's correct to do it this way, isn't it? I can write there value with just the one bit which I want to apply, the rest of the bits in the written value can be 0? Could this be the fix of our issue?

    Regards,

    Zdenek

  • Hello Zdenek,

    I would be interested to know if there is any functional difference between these lines for your code:

    HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV

    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV;

    Because based on what we are looking at for that register, they should be identical in functionality. Which essentially means you'd be reverting back to our proven TivaWare code. Which leads me to wonder if it was changed to begin with unless it was changed during initial development and not as a fix for an issue that was observed?

    I have no issue with that being the resolution to be clear but it begs the question of why the original TivaWare handler got modified in the first place.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    I wrote all the detailed information in previous messages, but I try to sum it up ...

    - we had a problem in timing in our RTOS. Sometimes one of the thread which should be run cyclically each 2 ms was started much earlier

    - the timing of the RTOS is controlled by SysTick timer interrupt, which is set to 1 ms cycle. As I found out sometimes the SysTick interrupt was raised additionally between  the normal 1ms cycles. There's the COUNT bit in SysTick Control and Status Register (STCTRL), which can be used to verify if the Systick interrupt was raised by SysTick timer timeout. In case of additional raising of the interrupt, the COUNT bit was zero, which means it was not raised by SysTick timer timeout.

    - it led me to the Interrupt Control and State register (INTCTRL), which can be used for raising SysTick interrupt. We use the  IntPendSet function from TivaWare to handle the INTCTRL register. But we use it just for raising of PendSV interrupt

    - as can be seen in the IntPendSet function, the PENDSV bit in the INTCTRL register is set by the code HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV. This piece of code means, that the actual value of INTCTRL register is firstly read, then the PENDSV bit is added and the result is written back, which is several instructions

    - we found out in the data sheet of the controller that writing of 0 value to any of RO or RW bits of INTCTRL register has no effect, just writing of 1 should influence the exception pending. So I reused the IntPendSet function of TivaWate and just changed the command in following way ... HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV. This way just the value of PENSV bit is 1, the rest is 0, but it should be correct according to data sheet information

    - I tested the FW with this change and the results were perfect. The SysTick interrupt worked correctly, there were no additional SysTick intrrupts and the rest of the system worked correctly

    - my understanding of the issue is that the HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV operation was interrupted after reading of INTCTRL register actual value (where PENDSTSET bit was 1), because it's not atomic instruction. In this interruption the SysTick pending was cleared. Then the operation is finished, but the read PENDSTSET bit was 1, we set also PENDSV bit to 1 and wrote back. And the PENDSTSET bit causes to raise additional Systick interrupt. 

    So back to you questions ...

    - there's a big difference between these 2 commands ...
    HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV
    HWREG(NVIC_INT_CTRL) |= NVIC_INT_CTRL_PEND_SV
    the first one is atomic and sets just the PENDSV bit to 1

    - the change was done as a fix for the issue and it really works. I change it this way because the available information shows me that it should be done this way

    Can you go through all the provided information and let me know if
    - you have all the necessary information for investigation
    - my fix is correct and could be used this way
    - what will be the following step
    please?

    Best regards,

    Zdenek

  • Hello Zdenek,

    Thanks for confirming the results of the change vs the original code. 

    I ran some additional experiments on the project I had put together previously and confirmed your findings.

    The two solutions to avoid the issue would be:

    1. As proposed, just use HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV
    2. Before calling IntPendSet, disable interrupts to that IntPendSet can resolve without an interrupt firing.

    For your application, 1. should be just fine to use to solve this issue. While 2. would be the most foolproof way, it may add a small amount of delay in the SysTick being serviced.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    Thank you for verifying and approving my findings. I will use the 1st way of fixing, HWREG(NVIC_INT_CTRL) = NVIC_INT_CTRL_PEND_SV.

    Since we know about the issue, do you consider to change it directly in the TivaWare and release a new version? Because this issue can appear also in projects of other users of the TivaWare library.

    Best regards,

    Zdenek Krejsa

  • Hello Zdenek,

    I will discuss it with my team if this should be added to our bug tracker for future revisions but my initial takeaway is that this is application specific and if I were to run into such an issue, I would instead disable interrupts before processing IntPendSet.

    Regardless of our decision to file it as a bug, we will not release a new version for a minor fix and this would not come until a larger update is required.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    It's application specific just in the way that it sometimes happens in our case, but it's probably by chance caused by some time conditions. May be it's appearing also in other applications but the users just do not notice that since it's quite demanding to find out that sometimes once in a few tens of minutes, there appears an additional systick interrupt.

    What's really important to me is the fact that the IntPendSet function behaves in a different way than it's described in the documentation. There's written that "The specified interrupt is pended", no warning about possibility of other interrupt to be set to pending state, no warning about necessity of interrupts disabling. 

    From my point of view there's not a need for interrupt disabling before the IntPendSet since the INTCTRL register provides a way for correct setting of 1 bit, one specific interrupt. Disabling of interrupts seems to be just a workaround, change of writing to INTCTRL register is the real fix.

    It took me a long time to find out that the root cause of my timing issue is in the IntPendSet function. Fixing of the issue would avoid other users of facing to similar issue and long investigation.

    Best regards,

    Zdenek Krejsa

  • Hello Zdenek,

    I've reviewed the request with our team and we are submitting a bug report for it. When the next TivaWare update is done, we will do a deeper analysis into the potential impact of the change. While I would not expect there to be an impact, as it is a fundamental change in how this function operates today, we do need to take due diligence to validate that the change would not break any existing applications and that will be part of the investigation before we approve the change. That said at minimum we can edit the documentation if no change is made to clarify that potential issue.

    Best Regards,

    Ralph Jacobi

  • Hello Ralph.

    Thank you for the cooperation.

    I think that there could be change all the expressions "HWREG(NVIC_INT_CTRL) |= " in the IntPendSet and IntPendClear function in the similar way, so the result would be "HWREG(NVIC_INT_CTRL) = "

    Best reghards,

    Zdenek Krejsa