AM335x ISR/HWIs blocked in BIOS 6.35.4.50 XDC 3.25.3.72

Moving this to the SYS/BIOS forum.

Frank,

Looking at that other thread, it seems you found many timing issues after connecting a CPLD.  And you tried updated BIOS and XDCTools versions, and things surprisingly got better.  But there is still an issue with interrupts not getting triggered as expected.  So… it seems that it wasn’t the case that the app stopped working once you switched versions (as stated in this thread), but instead things got better, but there are still issues.  Is this correct?

You mention a bug in an EDMA starterware driver.  Did you fix that, or swap that driver too?  

Is it just your DM timer interrupts that are not firing as expected?  Or other ISRs too?

And have you tried disconnecting the CPLD to see if the DM timer interrupts fire as expected?

And, can you describe more what you mean by “suspend and resume the program with the debugger, get 2 additional interrupts, etc.”?  Are you doing a connect/reconnect in CCS, or somehow a power management type suspend/resume?

Thanks,
Scott

Scott,

thanks for your reply.

Looking at that other thread, it seems you found many timing issues after connecting a CPLD.  And you tried updated BIOS and XDCTools versions, and things surprisingly got better.  But there is still an issue with interrupts not getting triggered as expected.  So… it seems that it wasn’t the case that the app stopped working once you switched versions (as stated in this thread), but instead things got better, but there are still issues.  Is this correct?

Actually, I have three different problems: GPMC, EDMA, ISRs. They only have in common, that I need all the things for my final project.

1st, and main problem is the GPMC timing that is different with the BIOS / XDC versions - it's fine that it runs fast with the new versions, but to be sure, I want to know why, respectively why that slow with the older version. (no ISRs, no EDMA here)

2nd, and meanwhile solved, is the problem with the edma.c driver for the EDMA programming I started to make things faster.

I had selected AM3359 [Cortex A] as device in the project properties, which led to

"C:/ti/am335x_sysbios_ind_sdk_1.1.0.6/sdk/starterware/drivers/edma.c", line 117: error #20: identifier "SOC_EDMA3_NUM_DMACH" is undefined

errors, since edma.h requires --defined am335x, and not --defined am3359 as generated.
Adding --defined am335x manually, I got some linker errors due to XDC misconfiguration.
Choosing Generic CortexA8 Device solves the problem. (no ISRs, no GPMC)

3rd issue are the INTs/HWIs not working with the new BIOS/XDC (no GPMC, no EDMA). To make the problem reproducable for you, I started to set up the project from the beginning.

Is it just your DM timer interrupts that are not firing as expected?  Or other ISRs too?

I set up a clock module triggered by the 32 kHz timer, and I found it affected too. Adding things step by step, I arrived at the following point: The problem seems to be somthing with the MMU (I need it for the access to the HWREGs of the timer etc., would be the next step):

Int main()
{
    /*
     * use ROV->SysMin to view the characters in the circular buffer
     */
    System_printf("enter main()\n");
    /* init mmu, otherwise crash on access to PRCM registers */
    /* works fine with XDCtools 3.25.3.72, BIOS 6.35.4.50 */
    /* no INTs / Clock with XDCtools 3.30.5.60, BIOS 6.41.0.26 */
    /* if mmuInit is called (Clock does not need this) */
    // mmuInit(applMmuEntries);

I attached the project files, details see there. Maybe the MMU code I copied is not up to date.

And have you tried disconnecting the CPLD to see if the DM timer interrupts fire as expected?

And, can you describe more what you mean by “suspend and resume the program with the debugger, get 2 additional interrupts, etc.”?  Are you doing a connect/reconnect in CCS, or somehow a power management type suspend/resume?

As stated, there's no relation to GPMC/CPLD. I just Start/Resume [F8] and Suspend [Alt F8] the program and look at some counters / timestamps in the watch window. In the project sent, you will see the counter tick_1ms incrementing or not.

Hope that helps you to find a reason for the missed ISRs. And may be, you have an idea for my problem #1 (I don't have an answer yet in the other thread.)

Thanks,

Frank

7128.houston.zip

Hi Scott,

meanwhile I updated my project to use the code from C:\ti\am335x_sysbios_ind_sdk_1.1.0.6\sdk\os_drivers\src\osdrv_mmu.c, and all the ISRs work as expected. It is the code

#if (ti_sysbios_family_arm_a8_Mmu___VERS >= 160)
        if (!attrs.bufferable && !attrs.cacheable)
            attrs.tex = 0; //tex is initialized to 1 and need this to force strongly ordered
#endif

that makes the difference.

Can you or anybody else please explain why? The BIOS API reference for ti.sysbios.family.arm.a8.Mmu is not really enlightening. Or should it be totally clear, that if ISRs do not work, the first thing one has to look for is MMU programming?

Thanks

Frank

Frank,

OK, great, thanks for the update.

I haven’t been able to determine why you needed to do this.  I sent an internal ping to find out why, and I'll reply back when I hear back…

Regards,
Scott

Frank,

I haven’t gotten a definitive answer, but from what I can tell, there were some MMU related changes recently, including the addition of the ‘tex’ attribute.  It seems the team putting together the SDK added this VERS check to selectively clear the new ‘tex’ attribute, to force strong ordering for the particular use case.  I think the earlier default was ‘normal’, and to avoid a compatibility break with previous software, they added this check and clear to select strongly ordered.  I haven’t been able to find any additional info on this, or why strong ordering is required.  But for your particular use case, it seems it really is required, and that is why this code got things working for you.  I’ve filed an enhancement request to better document the MMU settings: SDOCM00115573

Sorry I don’t have a more definitive answer for you why this is needed.

Regards,
Scott  

Scott,

thanks for your reply. Although my project runs now, I am still very curious what ISR behaviour has to do with MMU settings. It would be great if you could inform we what your team found out.

Another resulting question: Which strategy to use to handle such problems when changing to a new XDC/BIOS version, except to "never change a running system"?

Regards,

Frank

PS: Still waiting for an answer to my question #1, see

Frank,

Can you please clarify how you are initializing the MMU?  That SDK MMUInit function that you reference that forces strongly ordered: were you not using  it before you switched BIOS and XDCTools versions?  I thought you were using the SDK, and had just switched to later BIOS and XDCTools versions.  And if you *were* using the SDK’s MMU init previously, then you would have had the force of strong ordering already in place, and wouldn’t have seen a problem.

If you are not using the SDK’s MMU init, can you please post your initialization code?

In your ISRs, if you are accessing memory mapped peripheral registers, then it seems to make sense that strong ordering is necessary to ensure properly ordered and completed register writes.  Or, if you don’t have strong ordering enabled, that you’d need to use explicit memory barriers.

The only comment I have for strategy for changing versions is to change as little at a time as necessary, and verify things are OK before making additional changes.  You may have already done that, I’m not sure.

For your question #1 about GPMC timing, I have no specific idea, other than maybe the MMU init issue that we’ve been discussing.

I’ve sent additional pings internally to try to clarify why strong ordering is forced in that MMU init code.

Regards,
Scott

Scott,

my MMU initialisation is (and was) as suggested by the SDK examples, see e. g. C:\ti\am335x_sysbios_ind_sdk_1.1.0.6\sdk\examples\i2c_led or so. MMUInit() is one of the first functions called in main() before initialising the DMTIMER etc. See houston.zip in my posting from March 27th.

The change concerning the 'strongly ordered' occured somewhere between SDKs 1.1.0.1 and 1.1.0.4. My original code is from 1.0.0.4. Now I use the code from C:\ti\am335x_sysbios_ind_sdk_1.1.0.6\sdk\os_drivers\src\osdrv\mmu.c.

Yes, I have to access  memory mapped peripheral registers in my ISRs: I have to reset a flag to get a next ISR from the same source, see the code from my DMTimer ISR below.

If this access to the flag does not work correctly due to some MMU/Cache misconfiguration, this would explain that you don't get a second interrupt. Actually, I get 2 subsequent Interrupts but no 3rd one, isrcounter is incremented by 2 with each Start/Resume [F8] and Suspend [Alt F8] in CCS. Very strange. Maybe the debugger access plays a role too, I did not test if the program runs without CCS.

Meanwhile I have news concerning my problem #1: I again measured my GPMC timings and found them to be slow again - with my changed MMU initialisation. So maybe the two problems are both related to  MMU/Cache issues and you and your team find a solution for both questions...

Since there are other community members interested in this issue too, see details in the other thread:

Regards,

Frank

code snippets from my DMtimer programming:

/*

DMTimer7ModuleClkConfig() stolen from sdk/starterware/platform/evmAM335x/dmtimer.c

*/

void

hwiFxn_synctimer()

{

   /* clear the interrupt status from IRQSTATUS register */

   HWREG(synctimer_base + DMTIMER_IRQSTATUS) =

       DMTIMER_IRQSTATUS_OVF_IT_FLAG;

   /* store a timestamp */

   isrtimestamp[isrcounter & 0x07] = TimestampProvider_get32();

++isrcounter;

}

void

synctimer_cfg(unsigned period, unsigned pulsewidth)

{

   /* store arguments in local variables */

   synctimer_period = period;

   synctimer_pulsewidth = pulsewidth;

   /* compute the timer values according to period and pulse width */

   /* (The dmtimer counts up from negative values to zero.) */

   synctimer_reload =

       (unsigned)((long)-(TIMER_CLOCK_FREQUENCY / 1000000 * synctimer_period));

   synctimer_compare =

       (unsigned)((long)-(TIMER_CLOCK_FREQUENCY / 1000000 * synctimer_pulsewidth + 1));

   /* switch timer clock on */

   DMTimer7ModuleClkConfig();

   /* disable interrupts */

   HWREG(synctimer_base + DMTIMER_IRQENABLE_CLR) =

       DMTIMER_IRQENABLE_CLR_MAT_EN_FLAG | DMTIMER_IRQENABLE_CLR_OVF_EN_FLAG |

       DMTIMER_IRQENABLE_CLR_TCAR_EN_FLAG;

   /* stop the timer (clear all controls) */

   HWREG(synctimer_base + DMTIMER_TCLR) = 0;

   /* set reload register (period) */

   HWREG(synctimer_base + DMTIMER_TLDR) = synctimer_reload;

   /* set compare match register (pulse length) */

   HWREG(synctimer_base + DMTIMER_TMAR) = synctimer_compare;

   /* set counter register (ensure correct length of 1st pulse) */

   HWREG(synctimer_base + DMTIMER_TCRR) = synctimer_compare;

   /* enable the dmtimer overflow interrupt */

   HWREG(synctimer_base + DMTIMER_IRQENABLE_SET) = DMTIMER_IRQENABLE_SET_OVF_EN_FLAG;

   /* set timer control register (alphabetical order as in hw_dmtimer.h) */

   HWREG(synctimer_base + DMTIMER_TCLR) =

       /* auto reload, no capture, compare enable, PORGPOCFG = 0, no prescale */

       (DMTIMER_TCLR_AR_AUTO << DMTIMER_TCLR_AR_SHIFT) |

       (DMTIMER_TCLR_CAPT_MODE_SINGLE << DMTIMER_TCLR_CAPT_MODE_SHIFT) |

       (DMTIMER_TCLR_CE_ENABLE << DMTIMER_TCLR_CE_SHIFT) |

       (DMTIMER_TCLR_GPO_CFG_DRIVE0 << DMTIMER_TCLR_GPO_CFG_SHIFT) |

       (DMTIMER_TCLR_PRE_DISABLE << DMTIMER_TCLR_PRE_SHIFT) |

       /* toggle output, prescale 0, set pwm (positive pulse, 1st match is ignored) */

       (DMTIMER_TCLR_PT_TOGGLE << DMTIMER_TCLR_PT_SHIFT) |

       (0 << DMTIMER_TCLR_PTV_SHIFT) |

       (DMTIMER_TCLR_SCPWM_SET << DMTIMER_TCLR_SCPWM_SHIFT) |

       /* start timer, no capture, trigger output on overflow and match */

       (DMTIMER_TCLR_ST_START << DMTIMER_TCLR_ST_SHIFT) |

       (DMTIMER_TCLR_TCM_NOCAPTURE << DMTIMER_TCLR_TCM_SHIFT) |

       (DMTIMER_TCLR_TRG_OVERFLOWANDMATCH << DMTIMER_TCLR_TRG_SHIFT);

}

Frank,

Did you try changing SYS_MMU_ENTRY as Pratheesh suggested in the other thread?  Did this resolve problem #1?

Thanks,
Scott

Scott,

nice to have the topic in the TI-RTOS / SYS-BIOS forum again. Meanwhile, nearly all my problems concerning fast memory access and ISRs are more or less solved. I got fast CPU to CPLD writes, not so fast CPLD to CPU reads, and both fast EDMA reads and writes on AD muxed single access asynchronous GPMC controlled memory. And I have my ISRs working again, too. The key to all is to have the right MMU setting. To achieve this, I had to read a few pages in the ARM® Architecture Reference Manual ARMv7-A and ARMv7-R edition ARM DDI 0406C.c, and still not all is completely clear.

Details see my latest posting in the other thread:

Concerning SYS/BIOS:

The mmu init function in \am335x_sysbios_ind_sdk_1.1.0.6\sdk\os_drivers\src\osdrv_mmu.c is not able to set device type memory (TEX C B = 000 0 1) and should be updated in future releases. I had to use BIOS function Mmu_setFirstLevelDesc (see below) to set shareable device type memory. And of course you have to set this for the device (0x0100 0000), not for the GPMC configuration registers (0x5000 0000).

As stated before, the SYS/BIOS documentation on AM335x (A8) MMU settings is rather poor, some more sentences about memory types (stronly ordered, shareable device, normal, ...) and which type to use for external devices, memory mapped registers, DDR3 etc. would be very helpful (at least that the most common cases are covered).

Below the entries in my MMU table.

Regards,

Frank

Entries in ti_sysbios_family_arm_a8_Mmu_Module_State_0_tableBuf__A (@ 0x8001 8000 - 0x8001 C000)
(1 MB section descriptors: Phys.Addr.[31-20], NS, 0, nG, S(hareable), AP[2], TEX[2:0], AP[1:0],
impl. defd, Domain(bit 8 to 5), XN, C, B, 1, 0
e.g. 0x80001e0e = PA 0x8000 0000, NS = nG = S = AP[2] = 0, TEX[2:0] = 001, AP[1:0] = 11,
impl. defd = 1, Domain = XN = 0, C = 1, B = 1, 1 0 for section desc.

01000E06    (in my application) CPLD external device memory (shareable device for faster access)

40201E0E    OCRAM, set by SYS/BIOS initialisation (normal memory, cacheable / shareable)
40301E0E    OCRAM, set by SYS/BIOS initialisation    

44E00E02    memory mapped registers of peripherals (strongly ordered for correct interrupts)
...
49800E02    memory mapped registers EDMA3TC0
...
50000E02    memory mapped registers GPMC

80001E0E    DDR3, set by SYS/BIOS initialisation (normal memory, cacheable / shareable)
...

Hi Scott,

looking forward to read your work in the docs of the next SYS/BIOS release...

Thanks,

Frank

PS: Still a little mystique: Why I get 2 ISRs after start/stop with the debugger if the sections of memory mapped registers are _not_ set to be strongly ordered?

Hi Frank,

I can’t explain the 2 ISRs per start/stop.  But if timer register accesses happen out of the intended order it seems there can be all sorts of odd timings generated.

Regards,
Scott