AM3359: GPMC wait for EDMA blocks CPU

fmdhr

Part Number: AM3359
Other Parts Discussed in Thread: SYSBIOS, SYSCONFIG

Hi,

I have a problem with the following SYS/BIOS application:

The EDMA of my AM3359 is programmed to feed a FIFO in an external CPLD via the GPMC interface. The FIFO is read out using an external clock. To prevent a FIFO overflow, the CPLD is programmed to set the GPMC /WAIT signal if the FIFO is almost full. A GPIO "wait_enable" from the ARM to the CPLD allows to do so. This slows down / synchronises the EDMA with the external clock, while the CPU can do other things (of course must not use the GPMC / this EDMA channel meanwhile).

The SYS/BIOS clock function triggers a timeout, if the FIFO is not emptied and the EDMA transfer completion interrupt does not come within time, e.g. because the external clock signal is missing. The timeout resets the "wait_enable" signal to the CPLD, so that it releases the GPMC /WAIT, and finally the EDMA transfer runs to completion (now with FIFO overflow), and further error handling is done etc.

This all works fine ...

...until I add a third party USB driver software package. Now it happens, that the CPU (internal bus?) is blocked totally. No clock / timeout interrupt anymore, no chance to get the CPU stopped using the debugger if there is no external clock and the GPMC /WAIT is active:

CortxA8: Trouble Halting Target CPU: (Error -2062 @ 0x3E418)
Unable to halt device. Reset the device, and retry the operation.
If error persists, confirm configuration, power-cycle the board,
and/or try more reliable JTAG settings (e.g. lower TCLK). (
Emulation package 6.0.407.3)

If the external clock is switched on again, the CPU is halted at an access to a USB register at address 0x47401460 (a 10 ms polling triggered by a task), details see below.

The USB driver (some more than a few lines of code) neither uses GPMC nor EDMA, far as I can see.

How this can be explained? (A GPMC /WAIT to EDMA should not block the CPU access to an internal register!)

Are there any settings in the GPMC / EDMA / MMU (or whatever) configuration that do not match or may be in contradiction? Where to search first?

Thanks in advance,

Regards

Frank

Disassembly of halted CPU:

R12 = 0x47401460

          UsbdDc_GetVbusState(), Usbd_IsVbusPresent():
801bd1d8:   E59FC5F4            ldr        r12, [pc, #0x5f4] // R12 = 0x47401460
801bd1dc:   E5DCC000            ldrb       r12, [r12]
1611             vbus = T_TRUE;
801bd1e0:   E20CC018            and        r12, r12, #0x18   // debugger halts here
801bd1e4:   E35C0018            cmp        r12, #0x18
801bd1e8:   E3A00000            mov        r0, #0
801bd1ec:   03A00001            moveq      r0, #1
801bd1f0:   E12FFF1E            bx         lr

C Code:

USBD_DC_LINKAGE
T_BOOL
UsbdDc_GetVbusState(
   T_UsbdDeviceCore* devc
)
{
   T_BOOL vbus = T_FALSE;
   T_UINT8 reg8;

   TB_UNUSED_PARAM(devc);
#if (TRACE_ENABLE != 0)
   TRACEPrint(TRACE_LVL_USBD_EP, "%s [\n", __FUNCTION__);
#endif

   /* current VBUS level is indicated in DEVCTL register */
   reg8 = READ_REG8(TIMNT_USBOTG_REG8_DEVCTL);
   if ( TIMNT_DEVCTL_VBUS_ABOVEVALID == (reg8 & TIMNT_DEVCTL_VBUS_ABOVEVALID) ) {
       /* VBUS is at valid level */
       vbus = T_TRUE;
   }

#if (TRACE_ENABLE != 0)
   TRACEPrint(TRACE_LVL_USBD_EP, "%s ]\n", __FUNCTION__);
#endif
return vbus;

} /* UsbdDc_GetVbusState */

over 5 years ago

0 Frank Livingston over 5 years ago

TI__Mastermind 29388 points

Hi,

Is the a custom board, or a TI board?

Is your SYS/BIOS application based on any software provided in the PRSDK? If so, what version of PRSDK are you using?

In general I can't provide support on integration of 3rd party drivers. Does the 3rd party USB driver work in a stand-alone application without GPMC?

It looks like the hang occurs when the driver attempts to verify the USB VBUS voltage is valid. I think incorrect MMU settings would result in an ARM exception, not a hang on a USB core register read.

I've looped in a couple of colleagues who might have some more insights.

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

thanks for your reply. To answer your questions:

Frank Livingston said:
Is the a custom board, or a TI board?

It's our own board, runs well in other applications, no hardware problems.

Frank Livingston said:
Is your SYS/BIOS application based on any software provided in the PRSDK? If so, what version of PRSDK are you using?

We use am335x_sysbios_ind_sdk_01.01.01.01, SYS/BIOS 6.41.4.54, XDC 3.32.2.25. (Yes, not the latest ones, but No, we will not adapt all of our code to try a newer one.)

Frank Livingston said:
In general I can't provide support on integration of 3rd party drivers.

Clear. But we need a hint where to search the error.

Frank Livingston said:
Does the 3rd party USB driver work in a stand-alone application without GPMC?

Yes, it works good in another application with the same hardware and a similar GPMC / CPLD combination without this special /WAIT signalling.

Frank Livingston said:
It looks like the hang occurs when the driver attempts to verify the USB VBUS voltage is valid. I think incorrect MMU settings would result in an ARM exception, not a hang on a USB core register read.

I tried to read several memory mapped registers _before_ the /WAIT is released in my clock function (USB disabled):

void
pig_clockFxn()
{
    UInt    hwikey;

        if ( ++pig_tcount >= pig_timeout ) {
            //vusb = HWREG(0x47401460); // exception, USB not initialised / clocked
            vusb = HWREG(0x48000000); // reads OK, no lock
...
            /* disable wait to bring edma to completion */
            pig_disable_wait();
        }

}

I assume the problem can be located near the L3 interconnection system, and I also tried the following:

    /*
    give EDMATC0 a higher priority than mmu, pru_icss, etc. to ensure transfer
    see TRM 9.2.4.3.1 Initiator Priority Control for Interconnect
    0 = low, 1 = medium, 3 = high priority, initally all priorities are = 0
    OR
    give ARM higher priority so that cpu can override /WAIT on EDMA (?)
    */
#if EDMA_HIGH_PRIORITY
    HWREG(SOC_CONTROL_REGS + CONTROL_INIT_PRIORITY(0)) |=
      ((1 << CONTROL_INIT_PRIORITY_0_TCRD0_SHIFT) & CONTROL_INIT_PRIORITY_0_TCRD0) |
      ((1 << CONTROL_INIT_PRIORITY_0_TCWR0_SHIFT) & CONTROL_INIT_PRIORITY_0_TCWR0) |
      ((0 << CONTROL_INIT_PRIORITY_0_HOST_ARM_SHIFT) & CONTROL_INIT_PRIORITY_0_HOST_ARM);
#else
    HWREG(SOC_CONTROL_REGS + CONTROL_INIT_PRIORITY(0)) |=
      ((0 << CONTROL_INIT_PRIORITY_0_TCRD0_SHIFT) & CONTROL_INIT_PRIORITY_0_TCRD0) |
      ((0 << CONTROL_INIT_PRIORITY_0_TCWR0_SHIFT) & CONTROL_INIT_PRIORITY_0_TCWR0) |
      ((1 << CONTROL_INIT_PRIORITY_0_HOST_ARM_SHIFT) & CONTROL_INIT_PRIORITY_0_HOST_ARM);
#endif

In both cases I get the same behaviour (lock if USB is on, OK otherwise).

Unfortunately, the Chapter 10 in the TRM (SPRUH73P, p 1559 ff) is not very detailed.

Frank Livingston said:
I've looped in a couple of colleagues who might have some more insights.

Hope they have some good ideas, we still need your help.

Thanks & regards,

Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank et.al.,

no new ideas? It would be great if you can help me to find a solution.

I don't know if it matters, but we use some special MMU settings for the GPMC/CPLD, details see below.

Regards,

Frank

    Mmu_FirstLevelDescAttrs attrs;

    /*
    assumes TRE (TEX remap enabled) = 0 and AFE (access flag enable) = 0
    in the CP15_CONTROL_REGISTER (SCTLR)
    TEX[2:0] and C, B determine memory type:
    shareable device memory is 000 0 1
    */
    /* set default attributes */
    Mmu_initDescAttrs(&attrs);
    /* change attributes for cpld */
    attrs.type = Mmu_FirstLevelDesc_SECTION;
    attrs.cacheable = 0;
    attrs.bufferable = 1;
    attrs.tex = 0;
    attrs.domain = 0;
    attrs.imp = 1;      // implementation specific
    attrs.accPerm = 3; // access permission: read/write
    attrs.shareable = 0;

    /* program mmu for cpld access (MMUIinit cannot set this combination) */
    Mmu_disable();
    Mmu_setFirstLevelDesc((Ptr)&cpld_register, (Ptr)&cpld_register, &attrs);
    Mmu_enable();

SYS_MMU_ENTRY mmu_cfgtab[] =
{
    /* {virtual_address == phys_address, flags: bufferable, cacheable}, */
    /* GPMC controlled external memory CS0 */
    {(void*)0x01000000, 0},
    /* memory mapped registers - strongly ordered */
    {(void*)0x44E00000, 0}, // Clock Module, PRM, GPIO0, UART0, I2C0, - Non bufferable| Non Cacheable
    {(void*)0x47400000, 0}, // USB0 - Non bufferable | Non Cacheable
    {(void*)0x48000000, 0}, // UART1,UART2,I2C1,McSPI0,McASP0 CFG,McASP1 CFG,DMTIMER,GPIO1 -Non bufferable| Non Cacheable
    {(void*)0x48100000, 0}, // I2C2,McSPI1,UART3,UART4,UART5, GPIO2,GPIO3,MMC1 - Non bufferable| Non Cacheable
    {(void*)0x48200000, 0}, // INTCPS,MPUSS - Non bufferable| Non Cacheable
    {(void*)0x48300000, 0}, // PWM - Non bufferable| Non Cacheable
    {(void*)0x49000000, 0}, // EDMA3CC - Non bufferable | Non Cacheable
    {(void*)0x49800000, 0}, // EDMA3TC0 - Non bufferable | Non Cacheable
    {(void*)0x49900000, 0}, // EDMA3TC1 - Non bufferable | Non Cacheable
    {(void*)0x49a00000, 0}, // EDMA3TC2 - Non bufferable | Non Cacheable
    {(void*)0x4A000000, 0}, // L4 FAST CFG- Non bufferable| Non Cacheable
    {(void*)0x4A100000, 0}, // CPSW - Non bufferable | Non Cacheable
    {(void*)0x4A300000, 0}, // PRUSS1 - Non bufferable | Non Cacheable
    {(void*)0x50000000, 0}, // GPMC - Non bufferable | Non Cacheable
    // DDR3 memory mapping is initialised by sys/bios (as defined in platform)
    // {(void*)0x80000000, SYS_MMU_CACHEABLE | SYS_MMU_BUFFERABLE}, // DDR3 - bufferable | cacheable
    {(void*)0xFFFFFFFF, 0}   // 0xFFFFFFFF marks the end of the table
};

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi,

I don't yet have any update. I'm reaching out internally to a wider audience for help on this issue.

Regards,
Frank

0 Frank Livingston over 5 years ago in reply to Frank Livingston

TI__Mastermind 29388 points

Hi,

>> am335x_sysbios_ind_sdk_01.01.01.01

TI no longer supports the SYSBIOS Industrial SDK. Please see the following note on this topic: e2e.ti.com/.../580869.

>> Clear. But we need a hint where to search the error.

Understood, I'll do my best to help.

I don't completely understand what you're trying to do, so I have a few questions:

The GPMC writes to the CPLD FIFO?
The FIFO is read by the CPLD? The read timing is determined by the external clock?
What clock triggers the EDMA transfers to the GPMC?
Re-enabling the external clock forces the CPLD FIFO read, which forces the CPLD to de-assert the GPMC_WAIT signal, and the GPMC/EDMA resumes and finishes the current EDMA transfer?
The external clock is not connected to AM3359?
- Before resuming the external clock, you're unable to connect to the AM3359 over JTAG?
- After resuming the external clock, you're able to connect to the AM3359 over JTAG?
The SYS/BIOS clock interrupt & function stops executing, but is the associated hardware timer still running?

Here is some of the internal feedback I got:

Bus hangs can happen if the GPMC clock is not enabled. The hang happens between seemingly unrelated modules. In this scenario, the problem is caused by the error reporting mechanism.
For the L3 bus, only the slaves hold error reporting resources.
- Whenever a master does something wrong, the slave which is its endpoint will report and log the error.
- If a master does something bad to a null address which no slave controls, the bus designates an error collector which can log the masters error.
- In the case of the AM335x the GPMC will log errors for accesses which the USB issues into address hole areas.
- If the GPMC doesn't have a clock to process the error, the bus locks up until a clock comes.
One near sure-fix would be to clean up the USB driver which is likely generating some number of interconnect errors on a constant basis.

In case it's difficult to fix the 3rd party USB driver, the problem might be resolved by ensuring a clock stop of the interface never happens which logs the errors. It depends whether the functional clock is coupled to the error logic.Try forcing:

CM_PER_GPMC_CLKCTRL : write a 0x3
CM_PER_L3S_CLKSTCTRL : write a 2, then transition to a final 0
GPMC_SYSCONFIG : make sure Auto-idle is off and sidle=noidle

There are some interconnect error reporting register which aren't documented in the TRM which we can inspect if this doesn't work.

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

thanks to you and to your colleagues for the reply.

Frank Livingston said:
The GPMC writes to the CPLD FIFO?

Yes. The CPLD appears to the ARM as some registers / memory cells addressed via the GPMC (plus some GPIOs, e.g. the wait_enable pin).

Frank Livingston said:
The FIFO is read by the CPLD? The read timing is determined by the external clock?

The FIFO in the CPLD is read out (emptied) to an external DAC, clocked by the external clock.

Frank Livingston said:
What clock triggers the EDMA transfers to the GPMC?

A chain of 3 EDMA transfers is initiated by the ARM CPU: first: write some control registers, second: fill the FIFO, third: read back CPLD state register. The second transfer is slowed down indirectly by the external clock via the GPMC /WAIT signal generation. All fine without USB.

Frank Livingston said:
Re-enabling the external clock forces the CPLD FIFO read, which forces the CPLD to de-assert the GPMC_WAIT signal, and the GPMC/EDMA resumes and finishes the current EDMA transfer?

Yes. Additionally, EDMA/GPMC continues if I deassert the wait_enable signal to the CPLD, which disables the GPMC /WAIT generation of the CPLE. The CPLD logic (with fcX = FIFO fill counter) is:

gpmc_wait = !(!gpmc_cs & fifo_fc2 & fifo_fc1 & wait_enable);

But for this I need the clock interrupt of the CPU to write the GPIO, as it works without USB.

Frank Livingston said:
The external clock is not connected to AM3359?

No.

Frank Livingston said:
Before resuming the external clock, you're unable to connect to the AM3359 over JTAG?

I do all the tests using the JTAG: programm is loaded and started via the XDS560v2 debugger. If the external clock goes off while a FIFO transfer is running, I cannot stop the CPU:
CortxA8: Trouble Halting Target CPU: (Error -2062 @ 0x3E418) Unable to halt device. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 6.0.407.3)

Frank Livingston said:
After resuming the external clock, you're able to connect to the AM3359 over JTAG?

The CPU now stops, as reported:

   reg8 = READ_REG8(TIMNT_USBOTG_REG8_DEVCTL);
   if ( TIMNT_DEVCTL_VBUS_ABOVEVALID == (reg8 & TIMNT_DEVCTL_VBUS_ABOVEVALID) ) {
       /* VBUS is at valid level */
       vbus = T_TRUE;
   }

Frank Livingston said:
The SYS/BIOS clock interrupt & function stops executing, but is the associated hardware timer still running?

Two problems: Which timer is choosen by the BIOS (timer ID = ANY in BIOS configuration), and how to read it if I cannot access the ARM during the lock? I assume it is running.

Frank Livingston said:

Here is some of the internal feedback I got:

Bus hangs can happen if the GPMC clock is not enabled. The hang happens between seemingly unrelated modules. In this scenario, the problem is caused by the error reporting mechanism.

For the L3 bus, only the slaves hold error reporting resources.

Whenever a master does something wrong, the slave which is its endpoint will report and log the error.

If a master does something bad to a null address which no slave controls, the bus designates an error collector which can log the masters error.

In the case of the AM335x the GPMC will log errors for accesses which the USB issues into address hole areas.

If the GPMC doesn't have a clock to process the error, the bus locks up until a clock comes.

One near sure-fix would be to clean up the USB driver which is likely generating some number of interconnect errors on a constant basis.

In case it's difficult to fix the 3rd party USB driver, the problem might be resolved by ensuring a clock stop of the interface never happens which logs the errors. It depends whether the functional clock is coupled to the error logic.

Sorry, I did not completely understand all this. If I prevent the access to 0x47401460 reported above,
the USB driver hangs with another access into USB0_PHY_RX_DEBUG_REG.

Frank Livingston said:
Try forcing:

CM_PER_GPMC_CLKCTRL : write a 0x3

CM_PER_L3S_CLKSTCTRL : write a 2, then transition to a final 0

GPMC_SYSCONFIG : make sure Auto-idle is off and sidle=noidle

I tried these different initialisations, details see below. Behaviour did not change (still lockdown on missing clock, ARM continues with external clock resume).

I also asked our supplier of the USB stack: Unfortunately, all the programmers that know the AM3359 meanwhile have left the company.
Hope I get some more help here before all the AM335x experts have left TI.

Thanks and regards,
Frank

Remarks:
The code provided with the SDK does a repeated initialisation (not only) of the L3 registers (with each module again).
Writing 3 to CM_PER_GPMC_CLKCTRL (by code and with CCS) always reads back 2.

    HWREG(SOC_PRCM_REGS + CM_PER_GPMC_CLKCTRL) |=
                             CM_PER_GPMC_CLKCTRL_MODULEMODE_RESERVED; // = 3
                             // CM_PER_GPMC_CLKCTRL_MODULEMODE_ENABLE;
    /* Writing to CLKTRCTRL field of CM_PER_L3S_CLKSTCTRL register. */
    HWREG(SOC_CM_PER_REGS + CM_PER_L3S_CLKSTCTRL) |=
          CM_PER_L3S_CLKSTCTRL_CLKTRCTRL_SW_WKUP;
    /*Waiting for CLKTRCTRL field to reflect the written value. */
    while(CM_PER_L3S_CLKSTCTRL_CLKTRCTRL_SW_WKUP !=
          (HWREG(SOC_CM_PER_REGS + CM_PER_L3S_CLKSTCTRL) &
           CM_PER_L3S_CLKSTCTRL_CLKTRCTRL));
    //0x0 = NO_SLEEP : NO_SLEEP: Sleep transition cannot be initiated.
    //Wakeup transition may however occur.
    HWREG(SOC_CM_PER_REGS + CM_PER_L3S_CLKSTCTRL) = 0;

    HWREG(baseAddr + GPMC_SYSCONFIG) =
        ((GPMC_SYSCONFIG_AUTOIDLE_FREERUN << GPMC_SYSCONFIG_AUTOIDLE_SHIFT) &
            GPMC_SYSCONFIG_AUTOIDLE) |
        ((GPMC_SYSCONFIG_IDLEMODE_NOIDLE << GPMC_SYSCONFIG_IDLEMODE_SHIFT) &
            GPMC_SYSCONFIG_IDLEMODE);

registers with old initialisation:

0x44E00000:
CM_PER_CM_PER_L4LS_CLKSTCTRL
02004102
CM_PER_CM_PER_L3S_CLKSTCTRL
0000000A
CM_PER_CM_PER_L4FW_CLKSTCTRL
00000102
CM_PER_CM_PER_L3_CLKSTCTRL
0000005E   00070000
CM_PER_CM_PER_CPGMAC0_CLKCTRL
00000002
CM_PER_CM_PER_LCDC_CLKCTRL
00070000
CM_PER_CM_PER_USB0_CLKCTRL
00000002   00070000
CM_PER_CM_PER_TPTC0_CLKCTRL
00000002
CM_PER_CM_PER_EMIF_CLKCTRL
00000002
CM_PER_CM_PER_OCMCRAM_CLKCTRL
00000002
CM_PER_CM_PER_GPMC_CLKCTRL
00000002
CM_PER_CM_PER_MCASP0_CLKCTRL
00030000
CM_PER_CM_PER_UART5_CLKCTRL
00030000

0x50000000:
GPMC_GPMC_REVISION
00000060   00000000   00000000   00000000
GPMC_GPMC_SYSCONFIG
00000000
GPMC_GPMC_SYSSTATUS
00000001
GPMC_GPMC_IRQSTATUS
00000000
GPMC_GPMC_IRQENABLE
00000000   00000000   00000000   00000000   00000000
00000000   00000000   00000000   00000000
GPMC_GPMC_TIMEOUT_CONTROL
00001FF0
GPMC_GPMC_ERR_ADDRESS
00000000
GPMC_GPMC_ERR_TYPE
00000000   00000000
GPMC_GPMC_CONFIG
00000A00
GPMC_GPMC_STATUS
00000101   00000000   00000000
GPMC_GPMC_CONFIG1_0
00201200
GPMC_GPMC_CONFIG2_0
00060600
GPMC_GPMC_CONFIG3_0
22020210
GPMC_GPMC_CONFIG4_0

after new initialisation as suggested:

CM_PER_CM_PER_L3S_CLKSTCTRL
00000008
CM_PER_CM_PER_GPMC_CLKCTRL
00000002

GPMC_GPMC_SYSCONFIG
00000008

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank et. al.,

I additionally tried the following:

I let the USB driver run its normal initialisations, and I only hacked the USB task so that it normally gets the events, but does not react to it:

    while ( 1 ) {
       // wait for USBD event with 10ms Timeout
       ev = Event_pend(USBDTaskEvent, Event_Id_NONE, (USBD_EVENT_ISR | USBD_EVENT_TX_FREE | USBD_EVENT_TX_NEW | USBD_EVENT_OTHER), 10);
#if HACK_GPMC_WAIT_BLOCK || 1
        continue;
#endif

WIth this modification, the timeout works as expected (no CPU lock).

Then I modified my timeout clock function (btw., pig stands for PIxel Generator in the zoo of our applications):

volatile   uint32_t   reg;

void
pig_clockFxn()
{
    UInt    hwikey;

    /* timeout control */
    if ( pig_state == PIG_STATE_PRINT ) {
        hwikey = Hwi_disable();
        if ( ++pig_tcount >= pig_timeout ) {
            // reg = HWREG(0x47401000); // USB0, locks CPU
            reg = HWREG(0x48000000); // UART1...
            reg = HWREG(0x48200000); // INTCPS
            reg = HWREG(0x4A100000); // CPSW
            pig_state = PIG_STATE_ERROR;
            pig_error = PIG_ERROR_TIMEOUT;
            /* disable wait to bring edma to completion */
            pig_disable_wait();
            reg = HWREG(0x47401000); // OK, reads 0x4EA20800
            /* post of PRINTERR event is done in edma isr */
        }
        ++pig_time_done;
        Hwi_restore(hwikey);
    }

}

Only an access to the USB registers blocks the CPU while GPMC /WAIT is active, accesses to the registers of other peripherals do not block.

Why it is only the USB that hangs? What is the difference?

Hope this is a new question that may lead to an answer.

Thanks & regards,

Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank et. al.,

yet one more test result: Problem seems not to be limited to USB, CPU is locked also on access to GPMC registers. I think I should be able to read a GPMC configuration register while it is in use (and in /WAIT). Details below.

Any idea?

Regards,

Frank

void
pig_clockFxn()
{
    UInt    hwikey;

    /* timeout control */
    if ( pig_state == PIG_STATE_PRINT ) {
        hwikey = Hwi_disable();
        if ( ++pig_tcount >= pig_timeout ) {
            // reg = HWREG(0x47401000); // USB0, locks CPU
            reg = HWREG(0x48000000); // UART1...
            reg = HWREG(0x48200000); // INTCPS
            reg = HWREG(0x4A100000); // CPSW
            reg = HWREG(0x44E10000); // control module
            reg = HWREG(0x44E0D000); // ADC_TSC
            // reg = HWREG(0x50000000); // GPMC, locks
            pig_state = PIG_STATE_ERROR;
            pig_error = PIG_ERROR_TIMEOUT;
            /* disable wait to bring edma to completion */
            pig_disable_wait();
            //reg = HWREG(0x47401000); // OK, reads 0x4EA20800
            reg = HWREG(0x50000000); // OK
            /* post of PRINTERR event is done in edma isr */
        }
        ++pig_time_done;
        Hwi_restore(hwikey);
    }

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi again,

no new ideas?

Regards,

Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Seems that all AM335x experts have left TI.

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

Sorry for the delayed response, and thanks for your patience.

I don't have immediate additional feedback.

Can you please confirm my understanding of the results of your experiments? Please see my outline below.

I'll share this internally to see if your experiments help us find an answer.

Regards,
Frank

Modified the USB driver SYS/BIOS Task which responds to USB hardware-initiated SYS/BIOS Events. The USB-initiated Events still occur, and Event_pend() in the Task is still triggered by the Events. However, Task code which processes the Events has been removed.
Modified SYS/BIOS clock function to read the hardware Revision ID of various SoC IPs. The clock function checks whether a timeout counter is above a timeout threshold, and de-asserts the "wait enable" signal to the CPLD if the time out counter is above the threshold.
You observe a CPU lock occurs if either the USB Hardware Revision ID or GPMC Hardware Revision ID registers are read in the ISR.

0 Frank Livingston over 5 years ago in reply to Frank Livingston

TI__Mastermind 29388 points

Hi,

I'm trying to digest the register dump you shared. Please see the attached file with a summary. Can you please confirm my understanding?

Regards,
Frank

Fullscreen 1738.reg_compare.txt Download

// Compare regs before-and-after
//
CM_PER_GPMC_CLKCTRL:
    // -> 0x3
    // Writing 3 (RESERVED) to CM_PER_GPMC_CLKCTRL (by code and with CCS) always reads back 2.
    // OK?
    Before:                         
        CM_PER_GPMC_CLKCTRL         = 00000002
            Control the way mandatory clocks are managed.
            0x2 = ENABLE : Module is explicitly enabled. Interface clock (if not
            used for functions) may be gated according to the clock domain
            state. Functional clocks are guarantied to stay present. As long as in
            this configuration, power domain sleep transition cannot happen.

    After:
        CM_PER_GPMC_CLKCTRL         = 00000002


CM_PER_L3S_CLKSTCTRL:
    // write a -> 2, then transition to a final -> 0
    //  Code writes 0x2, waits for 0x2, then writes 0x0.
    //  OK.
    //
    Before:                         
        CM_PER_L3S_CLKSTCTRL        = 0000000A
            [3] CLKACTIVITY_L3S_GCLK = 1b
                This field indicates the state of the L3S_GCLK clock in the domain.
                0x1 = Act

          [1-0] CLKTRCTRL = 10b
                0x2 = SW_WKUP : SW_WKUP: Start a software forced wake-up
                transition on the domain.

    After:
        CM_PER_L3S_CLKSTCTRL        = 00000008
            [3] CLKACTIVITY_L3S_GCLK = 1b
                This field indicates the state of the L3S_GCLK clock in the domain.
                0x1 = Act
                
          [1-0] CLKTRCTRL = 00b
                0x0 = NO_SLEEP : NO_SLEEP: Sleep transition cannot be initiated.
                Wakeup transition may however occur.


GPMC_GPMC_SYSCONFIG:
    // -> make sure Auto-idle is off and sidle=noidle
    // (0<<0)&GPMC_SYSCONFIG_AUTOIDLE | (1<<3)&GPMC_SYSCONFIG_IDLEMODE
    //  GPMC_SYSCONFIG_AUTOIDLE?? assume this is a bit-field mask, 0x1<<0
    //  GPMC_SYSCONFIG_IDLEMODE?? assume this is a bit-field mask, 0x3<<3
    //
    Before:
    GPMC_SYSCONFIG                  = 00000000
        [4-3] SIDLEMODE = 00b
            Idle mode
            0h = Force-idle. An idle request is acknowledged unconditionally
        [0] AUTOIDLE = 0b
            Internal OCP clock gating strategy.
            0h = Interface clock is free-running
    
    After:
    GPMC_SYSCONFIG                  = 00000008
        [4-3] SIDLEMODE = 01b
            Idle mode
            1h = No-idle. An idle request is never acknowledged
        [0] AUTOIDLE = 0b
            Internal OCP clock gating strategy.
            0h = Interface clock is free-running

0 Frank Livingston over 5 years ago in reply to Frank Livingston

TI__Mastermind 29388 points

Hi,

Would it be possible for you to dump 0x100 bytes starting at the following addresses?

0x44001000
0x44001100
0x44806000
0x44807000

I assume it isn't possible for you to reproduce this issue on a TI EVM, but please correct me if I'm mistaken.

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

I was just trying not to get lost...

I will send you the information required in subsequent posts, below the first part.

Regards,

Frank

Frank Livingston said:
Modified the USB driver SYS/BIOS Task which responds to USB hardware-initiated SYS/BIOS Events. The USB-initiated Events still occur, and Event_pend() in the Task is still triggered by the Events. However, Task code which processes the Events has been removed.

Correct, with the "continue", the code that processes the events is never executed. The task structure is:

static void USBDtask(UArg arg0, UArg arg1)
{

/* initialisations */

AM3359_Usb0HwInit();

/* create USB device instance */

    /* main loop */

   while ( 1 ) {
       // wait for USBD event with 10ms Timeout
       ev = Event_pend(USBDTaskEvent, Event_Id_NONE, (USBD_EVENT_ISR | USBD_EVENT_TX_FREE | USBD_EVENT_TX_NEW | USBD_EVENT_OTHER), 20);
#if HACK_GPMC_WAIT_BLOCK || 1
        continue;
#endif

/* handle events, poll, ... */

} // main loop

}

May be, that due to the "not handling" of the events, they do not occur repeatedly. But there is (was) always the USBDpoll() that caused the lock.

Frank Livingston said:
Modified SYS/BIOS clock function to read the hardware Revision ID of various SoC IPs. The clock function checks whether a timeout counter is above a timeout threshold, and de-asserts the "wait enable" signal to the CPLD if the time out counter is above the threshold.

Frank Livingston said:
You observe a CPU lock occurs if either the USB Hardware Revision ID or GPMC Hardware Revision ID registers are read in the ISR.

Also right. My idea was to find out, if only the access to USB registers blocks the CPU. The results show, that access to USB and GPMC block the CPU, while other do not block.

volatile   uint32_t   reg;

void
pig_clockFxn()
{
    UInt    hwikey;

    /* timeout control */
    if ( pig_state == PIG_STATE_PRINT ) {
        hwikey = Hwi_disable();
        if ( ++pig_tcount >= pig_timeout ) {
            // reg = HWREG(0x47401000); // USB0, locks CPU
            reg = HWREG(0x48000000); // UART1...
            reg = HWREG(0x48200000); // INTCPS
            reg = HWREG(0x4A100000); // CPSW
            reg = HWREG(0x44E10000); // control module
            reg = HWREG(0x44E0D000); // ADC_TSC
            // reg = HWREG(0x50000000); // GPMC, locks
            pig_state = PIG_STATE_ERROR;
            pig_error = PIG_ERROR_TIMEOUT;
            /* disable wait to bring edma to completion */
            pig_disable_wait();
            //reg = HWREG(0x47401000); // OK, reads 0x4EA20800
            reg = HWREG(0x50000000); // OK
            /* post of PRINTERR event is done in edma isr */
        }
        ++pig_time_done;
        Hwi_restore(hwikey);
    }

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi,

here the dumps.

I first had to add the L3 registers to my MMU configuration to be able to read them. But even with this, I cannot read anything from 0x44806000 or 0x44807000 (typo?), included 0x44800600, 0x44800700 instead. In the files complete dumps from 0x4400000, 0x44800000, 0x1200 bytes each.

Hope you can find the information needed.

Frank Livingston said:
I assume it isn't possible for you to reproduce this issue on a TI EVM, but please correct me if I'm mistaken.

We have not used our EVM / ICE for many years, I don't know if they are still good. It might be possible to reproduce the problem on an EVM if a hardware connection from a GPIO (my "wait enable") to the GPMC /WAIT pin can be made, with EDMA read / write from / to nowhere, no CPLD etc. necessary.

For me, I do not see an advantage to change my hardware. If you want to set up an EVM project, I can send you parts of my code.

Regards,

Frank

(to be continued...)

SYS_MMU_ENTRY mmu_cfgtab[] =
{
    /* {virtual_address == phys_address, flags: bufferable, cacheable}, */
    /* GPMC controlled external memory CS0 */
    {(void*)0x01000000, 0},
    /* memory mapped registers - strongly ordered */
    {(void*)0x44000000, 0}, // L3 fast configuratio //n registers
    {(void*)0x44800000, 0}, // L3 slow configuration registers
    {(void*)0x44E00000, 0}, // Clock Module, PRM, GPIO0, UART0, I2C0, - Non bufferable| Non Cacheable
// ...

}

0x44001000

01 00 37 00 F1 79 34 00 FF 3F 00 00 00 00 00 00 FF 3F 00 00 01 00 00 00 00 00 00 00 00 00 00 00
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

0x44001100

01 00 37 00 6B 8A 19 00 03 00 00 00 02 00 00 00 03 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

0x44001000

00370001 003479F1 00003FFF 00000000 00003FFF 00000001 00000000 00000000
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????

0x44001100

00370001 00198A6B 00000003 00000002 00000003 00000001 00000000 00000000
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????

0x44800600

01 00 37 00 C6 39 7D 00 FF 1F 00 00 80 00 00 00 FF 1F 00 00 00 00 00 00 00 00 00 00 00 00 00 00
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

0x44800700

01 00 13 00 D5 DE 13 00 05 00 00 00 00 00 00 00 1E 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
02 00 00 00 02 00 00 00 01 00 08 00 02 08 00 00 01 00 00 00 1E 00 00 00 84 00 00 00 04 2F 40 30
00 00 00 00 17 00 00 00 20 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x44800600

00370001 007D39C6 00001FFF 00000080 00001FFF 00000000 00000000 00000000
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????

0x44800700

00130001 0013DED5 00000005 00000000 0000001E 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000002 00000002 00080001 00000802 00000001 0000001E 00000084 30402F04
00000000 00000017 00000020 00000002 00000000 00000000 00000000 00000000
0000001F 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

dump8_0x44000000.dat

dump8_0x44800000.dat

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

finally to the CM_PER_* and GPMC_* registers: Found that the two files are equal, and yes, it's all as you described in the file(s). Initalisation code and results also in my post from May, 25th.

Btw., the ((UNSHIFTED_BITFIELD_VALUE << BITFIELD_SHIFT) & BITFIELD_MASK) | ... is a syntax used throughout the include/hw/hw_<module>.h files in the TI SDK. A means to write a 32 bit value with (almost) as much code as possible, but with good intentions behind.

Regards,

Frank

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

Thanks for confirming my understanding and for sharing the dumps. We're inspecting the dumps now. I'll keep you posted on any findings.

Can you please describe the situation in which the dumps were taken? Which particular version of code, when, etc.

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

dumps were taken with all initialisations done, USB task doing nothing, code running somewhere in idle loop / main task.

Thought I get a message with first results of the dump inspections.

Hope my guesses with 0x0[67]00 instead of 0x[67]000 were right. Please let me know if you need more information.

Regards,

Frank

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

fmdhr said:
Hope my guesses with 0x0[67]00 instead of 0x[67]000 were right.

You were correct about this.

fmdhr said:
Thought I get a message with first results of the dump inspections.

The dumps show an L3S error and failed transaction being sent to GPMC. The GPMC interconnect error logging registers (GPMC_TARG) appear to show the initiator ID for SGX530. Are you using SGX530 in your application? This could be a 'stale' hold over error from the processor boot, or possibly we're misinterpreting the register fields containing the initiator ID.

The error registers are only one deep, so logged error information needs to be cleared before subsequent interconnect errors can be logged.

Can you please try the following?

Clear GPMC_TARG errors at start of the application.
Take register dumps (same dumps we requested before) before the USB is started. Clear GPMC_TARG errors after taking dumps.
Start USB. The application is still configured to not process USB events since otherwise a lockup occurs.
Take register dumps (same dumps we requested before) periodically (e.g. drive by timer). Clear GPMC_TARG errors after taking dumps in timer ISR.

GPMC_TARG errors (1 deep) can be cleared by writing 0xFFFFFFFF to address 0x44800748.

Please share these dumps in 32-bit format like you have here: https://e2e.ti.com/support/processors/f/791/p/904633/3375412#3375412

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

you write a bit mysterious. What is GPMC_TARG? Is GPMC_TARG == 0x44800748? Can you send some more background information please?

No, we do not use SGX530. Registers @ 0x56000000 are dead (read ???????? in CCS).

I will proceed and send you the requested dumps in a next post.

Regards

Frank

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

I'm not trying to be mysterious. We're taking dumps of interconnect error reporting registers to root cause the lock up. I haven't provided details of these registers since they aren't shared in the TRM. I'm following up internally to find out why this information isn't in the TRM.

Regards,
Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank,

as annonced, here the new dumps:

l3d_data_6dumps.dat

The six subsequent dumps were taken as follows:

Void main()
{
    Task_Params         taskParams;
    Types_FreqHz        freqhz;

    /* init mmu first, otherwise crash on access to memory mapped registers */
    mmu_init();
    l3d_init();
    l3d_dump();
    /* clear GPMC_TARG errors */
    HWREG(0x44800748) = 0xFFFFFFFF;
    l3d_dump();
    l3d_diff();

...

USBDInitialize(usbdacm_rxdata_cbfunc, PRODUCT_ID_COR8_ECE, usbdecm_rxdata_cbfunc);
l3d_dump();

...

    /* picture buffer management, pixel generator, cpld, gpme */
    pic_init();
    l3d_dump();

} // main

- after an aborted transfer (stop external clock, hang on USB/GPMC register read

- after an successfull transfer.

The only change in the dumped L3 registers happens after

/* clear GPMC_TARG errors */
HWREG(0x44800748) = 0xFFFFFFFF;

that is:

address, before, after, 0

4400110C   00000002   00000000   00000000
4480060C   00000080   00000000   00000000
44800708   00000005   00000001   00000000
44800748   00080001   00000000   00000000
00000000   00000000   00000000   00000000

You find this in the dump file at line 714 ff.

The behaviour remains unchanged: CPU hangs if USB task does its actions, access to USB / GPMC registers in timer task as reported before.

Note: Also the access to the L3 registers causes a CPU hang, if programmed before the wait_enable():

void
pig_clockFxn()
{
    UInt    hwikey;

    /* timeout control */
    if ( pig_state == PIG_STATE_PRINT ) {
        hwikey = Hwi_disable();
        if ( ++pig_tcount >= pig_timeout ) {
            // reg = HWREG(0x47401000); // USB0, locks CPU
            reg = HWREG(0x48000000); // UART1...
            reg = HWREG(0x48200000); // INTCPS
            reg = HWREG(0x4A100000); // CPSW
            reg = HWREG(0x44E10000); // control module
            reg = HWREG(0x44E0D000); // ADC_TSC
            // reg = HWREG(0x50000000); // GPMC, locks
            pig_state = PIG_STATE_ERROR;
            pig_error = PIG_ERROR_TIMEOUT;
            /* disable wait to bring edma to completion */
            pig_disable_wait();
            l3d_dump();
            l3d_diff();
            //reg = HWREG(0x47401000); // OK, reads 0x4EA20800
            reg = HWREG(0x50000000); // OK
            /* post of PRINTERR event is done in edma isr */
        }
        ++pig_time_done;
        Hwi_restore(hwikey);
    }

So, the problem still is not solved. Find below the code with the dumped data structure and the dump functions.

Regards,

Frank

/* dump L3 subsystem registers */

#define L3DUMP      1

#if L3DUMP

#define L3D_REGIONS     4
#define L3D_MAXDUMP     8
#define L3D_DUMPLEN     (8 + 8 + 8 + 64)

struct l3d_data {
    uint32_t    *address[L3D_REGIONS];
    uint32_t    length[L3D_REGIONS];
    uint32_t    dump[L3D_MAXDUMP * L3D_DUMPLEN];
    uint32_t    diff[L3D_MAXDUMP * L3D_DUMPLEN];
} l3d_data = {
    {
        (uint32_t *)0x44001000, (uint32_t *)0x44001100,
        (uint32_t *)0x44800600, (uint32_t *)0x44800700
    },
    { 8, 8, 8, 64 }
};

int32_t     l3d_cnt;
uint32_t    *l3d_ptr;

void
l3d_init()
{
    int32_t     i;

    for ( i = 0; i < L3D_MAXDUMP * L3D_DUMPLEN; ++i ) {
        l3d_data.dump[i] = 0;
        l3d_data.diff[i] = 0;
    }
    l3d_cnt = 0;
    l3d_ptr = l3d_data.dump;
}

void
l3d_dump()
{
    int32_t     i, j;

    if ( l3d_cnt >= L3D_MAXDUMP ) {
        return;
    }
    for ( i = 0; i < L3D_REGIONS; ++i ) {
        for ( j = 0; j < l3d_data.length[i]; ++j ) {
            *l3d_ptr++ = *(l3d_data.address[i] + j);
        }
    }
    ++l3d_cnt;
    /* clear GPMC_TARG errors */
    // HWREG(0x44800748) = 0xFFFFFFFF;
}

/* diffs between the dumps */
void
l3d_diff()
{
    int32_t     i, j, k;
    uint32_t    *p0, *p1, *pd;

    p0 = l3d_data.dump;
    p1 = l3d_data.dump + L3D_DUMPLEN;
    pd = l3d_data.diff;
    for ( i = 0; i < l3d_cnt - 1; ++i ) {
        for ( j = 0; j < L3D_REGIONS; ++j ) {
            for ( k = 0; k < l3d_data.length[j]; ++k, ++p0, ++p1 ) {
                /* avoid crash */
                if ( pd >= l3d_data.diff + L3D_MAXDUMP * L3D_DUMPLEN - 8) {
                    return;
                }
                /* log diffs: address, before, after, 0 */
                if ( *p1 != *p0 ) {
                    *pd++ = (uint32_t)(l3d_data.address[j] + k);
                    *pd++ = *p0;
                    *pd++ = *p1;
                    *pd++ = 0;
                }
            }
        }
        /* hole between dump runs */
        *pd++ = 0;
        *pd++ = 0;
        *pd++ = 0;
        *pd++ = 0;
    }
}

#else

/* do nothing */

void inline l3d_init() {}
void inline l3d_dump() {}
void inline l3d_diff() {}

#endif /* L3DUMP */

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank,

in a somehow mysterious way (hang in E-Mail, overlooked) I missed your latest post before sending my dumps, so I did not react on it, sorry. Just was not sure about GPMC_TARG.

To be not mysterios in my part, some more explanations:

The l3d_dump(), l3d_diff() call for a successful translation (20000 chained EDMA calls) is in the transfer completion routine, I did not listed that in my post.

The l3d_dump(), l3d_diff() after a 'failed' transfer come to execution after a CPU lock due to missing clock and a USB / GPMC / L3 register access (either in USB task or timer routine), optionally a try to halt the CPU by CCS debugger, and turning the external clock on again.

So, despite the (temporary, due to temporarily missing external clock) CPU lock, the transfer appears to be successful and no error is reported.

To correct or to be more clear:

Note: Also the access to the L3 registers in l3d_dump() causes a CPU lock, if programmed before the pig_disable_wait():

// l3d_dump(); // locks CPU
// l3d_diff();

/* disable wait to bring edma to completion */
pig_disable_wait();

            l3d_dump(); // OK
            l3d_diff();
            //reg = HWREG(0x47401000); // OK, reads 0x4EA20800
            reg = HWREG(0x50000000); // OK
            /* post of PRINTERR event is done in edma isr */
        }
        ++pig_time_done;
        Hwi_restore(hwikey);
    }

Hope my dumps help to find a solution.

Regards,

Frank

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

Thanks for shared the dumps and the additional details. I'll take a look and let you know what I find.

Regards,
Frank

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

I want to open week #8 of this issue with an old question:

Any news?

Thanks & regards,

Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

???

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Frank,

I'm following up. I'll try to have an update for you by tomorrow.

Regards,
Frank

0 Frank Livingston over 5 years ago in reply to fmdhr

TI__Mastermind 29388 points

Hi Frank,

Sorry for the delayed response.

I'm getting help interpreting the dumps.

I got some feedback on the requirement to add MMU for accessing the L3 registers. This shouldn't be necessary if the DAP port is used.

Run time polling via the debugger can happen and quick experiments if DAP is used.
For CCS, one needs to un-hide to show all connectable cores, then select the DAP instead of an ARM core.
From there system memory can be read without modifying MMU tables.
This view will not see inside cores local memory maps or caches, but it works well for registers
When using the debugger its easier to use these instead of adjusting system code.

Regards,
Frank

I gI see the customer needed to add a MMU entry to dump the registers. This should not be necessary if the DAP target is used. Run time polling via the debugger can happen and quick experiments if that is used. For CCS, one needs to unhide to show all connectable cores, then select the DAP instead of an ARM core. From there system memory can be read without modifying MMU tables. This view will not see inside cores local memory maps or caches, but it works well for registers. In Lauterbach its always available with a modifier and a separate connect is not needed. Most all of the cores allow run time accesses through a few different DAP (debug access port) structures. When using the debugger its always 10x easier to just use these instead of adjusting

0 fmdhr over 5 years ago in reply to Frank Livingston

Expert 1530 points

Hi Frank,

nice to get news again. As far as I can see, the DAP way is only a possibility to get the register data more easy, right? I'll try it next time...

Meanwhile, I am working on another part of the project, but the blocked CPU issue still needs a solution.

Regards,

Frank

0 fmdhr over 5 years ago in reply to fmdhr

Expert 1530 points

Hi Frank,

any news meanwhile?

Thanks,

Frank

Processors

Processors forum

AM3359: GPMC wait for EDMA blocks CPU