media controller PPB lockup

Matthijs van Duin

Other Parts Discussed in Thread: CCSTUDIO

It appears I've found a silicon bug which is not yet in the errata:

If a cortex-m3 of the media controller subsystem attempts to access an invalid address on the external PPB, it apparently locks up the entire bus. The debugger can still access memory through the core's DAP, but any read from the PPB times out. Since the debug registers are located there, debugging the core is no longer possible. After resetting the core through PRCM it will run again until it tries to access any address on the PPB, which remains stuck. The problem even persists after a global warm reset. To get the subsystem working again, I have to apply a global cold reset.

The only plausible explanation is that the PPB is being locked up by the debug logic, even though it also happens if no debugger is attached and nTRST is pulled low. Is there any way to reset the media controller's debug logic from software, or perhaps trigger a local cold reset? (Shutting off the power domain would probably work, but the PRCM won't do that unless its standby request is acknowledged, for which both cores need to go into deep-sleep, which requires setting a bit in a register which is, of course, on the PPB.)

over 10 years ago

0 Pavel Botev over 10 years ago

TI__Guru**** 170625 points

Matthijs van Duin said:
If a cortex-m3 of the media controller subsystem attempts to access an invalid address on the external PPB, it apparently locks up the entire bus.

The PPB is Cortex-M3 ARM specific, so I suggest you to check this in the ARM forums/support.

Matthijs van Duin said:
Is there any way to reset the media controller's debug logic from software, or perhaps trigger a local cold reset?

See DM814x TRM, section 2.7.25 Media Controller Power-On Reset Sequence and 2.7.26 Media Controller Warm Reset Sequence

Regards,
Pavel

0 Matthijs van Duin over 10 years ago in reply to Pavel Botev

Mastermind 8020 points

Pavel Botev said:

If a cortex-m3 of the media controller subsystem attempts to access an invalid address on the external PPB, it apparently locks up the entire bus.

The PPB is Cortex-M3 ARM specific, so I suggest you to check this in the ARM forums/support.

[/quote]

The bug I found is however not in the Cortex-M3 core but in the bus logic of the Media controller subsystem of the dm814x, so I don't understand in what other forum you're suggesting I should report this. (It is possible the same problem exists in other devices having this subsystem (dm816x and omap4) so it would probably be a good idea for someone at TI to check this, but it could also be specific to the subsystem's integration in the dm814x.)

Perhaps I could have been clearer in my original post that its primary purpose was to report a silicon bug. The secondary purpose was to ask if there was perhaps some way to get the subsystem in a usable state again if the bug is accidently triggered, other than a cold reset of the whole device, but I guess that's unlikely.

BTW, I also noticed that the lockup is also triggered by an access to 0xe0040000, which is where the TPIU would be located. Obviously the Media controller doesn't need a TPIU, however it is listed and marked 'present' in the Media controller's CoreSight ROM table. This means that any debugger which performs auto-detection of debug peripherals as per CoreSight spec will immediately trigger the bus lockup. Not nice.

Pavel Botev said:

Is there any way to reset the media controller's debug logic from software, or perhaps trigger a local cold reset?

See DM814x TRM, section 2.7.25 Media Controller Power-On Reset Sequence and 2.7.26 Media Controller Warm Reset Sequence

[/quote]

I obviously already know how to turn the subsystem on and, as I said quite clearly in my original post, once triggered the bus lockup is warm-reset insensitive. To clarify, I know that neither of the two types of reset I did ask about are possible in any official/documented way, but of course that doesn't exclude the possibility there might be an undocumented way, hence I asked.

0 Pavel Botev over 10 years ago in reply to Matthijs van Duin

TI__Guru**** 170625 points

Matthijs van Duin said:
The bug I found is however not in the Cortex-M3 core but in the bus logic of the Media controller subsystem of the dm814x, so I don't understand in what other forum you're suggesting I should report this.

I can not find anything regarding the Private Peripheral Bus (PPB) of the Cortex-M3 in the DM814x datasheet and TRM. But I can find detailed description of the Private Peripheral Bus (PPB) of the Cortex-M3 in the ARM site:

Cortex-M3 Device Generic User Guide:
http://infocenter.arm.com/help/topic/com.arm.doc.dui0552a/DUI0552A_cortex_m3_dgug.pdf

Cortex-M3 Technical Reference Manual (TRM):
http://infocenter.arm.com/help/topic/com.arm.doc.ddi0337i/DDI0337I_cortexm3_r2p1_trm.pdf

Thus I can conclude the Private Peripheral Bus (PPB) of the Cortex-M3 belongs to the Cortex-M3 ARM processor, not to the Cortex-M3 Subsystem (TI wrapper), also know as Media Controller.

Regards,
Pavel

0 Matthijs van Duin over 10 years ago in reply to Pavel Botev

Mastermind 8020 points

ok, let's try this one more time...

Pavel Botev said:

I can not find anything regarding the Private Peripheral Bus (PPB) of the Cortex-M3 in the DM814x datasheet and TRM.

The Media Controller subsystem of the DM814x is indeed quite poorly documented. Fortunately it is identical to the "Dual Cortex-M3 subsystem" of the OMAP4, which is documented in reasonable detail. The PPB, although TI gave it a different name, is described very tersely in section "7.3.6.4 Private Memory Space" of the omap4430 TRM for example.

But I can find detailed description of the Private Peripheral Bus (PPB) of the Cortex-M3 in the ARM site

Of course, since the PPB is one of the busses of the cortex-m3, and ARM's documentation tends to be reasonably detailed, you will find plenty of information there.

Thus I can conclude the Private Peripheral Bus (PPB) of the Cortex-M3 belongs to the Cortex-M3 ARM processor, not to the Cortex-M3 Subsystem (TI wrapper), also know as Media Controller.

Interesting reasoning: "ARM has plenty of documentation on this bus, while we have virtually no documentation on what we connected to this bus. Therefore, the problem must lie with ARM."

But let me then present the issue without making any reference to the PPB. For convenience, I've attached a zip file containing my target configuration, a simple GEL file to enable the M3 cores, and a debug launch config.

Connect to any DM814x target, preferably in a 'clean' state (e.g. no boot source), using attached files. It should connect to DAP, enable the m3 cores, and connect to their DAPs.

Select first the SECM3 for comparison, right-click "Connect to Target". Using the memory browser you can go to 0xe000e000 to see the nvic registers, you can view the rom table at 0xe00ff000, and some of ICECrusher's registers at 0xe0042f00. Any errors you may see if you scroll around and hit invalid addresses are transient: the bus fault is reported and that's it.

Now similarly connect to MC0 and try again. Once you hit any address in the range 0xe0040000-0xe0041fff or 0xe0043000-0xe00fdfff code composer hangs for a while, eventually spits out copious errors, etc, and sometimes crashes. The media controller is now undebuggable. Warm reset via a reset button on your target or the "System reset" menu option in CCS does not help, the media controller remains unresponsive to the debugger.

The fact that the problem remains when the cortex-m3 is reset is enough to show that the problem lies outside it. Since the secss m3 does not suffer from this problem, the issue lies specifically with the Media Controller.

Forget my questions, I would just like a confirmation that this is indeed a bug and will be looked into, or at least included in the errata.

mc_bug.zip

0 Pavel Botev over 10 years ago in reply to Matthijs van Duin

TI__Guru**** 170625 points

Matthijs,

Matthijs van Duin said:
The media controller is now undebuggable. Warm reset via a reset button on your target or the "System reset" menu option in CCS does not help, the media controller remains unresponsive to the debugger.

I made some tests on my own DM814x EVM, with TI-based GEL file (DM814x_PG2.x.gel) and Blackhawk USB560-M 20-pin JTAG emulator (the only one I have), CCS5.4.0

When I launch the target configuration, I have 4 cores in the debug window:

C674x_0
Cortex_M3_RTOS
Cortex_M3_ISS
Cortex_A8

I do not have SECM3 core, but it seems you are more interested in the Cortex_M3_RTOS (also known as Cortex-M3 core 0 / MC0) so let me focus on it.

As I mentioned, I am using DM814x_PG2.x.gel file, not dap.gel file, and I tried with both dm814x.ccxml and EVMDM8148.ccxml, both ccxml files give the same result.

The DM814x_PG2.x.gel file is attached:1638.DM814x_PG2.x.gel

First I power on the EVM and connect to Cortex-A8 core, then run two scripts from the DM814x_PG2.x.gel file:

1. Scripts -> Centaurus2 System Initialization -> Centaurus_System_Initialization_GP_Device

2. Scripts -> Centaurus2 CORE_CLKS Configurations -> DucatiClkEnable_API

Then I connect to the Cortex_M3_RTOS. Then when I type the address of 0xE000E000 into the "Memory Browser" window, I can see the NVIC registers values successfully. Then when I type the address of 0xE0040000 into the "Memory Browser" window, instead of viewing the TPIU registers values, I see "BAD0BAD0' values and the below console message output:

1362.m3_log.txt

Cortex_M3_RTOS: Error: (Error -1204 @ 0xE0040000) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1203 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Error: (Error -1204 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0) 
Cortex_M3_RTOS: Unable to determine target status after 20 attempts
Cortex_M3_RTOS: Failed to remove the debug state from the target before disconnecting.  There may still be breakpoint op-codes embedded in program memory.  It is recommended that you reset the emulator before you connect and reload your program before you continue debugging

Now, is the PPB lockup issue reproduced successfully on my side? Or you have something else?

If this is the issue you are interested in, this is how I restore the Cortex-M3 core, without power cycle the EVM:

1. Run -> Reset -> System Reset

2. Select the Cortex-A8 core

3. Suspend the Cortex-A8 core (as it is in Running state)

4. Now when Cortex-A8 is in Suspend state: Scripts -> Centaurus2 CORE_CLKS Configurations -> DucatiClkEnable_API

5. Select the Cortex_M3_RTOS core

6. Delete the 0xE0040000 value from the "Memory Browser" window and put (in example) 0xE00FF000

7. Connect to the Cortex_M3_RTOS core

Now the console output one error message (Cortex_M3_RTOS: Error connecting to the target: (Error -1170 @ 0x0) Unable to access the DAP. Reset the device, and retry the operation. If error persists, confirm configuration, power-cycle the board, and/or try more reliable JTAG settings (e.g. lower TCLK). (Emulation package 5.1.36.0)
, but after click into the "Retry" button, the connect operation is successful and I can successfully see the Cortex-M3 "Core Registers" from the "Registers" window and also the NVIC register (at address 0xE00FF000) from the "Memory Browser" window. This all means the Cortex_M3_RTOS core is now again fully functional without power cycle the EVM.

Let me know what do you think for this PPB lockup CCStudio solution.

BR
Pavel

0 Matthijs van Duin over 10 years ago in reply to Pavel Botev

Mastermind 8020 points

Pavel Botev said:

Now, is the PPB lockup issue reproduced successfully on my side?

This is indeed the PPB lockup.

If this is the issue you are interested in, this is how I restore the Cortex-M3 core, without power cycle the EVM:

1. Run -> Reset -> System Reset

[...] the Cortex_M3_RTOS core is now again fully functional without power cycle the EVM.

Let me know what do you think for this PPB lockup CCStudio solution.

I was a bit surprised a warm reset worked for you and investigated it a bit more (details below). However, as I said in my previous post my primary purpose was to report this as a bug. If there is a more suitable way to do so then please do tell, but none was immediately apparent to me.

Having to reset the whole device (whether warm or cold) as a result of a single invalid access is still something I'd rather avoid. The only benefit is when debugging with CCS, since it gets completely confused by a cold reset while connected, but I don't use CCS all that often.

On a more positive note, I've verified that unprivileged code on the cortex-m3 is denied all access to the external PPB, and in particular cannot trigger this bug, so that's very fortunate.

Anyhow, some final remarks on the warm reset (in)sensitivity, which spiked my curiosity. After some more testing it turns out to depend on PRCM configuration, specifically the power-state control of the power domain 'default' (PM_DEFAULT_PWRSTCTRL @ 0x48180b00):

According to the TRM (2.10.13.1), "This POWERSTATE field of this register needs to be written to ON state and keep it there." This sounds like an imperative to me rather than mere advice, hence I did this as part of my initialization. It also appears to be hardware enforced: any write to that register will turn the powerstate to ON and you can no longer turn it off.

As the TRM mentions, that register is warm reset insensitive, and as a result if the powerstate is ON the media controller is kept powered during warm reset, and therefore the reset-insensitive PPB lockup doesn't clear up. If however DEFAULT's power state register has never been touched and is still in its OFF state, as in your setup, then the power to the media controller (and the rest of the power domain) is probably briefly cut upon warm reset, hence the issue resolves.

I'm not sure what the implications are of leaving it off though. The TRM is virtually silent on power domain DEFAULT, most of the PRCM chapter seems to pretend it doesn't exist and places its modules in the ALWON domain instead. Although not mentioned in the TRM, part of the L3 interconnect (including the firewalls) resides in DEFAULT. Taking all this into consideration, it feels like it could potentially be bad idea to allow DEFAULT to be power-cycled on warm reset.

0 Pavel Botev over 10 years ago in reply to Matthijs van Duin

TI__Guru**** 170625 points

Matthijs,

Matthijs van Duin said:
However, as I said in my previous post my primary purpose was to report this as a bug.

I will report to the DM814x Silicon Errata team.

Matthijs van Duin said:

If there is a more suitable way to do so then please do tell, but none was immediately apparent to me.

Having to reset the whole device (whether warm or cold) as a result of a single invalid access is still something I'd rather avoid.

So, you need to know if there is more suitable way to bring the Cortex-M3 core to functional mode than reset (warm or cold) when make invalid access. I will check with the team if they have something more to add here.

P.S. I have found two wiki pages which has some info regarding this:

http://processors.wiki.ti.com/index.php/Cortex_M3_ITM_Support_for_Concerto_F28M35x#FAQ

I cannot view TPIU and ITM registers in 0xE0040000 region and 0xE0000000 region from Cortex M3' memory view in CCS. Why?

Ans: It is most likely that those memory region is not defined as valid memory in CCS's memory map setting. Please modify the device gel file to use GEL_MapAddStr function to add 0xE0040000(size of 0x1000) and 0xE0000000 (size of 0xF000) with R/W privilege.

http://processors.wiki.ti.com/index.php/EZSDK_Memory_Map

Hardware Limitations

Media Controller cannot access any program memory above 0xA0000000.
Media Controller cannot access any data memory above 0xE0000000.

Regards,
Pavel

0 Matthijs van Duin over 10 years ago in reply to Pavel Botev

Mastermind 8020 points

Pavel Botev said:

P.S. I have found two wiki pages which has some info regarding this

I'm sorry, but those pages discuss unrelated issues.

Pavel Botev said:

However, as I said in my previous post my primary purpose was to report this as a bug. If there is a more suitable way to do so then please do tell, but none was immediately apparent to me.

I will report to the DM814x Silicon Errata team.

[/quote]

Thank you.

Pavel Botev said:

Having to reset the whole device (whether warm or cold) as a result of a single invalid access is still something I'd rather avoid.

[/quote]

If a mechanism to resolve the bus lockup exists (other than global reset), that would be nice to have yes.

And, for the reasons explained in my previous post, it would also be useful to know what the consequences are of leaving the powerstate control of the DEFAULT power domain at 'OFF' rather than setting it to 'ON' as the TRM instructs.

0 Pavel Botev over 10 years ago in reply to Matthijs van Duin

TI__Guru**** 170625 points

Matthijs van Duin said:
And, for the reasons explained in my previous post, it would also be useful to know what the consequences are of leaving the powerstate control of the DEFAULT power domain at 'OFF' rather than setting it to 'ON' as the TRM instructs.

I have check the settings for this PM_DEFAULT_PWRSTCTRL register in the DM814x EZSDK that I am using.

I can tell that this register is leaved at it default state (POWERSTATE = 0) in the GEL file (which should be used with CCStudio):

ti-ezsdk_dm814x-evm_5_05_02_00/board-support/host-tools/DM814x_gel.zip/DM814x_PG2.x.gel

This register is leaved at its default state also in the DM814x u-boot, linux kernel and M3 firmware. Thus when booting the u-boot, the linux kernel and load the M3 firmware, I end up with fully working system and PM_DEFAULT_PWRSTCTRL[1:0] POWERSTATE is 0x0

ti-ezsdk_dm814x-evm_5_05_02_00/board-support/u-boot-2010.06-psp04.04.00.01

ti-ezsdk_dm814x-evm_5_05_02_00/board-support/linux-2.6.37-psp04.04.00.01

ti-ezsdk_dm814x-evm_5_05_02_00/board-support/prebuilt-images/dm814x-firmware_05_02_00_48/dm814x_hdvpss.xem3

ti-ezsdk_dm814x-evm_5_05_02_00/board-support/prebuilt-images/dm814x-firmware_05_02_00_48/dm814x_hdvicp.xem3

I will also check with our PRCM designer about his opinion.

Regards,
Pavel

0 Matthijs van Duin over 10 years ago in reply to Pavel Botev

Mastermind 8020 points

Pavel Botev said:

I will also check with our PRCM designer about his opinion.

Any news?

0 Pavel Botev over 10 years ago in reply to Matthijs van Duin

TI__Guru**** 170625 points

Matthijs van Duin said:
Any news?

I ping him again. Sorry for the delay.

Regards,
Pavel

0 Pavel Botev over 9 years ago in reply to Pavel Botev

TI__Guru**** 170625 points

Matthijs,

In DM814x, complete functionality of PRCM was not used and that resulted into such requirements.

This recommendations was made based on DV test cases with POWERONSTATE == ON/0x3.

We didn’t do any negative testing to see the behaviour in case POWERONSTATE != ON/0x3.

So I can recommend you to test your system without this requirement and see if you will have any issues. I think this requirement is not needed (POWERONSTATE == ON/0x3), as the DM814x EZSDK (u-boot, linux kernel) is working fine without it.

Regards,
Pavel

0 Matthijs van Duin over 9 years ago in reply to Pavel Botev

Mastermind 8020 points

I can imagine forcing powerstate on may sometimes be needed when actively doing power management by disabling clocks, since otherwise if all clocks go idle in that power domain the power would be cut, causing potentially undesirable loss of state.

I still think it's rather strange however that if you do any write to that register, hardware forces the powerstate on even across warm reset and no longer allows you to return the register to its cold-reset state, especially since you noted that the whole EZSDK apparently works fine without powerstate being forced on.

But, since things seem to work and we're not really doing much with power management in the processor anyway (it feels a bit futile when there's a 1 kW amplifier sitting next to it), I don't actually need that mystery cleared up and will just stay away from that register.

Thank you for your time!

Processors

Processors forum

media controller PPB lockup