OMAP-L138: looking for early boot phase debugging hints

Hi Brad,

thank you for this first hint, trying to connect via JTAG.

We do have an XDS100v2 JTAG adapter and the required connection port on our board too. But we never used it until the recent weeks. Indeed, we're able to build, flash and run out board without JTAG debugging, may sound funny, but true ;-)

Meanwhile I took the XDS100 out of my drawer, read some articles about connecting and configuring it in ccs5.5 and was able to connect the board and load the GEL file you pointed to, into the target configuration. I was not sure, whether to load it into the ICEPICK_C connection or into the ARM9_0 connection, so I put it into the 'Initialisation script' entry of both connections. I got no error messages, so I assume, it's ok.

Clicking the button 'Test Connection' yields 'good' results, the message tells me, that JTAG seems to work and the processor is visible. If I turn off the target power, this gets reported as well, so I think, this works.

When I open the Debug view in the studio, I see, that the DSP is connected (status 'Suspended') and the ARM and both PRUs are 'disconnected : unknown'. The disassembly window shows, that the DSP stands somewhere within its code - that does not look too bad, while I'm asking myself, how it got started. But that's not the question here.

When I connect the ARM target by context menu, it takes a while (my ccs is running inside a virtual box Linux, b/c we're not allowed to run native linux PCs in our department), then it tells me, that the pc points to 0x14 and cannot access the RAM:  "00000014:   ???? Page fault occurred reading 0x00000014 [code=0x6]"

This would explain why the board doesn't start, but why does it start if I press the reset button a second later.

I made other investigations regarding the bootmode configuration:  We switch between serial boot for bootstrap loading and flashing and 'normal' boot/run by configuring the BOOTMODE pins using a simple passive logik, which switches BOOT0, -1, -2, -3, -4  between 0x14 for 'serial' and 0x0E for 'NAND-8' flash boot. This logik is driven by a pin which is open for normal starting and driven by a GPIO of a cp2103 USB to serial adapter board. This all works pretty well and I checked the voltages at all this BOOTx pins. The values are all correct both for 'good', as well as for the two boards that don't start on POR.

My funny observation is, that 'serial' booting works perfectly when applying power. During the board flashing process, the master reset is actuated several times by the CP2103 GPIO and so the bug was not observed during the flash procedure. When powering up the board with the GPIO in "NAND-8" boot mode, it does not start. In this state I made the observations via JTAG, debug view in ccs.

My questions:

Why do I get the (JTAG-) DSP target connected first in the Studio? Is this a (changeable) default or does it come from the ARM, who seems to be unable to read from RAM and to run?

What happens within the first microseconds after applying power / releasing RESET?  Is the ARM CPU the first core running or is the DSP core somehow involved in the early boot procedure?

Is it a realistic idea to suspect the FLASH and/or RAM power supply for my problem? it is hard-connected to the OMAP ps, using 1.8V for both devices this time (modifyable).

And most important: are there some infos about the very early boot procedures in the OMAP-L1x which can give us hints about how to debug this problem?

Thank you for reading and - maybe - for further help.

Horst

Hello Brad,

meanwhile I managed to run the GEL diagnostic script on my broken board. I captured the console output for your reference:

- previous output, board power off

ICEPICK_C: Power Failure on Target CPU: (Error -180 @ 0xFFFFFF4C) The controller has detected a target power loss. The user must turn-on or connect the power supply for the target. (Emulation package 5.1.232.0)
ICEPICK_C: Failed to remove the debug state from the target before disconnecting.  There may still be breakpoint op-codes embedded in program memory.  It is recommended that you reset the emulator before you connect and reload your program before you continue debugging
ARM9_0: Error: Failed to get PRSC status
ARM9_0: Unable to determine target status after 20 attempts
ARM9_0: Failed to remove the debug state from the target before disconnecting.  There may still be breakpoint op-codes embedded in program memory.  It is recommended that you reset the emulator before you connect and reload your program before you continue debugging

- board powered, then GEL file executed, according description in processors.wiki.ti.com/.../OMAP-L1x_Debug_Gel_Files

- then output copied and pasted from console window:

ARM9_0: GEL Output:
---------------------------------------------
ARM9_0: GEL Output: |             Device Information            |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: DEV_INFO_00 = 0x1B7D102F
ARM9_0: GEL Output: DEV_INFO_01 = 0x00000000
ARM9_0: GEL Output: DEV_INFO_02 = 0x0000001E
ARM9_0: GEL Output: DEV_INFO_03 = 0x00000035
ARM9_0: GEL Output: DEV_INFO_04 = 0x00000000
ARM9_0: GEL Output: DEV_INFO_05 = 0x000003E0
ARM9_0: GEL Output: DEV_INFO_06 = 0x00000080
ARM9_0: GEL Output: DEV_INFO_07-DEV_INFO_08-DEV_INFO_09-DEV_INFO_10-DEV_INFO_11-DEV_INFO_12 = 0-0-4527369-13-17-15
ARM9_0: GEL Output: DEV_INFO_13,DEV_INFO_14,DEV_INFO_15,DEV_INFO_16 = 5,0,0,6246
ARM9_0: GEL Output: -----
ARM9_0: GEL Output: DEV_INFO_17 = 0x00030003
ARM9_0: GEL Output: DEV_INFO_18 = 0x00000000
ARM9_0: GEL Output: DEV_INFO_19 =ARM9_0: GEL Output: 0ARM9_0: GEL Output: 0ARM9_0: GEL Output: 0ARM9_0: GEL Output: 0ARM9_0: GEL Output: 0ARM9_0: GEL Output:
ARM9_0: GEL Output: -----
ARM9_0: GEL Output: DEV_INFO_20 = 0x30303864
ARM9_0: GEL Output: DEV_INFO_21 = 0x3830306B
ARM9_0: GEL Output: DEV_INFO_22 = 0x30303864
ARM9_0: GEL Output: DEV_INFO_23 = 0x3830306B
ARM9_0: GEL Output: -----
ARM9_0: GEL Output: DEV_INFO_24 = 0x0D00F011
ARM9_0: GEL Output: DEV_INFO_25 = 0x00451509
ARM9_0: GEL Output: DEV_INFO_06 = 0x00000080
ARM9_0: GEL Output: DEV_INFO_26 = 0x30CC0005
ARM9_0: GEL Output:

ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |               BOOTROM Info                |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: ROM ID: d800k008
ARM9_0: GEL Output: Silicon Revision 2.1
ARM9_0: GEL Output: Boot pins: 30
ARM9_0: GEL Output: Boot Mode: Emulation Debug
ARM9_0: GEL Output:
ROM Status Code: 0x00000000
Description:ARM9_0: GEL Output: No error
ARM9_0: GEL Output:
Program Counter (PC) = 0xFFFF0000
ARM9_0: GEL Output:
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |              Clock Information             |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output:
ARM9_0: GEL Output: PLLs configured to utilize crystal.
ARM9_0: GEL Output: ASYNC3 = PLL0_SYSCLK2
ARM9_0: GEL Output:
ARM9_0: GEL Output: NOTE:  All clock frequencies in following PLL sections are based
ARM9_0: GEL Output: off OSCIN = 24 MHz.  If that value does not match your hardware
ARM9_0: GEL Output: you should change the #define in the top of the gel file, save it,
ARM9_0: GEL Output: and then reload.
ARM9_0: GEL Output:
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |              PLL0 Information             |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output:
ARM9_0: GEL Output: PLL0_SYSCLK1 = 24 MHz
ARM9_0: GEL Output: PLL0_SYSCLK2 = 12 MHz
ARM9_0: GEL Output: PLL0_SYSCLK3 = 8 MHz
ARM9_0: GEL Output: PLL0_SYSCLK4 = 6 MHz
ARM9_0: GEL Output: PLL0_SYSCLK5 = 8 MHz
ARM9_0: GEL Output: PLL0_SYSCLK6 = 24 MHz
ARM9_0: GEL Output: PLL0_SYSCLK7 = 4 MHz
ARM9_0: GEL Output:
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |              PLL1 Information             |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output:
ARM9_0: GEL Output: PLL1_SYSCLK1 = 24 MHz
ARM9_0: GEL Output: PLL1_SYSCLK2 = 24 MHz
ARM9_0: GEL Output: PLL1_SYSCLK3 = 24 MHz
ARM9_0: GEL Output:
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |              PSC0 Information             |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output:
ARM9_0: GEL Output: State Decoder:
ARM9_0: GEL Output:  0 = SwRstDisable (reset asserted, clock off)
ARM9_0: GEL Output:  1 = SyncReset (reset assered, clock on)
ARM9_0: GEL Output:  2 = Disable (reset de-asserted, clock off)
ARM9_0: GEL Output:  3 = Enable (reset de-asserted, clock on)
ARM9_0: GEL Output: >3 = Transition in progress
ARM9_0: GEL Output:
ARM9_0: GEL Output: Module 0:    EDMA3CC (0)        STATE = 0
ARM9_0: GEL Output: Module 1:    EDMA3 TC0          STATE = 0
ARM9_0: GEL Output: Module 2:    EDMA3 TC1          STATE = 0
ARM9_0: GEL Output: Module 3:    EMIFA (BR7)        STATE = 0
ARM9_0: GEL Output: Module 4:    SPI 0              STATE = 0
ARM9_0: GEL Output: Module 5:    MMC/SD 0           STATE = 0
ARM9_0: GEL Output: Module 6:    AINTC              STATE = 3
ARM9_0: GEL Output: Module 7:    ARM RAM/ROM        STATE = 3
ARM9_0: GEL Output: Module 9:    UART 0             STATE = 0
ARM9_0: GEL Output: Module 10:    SCR 0 (BR0/1/2/8)  STATE = 3
ARM9_0: GEL Output: Module 11:    SCR 1 (BR4)        STATE = 3
ARM9_0: GEL Output: Module 12:    SCR 2 (BR3/5/6)    STATE = 3
ARM9_0: GEL Output: Module 13:    PRUSS              STATE = 0
ARM9_0: GEL Output: Module 14:    ARM                STATE = 3
ARM9_0: GEL Output: Module 15:    DSP                STATE = 3
ARM9_0: GEL Output:
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output: |              PSC1 Information             |
ARM9_0: GEL Output: ---------------------------------------------
ARM9_0: GEL Output:
ARM9_0: GEL Output: State Decoder:
ARM9_0: GEL Output:  0 = SwRstDisable (reset asserted, clock off)
ARM9_0: GEL Output:  1 = SyncReset (reset assered, clock on)
ARM9_0: GEL Output:  2 = Disable (reset de-asserted, clock off)
ARM9_0: GEL Output:  3 = Enable (reset de-asserted, clock on)
ARM9_0: GEL Output: >3 = Transition in progress
ARM9_0: GEL Output:
ARM9_0: GEL Output: Module 0:    EDMA3CC (1)        STATE = 0
ARM9_0: GEL Output: Module 1:    USB0 (2.0)         STATE = 0
ARM9_0: GEL Output: Module 2:    USB1 (1.1)         STATE = 0
ARM9_0: GEL Output: Module 3:    GPIO               STATE = 0
ARM9_0: GEL Output: Module 4:    UHPI               STATE = 0
ARM9_0: GEL Output: Module 5:    EMAC               STATE = 0
ARM9_0: GEL Output: Module 6:    DDR2 and SCR F3    STATE = 0
ARM9_0: GEL Output: Module 7:    MCASP0 + FIFO      STATE = 0
ARM9_0: GEL Output: Module 8:    SATA               STATE = 0
ARM9_0: GEL Output: Module 9:    VPIF               STATE = 0
ARM9_0: GEL Output: Module 10:    SPI 1              STATE = 0
ARM9_0: GEL Output: Module 11:    I2C 1              STATE = 0
ARM9_0: GEL Output: Module 12:    UART 1             STATE = 0
ARM9_0: GEL Output: Module 13:    UART 2             STATE = 0
ARM9_0: GEL Output: Module 14:    MCBSP0 + FIFO      STATE = 0
ARM9_0: GEL Output: Module 15:    MCBSP1 + FIFO      STATE = 0
ARM9_0: GEL Output: Module 16:    LCDC               STATE = 0
ARM9_0: GEL Output: Module 17:    eHRPWM (all)       STATE = 0
ARM9_0: GEL Output: Module 18:    MMC/SD 1           STATE = 0
ARM9_0: GEL Output: Module 19:    UPP                STATE = 0
ARM9_0: GEL Output: Module 20:    eCAP (all)         STATE = 0
ARM9_0: GEL Output: Module 21:    EDMA3 TC2          STATE = 0
ARM9_0: GEL Output: Module 24:    SCR-F0 Br-F0       STATE = 3
ARM9_0: GEL Output: Module 25:    SCR-F1 Br-F1       STATE = 3
ARM9_0: GEL Output: Module 26:    SCR-F2 Br-F2       STATE = 3
ARM9_0: GEL Output: Module 27:    SCR-F6 Br-F3       STATE = 3
ARM9_0: GEL Output: Module 28:    SCR-F7 Br-F4       STATE = 3
ARM9_0: GEL Output: Module 29:    SCR-F8 Br-F5       STATE = 3
ARM9_0: GEL Output: Module 30:    Br-F7 (DDR Contr)  STATE = 3
ARM9_0: GEL Output: Module 31:    L3 RAM, SCR-F4, Br-F6 STATE = 3

maybe this helps or gives some clues at least ...

kind regards

Horst

Hi Brad,

thank you for this first hint, trying to connect via JTAG.

...

please see my following two messages regarding my investigations to further isolate the problem.

This question here only in order to remove the "TI thinks this solves the problem" status flag.

Horst

Hi Brad and Rahul,

first thank you for that first hint, to try jtag debugging. The prpoblem is is not too urgent for us, so the response delay of two weeks is not a problem.

I learned a bit when trying our  XDS100v2  JTAG adapter, together with our ccs 5.5 installation, running within a virtualbox, executing Ubuntu 16.04 on a Win7-box.  In this confuguration, XDS100v2 is not usable in an acceptable manner. Everything "seems to work", as documented, but it is running soooooo slow (about one minute per instruction when singel-stepping) that it yields no reasonable results or understanding of the problem. But in principle, it works. Maybe that the XDS100 probe runs much faster and more usable on a native LINUX box without VirtualBox ***and*** VirusScanner overhead.

Wrong bootmode: it is true, that the bootmode is set to jtag (0x1E) when the XDS100 is active. But this seems to come from the adapter / debugger settings within ccs. I can't find the switch / tab / config file, where to modify the bootmode when using a jtag adapter - but I do believe, that it's possible somehow. Otherwise debugging a problem of my kind wouldn't be possible at all.

But, instead, I got further understanding by (simply!) googling and reading.

In the weekend after my first post, I googled for similar bug reports / community posts, regarding boot problems with TI ARM processors and I found the chip errata of the AM1808 Sitara chip, which is a close relative to the OMAP-L, the same ARM core, but no DSP companion. In this document I found issue 2.0.9, which discribes exactly our problem. Then I read the chip erratum for the OMAP-L138  SPRZ301M  (sorry, that I haden't this idea some  month earlier) and that one has exactly the same 'anormality', described in "Advisory 2.3.23". So I thought about a "secondary reset" fix.

In our hardware, we use a TPS3707-33DR  brown-out detector and Reset-Timer, and the Master-Reset input #MR, pin 1, is connected to a pushbutton an a reset-pin of the RS232 console connector too.

My idea to generate a secondary reset without too much hardware redesign, is to change the TPS3707 into a TPS3705, which incorporates a watchdog timer. I connect the OMAP ball T18, "CLKOUT", to the WDI input of the TPS3705 (pin 6)  and the watchdog ouput  #WDO  directly, and parallel to the pushbutton and the (externally gpio-controlled) RESET input pin of the console connector.

I hope, that the temporary shortcut of the TPS3705 #WDO output (if someone pushes the RESET button or our script controlled board flash system pulles that output down for some hundred milliseconds) does not destroy the TPS3705. And if the OMAP doesn't start and does not supply CLKOUT within 1.5s after POR, it will get a second reset pulse of 200ms from the TPS3705. This will repeat until the OMAP runs.

Just yesterday we got five samples of the TPS3705-DR33 chip, so I wasn't able to test that meanwhile, and so I ask you: How do you think about that workaround?

kind regards

Horst

Hello Rahul,

thank you for youer kind reminder :-)

We are still working on that issue. Soldering two pieces of wire and exchanging the TPS3707 chip is an easy job for me, but - due to shortage of manpower - we have not been able to modify the u-boot code in order to activate the OMAP's CLKOUT pin within the 1.2 s after power on.

We used that pin several month ago on the LCDK evaluation board to clock our ADS1274 ADC and this code is still in our software. But this activation occures at LINUX boot time only and this is more than 12s after power up, too late in order to help here.

So we (my software buddy and me) will modify the early initialisation of the u-boot this week, in order to set up the pin-muxer of ball T18 for function as "CLKOUT". As I understand the description in the technical reference manual SPRUH77-A, it should be sufficient to enable the pin function in the SYSCFG PINMUX13_7_4 bitfield with 0x1 an. The OCSRC field of the OCSCLK configuration register of PLLC0 has a default setting of 0x14, which should feed through the 24MHz external clock frequency to the CLKOUT pin T18. The OSCDIV register is also set by default to 0x8000 (16bit), which means enabled and a divider ration of 1:1. 

When we edit the sources, we will set these two registers too, b/c we don't know for sure wether they get modified by the u-boot standard code, but we will grep for those register names and take a look into the sources first.

Do you agree with that idea of setup? We will code and try this tomorrow or wednesday and report our results.

Regards,

Horst