This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PROCESSOR-SDK-AM65X: MCU_PLL0 idle after reset

Part Number: PROCESSOR-SDK-AM65X

Dear TI team,

we're facing an issue with MCU_PLL0 after a reset of the device.

During cold boot (initial power-up of the board), MCU_PLL0 is configured to generate a 400 MHz clock for the R5 processor at the time when the ROM enters the SBL.

After triggering a system reset, either via JTAG (CCS: "System Reset" / CTRL+SHIFT+S) or via Sciclient_pmDeviceReset(SCICLIENT_SERVICE_WAIT_FOREVER), the MCU_PLL0 comes up with the IDLE bit set in MCU_PLL0_CFG, causing the outputs to operate in bypass mode. This causes the OSPI for example to operate at 6.25 MHz instead of 33 MHz. The R5f is apparently also running considerably slower, since for example the checksum calculation in our own code takes ~16 times as long (would nicely match 400 MHz vs. 25 MHz).

Resetting the IDK via the PORz button does NOT show this problem.

You can verify this behaviour on an IDK with a SD card containing just the SBL and SYSFW (I'm using 06.01). If you cold boot the target you get the following output on the 2nd COM PORT (MCU_UART1):

SBL Revision: 01.00.09.02 (Oct 20 2019 - 15:14:40)
SYSFW  ver: 19.7.1-v2019.07a (Terrific Llam

 SD Boot - File open fails

If you trigger a system reset via CCS, booting takes a bit longer, and you get output like this:

æ˜à˜øæfžxþ†žþ†žþžøž˜øøøøø†øøø†ø†øø†ø€ø€†þ˜ž`þøøøøøøø†øøæ†˜àøfø˜øø`ø˜ø`øø††æ€æ€˜€SYSFW  ver: 19.7.1-v2019.07a (Terrific Llam

 SD Boot - File open fails

The first line with the SBL Revision is garbled.

If you configure the terminal for 57600 baud (actually you'd need 60000, but I got useful output with 57600) you can see the first line but not the rest:

SBL Revision: 01.00.09.02 (Oct 20 2019 - 15:14:40)
ñòñññôôõôõ÷ôôö÷õ÷ôöõõôôö÷ôõôôöðôõõöõöõôòöôöòòóòòóôñðôñ÷÷ôôöôñööôô÷ôô÷ôõôööõôòòó

The same ist true for the 3rd UART (WKUP_UART0):

On a cold boot with 115200 you get:

0x40000B
                              0x800004

On a system reset with 115200 you get:

                                      øààà€†xþ~xxp
                                                  ð
                                                   ó
                                                    þ

If you configure the terminal for 57600, you get valid output on a system reset:

0x40000B
                        0x800004

The MCU_PLL0 is running as intended halfway through the SBL (probably somewhere within SBL_SciClientInit(), because the "SYSFW  ver" is output correctly), but up to that point it is outputting 25 MHz on all outputs.

It seems that we have this issue only if the booting progressed through SBL, because if I cold boot the target without an SD card, break in the debugger, execute the system reset, and break in the debugger again, the IDLE bit is NOT set.

Why is MCU_PLL0 sometimes in IDLE state following a reset?

Are there any known issues with regard to the reset state of MCU_PLL0?

I've been told that the PORz on our own hardware does NOT work, but I still need to verify the exact behaviour.

Regards,

Dominic

  • Dominic, 

    sorry for the slow response. There are no issues with MCU_PLL0. Problems are most likely with the two reset method - CCS->SystemReset, and Sciclient_pmDeviceReset. I had a email exchange with the CCS team on the CCS reset, and also need to confirm the sysfw function it was a warmreset API. If it is WarmReset, then I expect the MCU_PLL0 is isolated and shall remains locked.

    By the way, it seems your sysfw version  (19.7.1-v2019.07a) is quite old. 

    Jian

  • Hello Jian,

    thanks for the clarification.

    I believe this is at least a documentation issue. Chapter 5.3.2 from TRM Rev. E contains a lot of details about things that are abstracted away from the user, since the processor SDK RTOS uses Sciclient (SYSFW) to control stuff like power and clocking. Even now that you told me that a WarmReset keeps MCU_PLL0 isolated I'm not able to find that information in the TRM (and I'm not sure what the implications of an isolated MCU_PLL0 are).

    I've seen differences in the MCU_PLLCTRL0_RSTYPE and PLLCTRL_RSTYPE registers depending on whether I've used CCS reset (IIRC one of the EMUn bits was set) and an API reset. I wont be able to verify that before Monday.

    If the CCS reset and the Sciclient_pmDeviceReset are both warm resets, what should the boot sequence look like from there? From a user perspective it has been very similar to a POR reset so far, i.e. some ROM code loads the SBL (or whatever you put there) from OSPI (or SD card), which then starts executing on the R5f. The initialization sequence from there is the same, whether this was the first boot or a reset. It's only recently that we found out that MCU_PLL0 was in IDLE mode, outputting only 25 MHz, resulting in only 1/5th of the OSPI clock.

    jian35385 said:

    By the way, it seems your sysfw version  (19.7.1-v2019.07a) is quite old. 

    Up until Friday last week, that was the latest sysfw version available (from processor SDK RTOS 06.01). We're right now working on migrating to SDK RTOS 06.03.

    Regards,

    Dominic

  • Dominic Rath said:
    I've seen differences in the MCU_PLLCTRL0_RSTYPE and PLLCTRL_RSTYPE registers depending on whether I've used CCS reset (IIRC one of the EMUn bits was set) and an API reset. I wont be able to verify that before Monday.

    When I reset via the Sciclient API the RSTYPE register for both WKUP_PLLCTRL0_RSTYPE and PLLCTRL_RSTYPE contain 0x2 (RSTIN, External Warm Reset device pin).

    When I reset via CCS "System Reset" both RSTYPE registers contain 0x10000000 (Emulation chip 0 reset).

    jian35385 said:
    If it is WarmReset, then I expect the MCU_PLL0 is isolated and shall remains locked.

    Can you sum up the behaviour of the SoC including initialization performed by ROM upon a WarmReset? The issue that I'm seeing is that upon a POR the ROM (R5? DMSC?) initializes MCU_PLL0 so that the R5f runs at 400 MHz and the OSPI reference is clocked at 133 MHz.  Upon a warm reset of both main and MCU domain, the R5f and the OSPI reference come up running at 25 MHz.

    Regards,

    Dominic

  • Dominic, 

    Can you confirm above notes you posted on 5/4 was based on the old or new SYSFW? From a jira tickets, I noted that Sciclient_pmDeviceReset() call was implemented and tested in the 2019.09 release. So I am interested to know what is the behavior of this API from the newer version. 

    I can confirm the SYSFW API call is doing a WARMRESET, but I did not tell you that MCU_PLL0 isolation bit need to be set, in order to retain MCU_PLL0 MMR content, we need to set LPSC 10 of WKUP_PSC0 to be isolated from MCU_WARMRESET. To do so,   

         WKUP_PLLCTRL0_RSISO = 0x1;  //enables sysclk during mcu_warmreset;

         WKUP_PSC0_MDCTL10[12] = 1h;  //enables isolation on LPSC10 

    Table 5-1441 shows that MCU_PLL_MMR0 is on the LPSC_MCU_COMMON of WKUP_PSC0; then Table 5-1443 shows LPSC 10 supports reset isolation.  

    Section  5.3.2.2 of the TRM is not very clear in terms describing MCU_RESET as a warmreset source. We may improve this section based on your feedback. 

    Jian

  • jian35385 said:
    Can you confirm above notes you posted on 5/4 was based on the old or new SYSFW? From a jira tickets, I noted that Sciclient_pmDeviceReset() call was implemented and tested in the 2019.09 release. So I am interested to know what is the behavior of this API from the newer version. 

    What I wrote about *_RSTYPE after a Sciclient_pmDeviceReset() was based on the processor SDK 06.01.

    I just re-ran this test with processor SDK 06.03 with the same result, i.e. 0x2 in both RSTYPE registers.

    I assume that what you're trying to say is that the Sciclient_pmDeviceReset() shouldn't have worked with the sysfw from SDK 06.01?

    jian35385 said:

    I can confirm the SYSFW API call is doing a WARMRESET, but I did not tell you that MCU_PLL0 isolation bit need to be set, in order to retain MCU_PLL0 MMR content, we need to set LPSC 10 of WKUP_PSC0 to be isolated from MCU_WARMRESET. To do so,   

         WKUP_PLLCTRL0_RSISO = 0x1;  //enables sysclk during mcu_warmreset;

         WKUP_PSC0_MDCTL10[12] = 1h;  //enables isolation on LPSC10 

    I guess you wanted me to configure these two registers, then trigger the warm reset via Sciclient_pmResetDevice()? I seem to be unable to write to WKUP_PLLCTRL0_RSISO, neither using code on the R5f nor using the debugger on either the R5f or the DMSC.

    Setting WKUP_PSC0_MDCTL10[12] worked, but the system was then stuck somewhere in the R5f ROM.

    Are there any special steps necessary to write to the WKUP_PLLCTRL0_RSISO register?

    Regards,

    Dominic

  • Dominic, 

    I tried on my board then realized we first need to write a key to the WKUP_PLLCTRL0_RSCTRL register. See Section 5.4.4.12.2.8 or search the register name for the key value. I verified once the key is written, WKUP_PLLCTRL0_RSISO can be written. 

    Please try the isolation sequence then issue warmreset, lets see what happens. 

    Regards

    Jian

  • Hello Jian,

    I was suspecting something like that, but I searched for "kick" and "lock" as in the PLL registers. The description of the *RSCTRL register in the TRM could really use an update:

    Reset Control Register
    This register contains a key that enables writes to the upper part of the . This key also enables writes to
    the and registers.

    The RSCTRL register turned from 0x10003 to 0x1000c after writing the key, and I was able to write the RSISO register.

    Unfortunately the behaviour is still the same, i.e. after triggering a reset via Sciclient the R5f is stuck somewhere in the ROM.

    Regards,

    Dominic

  • Dominic, 

    I had an offline discussion with our SYSFW team. There are a few additional software items that I am checking with our software teams. One suspect I had was that your boot interface failed, either due to not being reset, or not working with PLL in bypass. 

    I will get back once I get confirmation. 

    regards

    Jian

  • I'm not sure if we're on the same page here anymore. I can boot after a WarmReset, but it takes considerably longer. I just can't boot after configuring the reset isolation bits, but then I'm not sure WHY I should configure these bits.

    The issue I'm seeing is that the ROM is using the wrong clocking after a WarmReset, which causes the OSPI to use a considerably lower clock (factor 25/133=~1/5). The R5f is also running at a much lower clock (factor 25/400=1/16). When the ROM then enters the first user code ("SBL") on the R5f, the clocks are still slow, until the SYSFW has been loaded and the Sciclient initialized.

    WHY doesn't the ROM configure the clocks after a WarmReset the way it configured them after a POR? This looks like a bug in the AM65x ROM? I tried the same with SR2.0 hardware, and I was having the same issue.

    Is there a workaround, e.g. something that I can configure BEFORE executing the WarmReset, that ensures that the ROM uses the correct clocking? I believe that's what you're trying to tell me with the MCU_PLL0 reset isolation?

    Is it "safe" to just re-enable MCU_PLL0 by clearing the IDLE bit? Our setup uses a very small first-stage bootloader that gets loaded by ROM, and then loads the actual SBL or a fallback copy. That small first-stage bootloader could make sure that the IDLE bit gets cleared, thus mitigating the effect of the slow clock. I tried this a few times, and it worked, but I believe the "expected" way to configure PLLs is using the Sciclient.

    Regards,

    Dominic

  • Posting this information on behalf of Piyali:

    Hi Dominic

    I gave this a try at my end (albeit with the UART boot mode based remote AM65x setup). I see that when I load the SBL and then load the SYSFW and then run a system reset I checked the status of the MCU PLL. I see that the MCU PLL still remains in a good condition for boot. (not in idle)

    Before System reset:

    At address: 0x40d00000

     

    61820206           ????????             00000005           00000005

    68EF3491           D172BC5A          ????????             ????????

    0001E009           00000000           00010300           00000001

    00000019           03100419           C0000003           C0000000

    00000000           00000000           ????????             ????????

    ????????             ????????             ????????             ????????

     

    Just after system reset without re-loading SBL:

     

    At address: 0x40d00000

     

    61820206           ????????             00000005           00000005

    68EF3491           D172BC5A          ????????             ????????

    0001E009           00000000           00010300           00000001

    00000019           03100419           C0000003           C0000000

    00000000           00000000           ????????             ????????

    ????????             ????????             ????????             ????????

    Can you please help also check in your setup if you see the same? I will try and get a colleague of mine to try this on their setup with MMC SD boot mode.

    Thanks and Regards

    Piyali

  • Hello Piyali,

    I've seen this issue with QSPI and SD boot mode. I haven't used UART boot mode yet. On our custom hardware only QSPI boot mode is available.

    Did you verify that your reset was actually a warm reset or an emulation reset like in my case? Did you look at WKUP_PLLCTRL0_RSTYPE and PLLCTRL_RSTYPE?

    I wont be in the office until January 7th to re-run some tests with an IDK, but on our custom hardware the problem still existed with SDK 07.00. We're in the process of porting our custom hardware specific changes to SDK 07.01 and we'll try with the latest version again.

    It would be great if this thread could be kept open until the issue is solved.

    Regards,

    Dominic

  • Dominic

    Just FYI, we tried this sequence with 7.1 SDK (confirmed it is a warm reset) and with MMC SD boot the issue was not observed.

    Please do try this with the latest SDK and let us know if you continue to face issues.

    Regards

    Piyali

  • Dominic,

    We spend some more time on this today and found that is created by a bug in SDK 6.3 which was bypassing SYSFW routines and setting up the MCU PLL clock. The setup was incorrect leading to an inconsistent PLL state. The issue is fixed in the 7.1 SDK and is tested to be working well.

    Regards

    Piyali

  • Hello Piyali,

    thanks for looking into this.

    I can confirm that with SDK 07.01 the issue disappears when booting the IDK from MMCSD.

    We're still in the process of migrating our own custom hardware from SDK 07.00 to SDK 07.01. Would you happen to know where (which file, maybe which commit) exactly this bug was fixed?

    Regards,

    Dominic

  • I believe I've found the commit that switched the PLL configuration to SYSFW, but we've seen new failures with SDK 07.01 due to this change. There is commit 01a010b197879a071bd484947dcae8e9afe97ed5 in the PDK repository (PRSDK-8607: Board: Removed CSL based PLL config functions in AM65x Board library) that changes the generic code in packages/ti/board/src/evmKeystone3/board_pll.c.

    Unfortunately only the (k3-)generic code got changed, but the am65xx_idk/evm specific code (e.g. am65xx_idk/am65xx_idk_pll.c) was left untouched. This looks like dead code to me, that should be removed to avoid confusion. Or is there a reason for keeping these files around?

    We also ran into a new issue with the 07.01 SDK because apparently the GTC is now clocked at 200 MHz instead of 250 MHz like before. I'll create a new issue for this, but I guess the commit that switched the PLL configuration caused this problem.

    Regards,

    Dominic

  • Dominic

    There seems to be no reason for the content of these files to be around. They are not used.
    I will request my colleague to remove this.

    The GTC can get clocks from multiple sources. The PLLs are programmed by default as per software-dl.ti.com/.../pll_data.html.

    The board library additionally tries to set the following clock :

    TISCI_DEV_MCU_CPSW0 -> TISCI_DEV_MCU_CPSW0_BUS_RGMII_MHZ_250_CLK -> 250 MHz
    Note this is the CPSW PLL CLOCKOUT.

    For GTC the clock can be either from:
    MAINHSDIV_CLKOUT3 (mux option 1) - 250 MHz
    or CPSWHSDIV_CLKOUT2 (mux option 0) - 200 MHz.

    Can you please check what is the value of GTCCLK_SEL in your setup?

    You can use Sciclient_pmSetModuleClkParent to change the parent of the GTC clock.

    int32_t Sciclient_pmSetModuleClkParent(uint32_t moduleId,
                                           uint32_t clockId,
                                           uint32_t parent,
                                           uint32_t timeout)

    moudleId:
    #define TISCI_DEV_GTC0 61

    clockId:
    #define TISCI_DEV_CBASS_INFRA0_BUS_GTC_CLOCK_1_CLK 0

    parent -> options:
    #define TISCI_DEV_CBASS_INFRA0_BUS_GTC_CLOCK_1_CLK_PARENT_ADPLLM_HSDIV_WRAP_MCU_1_BUS_HSDIV_CLKOUT2_CLK 1 --> 200 MHz
    #define TISCI_DEV_CBASS_INFRA0_BUS_GTC_CLOCK_1_CLK_PARENT_ADPLLLJM_HSDIV_WRAP_MAIN_0_BUS_HSDIV_CLKOUT3_CLK 2 --> 250 MHz

    Thanks and Regards
    Piyali