AM3354: Sometimes can not jump from boot to Linux

Part Number: AM3354

Tool/software:

Dear Sir/Madam,

I use AM3354 in our product to run Linux OS. The board was designed by us with discrete components.

We had run 2 round of prototype build and 30pcs for each round, the boards from both round are runs normally for the Linux OS + application.

Now we are in small batch production phase, however part of them (5 out of 30pcs) have the issue: Bootloader on NOR Flash works well to read the image file from NAND Flash and load to DDR3, the checksum calculation for image file is also pass, however, when jump from boot to Linux, the Linux startup failed. There is no information print out after "jump to Linux". This issue happens occasionally, maybe the Linux startup failed in 3 times out of 10 times power on/off.

Question: Whether you can provide me a checklist for electronic side to investigate this issue, to let me understand which part lead to this issue? What I can do to check or ensure whether this issue is on electronic side?

Thank you.

  • Hi Wenhai,

    There is no information print out after "jump to Linux".

    What is "jump to Linux"? Is it the "Starting kernel ..." message?

  • Hi Bin,

    Yes, you are right, it is the starting kernel. You can see part of the log below, when issue happened, it stop at "[2976] linux.c :26 jumping to Linux".

    ***************************************

    [6 ] main.c :127 Bootloader Version 1.6.2.0
    O2HOST_SYNCHRO
    [2233 ] main.c :145 Starting BootLdr_1.6.2.0
    [2242 ] bsp_eeprom.c :45 reading 512 bytes at 0x0000
    [2306 ] main.c :155 loading oftree
    [2312 ] bsp_nand.c :1153 Reading 64257 bytes from 0x00760000
    [2327 ] bsp_nand.c :1233 done reading
    [2333 ] main.c :161 loading zImage
    [2339 ] bsp_nand.c :1153 Reading 4191904 bytes from 0x00160000
    [2844 ] bsp_nand.c :1233 done reading
    [2895 ] bsp_eeprom.c :45 reading 512 bytes at 0x0200
    [2959 ] main.c :174 booting linux
    [2965 ] linux.c :20 disable caches
    [2971 ] linux.c :24 turn off MMU
    [2976 ] linux.c :26 jumping to Linux
    [ 0.000000] Booting Linux on physical CPU 0x0
    [ 0.000000] Linux version 5.10.166 (oe-user@oe-host) (arm-oe-linux-gnueabi-gcc (Sourcery CodeBench 12.0.17) 9.5.0, GNU ld (Sourcery CodeBench 12.0.17) 2.34.0.20200910) #1 SMP PREEMPT Thu May 16 07:11:14 UTC 2024
    [ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
    [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

    ......

    ***************************************

  • Hi Wenhai,

    You can see part of the log below, when issue happened, it stop at "[2976] linux.c :26 jumping to Linux".

    It seems like kernel is not starting. Does your board has JTAG access? You might want to connect a debugger to see where the code gets stuck.

    Since you only have a small amount of boards which have the issue, you might want to check board assembly, voltage of the power rails, and DDR configuration, etc.

  • Hi Bin,

    We are trying to debug the board with JTAG to see where it get stuck.

    For the voltage of power rails, we have checked and no issue identified.

    Whether the power-up sequence will lead to this issue? The PMIC we used is TPS65910A3.

    For DDR configuration, do you have further items that need to be checked.

    Thanks a lot.

  • Hi Bin,

    I find an interesting scenario: When the kernel not startup normally, if I reset the board by pull down the PWRONRSTn pin of AM3354, the kernel will keep not startup. even I try 20 times. When the kernel can startup normally, the kernel will startup normally for each time (I tried 20 times) if I reset the board by PWRONRSTn. So without power off/on the board, the status for startup normal/abnormal will be stable.

    With reset by pull down the PWRONRSTn pin, the power rails from PMIC is not impacted, but for power off/on the power rails will be re-applied. With power off/on operation, the kernel startup or not is occasionally.

    I have checked the power-up sequence of PMIC, but no finding on that point. Do you have any suggestion?

    Thank you.

  • Hi Wenhai,

    if I reset the board by pull down the PWRONRSTn pin of AM3354, the kernel will keep not startup. even I try 20 times.

    In this case, your bootloader can always restart, but kernel is unable to boot?

    I don't know what else to check.

    Did you follow guidance in the TRM section 8.1.4.3.6 "Internal RTC LDO" about RTC? I see the TRM Table 8-25 "Reset Sources" shows PWRONRSTn pin doesn't reset RTC.

  • Hi Bin,

    "In this case, your bootloader can always restart, but kernel is unable to boot?"

    Yes, If this cycle power on was kernel starting normally, then if I reset the board without power off, bootloader and kernel work normally for each time;

    If this cycle power on was kernel can not starting, then if I reset the board without power off, bootloader can normally restart, but kernel is unable to boot for each time. 

    The reset on PWRONRSTn also applied on RTCPWRONRSTn in my design, PWRONRSTn pull-up with 3V3, RTCPWRONRSTn pull-up with 1.8V, when perform the reset action, both of them pull-down to ground.

  • Hi Wenhai,

    Thanks for the clarification.

    I don't have any other recommendations, other than connect a JTAG debugger to see where the execution gets stuck.

  • Hi Bin,

    Thanks a lot. I will do this and feedback to you if have an update.

  • Hi Bin,

    Finally we solved the issue.

    It is caused by not following TRM section 8.1.4.3.6 "Internal RTC LDO". We connect CAP_VDD_RTC to 1uF decoupling capacitor to ground, but RTC_KALDO_ENn is connected to 1.8V instead of ground, this is the mistake. In our SW application, we use external RTC chip instead of the internal RTC, so we don't indentified this issue during development. However, in the initialization of the Linux, the internal RTC was enabled, maybe sometimes the Linux detected this issue and not startup. I want to check with you:

    1. Why in my fault configuration connection, the Linux still can normally startup for most of times, do you have further information on that?

    2. As you know in 8.1.4.3.6, there are 5 conditions need to meet to "use RTC functionality and RTC-only mode", I don't want to modify my PCB to meet condition d and e, and also in my SW application keep to use external RTC. Whether there will have some issue prevent my device normal use, if only a,b and c followed?

    • a. RTC_KALDO_ENn is grounded (meet after BOM change)
    • b. CAP_VDD_RTC is connected to 1uF decoupling capacitor to ground (meet)
    • c. RTC_PWRONRSTn is connected to 1.8V RTC power on reset (meet)
    • d. PMIC_POWER_EN is connected to power input of PMIC (PMIC_POWER_EN is left float on the board, not connected)
    • e. EXT_WAKEUP0 is connected to a wakeup source (EXT_WAKEUP connected to ground through 22R resistor).

    Thank you.

  • Hi Wenhai,

    glad to hear that you were able to figure out what was wrong with your system and got it working.

    As for the continued discussion, please note that Bin is out of the office this week (U.S. holidays), returning next week. Please allow some extra time for a response here.

    Regards, Andreas

  • Hi Wenhai,

    Glad to hear that you found the root cause.

    I am not an RTC expert to answer all your RTC design questions. But can you please try to apply the following kernel patch to see if resolves the Linux boot issue? The patch disables the internal RTC module in kernel.

    diff --git a/arch/arm/boot/dts/am33xx-l4.dtsi b/arch/arm/boot/dts/am33xx-l4.dtsi
    index b2b44b9b5239..06cb455b679f 100644
    --- a/arch/arm/boot/dts/am33xx-l4.dtsi
    +++ b/arch/arm/boot/dts/am33xx-l4.dtsi
    @@ -438,6 +438,7 @@ target-module@3e000 {                       /* 0x44e3e000, ap 35 60.0 */
                            #address-cells = <1>;
                            #size-cells = <1>;
                            ranges = <0x0 0x3e000 0x1000>;
    +                       status = "disabled";
     
                            rtc: rtc@0 {
                                    compatible = "ti,am3352-rtc", "ti,da830-rtc";