This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DRA829V: Boot problems going from 2023.04 to 2024.04

Part Number: DRA829V

Tool/software:

Hello experts!

We are facing a boot issue when trying to lift our U-Boot from 2023.04 to 2024.04.

Custom board with 2GB of RAM. Yocto build system based on SDK10.

We use USB boot and dfu process to transfer the four boot files to the board.

In 2024.04 the dfu process now hangs after transfer of sysfw.itb:

dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 0451:6163
Run-time device DFU version 0110
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0110
Device returned transfer size 512
Copying data from PC to DFU device
Download        [=========================] 100%       281108 bytes
Download done.
state(6) = dfuMANIFEST-SYNC, status(0) = No error condition is present
dfu-util: unable to read DFU status after completion
dfu-util: can't detach
Resetting USB to switch back to runtime mode
+ set +x
Found DFU: [16de:0057] ver=0224, devnum=117, cfg=1, intf=0, path="5-2", alt=0, name="sysfw.itb", serial="UNKNOWN"
+ dfu-util -R -d 0451:6163 -a sysfw.itb -D /home/bomellberg/se3linux/yocto-base/build/tmp/deploy/images/asp3/sysfw-asp3-hs-fs.itb
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 16de:0057
Run-time device DFU version 0110
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0110
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=========================] 100%       269718 bytes
Download done.
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(2) = dfuIDLE, status(0) = No error condition is present
Done!
Resetting USB to switch back to runtime mode
+ set +x

At this point, dfu-util -l does not show any dfu device:

bomellberg@bosse-buildcom:~/se3linux/yocto-base/build$ dfu-util -l
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

There is no debug output on the main debug port. The wakeup uart responds as normal:

0x710002
0xB10004
0x4003007
0x4400B04
0x71000B
0xB10004

Attaching a J-Link debugger gives me this information on the wakeup-mcu:

bomellberg@bosse-buildcom:~$ gdb-multiarch -ex 'target extended-remote :3336'
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Remote debugging using :3336
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
0x41c0943c in ?? ()
─── Assembly ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 0x41c0943c  ? b.n      0x41c0943c
 0x41c0943e  ? nop
 0x41c09440  ? cmp      r7, #139        ; 0x8b
 0x41c09442  ? rors     r3, r0
 0x41c09444  ? push     {r4, r5, r6, lr}
 0x41c09446  ? mov      r6, r1
 0x41c09448  ? add.w    r1, r0, #63     ; 0x3f
 0x41c0944c  ? mov      r5, r0
 0x41c0944e  ? bic.w    r1, r1, #63     ; 0x3f
 0x41c09452  ? movs     r0, #64 ; 0x40
─── Breakpoints ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─── Expressions ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─── History ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─── Memory ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─── Registers ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
No registers to show (check the "dashboard registers -style list" attribute)
─── Source ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─── Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[0] from 0x41c0943c
─── Threads ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[1] id 0 from 0x41c0943c
─── Variables ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
>>>

As shown above, the program counter is stuck at: cpsr: 0x200001f3 pc: 0x41c0943c

Could you help out in debugging this issue?

Best regards,

/Bo

  • Hello Praveen,

    I have tried on HS-FS as well as GP variants. Neither of them boots beyond sysfw.itb which
    makes me suspect memory setup issues.

    The link you posted describes what we are use to see. A working DFU process. With U-Boot 2024.04 we are stuck.

    Regards,

    /Bo

  • Please note that the sysfw.the file is specific to each variant, so select the correct file for the DFU.

    Are you using the pre-built boot binaries provided on ti.com? If so, can you point to the release binaries you are using?

    We just verified the SDK 10.1 provided boot files on our TI EVM using DFU boot, and it worked fine.

    Thanks.

  • We are building our own boot files since over two years and I have updated the recipe for the firmware to the latest sdk10 scarthgap tag.

    I understand that the DM and TIFS firmware are proprietary but how do you expect customers to be able to debug situations like this without any debug output or source code?

    The only thing I have as a clue is the PC noted above and that doesnt tell me anything since I dont have any symbol files.

  • Do you support any other boot mode?

    Generally, TIFS is platform-independent. You must not be seeing any issues using it on your custom board. We are saying this based on the experience of having this used by other customers who already have SDK 10 on their platform.

    There is no debug output on the main debug port.

    Since you do not see any prints on the UART, we suspect it is likely mainly something specific to your modification on the MCU R5F SPL code (tiboot3.bin file). Hope you are using the stock sysfw.itb file? 

    As shown above, the program counter is stuck at: cpsr: 0x200001f3 pc: 0x41c0943c

    This is the MSRAM address where the SPL code is loaded, so suggest looking into your tiboot3.bin's map file to figure out the function where it is stalling.

    Thanks.

  • Do you support any other boot mode?

    A normal boot is from qspi. During our DFU process we copy the four bootfiles into qspi flash to be used when booting normally.

    Hope you are using the stock sysfw.itb file? 

    Our sysfw.itb is assembled during build from the binman files. It looks like this:

    bomellberg@bosse-buildcom:~/se3linux/yocto-base/build/tmp/deploy/images/asp3$ dumpimage -l sysfw-asp3-hs-fs.itb
    FIT description: SYSFW and Config fragments
    Created:         Mon Mar 17 16:46:20 2025
     Image 0 (sysfw.bin)
      Description:  sysfw
      Created:      Mon Mar 17 16:46:20 2025
      Type:         Firmware
      Compression:  uncompressed
      Data Size:    263828 Bytes = 257.64 KiB = 0.25 MiB
      Architecture: ARM
      OS:           Unknown OS
      Load Address: unavailable
     Image 1 (board-cfg.bin)
      Description:  board-cfg
      Created:      Mon Mar 17 16:46:20 2025
      Type:         Firmware
      Compression:  uncompressed
      Data Size:    29 Bytes = 0.03 KiB = 0.00 MiB
      Architecture: ARM
      OS:           Unknown OS
      Load Address: unavailable
     Image 2 (pm-cfg.bin)
      Description:  pm-cfg
      Created:      Mon Mar 17 16:46:20 2025
      Type:         Firmware
      Compression:  uncompressed
      Data Size:    2 Bytes = 0.00 KiB = 0.00 MiB
      Architecture: ARM
      OS:           Unknown OS
      Load Address: unavailable
     Image 3 (rm-cfg.bin)
      Description:  rm-cfg
      Created:      Mon Mar 17 16:46:20 2025
      Type:         Firmware
      Compression:  uncompressed
      Data Size:    3710 Bytes = 3.62 KiB = 0.00 MiB
      Architecture: ARM
      OS:           Unknown OS
      Load Address: unavailable
     Image 4 (sec-cfg.bin)
      Description:  sec-cfg
      Created:      Mon Mar 17 16:46:20 2025
      Type:         Firmware
      Compression:  uncompressed
      Data Size:    349 Bytes = 0.34 KiB = 0.00 MiB
      Architecture: ARM
      OS:           Unknown OS
      Load Address: unavailable
    

    The DM and SYSFW versions are defined in the ti-linux-fw.inc file:

    # Firmware versions
    CORESDK_RTOS_VERSION = "08.02.00.04"
    PRUETH_FW_AM65X_VERSION = "08.00.00.20"
    PRUETH_FW_AM65X_SR2_VERSION = "02.02.13.00"
    GOODIX_FW_VERSION = "1.0.0.0"
    CADENCE_MHDP_FW_VERSION = "2.1.0"
    IMG_DEC_FW_VERSION = "1.0"
    CNM_WAVE521_FW_VERSION = "1.0.4"
    TI_DM_FW_VERSION = "10.00.01"
    TI_SYSFW_VERSION = "09.02.08"
    
    TI_LINUX_FW_SRCREV ?= "83d5c6a2349436185fb49922da40d4201fb8d2c3"
    SRCREV = "${TI_LINUX_FW_SRCREV}"
    
    BRANCH ?= "ti-linux-firmware"
    
    SRC_URI = "git://git.ti.com/git/processor-firmware/ti-linux-firmware.git;protocol=https;branch=${BRANCH}"
    

    The hash used is the tag that corresponds to the Scarthgap version of Yocto.

    This is the MSRAM address where the SPL code is loaded, so suggest looking into your tiboot3.bin's map file to figure out the function where it is stalling.

    OK, I will try to find that and get back to you.

    EDIT: These are the map files that I find in the build system:

    bomellberg@bosse-buildcom:~/se3linux/yocto-base/build$ cat ./tmp-k3r5/work/asp3_k3r5-poky-eabi/u-boot-ti-as3/2024.04+git/u-boot-ti-as3-2024.04+git/tiboot3_hs_fs.map
    ImagePos    Offset      Size  Name
    00000000  00000000  00044e4c  tiboot3_hs_fs
    00000000   00000000  000006d4  ti-secure-rom
    000006d4   000006d4  00044778  u-boot-spl
    
    bomellberg@bosse-buildcom:~/se3linux/yocto-base/build$ cat ./tmp-k3r5/work/asp3_k3r5-poky-eabi/u-boot-ti-as3/2024.04+git/u-boot-ti-as3-2024.04+git/sysfw_hs_fs.map
    ImagePos    Offset      Size  Name
    00000000  00000000  00040694  sysfw_hs_fs
    00000000   00000000  00000694  ti-fs-cert-fs.bin
    00000694   00000694  00040000  ti-fs-firmware-j721e-hs-fs-enc.bin

    I can't read much from those. Any help is appreciated.

    I also tried a later hash for the ti-linux-firmware, with DM and SYSFW versions 10.01.08 but I'm still stalling.

    EDIT2: Inspecting the SPL u-boot-spl.map shows that it is stuck in some lib/div64:

     .text.__div64_32
                    0x41c093f6       0x70 lib/div64.o
                    0x41c093f6                __div64_32
    

    Does this tell you anything?

    Regards,

    /Bo

  • Loading the u-boot-spl file into gdb shows that it has hanged in the hang() process. Makes sense.

    >>> add-symbol-file ~/se3linux/yocto-base/build/tmp-k3r5/work/asp3_k3r5-poky-eabi/u-boot-ti-as3/2024.04+git/u-boot-ti-as3-2024.04+git/spl/u-boot-spl
    add symbol table from file "/home/bomellberg/se3linux/yocto-base/build/tmp-k3r5/work/asp3_k3r5-poky-eabi/u-boot-ti-as3/2024.04+git/u-boot-ti-as3-2024.04+git/spl/u-boot-spl"
    (y or n) y
    Reading symbols from /home/bomellberg/se3linux/yocto-base/build/tmp-k3r5/work/asp3_k3r5-poky-eabi/u-boot-ti-as3/2024.04+git/u-boot-ti-as3-2024.04+git/spl/u-boot-spl...
    >>> bt
    #0  hang () at /usr/src/debug/u-boot-ti-as3/2024.04+git/lib/hang.c:33
    #1  0x41c13436 in sysreset_walk_halt (type=type@entry=SYSRESET_COLD) at /usr/src/debug/u-boot-ti-as3/2024.04+git/drivers/sysreset/sysreset-uclass.c:110
    #2  0x41c1346a in do_reset (cmdtp=cmdtp@entry=0x0, flag=flag@entry=0, argc=argc@entry=0, argv=argv@entry=0x0) at /usr/src/debug/u-boot-ti-as3/2024.04+git/drivers/sysreset/sysreset-uclass.c:137
    #3  0x41c0a048 in panic_finish () at /usr/src/debug/u-boot-ti-as3/2024.04+git/lib/panic.c:29
    #4  0x41c0a054 in panic_str (str=str@entry=0x41c36755 "No serial driver found") at /usr/src/debug/u-boot-ti-as3/2024.04+git/lib/panic.c:38
    #5  0x41c1e1bc in serial_find_console_or_panic () at /usr/src/debug/u-boot-ti-as3/2024.04+git/drivers/serial/serial-uclass.c:152
    #6  serial_init () at /usr/src/debug/u-boot-ti-as3/2024.04+git/drivers/serial/serial-uclass.c:192
    #7  0x41c01a42 in preloader_console_init () at /usr/src/debug/u-boot-ti-as3/2024.04+git/common/spl/spl.c:831
    #8  0x41c007e2 in board_init_f (dummy=<optimized out>) at /usr/src/debug/u-boot-ti-as3/2024.04+git/arch/arm/mach-k3/j721e_init.c:306
    #9  0x41c01220 in _main () at /usr/src/debug/u-boot-ti-as3/2024.04+git/arch/arm/lib/crt0.S:127
    Backtrace stopped: previous frame identical to this frame (corrupt stack?)
    

    From the backtrace it looks like we have a problem with our serial console setup.

    Do you know of any differences between the U-Boot versions 2023.04 to 2024.04 regarding this?

    Regards,

    /Bo

  • Yes, it is now confirmed that there was an alias missing in the dts.

    Adding

            serial0 = &wkup_uart0;
            serial1 = &mcu_uart0;
            serial2 = &main_uart0;
    enables the serial console and we get further in the boot process:
    U-Boot SPL 2024.04-ti-gea67cbeaca21 (Mar 18 2025 - 10:37:12 +0000)
    SYSFW ABI: 4.0 (firmware rev 0x000a '10.1.6--v10.01.06 (Fiery Fox)')
    MISC init for esm@700000 failed: -19
    ESM PMIC init failed: -19
    Trying to boot from DFU
    #######################################################DOWNLOAD ... OK
    Ctrl+C to exit ...
    Authentication passed
    Authentication passed
    Authentication passed
    Loading Environment from nowhere... OK
    init_env from device 18 not supported!
    Authentication passed
    Authentication passed
    Starting ATF on ARM64 core...
    
    NOTICE:  BL31: v2.10.0(release):v2.10.0-367-g00f1ec6b87-dirty
    NOTICE:  BL31: Built : 07:57:12, Oct 23 2024
    I/TC:
    I/TC: OP-TEE version: 4.2.0-dev (gcc version 13.3.0 (GCC)) #1 Tue Oct 22 10:29:57 UTC 2024 aarch64
    I/TC: WARNING: This OP-TEE configuration might be insecure!
    I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
    I/TC: Primary CPU initializing
    I/TC: GIC redistributor base address not provided
    I/TC: Assuming default GIC group status and modifier
    I/TC: SYSFW ABI: 4.0 (firmware rev 0x000a '10.1.6--v10.01.06 (Fiery Fox)')
    I/TC: HUK Initialized
    I/TC: Activated SA2UL device
    I/TC: Enabled firewalls for SA2UL TRNG device
    I/TC: SA2UL TRNG initialized
    I/TC: SA2UL Drivers initialized
    I/TC: Primary CPU switching to normal world boot
    
    U-Boot SPL 2024.04-ti-gea67cbeaca21 (Mar 18 2025 - 10:37:12 +0000)
    SYSFW ABI: 4.0 (firmware rev 0x000a '10.1.6--v10.01.06 (Fiery Fox)')
    The value of PLL8_SS_CTRL register 0x80000001
    The value of PLL7_SS_CTRL register 0x80000001
    Successfully set the A72 clock frequency to 1000000000
    Successfully set the MSMC clock frequency to 500000000
    Trying to boot from DFU
    ############DOWNLOAD ... OK
    Ctrl+C to exit ...
    Authentication passed
    Authentication passed
    


    It is now stuck in handing over to the A72, but this thread can be closed. Thank you for your assistance!
    /Bo