MCU-PLUS-SDK-AM243X: starting of big appimages fails when BOOTLOADER_MEDIA_MEM selected

Part Number: MCU-PLUS-SDK-AM243X

Tool/software:

Hi TI-Team,

I am trying to get a SBL OSPI bootloader running on a custom AM243x-Board. I had problems with authentication for appimages when selecting BOOTLOADER_MEDIA_FLASH. Because of this, and as I want to support encryption later on, I switched to BOOTLOADER_MEDIA_MEM. I did the following:

  • added DDR support and manually loaded the data into a big enough location on DDRAM (split into reading block-wise via Flash_read(...))
  • passed the arguments as following to the bootloader, directly before Bootloader_open(..) is called
bootParams.bufIoTempBufSize = appimage_size;
bootParams.bufIoTempBuf = p_appimage;
bootParams.bufIoDeviceIndex = ~0;
bootParams.memArgsAppImageBaseAddr = (uintptr_t)p_appimage;

I also double checked if the appimage was successfully loaded with saveRaw() and scripting console, which was the case. Parsing of the boot image works fine:

  • authentication/signature checking works
  • the correct cores get initialized
  • BUT
  • afterwards no firmware is started
  • I do not see any firmware specific output on Debug-UART (only the messages from Bootloader)
  • If I start the same project manually via CCS I see the messages

I took the ipc_notify_echo_am243x-evm_system_freertos_nortos to replicate the behavior. I increased size of R5FSS0-1 firmware image to ~1MB (by adding a dummy array) and then I have the same behavior as for my custom firmware. No firmware Image is started. As soon as I decrease the firmware image size I see the outputs again and the image is started successfully.

I wonder whether there is somewhere a size limitation when using BOOTLOADER_MEDIA_MEM, as it is also somewhere described as Boot media is SOC memory. Maybe my test to load from DDR cannot work? However, then I would have a problem loading the firmware images we have which are nearly 2 MB and could be encrypted in future.

Here the test project to reproduce the behavior: multicore-project-ipc.zip

Thanks in advance for any help!

  • Hello,

    Could you please share the boot logs? Also, have you tried attaching to the R5FSS0-1 core after the boot to see the address it is suspended at. If the address belongs to the ".text" section address space then we can at least say the firmware is booting.

    Regards,

    Prashant

  • Hello,

    I do not see the UART logs with your shared images but could see using CCS that everything booted just fine. The reason is probably that you have the UART logging assigned to UART5 & not UART0.

    ❯ grep -r debug_log -- multicore-project-ipc/ipc_notify_echo_am243x-evm_r5fss0-0_freertos_ti-arm-clang/example.syscfg
    const debug_log = scripting.addModule("/kernel/dpl/debug_log");
    debug_log.enableCssLog = false;
    debug_log.enableUartLog = true;
    debug_log.enableSharedMemLog = true;
    debug_log.enableSharedMemLogReader = true;
    debug_log.uartLog.$name = "CONFIG_UART0";
    debug_log.uartLog.UART.$assign = "USART5";
    debug_log.uartLog.child = uart_v0_template1;
    debug_log.uartLog.UART.RXD.$suggestSolution = "GPMC0_ADVn_ALE";
    debug_log.uartLog.UART.TXD.$suggestSolution = "GPMC0_OEn_REn";

    Regards,

    Prashant

  • Hi Prashant,

    the change for UART logging was on purpose, as on our custom board that's the UART accessable for logging. I also changed the uart log in SBL_OSPI to USART5 and I see the messages from bootloader.

    The following messages are printed (I guess that are the boot-logs you requested)

    DMSC Firmware Version 9.1.6--v09.01.06 (Kool Koala)
    DMSC Firmware revision 0x9
    DMSC ABI revision 3.1
    
    INFO: Bootloader_loadSelfCpu:207: CPU r5f0-0 is initialized to 800000000 Hz !!!
    INFO: Bootloader_loadSelfCpu:207: CPU r5f0-1 is initialized to 800000000 Hz !!!
    [BOOTLOADER_PROFILE] Boot Media       : NOR SPI FLASH
    [BOOTLOADER_PROFILE] Boot Media Clock : 200.000 MHz
    [BOOTLOADER_PROFILE] Boot Image Size  : 1078 KB
    [BOOTLOADER_PROFILE] Cores present    :
    m4f0-0
    r5f1-0
    r5f1-1
    r5f0-0
    r5f0-1
    [BOOTLOADER PROFILE] SYSFW init                       :      12153us
    [BOOTLOADER PROFILE] System_init                      :     370553us
    [BOOTLOADER PROFILE] Drivers_open                     :        313us
    [BOOTLOADER PROFILE] Board_driversOpen                :         56us
    [BOOTLOADER PROFILE] Sciclient Get Version            :      10035us
    [BOOTLOADER PROFILE] CPU load                         :     241461us
    [BOOTLOADER_PROFILE] SBL Total Time Taken             :     634575us
    
    Image loading done, switching to application ...
    INFO: Bootloader_runCpu:155: CPU m4f0-0 is initialized to 400000000 Hz !!!
    INFO: Bootloader_runCpu:155: CPU r5f1-0  is initialized to 800000000 Hz !!!
    INFO: Bootloader_runCpu:155: CPU r5f1-1 is initialized to 800000000 Hz !!!
    INFO: Bootloader_runSelfCpu:217: All done, reseting self ...

    However, afterwards I get no single output on UART showing that firmware is alive. I also attached to the board and conneccted to several cores. R5_0_0 looks strange?

    When I switch back to a smaller firmware on R5FSS-0-1 by modifying the added dummy buffer:

    #define BOOTLOADER_APPIMAGE_MAX_FILE_SIZE (256)
    uint8_t g_appimage_buffer[BOOTLOADER_APPIMAGE_MAX_FILE_SIZE]  = { 'x'};;

    I see the output of the firmware again on the UART:

    DMSC Firmware Version 9.1.6--v09.01.06 (Kool Koala)
    DMSC Firmware revision 0x9
    DMSC ABI revision 3.1
    
    INFO: Bootloader_loadSelfCpu:207: CPU r5f0-0 is initialized to 800000000 Hz !!!
    INFO: Bootloader_loadSelfCpu:207: CPU r5f0-1 is initialized to 800000000 Hz !!!
    [BOOTLOADER_PROFILE] Boot Media       : NOR SPI FLASH
    [BOOTLOADER_PROFILE] Boot Media Clock : 200.000 MHz
    [BOOTLOADER_PROFILE] Boot Image Size  : 118 KB
    [BOOTLOADER_PROFILE] Cores present    :
    m4f0-0
    r5f1-0
    r5f1-1
    r5f0-0
    r5f0-1
    [BOOTLOADER PROFILE] SYSFW init                       :      12153us
    [BOOTLOADER PROFILE] System_init                      :     370499us
    [BOOTLOADER PROFILE] Drivers_open                     :        313us
    [BOOTLOADER PROFILE] Board_driversOpen                :         56us
    [BOOTLOADER PROFILE] Sciclient Get Version            :      10024us
    [BOOTLOADER PROFILE] CPU load                         :      66597us
    [BOOTLOADER_PROFILE] SBL Total Time Taken             :     459645us
    
    Image loading done, switching to application ...
    INFO: Bootloader_runCpu:155: CPU m4f0-0 is initialized to 400000000 Hz !!!
    INFO: Bootloader_runCpu:155: CPU r5f1-0  is initialized to 800000000 Hz !!!
    INFO: Bootloader_runCpu:155: CPU r5f1-1 is initialized to 800000000 Hz !!!
    INFO: Bootloader_runSelfCpu:217: All done, reseting self ...
    
    [IPC NOTIFY ECHO] Message exchange started by main core !!!
    [m4f0-0]     0.032062s : [IPC NOTIFY ECHO] Remote Core waiting for messages from main core ... !!!
    [r5f0-1]     0.002118s : [IPC NOTIFY ECHO] Remote Core waiting for messages from main core ... !!!
    [r5f0-1]     4.378178s : [IPC NOTIFY ECHO] Remote core has echoed all messages !!!
    [r5f1-0]     0.024128s : [IPC NOTIFY ECHO] Remote Core waiting for messages from main core ... !!!
    [r5f1-0]     4.400953s : [IPC NOTIFY ECHO] Remote core has echoed all messages !!!
    [r5f1-1]     0.017122s : [IPC NOTIFY ECHO] Remote Core waiting for messages from main core ... !!!
    [r5f1-1]     4.393575s : [IPC NOTIFY ECHO] Remote core has echoed all messages !!!
    [IPC NOTIFY ECHO] All echoed messages received by main core from 4 remote cores !!!
    [IPC NOTIFY ECHO] Messages sent to each core = 1000000
    [IPC NOTIFY ECHO] Number of remote cores = 4
    All tests have passed!!
    [m4f0-0]     6.722249s : [IPC NOTIFY ECHO] Remote core has echoed all messages !!!
    

    And if I connect in CCS I see some valid addresses on all cores

  • Hi Dominik,

    Thanks for the attachments!!

    It does look like the R5FSS0-0 core did not jump to the IPC example at all. However, the previously shared images are booting just fine for me.

    So, this definitely is an issue with the SBL.

    Could you once share your SBL image? And you are using MCU+ SDK v09_01_00_41?

  • Can you also once try keeping the media as BOOTLOADER_MEDIA_FLASH, skip the authentication to overcome the authentication failure & see if the issue still occurs?

  • Hi again,

    I am using latest MCU+ SDK v09_02_00_50

    Normally we would use a custom SBL OSPI but I added the behavior also to SBL OSPI of 09_02, here the adjusted SBL OSPI + the appimage that is not booting

    sbl_ospi.zip

    With the adjusted SBL OSPI I got the same problems, no output, same behavior in CCS

    DMSC Firmware Version 9.2.7--v09.02.07 (Kool Koala)
    DMSC Firmware revision 0x9
    DMSC ABI revision 3.1
    
    [BOOTLOADER_PROFILE] Boot Media       : NOR SPI FLASH
    KPI_DATA: [BOOTLOADER_PROFILE] Boot Media Clock : 200.000 MHz
    KPI_DATA: [BOOTLOADER_PROFILE] Boot Image Size  : 1078 KB
    [BOOTLOADER_PROFILE] Cores present    :
    m4f0-0
    r5f1-0
    r5f1-1
    r5f0-0
    r5f0-1
    KPI_DATA: [BOOTLOADER PROFILE] SYSFW init                       :     223077us
    KPI_DATA: [BOOTLOADER PROFILE] System_init                      :    1348496us
    KPI_DATA: [BOOTLOADER PROFILE] Drivers_open                     :      60375us
    KPI_DATA: [BOOTLOADER PROFILE] Board_driversOpen                :        174us
    KPI_DATA: [BOOTLOADER PROFILE] Sciclient Get Version            :      13106us
    KPI_DATA: [BOOTLOADER PROFILE] CPU load                         :     481891us
    KPI_DATA: [BOOTLOADER_PROFILE] SBL Total Time Taken             :    2127122us
    
    Image loading done, switching to application ...
    
    

    In our custom bootloader I currently use BOOTLOADER_MEDIA_FLASH and disabled the authentication. That's working fine. But I must do authentication checks and I also want to use BOOTLOADER_MEDIA_MEM as I want to decrypt later on at bootup. That's why I am actually facing all those problems

  • Hello,

    I have identified the issue. It is basically caused by the overwriting of the loaded appimage in memory.

    In the boot flow, the SBL OSPI loads the R5FSS0-0 RPRC image at the very last stage after everything is loaded. This means the memory occupied by the appimage in the RAM must not be overwritten by any of the loaded other cores RPRC images.

    However, you have the buffer in R5FSS0-1 image allocated to the address 0x80000000 which is the same address at which you have the appimage buffer allocated in SBL OSPI & so the conflict happens.

    Please resolve this conflict by relocating one of the buffer & then you should not see the issue.

  • BTW, I think a more clean solution for reading the appimage in RAM from OSPI & then authenticating would have the following design:

    1) Add another bootloader memory instance like so

    With this, there will be two bootloader instances:

    • CONFIG_BOOTLOADER_FLASH0 => Solely for reading the appimage in RAM from OSPI.
    • CONFIG_BOOTLOADER_MEM0 => For authenticating the appimage & loading the RPRC images.

    2) Apply the following diff

    diff --git a/examples/drivers/boot/sbl_ospi/am243x-evm/r5fss0-0_nortos/main.c b/examples/drivers/boot/sbl_ospi/am243x-evm/r5fss0-0_nortos/main.c
    index eb136ebf..7d15c98a 100644
    --- a/examples/drivers/boot/sbl_ospi/am243x-evm/r5fss0-0_nortos/main.c
    +++ b/examples/drivers/boot/sbl_ospi/am243x-evm/r5fss0-0_nortos/main.c
    @@ -54,6 +54,26 @@ void loop_forever(void)
             ;
     }
     
    +uint8_t* gAppImageBuf = (uint8_t*)0x90000000;
    +
    +void test_func() {
    +    Bootloader_BootImageInfo bootImageInfo;
    +    Bootloader_Params bootParams;
    +    Bootloader_Handle bootHandle;
    +
    +    Bootloader_Params_init(&bootParams);
    +    Bootloader_BootImageInfo_init(&bootImageInfo);
    +
    +    bootHandle = Bootloader_open(CONFIG_BOOTLOADER_FLASH0, &bootParams);
    +
    +    Bootloader_Config *config = (Bootloader_Config *)bootHandle;
    +
    +    config->fxns->imgSeekFxn(0, config->args);
    +
    +    /* Read 2MB data. Correct implementation would be determining the appimge size */
    +    config->fxns->imgReadFxn((void *)gAppImageBuf, 0x200000, config->args);
    +}
    +
     int main(void)
     {
         int32_t status;
    @@ -126,6 +146,9 @@ int main(void)
     
         status = Sciclient_getVersionCheck(1);
         Bootloader_profileAddProfilePoint("Sciclient Get Version");
    +
    +    test_func();
    +
         if(SystemP_SUCCESS == status)
         {
             Bootloader_BootImageInfo bootImageInfo;
    @@ -135,7 +158,9 @@ int main(void)
             Bootloader_Params_init(&bootParams);
             Bootloader_BootImageInfo_init(&bootImageInfo);
     
    -        bootHandle = Bootloader_open(CONFIG_BOOTLOADER_FLASH0, &bootParams);
    +        bootParams.memArgsAppImageBaseAddr = (uint32_t)gAppImageBuf;
    +
    +        bootHandle = Bootloader_open(CONFIG_BOOTLOADER_MEM0, &bootParams);
             if(bootHandle != NULL)
             {
                 /* Initialize PRU Cores if applicable */
    

    The patch is on top of SDK v10_00_00_20.

  • Hi Prashant,

    thank you very much for your time and investigating the issue! You are completely right, there was no clear separation of the DDR area used in firmware and the area used in bootloader.

    I relocated the buffer to an area explicitly reserved for bootloader now. I am now able to load even our custom images and do the authentication.

    Also thank you for suggesting the alternative way of adding and additional bootloader, which looks like a very clean solution. I already tested it and it works just fine with v10_00_00_20.

    One question is left according to the CONFIG_BOOTLOADER_FLASH0. Is there any chance to overwrite the address where the image should be loaded from in source code/at runtime? We intend to have 2 images in flash and depending on flags etc. we decide to either load image A or B but for this I would need to specify the address of image in flash.

    I've seen that there is a struct Bootloader_FlashArgs in bootloader_flash.h. Do I need to reinterpret the  void *args; pointer in Bootloader_Config struct?

  • Hi Dominic,

    One approach is to have as many bootloader instances as required for the images at different locations. For your use case, you will have two bootloader instances for FLASH & one for MEM. Then, you may choose which instance to use like so.

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1327528/tmds64evm-sbl-to-switch-and-launch-user-applications

    The benefit of this approach is that everything is nicely abstracted.

    Another approach is to just manipulate the "appImageOffset" of the "Bootloader_FlashArgs" struct like so

    https://e2e.ti.com/support/microcontrollers/arm-based-microcontrollers-group/arm-based-microcontrollers/f/arm-based-microcontrollers-forum/1345730/am2432-boot-image-offset/5127496#5127496

    Regards,

    Prashant

  • Hi Prashant,

    I just tested with "Bootloader_FlashArgs" that works fine.

    However, I will try the other solution as you are right, that's nicely abstracted. You really helped me with my issue, I now have a better understanding and can fix on my own, thanks for all the information and the good support!