• State TI Thinks Resolved
  • +1 person also asked this people also asked this
  • Locked Locked
  • Replies 7 replies
  • Answers 1 answer
  • Subscribers 103 subscribers
  • Views 524 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

am64x: How to minimize boot time?

Harry Waschkeit
Prodigy 20 points
Other Parts Discussed in Thread: AM625

Hi support team,

we have an am64x based CPU board which shall be used as base for a customer board with a strict boot time requirement: Linux must be able to process network traffic within 6s after board power-up.

In particular, DHCP requests have to be answered from then on, followed by other network traffic over two interfaces.

I guess, with this time constraints that is generally only possible via the three direct Gb Ethernet ports, opposed to the potentially additional ones via PRU-ICSS subsystems which I would expect to require additional setup time, eventually exceeding the time frame.

I already stripped down all boot loader images (SPLs for R5F and A53 as well as A53 U-Boot) to the bare minimum with respect to subsystems and drivers which means ~2s until U-Boot can start with loading OS images.
The latter are set up as FIT image containing the stripped down kernel image, a device tree image and an initramfs directly serving for the actual application.

At the moment I cannot implement the whole boot process, unfortunately, because the current board used for evaluation behaves differently than the final platform would.

One question that arose first was: how to even measure start-up time?

Luckily, I found the GTC already running once the R5F SPL emits boot messages, so I could emit time stamps here and there to strip down image contents.
However, there must be some delay until GTC gets even started according to the reference manual: the reset state of GTC is said to be "off".
So, one question is: is it possible to determine this delay given processor speed (main clock) and speed of QSPI flash?

Another question is: are there means to reduce image sizes further in order to decrease load time as well as validation time?
(I found that a lot of the time is spent with CRC checking the image to be started next)

I had a look at the involved object files for my images and couldn't easily see further potential for minimizing them.
Any hints about relevant compile time switches would be greatly appreciated :-)

Or do other possibilities exists to speed up booting which I am not aware of?

Thanks in advance.

Best regards,
Harry

  • 0
    TI__Mastermind 27995 points

    Hi Harry,

    I have assigned your queries to our expert Hong. He is currently out for holidays, so expect the response after Jan 2nd. 

    In the meantime, I found these references to reduce Linux boot time, that might be useful.
    bootlin.com/.../boot-time-slides.pdf
    bootlin.com/.../opdenacker-timing-boot-time-reduction-techniques.pdf

    Best Regards,

    Suren

  • 0 in reply to Suren Porwar
    Prodigy 20 points

    Hi Suren,

    I was already aware of the two documents, they contain precious information for optimizations for sure.

    Thanks for this information nevertheless :-)

    Best regards,
    Harry

  • 0 in reply to Harry Waschkeit
    TI__Guru 68290 points

    Hello Harry,
    In addition to the boot-time optimizations you've already tried, and general technics as listed in the referenced links shared by Suren, here are some other ideas. Please note that these are listed as reference only, and not available from TI Linux SDK. Let's use the Linux flow as well as "Image Formats" for our discussions.
    software-dl.ti.com/.../UG-General-Info.html

    1. By default, R5-SPL/A53-SPL/A53 u-boot etc... are built with the same "am64x_evm_r5_defconfig" for R5 build, and "am64x_evm_a53_defconfig" for A53 build such that the same set SPL/u-boot binary support all boot modes/media onTI AM64x EVM board. For user's own HW board, it is possible to keep only supported boot mode/media by removing unused boot mode/media support when building R5/A53 u-boot binaries.


    2. Boot flow customerization:
    Let's take a closer look at "Image Formats" along with Linux boot flow diagram.
    By default,
    a/. "tispl.bin" = "ATF + OPTEE + A53 SPL"
    b/. "u-boot.img" = "A53 u-boot"

    => The standard boot flow: RBL => R5-SPL on R5 => on A53: ATF => OPTEE => A53 SPL => A53 u-boot => Linux kernel/DTB

    It is possible "in theory" to customize the boot flow as
    RBL => R5-SPL on R5 => on A53: ATF => Linux kernel/DTB, where "tispl.bin" = "ATF + Kernel/DTB"

    This is "feasible" due to the fact that most of HW dependent configurations (clocks, PMIC, DDR...) are configured in R5-SPL which makes possible for R5-SPL loading A53 ATF first, and then loading/launching kernel/dtb...

    The customized boot flow would need deeper SPL/u-boot customizations, and the above is an option for your reference.

    Best,
    -Hong

  • 0 in reply to Hong Guan64
    Prodigy 20 points

    Hello Hong,

    thanks for your explanation and suggestions :-)

    ad 1.

    I think I got already rid of all unneeded boot methods in all of the defconfig files slimming the resulting binaries quite a bit, but I will double check that nevertheless.

    ad 2.

    Removing A53 SPL & U-Boot sounds very promising for further boot time saving.
    However, I cannot tell how much effort it would be to implement that.

    You wrote that "most of HW dependent configurations" are done already in R5 SPL anyway, leaving the question "and which aren't?" ;-)
    Or put in other words: can you please elaborate a bit more on circumstances what could be missing and when?

    This would help estimating effort for that in case we are forced to take that route because this would come on top of the necessary adaptions to the SDK to construct the desired images, and to further configuration modifications for ATF/SPL/....

    If I understood you correctly ATF would in this scenario directly load the OS blob (as FIT image) avoiding the detour via SPL and U-Boot, right?
    In that case ATF would have to be parameterized to use the FIT image as BL33, I guess?
    (just as a side note: secure boot is not relevant in that project)

    And last but not least I want to point to my question concerning boot time measurement via GTC, as it might got overlooked:

    Harry Waschkeit said:

    One question that arose first was: how to even measure start-up time?

    Luckily, I found the GTC already running once the R5F SPL emits boot messages, so I could emit time stamps here and there to strip down image contents.
    However, there must be some delay until GTC gets even started according to the reference manual: the reset state of GTC is said to be "off".
    So, one question is: is it possible to determine this delay given processor speed (main clock) and speed of QSPI flash?

    Do you have any information on that topic?
    At least any pointers to code parts where GTC gets started so that I would have the chance to add some externally visible trigger (e.g. GPIO) for an oscilloscope to measure delays like from CPU reset release or even from power-up.

    Thanks again for your help! :-)

    Best regards,
    Harry

  • 0 in reply to Hong Guan64
    Prodigy 20 points

    Hi Hong,

    gentle ping for my additional questions and concerns.

    Best regards,
    Harry

  • 0 in reply to Harry Waschkeit
    TI__Guru 68290 points

    Hello Harry,

    Harry Waschkeit said:
    can you please elaborate a bit more on circumstances what could be missing and when?

    HW dependent configurations, such as clocks, PMIC, DDR, "should" be in general setup in the board file build for R5-SPL based upon, for example, the "*r5*_defcong", and *r5*.dtb files...

    Harry Waschkeit said:
    If I understood you correctly ATF would in this scenario directly load the OS blob (as FIT image) avoiding the detour via SPL and U-Boot, right?
    In that case ATF would have to be parameterized to use the FIT image as BL33, I guess?
    (just as a side note: secure boot is not relevant in that project)

    Let's chat more on the fast boot customization "proposal" in my last reply.
    a/. In the standard SDK, "tispl.bin" = "ATF + OPTEE + A53 SPL"
    => The boot flow: RBL => R5-SPL on R5 => on A53: ATF => OPTEE => A53-SPL => A53 u-boot => Linux kernel/DTB
    b/. It is possible "in theory" to customize the fast boot flow as
    RBL => R5-SPL on R5 => on A53: ATF => Linux kernel/DTB, where "tispl.bin" = "ATF + Kernel/DTB"
    OR
    RBL => R5-SPL on R5 => on A53: ATF => OPTEE => Linux kernel/DTB, where "tispl.bin" = "ATF + OPTEE + Kernel/DTB"

    Now let's take a deeper look at the boot flow on "tispl.bin" = "ATF + OPTEE + A53-SPL"  loading by R5-SPL in the standard SDK
    1. "tispl.bin" is a FIT image, where the load address of each binary are in the FIT header (refer to [1] [2] below)
    2. "tispl.bin" loading sequence by R5-SPL:
    - R5-SPL loads "tispl.bin", parses the FIT header, and loads each binary blob to the pre-allocated memory
    - R5-SPL brings A53 out out of RESET
    - ATF is the first one running on A53
    - ATF brings-up OPTEE, A53-SPL in sequence (refer to [3] below)

    3. Some additional reference on binary blob (ATF/OPTEE/A53-SPL) load addr and loading sequence...
    [1]. How to generate *.its which is used to create the FIT image "tispl.bin" in SDK
    git.ti.com/gitweb
    For example:
    load = <0x9e800000>; => OPTEE addr
    load = <0x80080000>; => A53-SPL addr

    [2]. The matching binary blob (ATF/OPTEE/A53-SPL) load addr in ATF
    git.ti.com/gitweb
    For example:
    BL32_BASE ?= 0x9e800000 => OPTEE addr
    PRELOADED_BL33_BASE ?= 0x80080000 => A53-SPL addr

    [3]. OPTEE(BL32) and A53-SPL(BL33) loading by ATF running on A53
    git.ti.com/gitweb
    bl32_image_ep_info.pc = BL32_BASE; => OPTEE
    bl33_image_ep_info.pc = PRELOADED_BL33_BASE; => A53-SPL

    So the high-level changes for the fast boot customization "proposal" for your reference ONLY:
    1. "tispl.bin": replace A53-SPL/DTB with Kernel/DTB...
    2. ATF loading sequence: OPTEE, Kernel/DTB...
    3. "u-boot.img" on A53 is skipped...


    Best,
    -Hong

  • 0 in reply to Hong Guan64
    TI__Guru 69602 points
    Harry Waschkeit said:
    Removing A53 SPL & U-Boot sounds very promising for further boot time saving.
    However, I cannot tell how much effort it would be to implement that.
    Hong Guan64 said:
    So the high-level changes for the fast boot customization "proposal" for your reference ONLY:
    1. "tispl.bin": replace A53-SPL/DTB with Kernel/DTB...
    2. ATF loading sequence: OPTEE, Kernel/DTB...
    3. "u-boot.img" on A53 is skipped...

    We now have an E2E FAQ on this very topic, one should be able to adopt this for AM64x without too much extra work:

    [FAQ] AM625: How to boot from R5 U-Boot SPL directly into the Linux Kernel, skipping A53 SPL and A53 U-Boot (Falcon Mode)
    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1205638/faq-am625-how-to-boot-from-r5-u-boot-spl-directly-into-the-linux-kernel-skipping-a53-spl-and-a53-u-boot-falcon-mode

    Regards, Andreas