Hi there,
I am trying to optimize to boot time of the linux image. I observe that while jumping from u-boot to kernel it is taking 4 to 5 seconds.
My desired target is to reach the boot up (SPL / u-boot / kernel / user-space under 10 seconds, but the u-boot it self is taking around 6 to 7 seconds.
the logs of the tisdk-default-image-am437x-evm where the jumping of u-boot to kerel is taking around 4 to 5 seconds
[2023-06-14 12:52:56.175] WDT: Not found! [2023-06-14 12:52:56.175] Trying to boot from MMC1 [2023-06-14 12:52:56.272] SPL: Please implement spl_start_uboot() for your board [2023-06-14 12:52:56.272] SPL: Direct Linux boot not active! [2023-06-14 12:52:56.607] [2023-06-14 12:52:56.607] [2023-06-14 12:52:56.607] U-Boot 2021.01-g44a87e3ab8 (Jun 02 2022 - 22:28:36 +0000) [2023-06-14 12:52:56.623] [2023-06-14 12:52:56.623] CPU : AM437X-GP rev 1.2 [2023-06-14 12:52:56.623] Model: TI AM437x SK EVM [2023-06-14 12:52:56.623] DRAM: 1 GiB [2023-06-14 12:52:56.735] PMIC: TPS65218 [2023-06-14 12:52:56.735] NAND: 0 MiB [2023-06-14 12:52:57.140] MMC: OMAP SD/MMC: 0 [2023-06-14 12:52:57.140] Loading Environment from FAT... *** Warning - bad CRC, using default environment [2023-06-14 12:52:57.231] [2023-06-14 12:52:57.231] Net: eth2: ethernet@4a100000 [2023-06-14 12:52:57.263] Hit any key to stop autoboot: 2 1 0 [2023-06-14 12:52:59.344] switch to partitions #0, OK [2023-06-14 12:52:59.344] mmc0 is current device [2023-06-14 12:52:59.424] SD/MMC found on device 0 [2023-06-14 12:52:59.424] Failed to load 'boot.scr' [2023-06-14 12:52:59.424] 717 bytes read in 1 ms (700.2 KiB/s) [2023-06-14 12:52:59.441] Loaded env from uEnv.txt [2023-06-14 12:52:59.441] Importing environment from mmc0 ... [2023-06-14 12:52:59.519] switch to partitions #0, OK [2023-06-14 12:52:59.519] mmc0 is current device [2023-06-14 12:52:59.599] SD/MMC found on device 0 [2023-06-14 12:52:59.859] 4989440 bytes read in 250 ms (19 MiB/s) [2023-06-14 12:52:59.871] 112940 bytes read in 8 ms (13.5 MiB/s) [2023-06-14 12:52:59.871] ## Flattened Device Tree blob at 88000000 [2023-06-14 12:52:59.871] Booting using the fdt blob at 0x88000000 [2023-06-14 12:52:59.871] Loading Device Tree to 8ffe1000, end 8ffff92b ... OK [2023-06-14 12:52:59.886] [2023-06-14 12:52:59.886] Starting kernel ... [2023-06-14 12:52:59.886] [2023-06-14 12:53:04.099] [ 0.000000] Booting Linux on physical CPU 0x0 [2023-06-14 12:53:04.099] [ 0.000000] Linux version 5.10.100-rt62-g204ec708dc (oe-user@oe-host) (arm-none-linux-gnueabihf-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025, GNU ld (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209) #1 PREEMPT_RT Thu Jun 2 22:32:00 UTC 2022 [2023-06-14 12:53:04.144] [ 0.000000] CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d [2023-06-14 12:53:04.144] [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [2023-06-14 12:53:04.159] [ 0.000000] OF: fdt: Machine model: TI AM437x SK https://e2e.ti.com/tinymce//apis/embeddables/configure?typeId=dc8ab71f-3b98-42d9-b0f6-e21e02a0f8e2&id=undefined#EVM [2023-06-14 12:53:04.159] [ 0.000000] Memory policy: Data cache writeback [2023-06-14 12:53:04.159] [ 0.000000] efi: UEFI not found.
the logs for the image we have created and the boot up time in there is also taking round 4 seconds of time while jumping from u-boot to kernel
[2023-06-14 12:10:25.127] U-Boot SPL 2020.01-g999 (Jun 13 2023 - 12:22:51 +0000) [2023-06-14 12:10:25.127] WDT: Not found! [2023-06-14 12:10:25.143] Trying to boot from MMC1 [2023-06-14 12:10:25.241] SPL: Please implement spl_start_uboot() for your board [2023-06-14 12:10:25.241] SPL: Direct Linux boot not active! [2023-06-14 12:10:25.799] [2023-06-14 12:10:25.799] [2023-06-14 12:10:25.799] U-Boot 2020.01-g999 (Jun 13 2023 - 12:22:51 +0000) [2023-06-14 12:10:25.799] [2023-06-14 12:10:25.799] CPU : AM437X-GP rev 1.2 [2023-06-14 12:10:25.799] Model: TI AM437x SK EVM [2023-06-14 12:10:25.799] DRAM: 1 GiB [2023-06-14 12:10:25.912] PMIC: TPS65218 [2023-06-14 12:10:25.912] NAND: 0 MiB [2023-06-14 12:10:26.311] MMC: OMAP SD/MMC: 0 [2023-06-14 12:10:26.326] Loading Environment from FAT... OK [2023-06-14 12:10:26.424] Net: eth0: ethernet@4a100000 [2023-06-14 12:10:26.438] Hit any key to stop autoboot: 0 [2023-06-14 12:10:26.518] switch to partitions #0, OK [2023-06-14 12:10:26.536] mmc0 is current device [2023-06-14 12:10:26.598] SD/MMC found on device 0 [2023-06-14 12:10:26.615] 1396 bytes read in 3 ms (454.1 KiB/s) [2023-06-14 12:10:26.615] Loaded env from uEnv.txt [2023-06-14 12:10:26.615] Importing environment from mmc0 ... [2023-06-14 12:10:26.695] switch to partitions #0, OK [2023-06-14 12:10:26.695] mmc0 is current device [2023-06-14 12:10:26.777] SD/MMC found on device 0 [2023-06-14 12:10:26.777] ROOTFS_A exist [2023-06-14 12:10:26.793] Saving Environment to FAT... OK [2023-06-14 12:10:27.094] 5575168 bytes read in 275 ms (19.3 MiB/s) [2023-06-14 12:10:27.111] 112940 bytes read in 9 ms (12 MiB/s) [2023-06-14 12:10:27.111] ## Flattened Device Tree blob at 88000000 [2023-06-14 12:10:27.111] Booting using the fdt blob at 0x88000000 [2023-06-14 12:10:27.126] Loading Device Tree to 8ffe1000, end 8ffff92b ... OK [2023-06-14 12:10:27.126] [2023-06-14 12:10:27.126] Starting kernel ... [2023-06-14 12:10:27.126] [2023-06-14 12:10:27.126] [2023-06-14 12:10:27.126] at end of Starting kernel ... [2023-06-14 12:10:27.143] [2023-06-14 12:10:27.143] [2023-06-14 12:10:27.143] at announce_and_cleanup 1 ... [2023-06-14 12:10:27.143] [2023-06-14 12:10:27.143] point-1 [2023-06-14 12:10:27.143] point-1.1 [2023-06-14 12:10:27.143] point-2 the fake value is 0 [2023-06-14 12:10:27.143] [2023-06-14 12:10:27.143] before kernel_entry ..at the address of 82000000 the machid is 0 and r2 is 8ffe1000. [2023-06-14 12:10:27.159] [2023-06-14 12:10:32.087] [ 0.000000] I am the first statement in start_kernel in main.c [2023-06-14 12:10:32.103] [ 0.000000] at the start of smp_setup_processor_id [2023-06-14 12:10:32.103] [ 0.000000] Booting Linux on physical CPU 0x0 [2023-06-14 12:10:32.118] [ 0.000000] Linux version 5.10.162-rt79-g999 (oe-user@oe-host) (arm-poky-linux-gnueabi-gcc (GCC) 9.5.0, GNU ld (GNU Binutils) 2.34.0.20200910) #1 PREEMPT_RT Wed Jun 14 06:22:44 UTC 2023 [2023-06-14 12:10:32.150] [ 0.000000] CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d
To verify I have done the code changes and added few logs as well to identify where it is taking most of the time the added logs screen shots are below
- in u-boot-ti-staging/arch/arm/lib/bootm.c
- in static void boot_jump_linux(bootm_headers_t *images, int flag)
/* Subcommand: GO */
static void boot_jump_linux(bootm_headers_t *images, int flag)
{
#ifdef CONFIG_ARM64
void (*kernel_entry)(void *fdt_addr, void *res0, void *res1,
void *res2);
int fake = (flag & BOOTM_STATE_OS_FAKE_GO);
kernel_entry = (void (*)(void *fdt_addr, void *res0, void *res1,
void *res2))images->ep;
debug("## Transferring control to Linux (at address %lx)...\n",
(ulong) kernel_entry);
bootstage_mark(BOOTSTAGE_ID_RUN_OS);
announce_and_cleanup(fake);
printf("\nat announce_and_cleanup ...\n\n");
if (!fake) {
#ifdef CONFIG_ARMV8_PSCI
armv8_setup_psci();
#endif
do_nonsec_virt_switch();
update_os_arch_secondary_cores(images->os.arch);
#ifdef CONFIG_ARMV8_SWITCH_TO_EL1
armv8_switch_to_el2((u64)images->ft_addr, 0, 0, 0,
(u64)switch_to_el1, ES_TO_AARCH64);
#else
if ((IH_ARCH_DEFAULT == IH_ARCH_ARM64) &&
(images->os.arch == IH_ARCH_ARM))
armv8_switch_to_el2(0, (u64)gd->bd->bi_arch_number,
(u64)images->ft_addr, 0,
(u64)images->ep,
ES_TO_AARCH32);
else
armv8_switch_to_el2((u64)images->ft_addr, 0, 0, 0,
images->ep,
ES_TO_AARCH64);
#endif
}
#else
unsigned long machid = gd->bd->bi_arch_number;
char *s;
void (*kernel_entry)(int zero, int arch, uint params);
unsigned long r2;
int fake = (flag & BOOTM_STATE_OS_FAKE_GO);
kernel_entry = (void (*)(int, int, uint))images->ep;
#ifdef CONFIG_CPU_V7M
ulong addr = (ulong)kernel_entry | 1;
kernel_entry = (void *)addr;
#endif
s = env_get("machid");
if (s) {
if (strict_strtoul(s, 16, &machid) < 0) {
debug("strict_strtoul failed!\n");
return;
}
printf("Using machid 0x%lx from environment\n", machid);
}
debug("## Transferring control to Linux (at address %08lx)" \
"...\n", (ulong) kernel_entry);
bootstage_mark(BOOTSTAGE_ID_RUN_OS);
announce_and_cleanup(fake);
printf("\nat announce_and_cleanup 1 ...\n\n");
printf("point-1\n");
if (IMAGE_ENABLE_OF_LIBFDT && images->ft_len){
printf("point-1.1\n");
r2 = (unsigned long)images->ft_addr;
}
else{
printf("point-1.2\n");
r2 = gd->bd->bi_boot_params;
printf("print1.3\n");
}
printf("point-2 the fake value is %d \n", fake);
if (!fake) {
#ifdef CONFIG_ARMV7_NONSEC
if (armv7_boot_nonsec()) {
printf("point-3\n");
armv7_init_nonsec();
printf("point-4\n");
secure_ram_addr(_do_nonsec_entry)(kernel_entry,
0, machid, r2);
printf("point-5\n");
} else
#endif
printf("\n before kernel_entry ..at the address of %lx the machid is %lx and r2 is %lx.\n\n", kernel_entry, machid, r2);
kernel_entry(0, machid, r2);
printf("point-6\n");
}
#endif
printf("\n at end of kernel_entry ...\n\n");
}
- in linux-ti-staging-rt/init/main.c
asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
{
printk("I am the first statement in start_kernel in main.c" );
char *command_line;
char *after_dashes;
set_task_stack_end_magic(&init_task);
smp_setup_processor_id();
debug_objects_early_init();
cgroup_init_early();
local_irq_disable();
early_boot_irqs_disabled = true;
- in linux-ti-staging-rt/arch/arm/kernel/setup.c
void __init smp_setup_processor_id(void)
{
int i;
pr_info("at the start of smp_setup_processor_id");
u32 mpidr = is_smp() ? read_cpuid_mpidr() & MPIDR_HWID_BITMASK : 0;
u32 cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
cpu_logical_map(0) = cpu;
for (i = 1; i < nr_cpu_ids; ++i)
cpu_logical_map(i) = i == cpu ? 0 : i;
/*
* clear __my_cpu_offset on boot CPU to avoid hang caused by
* using percpu variable early, for example, lockdep will
* access percpu variable inside lock_release
*/
set_my_cpu_offset(0);
pr_info("Booting Linux on physical CPU 0x%x\n", mpidr);
}
From the logs and mapping that logs the code changes mentioned above we observe that the u-boot is jumping from
