AM3352: U-boot: I get error from time to time while booting

Hi Hong

I have added "#define DEBUG" at the top of "/lib/initcall.c" and this is what I get during booting procedure:

U-Boot SPL 2019.01-gc14892445a-dirty (Sep 16 2020 - 08:39:31 +0200)
Trying to boot from MMC1
initcall: 8080dd19
initcall: 8083da59
No valid device tree binary found - please append one to U-Boot binary, use u-boot-dtb.bin or define CONFIG_OF_EMBED. For sandbox, use -d <file.dtb>
initcall sequence 80844078 failed at call 8083da59 (err=-1)
### ERROR ### Please RESET the board ###

In “u-boot.sym” I found it's pointing to this line:

8083da58 g     F .text_rest     00000028 fdtdec_setup

Additionally, I attach the file with log when It's booting correctly:

U-Boot SPL 2019.01-gc14892445a-dirty (Sep 16 2020 - 08:39:31 +0200)
Trying to boot from MMC1
initcall: 8080dd19
initcall: 8083da59
initcall: 80812f73
initcall: 8080dd0d
initcall: 8080decf
initcall: 8080dedb
initcall: 8080deff
initcall: 8080dd31
initcall: 8080dedf
initcall: 8080dec9
initcall: 80801b39
initcall: 8083f315
initcall: 8082907d
initcall: 8080dead
initcall: 80821c09
initcall: 8083e4a1


U-Boot 2019.01-gc14892445a-dirty (Sep 16 2020 - 08:39:31 +0200)

initcall: 8080dd15
initcall: 80801a61
CPU  : AM335X-GP rev 2.1
initcall: 8080e155
Model: GATEWAY 3
initcall: 8080df07
initcall: 8080de19
initcall: 8080de9d
DRAM:  initcall: 80801af1
initcall: 8080de19
initcall: 8080de19
initcall: 8080de19
initcall: 8080df61
initcall: 8080dd35
initcall: 8080dd49
initcall: 8080dee3
initcall: 8080dee7
initcall: 8080dd77
initcall: 8080dda7
initcall: 8080de75
initcall: 8080ddbb
initcall: 8080ddc9
initcall: 8080dddf
initcall: 8080deeb
initcall: 8080deef
initcall: 8080df03
initcall: 8080df91
initcall: 80801b07
initcall: 8080df19
128 MiB
initcall: 8080def3
initcall: 8080de19
initcall: 8080de49
initcall: 8080def7
initcall: 8080defb
initcall: 8080de23
]!ý ]ýý ]ýý`]ýý`]#ý Uýýtýýýoýx7ý d k 9SýýýýýýRýRýýg4VýýýzrýýýF}ýrzCý>ý5c^,N/ý·KoR?ýýsýfýxýxýýýinitcal
l: 8080e0d5 (relocated to 87f840d5)
initcall: 8080e117 (relocated to 87f84117)
initcall: 8080e0b9 (relocated to 87f840b9)
initcall: 8080dd11 (relocated to 87f83d11)
initcall: 8080e0ad (relocated to 87f840ad)
initcall: 8080e11b (relocated to 87f8411b)
initcall: 8080e113 (relocated to 87f84113)
initcall: 8080e097 (relocated to 87f84097)
initcall: 80801cdd (relocated to 87f77cdd)
initcall: 80834735 (relocated to 87faa735)
initcall: 80815745 (relocated to 87f8b745)
initcall: 8080e08d (relocated to 87f8408d)
initcall: 8080e123 (relocated to 87f84123)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080e11f (relocated to 87f8411f)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080e075 (relocated to 87f84075)
MMC:
initcall: 8080e02d (relocated to 87f8402d)
Loading Environment from FAT... initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080e129 (relocated to 87f84129)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 80815759 (relocated to 87f8b759)
initcall: 8080e023 (relocated to 87f84023)
initcall: 808123dd (relocated to 87f883dd)
initcall: 80801b1f (relocated to 87f77b1f)
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080160d (relocated to 87f7760d)
initcall: 8080e019 (relocated to 87f84019)
initcall: 8080e001 (relocated to 87f84001)
initcall: 80801cf9 (relocated to 87f77cf9)
<ethaddr> not set. Validating first E-fuse MAC
initcall: 8080de19 (relocated to 87f83e19)
initcall: 8080dfed (relocated to 87f83fed)
Net:   Could not get PHY for ethernet@4a100000: addr 0
eth0: ethernet@4a100000
initcall: 8080dfe5 (relocated to 87f83fe5)
Hit any key to stop autoboot:  0
Unknown command 'if test ${boot_fit} -eq 1; then run update_to_fit; fi; run findfdt; run init_console
; run envboot; run distro_bootcmd
' - try 'help' or use 'run' command
=>

How should I solve this problem?

Best,

-Maciej

Hi Hong,

Hong Guan64 said:

These being said, it points to the issue from detection of dtb embedded in u-boot.img file.

We understand the issue, but you must remember It happens 2 per 3 times we boot our custom board.

Hong Guan64 said:

like DDR stability...

1.  We doubled checked values in the EMIF tool. Please find attached EMIF's spreadsheet AM335x_EMIF_Configuration_Tool_v3.xlsx and datasheet to DDR chip, we use(W631GU6MB-12) link
   
   
2.  We used generated GEL files to perform the Scripts->AM335xDDRTests, everything was correct.
   
   
3.  We performed Software leveling. Bellow, please find values.
   1. before optimalization:

      #define  DATA_PHY_RD_DQS_SLAVE_RATIO                     0x00000040
      #define  DATA_PHY_FIFO_WE_SLAVE_RATIO                    0x000000F3
      #define  DATA_PHY_WR_DQS_SLAVE_RATIO                    0x0000007C
      #define  DATA_PHY_WR_DATA_SLAVE_RATIO                    0x000000BC

   2. after optimalization:

      #define  DATA_PHY_RD_DQS_SLAVE_RATIO                     0x0000003A
      #define  DATA_PHY_FIFO_WE_SLAVE_RATIO                    0x00000101
      #define  DATA_PHY_WR_DQS_SLAVE_RATIO                    0x00000083
      #define  DATA_PHY_WR_DATA_SLAVE_RATIO                    0x000000C3

Unfortunately, when we use these values to launch u-boot, it didn't even start. We use DDR configuration with one chip, so it shouldn't be nessesery to perform Software leveling.

How can we troubleshoot these issues?

What can we do to diagnose the reason why our u-boot configuration doesn't boot correctly every time?

Best

Maciej

Hi Maciej,

Note that:

1. #define FDT_MAGIC 0xd00dfeed

2. fdt_blob is appended to the end of u-boot binary, where u-boot end address (_end)  is retrievable from u-boot.sym file. The below is from my u-boot build as an example.

80877af4 g     O .hash 00000000 _end

If JTAG access is available on your setup, some debug points to catch fdt_magic(fdt) != FDT_MAGIC which triggers fdt_check_header() return fails.

- add spin loop at the start of u-boot;

- attach JTAG (no intrusive, just JTAG attach, no power-cycle board);

- check if @fdt_blob (ie, 0x80877af4 in my case) == FDT_MAGIC?

- setup HW breakpoint @fdt_blob (ie, 0x80877af4) triggering upon any RD/WR access;

- break spin loop, and run to see if HW breakpoint is hit to catch any code writing to it.

Best,

-Hong

Hi Hong,

Hong Guan64 said:

fdt_blob is appended to the end of u-boot binary, where u-boot end address (_end)  is retrievable from u-boot.sym file. The below is from my u-boot build as an example.

My u-boot end address is 0x8086A4C4

Hong Guan64 said:

add spin loop at the start of u-boot;

I have placed the spin loop at the end of the function enable_board_pin_mux() in mux.c

At the beginning of the program, the 0x8086A4C4 contains value 0x00000000

When u-boot starts correctly, the 0x8086A4C4 memory location contains value 0xD00DFEED
When u-boot doesn't start correctly, the 0x8086A4C4 memory location contains value 0xF0DD8580

It's important to notice that this value is totally repeatable, each time the u-boot doesn't start correctly we have the same value at this address.

Hong Guan64 said:

- setup HW breakpoint @fdt_blob (ie, 0x80877af4) triggering upon any RD/WR access;

Hong Guan64 said:

- setup HW breakpoint @fdt_blob (ie, 0x80877af4) triggering upon any RD/WR access;

Unfortunately, I was not able to stop the debugger using HW Breakpoint as you stated, so I have no idea which function changes this value.

I'm still not able to find the solution, any help is appreciated.

Can you somehow explain how do we get this 0xF0DD8580 value each time u-boot doesn't start correctly?

Best,

Maciej

Hi Hong,

We added the spin loop at the very early SPl code (board_early_init_f()).

We debugged the code line by line to find the place where the change occurs at the "_end" address.

Before the "spl_load_fit_image()" function, the "_end" address contains value "0xD00DFEED", which is a correct value.

After the "spl_load_fit_image()", the "_end" address contains value "0xF0DD8580", which is not a correct value.

We used the printf function to show how the "spl_fit_append_fdt", "spl_load_fit_image" and "spl_load_simple_fit_ex", are executed.

Bellow, we enclosed files with functions we modified (Keep in mind that "0x8086a4ec" is our "_end" address):

• Fullscreen spl_fit_append_fdt.txt Download

  static int spl_fit_append_fdt(struct spl_image_info *spl_image,
  			      struct spl_load_info *info, ulong sector,
  			      void *fit, int images, ulong base_offset)
  {
      printf("spl_fit_append_fdt: init: %x\n", *(int*)0x8086a4ec);
  	struct spl_image_info image_info;
  	int node, ret, index = 0;
  
  	/* Figure out which device tree the board wants to use */
  	node = spl_fit_get_image_node(fit, images, FIT_FDT_PROP, index++);
  	if (node < 0) {
  		debug("%s: cannot find FDT node\n", __func__);
  		return node;
  	}
  
  	/*
  	 * Read the device tree and place it after the image.
  	 * Align the destination address to ARCH_DMA_MINALIGN.
  	 */
  	image_info.load_addr = spl_image->load_addr + spl_image->size;
  	ret = spl_load_fit_image(info, sector, fit, base_offset, node,
  				 &image_info);
  
  	if (ret < 0)
  		return ret;
  
  	/* Make the load-address of the FDT available for the SPL framework */
  	spl_image->fdt_addr = (void *)image_info.load_addr;
  #if !CONFIG_IS_ENABLED(FIT_IMAGE_TINY)
  #if defined(CONFIG_OF_LIBFDT_OVERLAY)
  	void *tmpbuffer;
  	/*
  	 * allocate 64kB of memory. This will be used to store the DT overlay
  	 * before it is applied. It may not be used depending on how the
  	 * overlay is stored.
  	 */
  	tmpbuffer = malloc(64 * 1024);
  	if (!tmpbuffer)
  		debug("%s: unable to allocate space for overlays\n", __func__);
  
  	for (; ; index++) {
  		node = spl_fit_get_image_node(fit, images, FIT_FDT_PROP, index);
  		if (node == -E2BIG) {
  			debug("%s: No additional FDT node\n", __func__);
  			return 0;
  		} else if (node < 0) {
  			debug("%s: unable to find FDT node %d\n", __func__,
  			      index);
  			continue;
  		}
  
  		image_info.load_addr = (ulong) tmpbuffer;
  		ret = spl_load_fit_image(info, sector, fit, base_offset, node,
  					 &image_info);
  		if (ret < 0)
  			return ret;
  
  		/* Make room in FDT for changes coming from the overlay */
  		ret = fdt_increase_size(spl_image->fdt_addr, image_info.size);
  		if (ret < 0)
  			return ret;
  
  		ret = fdt_overlay_apply_verbose(spl_image->fdt_addr,
  						(void *)image_info.load_addr);
  		if (ret) {
  			pr_err("failed to apply DT overlay %s\n",
  			       fit_get_name(fit, node, NULL));
  			return ret;
  		}
  
  		debug("%s: DT overlay %s applied\n", __func__,
  		      fit_get_name(fit, node, NULL));
  	}
  	if (tmpbuffer)
  		free(tmpbuffer);
  #endif
  	/* Try to make space, so we can inject details on the loadables */
  	ret = fdt_shrink_to_minimum(spl_image->fdt_addr, 8192);
  	if (ret < 0)
  		return ret;
  #endif
  
  	return ret;
  }
  

• Fullscreen spl_load_fit_image.txt Download

  static int spl_load_fit_image(struct spl_load_info *info, ulong sector,
  			      void *fit, ulong base_offset, int node,
  			      struct spl_image_info *image_info)
  {
      printf("spl_load_fit_image: init: %x\n", *(int*)0x8086a4ec);
  	int offset;
  	size_t length;
  	int len;
  	ulong size;
  	ulong load_addr, load_ptr;
  	void *src;
  	ulong overhead;
  	int nr_sectors;
  	int align_len = ARCH_DMA_MINALIGN - 1;
  	uint8_t image_comp = -1, type = -1;
  	const void *data;
  	bool external_data = false;
  
  	if (IS_ENABLED(CONFIG_SPL_FPGA_SUPPORT) ||
  	    (IS_ENABLED(CONFIG_SPL_OS_BOOT) && IS_ENABLED(CONFIG_SPL_GZIP))) {
  		if (fit_image_get_type(fit, node, &type))
  			puts("Cannot get image type.\n");
  		else{
  			debug("%s ", genimg_get_type_name(type));
  			printf("spl_load_fit_image: genimg_get_type_name: %s\n", genimg_get_type_name(type));
  		}
  		printf("spl_load_fit_image: IS_ENABLED(CONFIG_SPL_FPGA_SUPPORT): %x\n", *(int*)0x8086a4ec);
  	}
  	
  
  	if (IS_ENABLED(CONFIG_SPL_OS_BOOT) && IS_ENABLED(CONFIG_SPL_GZIP)) {
  		if (fit_image_get_comp(fit, node, &image_comp))
  			puts("Cannot get image compression format.\n");
  		else{
  			debug("%s ", genimg_get_comp_name(image_comp));
  			printf("spl_load_fit_image: genimg_get_comp_name: %s\n", genimg_get_comp_name(image_comp));
  		}
  		printf("spl_load_fit_image: IS_ENABLED(CONFIG_SPL_OS_BOOT): %x\n", *(int*)0x8086a4ec);
  	}
  	
  	
  	if (fit_image_get_load(fit, node, &load_addr)){
  		load_addr = image_info->load_addr;
  		printf("spl_load_fit_image: fit_image_get_load: %x\n", *(int*)0x8086a4ec);
  	}
  
  	if (!fit_image_get_data_position(fit, node, &offset)) {
  		external_data = true;
  		printf("spl_load_fit_image: fit_image_get_data_position: %x\n", *(int*)0x8086a4ec);
  	} else if (!fit_image_get_data_offset(fit, node, &offset)) {
  		offset += base_offset;
  		external_data = true;
  		printf("spl_load_fit_image: !fit_image_get_data_offset: %x\n", *(int*)0x8086a4ec);
  	}
  	
  
  	if (external_data) {
  		/* External data */
  	    printf("spl_load_fit_image: external_data: %x\n", *(int*)0x8086a4ec);
  		if (fit_image_get_data_size(fit, node, &len)){
  			return -ENOENT;
  		    printf("spl_load_fit_image: fit_image_get_data_size: %x\n", *(int*)0x8086a4ec);
  		}
  
  		load_ptr = (load_addr + align_len) & ~align_len;
  		length = len;
  		printf("spl_load_fit_image: load_addr: %x\n", load_addr);
  		printf("spl_load_fit_image: align_len: %x\n", align_len);
  
  		overhead = get_aligned_image_overhead(info, offset);
  		nr_sectors = get_aligned_image_size(info, length, offset);
  	    //loop
  //	    asm("test: nop");
  //	    asm("       nop");
  //	    asm("  b test ");
  //	    asm(" nop");
  //	    asm(" nop");
  		printf("spl_load_fit_image: info: %x\n", info);
  		printf("spl_load_fit_image: load_ptr: %x\n", load_ptr);
  		printf("spl_load_fit_image: length: %d\n", length);
  		printf("spl_load_fit_image: offset: %d\n", offset);
  		printf("spl_load_fit_image: nr_sectors: %d\n", nr_sectors);
  		if (info->read(info,
  			       sector + get_aligned_image_offset(info, offset),
  			       nr_sectors, (void *)load_ptr) != nr_sectors)
  			return -EIO;
  		printf("spl_load_fit_image: info->read: %x\n", *(int*)0x8086a4ec);
  		debug("External data: dst=%lx, offset=%x, size=%lx\n",
  		      load_ptr, offset, (unsigned long)length);
  		src = (void *)load_ptr + overhead;
  	} else {
  		/* Embedded data */
  		if (fit_image_get_data(fit, node, &data, &length)) {
  			puts("Cannot get image data/size\n");
  			return -ENOENT;
  		}
  		printf("spl_load_fit_image: fit_image_get_data: %x\n", *(int*)0x8086a4ec);
  		debug("Embedded data: dst=%lx, size=%lx\n", load_addr,
  		      (unsigned long)length);
  		src = (void *)data;
  	}
  	printf("spl_load_fit_image: after external_data: %x\n", *(int*)0x8086a4ec);
  
  #ifdef CONFIG_SPL_FIT_SIGNATURE
  	printf("CONFIG_SPL_FIT_SIGNATURE: spl_load_fit_image: %x\n", *(int*)0x8086a4ec);
  	printf("## Checking hash(es) for Image %s ... ",
  	       fit_get_name(fit, node, NULL));
  	if (!fit_image_verify_with_data(fit, node,
  					 src, length))
  		return -EPERM;
  	puts("OK\n");
  #endif
  
  #ifdef CONFIG_SPL_FIT_IMAGE_POST_PROCESS
  	printf("CONFIG_SPL_FIT_IMAGE_POST_PROCESS: spl_load_fit_image: %x\n", *(int*)0x8086a4ec);
  	board_fit_image_post_process(&src, &length);
  #endif
  
  	if (IS_ENABLED(CONFIG_SPL_GZIP) && image_comp == IH_COMP_GZIP) {
  		size = length;
  		if (gunzip((void *)load_addr, CONFIG_SYS_BOOTM_LEN,
  			   src, &size)) {
  			puts("Uncompressing error\n");
  			return -EIO;
  		}
  		length = size;
  		printf("spl_load_fit_image: IS_ENABLED(CONFIG_SPL_GZIP): %x\n", *(int*)0x8086a4ec);
  	} else {
  		memcpy((void *)load_addr, src, length);
  		printf("spl_load_fit_image: memcpy: %x\n", *(int*)0x8086a4ec);
  	}
  	
  
  	if (image_info) {
  		image_info->load_addr = load_addr;
  		image_info->size = length;
  		image_info->entry_point = fdt_getprop_u32(fit, node, "entry");
  	}
  	return 0;
  }
  

• Fullscreen spl_load_simple_fit_ex.txt Download

  int spl_load_simple_fit_ex(struct spl_image_info *spl_image,
  			   struct spl_load_info *info, ulong sector,
  			   void *fit, void *load_only_addr)
  {
      printf("spl_load_simple_fit_ex: init: %x\n", *(int*)0x8086a4ec);
  	int sectors;
  	ulong size;
  	unsigned long count;
  	struct spl_image_info image_info;
  	int node = -1;
  	int images, ret;
  	int base_offset, hsize, align_len = ARCH_DMA_MINALIGN - 1;
  	int index = 0;
  
  	/*
  	 * For FIT with external data, figure out where the external images
  	 * start. This is the base for the data-offset properties in each
  	 * image.
  	 */
  	size = fdt_totalsize(fit);
  	size = (size + 3) & ~3;
  	size = board_spl_fit_size_align(size);
  	base_offset = (size + 3) & ~3;
  
  	/*
  	 * So far we only have one block of data from the FIT. Read the entire
  	 * thing, including that first block, placing it so it finishes before
  	 * where we will load the image.
  	 *
  	 * Note that we will load the image such that its first byte will be
  	 * at the load address. Since that byte may be part-way through a
  	 * block, we may load the image up to one block before the load
  	 * address. So take account of that here by subtracting an addition
  	 * block length from the FIT start position.
  	 *
  	 * In fact the FIT has its own load address, but we assume it cannot
  	 * be before CONFIG_SYS_TEXT_BASE.
  	 *
  	 * For FIT with data embedded, data is loaded as part of FIT image.
  	 * For FIT with external data, data is not loaded in this step.
  	 */
  
  	/*
  	 * If load_only_addr is set this means we are operating in "loader
  	 * mode", in which we a) force a specific destination address, and b)
  	 * skip the standard post-loading image processing steps.
  	 */
  	if (load_only_addr) {
  		fit = load_only_addr;
  	} else {
  		hsize = (size + info->bl_len + align_len) & ~align_len;
  		fit = spl_get_load_buffer(-hsize, hsize);
  	}
  	printf("spl_load_simple_fit_ex: load_only_addr: %x\n", *(int*)0x8086a4ec);
  
  	sectors = get_aligned_image_size(info, size, 0);
  	count = info->read(info, sector, sectors, fit);
  	debug("fit read sector %lx, sectors=%d, dst=%p, count=%lu, size=0x%lx\n",
  	      sector, sectors, fit, count, size);
  	printf("spl_load_simple_fit_ex: info->read: %x\n", *(int*)0x8086a4ec);
  	if (count == 0)
  		return -EIO;
  
  	/*
  	 * Perform early successful exit if we are in "loader mode", meaning
  	 * extracting the (U-Boot) image is not desired.
  	 */
  	if (load_only_addr)
  		return 0;
  
  	/* find the node holding the images information */
  	images = fdt_path_offset(fit, FIT_IMAGES_PATH);
  	if (images < 0) {
  		debug("%s: Cannot find /images node: %d\n", __func__, images);
  		return -1;
  	}
  
  #ifdef CONFIG_SPL_FPGA_SUPPORT
  	node = spl_fit_get_image_node(fit, images, "fpga", 0);
  	printf("spl_load_simple_fit_ex: spl_fit_get_image_node: %x\n", *(int*)0x8086a4ec);
  	if (node >= 0) {
  		/* Load the image and set up the spl_image structure */
  		ret = spl_load_fit_image(info, sector, fit, base_offset, node,
  					 spl_image);
  		printf("spl_load_simple_fit_ex: spl_load_fit_image(fpga): %x\n", *(int*)0x8086a4ec);
  		if (ret) {
  			printf("%s: Cannot load the FPGA: %i\n", __func__, ret);
  			return ret;
  		}
  
  		debug("FPGA bitstream at: %x, size: %x\n",
  		      (u32)spl_image->load_addr, spl_image->size);
  
  		ret = fpga_load(0, (const void *)spl_image->load_addr,
  				spl_image->size, BIT_FULL);
  		if (ret) {
  			printf("%s: Cannot load the image to the FPGA\n",
  			       __func__);
  			return ret;
  		}
  
  		puts("FPGA image loaded from FIT\n");
  		node = -1;
  	}
  #endif
  
  	/*
  	 * Find the U-Boot image using the following search order:
  	 *   - start at 'firmware' (e.g. an ARM Trusted Firmware)
  	 *   - fall back 'kernel' (e.g. a Falcon-mode OS boot
  	 *   - fall back to using the first 'loadables' entry
  	 */
  	if (node < 0)
  		node = spl_fit_get_image_node(fit, images, FIT_FIRMWARE_PROP,
  					      0);
  #ifdef CONFIG_SPL_OS_BOOT
  	printf("spl_load_simple_fit_ex: CONFIG_SPL_OS_BOOT: %x\n", *(int*)0x8086a4ec);
  	if (node < 0)
  		node = spl_fit_get_image_node(fit, images, FIT_KERNEL_PROP, 0);
  	printf("spl_load_simple_fit_ex: spl_fit_get_image_node(FIT_KERNEL_PROP): %x\n", *(int*)0x8086a4ec);
  #endif
  	if (node < 0) {
  		debug("could not find firmware image, trying loadables...\n");
  		node = spl_fit_get_image_node(fit, images, "loadables", 0);
  		printf("spl_load_simple_fit_ex: spl_fit_get_image_node(loadables): %x\n", *(int*)0x8086a4ec);
  		/*
  		 * If we pick the U-Boot image from "loadables", start at
  		 * the second image when later loading additional images.
  		 */
  		index = 1;
  	}
  	if (node < 0) {
  		debug("%s: Cannot find u-boot image node: %d\n",
  		      __func__, node);
  		return -1;
  	}
  
  	/* Load the image and set up the spl_image structure */
  	ret = spl_load_fit_image(info, sector, fit, base_offset, node,
  				 spl_image);
  	printf("spl_load_simple_fit_ex: spl_load_fit_image: %x\n", *(int*)0x8086a4ec);
  	if (ret)
  		return ret;
  
  	/*
  	 * For backward compatibility, we treat the first node that is
  	 * as a U-Boot image, if no OS-type has been declared.
  	 */
  	if (!spl_fit_image_get_os(fit, node, &spl_image->os)){
  		debug("Image OS is %s\n", genimg_get_os_name(spl_image->os));
  	    printf("spl_load_simple_fit_ex: spl_fit_image_get_os: %x\n", *(int*)0x8086a4ec);
  	}
  #if !defined(CONFIG_SPL_OS_BOOT)
  	else
  		spl_image->os = IH_OS_U_BOOT;
  #endif
  
  	/*
  	 * Booting a next-stage U-Boot may require us to append the FDT.
  	 * We allow this to fail, as the U-Boot image might embed its FDT.
  	 */
      printf("spl_load_simple_fit_ex: spl_image->os: %d\n", spl_image->os);
  	
  	if (spl_image->os == IH_OS_U_BOOT){
  		spl_fit_append_fdt(spl_image, info, sector, fit,
  				   images, base_offset);
  		printf("spl_load_simple_fit_ex: spl_fit_append_fdt: %x\n", *(int*)0x8086a4ec);
  	}
  	
  	/* Now check if there are more images for us to load */
  	for (; ; index++) {
  		uint8_t os_type = IH_OS_INVALID;
  
  		node = spl_fit_get_image_node(fit, images, "loadables", index);
  		if (node < 0)
  			break;
  
  		ret = spl_load_fit_image(info, sector, fit, base_offset, node,
  					 &image_info);
  		if (ret < 0)
  			continue;
  
  		if (!spl_fit_image_get_os(fit, node, &os_type))
  			debug("Loadable is %s\n", genimg_get_os_name(os_type));
  
  		if (os_type == IH_OS_U_BOOT) {
  			spl_fit_append_fdt(&image_info, info, sector,
  					   fit, images, base_offset);
  			spl_image->fdt_addr = image_info.fdt_addr;
  		}
  
  		/*
  		 * If the "firmware" image did not provide an entry point,
  		 * use the first valid entry point from the loadables.
  		 */
  		if (spl_image->entry_point == FDT_ERROR &&
  		    image_info.entry_point != FDT_ERROR)
  			spl_image->entry_point = image_info.entry_point;
  
  		/* Record our loadables into the FDT */
  		if (spl_image->fdt_addr)
  			spl_fit_record_loadable(fit, images, index,
  						spl_image->fdt_addr,
  						&image_info);
  	}
  
  	/*
  	 * If a platform does not provide CONFIG_SYS_UBOOT_START, U-Boot's
  	 * Makefile will set it to 0 and it will end up as the entry point
  	 * here. What it actually means is: use the load address.
  	 */
  	if (spl_image->entry_point == FDT_ERROR || spl_image->entry_point == 0)
  		spl_image->entry_point = spl_image->load_addr;
  
  	spl_image->flags |= SPL_FIT_FOUND;
  
  #ifdef CONFIG_SECURE_BOOT
  	board_spl_fit_post_load((ulong)fit, size);
  #endif
  	return 0;
  }
  

These are the results we had, during the debug session of our u-boot, including added printf functions.

spl_load_simple_fit_ex: init: edfe0dd0
spl_load_simple_fit_ex: load_only_addr: edfe0dd0
spl_load_simple_fit_ex: info->read: edfe0dd0
spl_load_simple_fit_ex: CONFIG_SPL_OS_BOOT: edfe0dd0
spl_load_simple_fit_ex: spl_fit_get_image_node(FIT_KERNEL_PROP): edfe0dd0
spl_load_fit_image: init: edfe0dd0
spl_load_fit_image: !fit_image_get_data_offset: edfe0dd0
spl_load_fit_image: external_data: edfe0dd0
spl_load_fit_image: load_addr: 80800000
spl_load_fit_image: align_len: 3f
spl_load_fit_image: info: 81fffe5c
spl_load_fit_image: load_ptr: 80800000
spl_load_fit_image: length: 435436
spl_load_fit_image: offset: 872
spl_load_fit_image: nr_sectors: 435476
spl_load_fit_image: info->read: 8085de10
spl_load_fit_image: after external_data: 8085de10
spl_load_fit_image: memcpy: 8085de10
spl_load_simple_fit_ex: spl_load_fit_image: 8085de10
spl_load_simple_fit_ex: spl_fit_image_get_os: 8085de10
spl_load_simple_fit_ex: spl_image->os: 0

--------------------------

spl_load_simple_fit_ex: init: edfe0dd0
spl_load_simple_fit_ex: load_only_addr: edfe0dd0
spl_load_simple_fit_ex: info->read: edfe0dd0
spl_load_simple_fit_ex: CONFIG_SPL_OS_BOOT: edfe0dd0
spl_load_simple_fit_ex: spl_fit_get_image_node(FIT_KERNEL_PROP): edfe0dd0
spl_load_fit_image: init: edfe0dd0
spl_load_fit_image: !fit_image_get_data_offset: edfe0dd0
spl_load_fit_image: external_data: edfe0dd0
spl_load_fit_image: load_addr: 80800000
spl_load_fit_image: align_len: 3f
spl_load_fit_image: info: 81fffe5c
spl_load_fit_image: load_ptr: 80800000
spl_load_fit_image: length: 435436
spl_load_fit_image: offset: 872
spl_load_fit_image: nr_sectors: 435476
spl_load_fit_image: info->read: 8085de10
spl_load_fit_image: after external_data: 8085de10
spl_load_fit_image: memcpy: 8085de10
spl_load_simple_fit_ex: spl_load_fit_image: 8085de10
spl_load_simple_fit_ex: spl_fit_image_get_os: 8085de10
spl_load_simple_fit_ex: spl_image->os: 17
spl_fit_append_fdt: init: 8085de10
spl_load_fit_image: init: 8085de10
spl_load_fit_image: fit_image_get_load: 8085de10
spl_load_fit_image: !fit_image_get_data_offset: 8085de10
spl_load_fit_image: external_data: 8085de10
spl_load_fit_image: load_addr: 8086a4ec
spl_load_fit_image: align_len: 3f
spl_load_fit_image: info: 81fffe5c
spl_load_fit_image: load_ptr: 8086a500
spl_load_fit_image: length: 54760
spl_load_fit_image: offset: 436308
spl_load_fit_image: nr_sectors: 54780
spl_load_fit_image: info->read: 8085de10
spl_load_fit_image: after external_data: 8085de10
spl_load_fit_image: memcpy: edfe0dd0
spl_load_simple_fit_ex: spl_fit_append_fdt: edfe0dd0

Please notice that when the u-boot starts correctly, the "spl_load_fit_image()" function is called twice, and when It's not starting correctly, the "spl_load_fit_image()" function is called only once.

We found that spl_image->os contains different values. It has "u-boot" when it starts correctly and has an "invalid os" value when it doesn't.

The questions are:

• Why the "spl_load_fit_image()" function is called twice or only once?
• Why do we have a different value in spl_image->os variable?
• Additionally, how to disable Watchdog that resets the debug session?

Hi Hong

I would like to join the conversation.
Thank you for the detailed answer.
Unfortunately, we still don't know the reason for the random behavior we experience on our custom board.

Why do we experience such random behavior?
Is it a problem with SDRAM settings?
or
Is it a misconfiguration of our U-boot instance?

What are the next steps we can conduct to find the issue?

Why do we get different values in spl_image->os?
When the u-boot starts correctly we get 17 (U-BOOT) but when it doesn't start correctly we get 0 (INVALID OS).

/*
* For backward compatibility, we treat the first node that is
* as a U-Boot image, if no OS-type has been declared.
*/
if (!spl_fit_image_get_os(fit, node, &spl_image->os)){
     debug("Image OS is %s\n", genimg_get_os_name(spl_image->os));
}

What can we do to get rid of this unwanted behavior?

What is even more interesting, when I overwrite spl_image->os (as below), I get a 100% correct booting.

/*
 * For backward compatibility, we treat the first node that is
 * as a U-Boot image, if no OS-type has been declared.
 */
if (!spl_fit_image_get_os(fit, node, &spl_image->os)){
    debug("Image OS is %s\n", genimg_get_os_name(spl_image->os));
}
#if !defined(CONFIG_SPL_OS_BOOT)
else
    spl_image->os = IH_OS_U_BOOT;
#endif

spl_image->os = IH_OS_U_BOOT; // added line to overwrite decoded variable

Regards
Adam