PROCESSOR-SDK-AM335X: "omap-elm: uncorrectable ECC errors" in U-Boot SPL

OneLoop

Part Number: PROCESSOR-SDK-AM335X
Other Parts Discussed in Thread: AM3356

Hi everyone,

I'm struggling with a weird NAND ECC error reported by U-Boot SPL on an AM335x custom board.

U-Boot SPL continuously reports a huge amount of "omap-elm: uncorrectable ECC errors" when booting from NAND, however they seem no real errors, since SPL is able to correctly load U-Boot proper and then U-Boot proper correctly loads the Linux Kernel.

Here's a reduced version of U-Boot output:

U-Boot SPL 2021.01-g9e33c11f9e (Jan 05 2023 - 23:24:05 +0000)
Trying to boot from NAND
omap-elm: uncorrectable ECC errors
omap-elm: uncorrectable ECC errors
omap-elm: uncorrectable ECC errors
omap-elm: uncorrectable ECC errors
.... [REPEATED 1789 TIMES!]


U-Boot 2021.01-g9e33c11f9e (Jan 05 2023 - 23:24:05 +0000)

CPU  : AM335X-GP rev 2.1
Model: Automata AM335x F6 PAC
DRAM:  512 MiB
WDT:   Started with servicing (60s timeout)
NAND:  1024 MiB
Loading Environment from NAND... OK
Net:   eth2: ethernet@4a100000
Hit any key to stop autoboot:  2
=> saveenv
Saving Environment to NAND... Erasing redundant NAND...

Erasing at 0x1e0000 -- 100% complete.
Writing to redundant NAND... OK
OK
=> saveenv
Saving Environment to NAND... Erasing NAND...

Erasing at 0x1c0000 -- 100% complete.
Writing to NAND... OK
OK
=> nand info

Device 0: nand0, sector size 128 KiB
  Page size       2048 b
  OOB size          64 b
  Erase size    131072 b
  subpagesize      512 b
  options     0x4000400c
  bbt options 0x00008000
=>

Some details about my setup:

Processor: AM3356
NAND Chip is a Micron MT29F8G08ADADAH4-IT:D (SLC 8G 1GX8)
Official TI's U-Boot: https://git.ti.com/git/ti-u-boot/ti-u-boot.git (branch ti-u-boot-2021.01)
Build system: Yocto Dunfell 3.1.24

In order to enable support for my custom board, I followed the U-Boot Board Port Guide available here.

Initial pin muxing is done in my mux.c file:

static struct module_pin_mux nand_pin_mux[] = {
	{OFFSET(gpmc_ad0), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD0 */
	{OFFSET(gpmc_ad1), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD1 */
	{OFFSET(gpmc_ad2), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD2 */
	{OFFSET(gpmc_ad3), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD3 */
	{OFFSET(gpmc_ad4), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD4 */
	{OFFSET(gpmc_ad5), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD5 */
	{OFFSET(gpmc_ad6), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD6 */
	{OFFSET(gpmc_ad7), (MODE(0) | PULLUP_EN | RXACTIVE)},    /* NAND AD7 */
	{OFFSET(gpmc_wait0), (MODE(0) | RXACTIVE | PULLUP_EN)},  /* NAND WAIT */
	{OFFSET(gpmc_wpn), (MODE(7) | PULLUP_EN | RXACTIVE)},    /* NAND_WPN */
	{OFFSET(gpmc_csn0), (MODE(0) | PULLUDEN)},               /* NAND_CS0 */
	{OFFSET(gpmc_advn_ale), (MODE(0) | PULLUDEN)},           /* NAND_ADV_ALE */
	{OFFSET(gpmc_oen_ren), (MODE(0) | PULLUDEN)},            /* NAND_OE */
	{OFFSET(gpmc_wen), (MODE(0) | PULLUDEN)},                /* NAND_WEN */
	{OFFSET(gpmc_be0n_cle), (MODE(0) | PULLUDEN)},           /* NAND_BE_CLE */
	{-1},
};

U-Boot is configured like this (only the relevant parts are shown):

Board header file (include/configs/am335x_f6.h)

#ifdef CONFIG_MTD_RAW_NAND
/* NAND: device related configs */
# define CONFIG_SYS_NAND_5_ADDR_CYCLE
# define CONFIG_SYS_NAND_PAGE_COUNT	(CONFIG_SYS_NAND_BLOCK_SIZE / \
					 CONFIG_SYS_NAND_PAGE_SIZE)
# define CONFIG_SYS_NAND_PAGE_SIZE	2048
# define CONFIG_SYS_NAND_OOBSIZE	64
# define CONFIG_SYS_NAND_BLOCK_SIZE	(128*1024)
/* NAND: driver related configs */
# define CONFIG_SYS_NAND_BAD_BLOCK_POS	NAND_LARGE_BADBLOCK_POS
# define CONFIG_SYS_NAND_ECCPOS		{ 2,  3,  4,  5,  6,  7,  8,  9, \
					 10, 11, 12, 13, 14, 15, 16, 17, \
					 18, 19, 20, 21, 22, 23, 24, 25, \
					 26, 27, 28, 29, 30, 31, 32, 33, \
					 34, 35, 36, 37, 38, 39, 40, 41, \
					 42, 43, 44, 45, 46, 47, 48, 49, \
					 50, 51, 52, 53, 54, 55, 56, 57, }

# define CONFIG_SYS_NAND_ECCSIZE	512
# define CONFIG_SYS_NAND_ECCBYTES	14
# define CONFIG_SYS_NAND_ONFI_DETECTION
# define CONFIG_NAND_OMAP_ECCSCHEME	OMAP_ECC_BCH8_CODE_HW
# define CONFIG_SYS_NAND_U_BOOT_OFFS	0x000c0000 /* u-boot proper offset 512KiB */
/* NAND: SPL related configs */
# ifdef CONFIG_SPL_OS_BOOT	/* Falcon Mode */
#  define CONFIG_SYS_NAND_SPL_KERNEL_OFFS	0x00200000 /* kernel offset 2MiB */
# endif
#endif /* !CONFIG_MTD_RAW_NAND */

Kconfig (.config)

CONFIG_NAND_OMAP_GPMC=y
CONFIG_NAND_OMAP_GPMC_PREFETCH=y
CONFIG_NAND_OMAP_ELM=y
CONFIG_SPL_NAND_AM33XX_BCH=y

CONFIG_SPL_NAND_SUPPORT=y
CONFIG_SPL_NAND_DRIVERS=y
CONFIG_SPL_NAND_ECC=y
# CONFIG_SPL_NAND_SIMPLE is not set
CONFIG_SPL_NAND_BASE=y

CONFIG_DM=y
CONFIG_SPL_DM=y
CONFIG_OF_CONTROL=y
# CONFIG_SPL_OF_CONTROL is not set

DTS used by U-Boot Proper

/* NAND */
&am33xx_pinmux {
	nand_flash_pins: nand_flash_pins {
		pinctrl-single,pins = <
			AM33XX_IOPAD(0x81c, PIN_INPUT | MUX_MODE0)		/* (T9) gpmc_ad7.gpmc_ad7 */
			AM33XX_IOPAD(0x818, PIN_INPUT | MUX_MODE0)		/* (R9) gpmc_ad6.gpmc_ad6 */
			AM33XX_IOPAD(0x814, PIN_INPUT | MUX_MODE0)		/* (V8) gpmc_ad5.gpmc_ad5 */
			AM33XX_IOPAD(0x810, PIN_INPUT | MUX_MODE0)		/* (U8) gpmc_ad4.gpmc_ad4 */
			AM33XX_IOPAD(0x80c, PIN_INPUT | MUX_MODE0)		/* (T8) gpmc_ad3.gpmc_ad3 */
			AM33XX_IOPAD(0x808, PIN_INPUT | MUX_MODE0)		/* (R8) gpmc_ad2.gpmc_ad2 */
			AM33XX_IOPAD(0x804, PIN_INPUT | MUX_MODE0)		/* (V7) gpmc_ad1.gpmc_ad1 */
			AM33XX_IOPAD(0x800, PIN_INPUT | MUX_MODE0)		/* (U7) gpmc_ad0.gpmc_ad0 */
			AM33XX_IOPAD(0x870, PIN_INPUT | MUX_MODE0)		/* (T17) gpmc_wait0.gpmc_wait0 */
			AM33XX_IOPAD(0x87c, PIN_OUTPUT | MUX_MODE0)		/* (V6) gpmc_csn0.gpmc_csn0 */
			AM33XX_IOPAD(0x88c, PIN_INPUT | MUX_MODE0)		/* (V12) gpmc_clk.gpmc_clk */
			AM33XX_IOPAD(0x890, PIN_OUTPUT | MUX_MODE0)		/* (R7) gpmc_advn_ale.gpmc_advn_ale */
			AM33XX_IOPAD(0x894, PIN_OUTPUT | MUX_MODE0)		/* (T7) gpmc_oen_ren.gpmc_oen_ren */
			AM33XX_IOPAD(0x898, PIN_OUTPUT | MUX_MODE0)		/* (U6) gpmc_wen.gpmc_wen */
			AM33XX_IOPAD(0x89c, PIN_OUTPUT | MUX_MODE0)		/* (T6) gpmc_be0n_cle.gpmc_be0n_cle */
		>;
	};
};

&elm {
	status = "okay";
};

&gpmc {
	pinctrl-names = "default";
	pinctrl-0 = <&nand_flash_pins>;
	ranges = <0 0 0x08000000 0x1000000>;	/* CS0: 16MB for NAND */
	status = "okay";
	nand@0,0 {
		compatible = "ti,omap2-nand";
		reg = <0 0 4>; /* CS0, offset 0, IO size 4 */
		interrupt-parent = <&gpmc>;
		interrupts = <0 IRQ_TYPE_NONE>, /* fifoevent */
		             <1 IRQ_TYPE_NONE>; /* termcount */
		/* Rule: 2 + (PAGESIZE / 512) * ECC_BYTES <= OOBSIZE */
		ti,nand-ecc-opt = "bch8"; /* page size: 2048, OOB size: 64 => ok ECC_BYTES: 14 (bch8) */
		ti,nand-xfer-type = "prefetch-polled";
		ti,elm-id = <&elm>;
					 
		/* NAND generic properties */
		nand-bus-width = <8>;
		rb-gpios = <&gpmc 0 GPIO_ACTIVE_HIGH>; /* gpmc_wait0 */
		
		/* GPMC properties */
		gpmc,device-nand = "true";
		gpmc,device-width = <1>;
		gpmc,sync-clk-ps = <0>;
		gpmc,cs-on-ns = <0>;
		gpmc,cs-rd-off-ns = <40>;
		gpmc,cs-wr-off-ns = <40>;
		gpmc,adv-on-ns = <0>;
		gpmc,adv-rd-off-ns = <40>;
		gpmc,adv-wr-off-ns = <40>;
		gpmc,we-on-ns = <0>;
		gpmc,we-off-ns = <20>;
		gpmc,oe-on-ns = <0>;
		gpmc,oe-off-ns = <20>;
		gpmc,access-ns = <30>;
		gpmc,rd-cycle-ns = <40>;
		gpmc,wr-cycle-ns = <40>;
		gpmc,wait-on-read = "true";
		gpmc,wait-on-write = "true";
		gpmc,bus-turnaround-ns = <0>;
		gpmc,cycle2cycle-delay-ns = <90>;
		gpmc,cycle2cycle-diffcsen;
		gpmc,clk-activation-ns = <0>;
		gpmc,wr-access-ns = <40>;
		gpmc,wr-data-mux-bus-ns = <0>;

		#address-cells = <1>;
		#size-cells = <1>;

		/* Partitions */
		/* Defined in U-Boot environment variable mtdparts */
	};
};

/* Cryptography modules */
/* &sham is required by NAND */
&sham {
	status = "okay";
};

Please note that I've enabled "Driver Model" (CONFIG_DM / CONFIG_SPL_DM) for both SPL and U-Boot Proper while "Run-time configuration via Device Tree" (CONFIG_OF_CONTROL) is enabled only for U-Boot Proper. This reflects the settings found in the am335x_evm template.

Do you have any idea? I think the problem is related to a configuration mismatch between SPL and U-Boot Proper but I can't spot it.
Or maybe, is there any known bug in omap-elm.c or am335x_spl_bch.c when the SPL build is performed?

Thank you and if you need any more information, don’t hesitate to ask.

Best regards,

OneLoop

Edited Mar 27, 2023 02:55 PM GMT: add nand info output to the first code block.

over 2 years ago

0 Hong Guan64 over 2 years ago

TI__Guru 70900 points

Hello,
1. do we know if the NAND device has an internal built-in ECC?
2. What is the output from "nand info" cmd @u-boot
3. If you're able to boot kernel, what is the output "dmesg | grep -i nand"

AM335x TRM <26.1.8.4 NAND> is a good reference on NAND ECC supoort by ROM.
ECC on AM335x SoC support is using the GPMC and ELM hardware.
In addition ECC computation done by the ROM can be turned off completely by using SYSBOOT[9]. This is particularly useful when interfacing with NAND devices that have built in ECC engines.

One reference on BCH scheme vs OOB size in u-boot/kernel
www.kernel.org/.../gpmc-nand.txt

Best,
-Hong

0 Leon Pollak over 2 years ago in reply to Hong Guan64

Intellectual 960 points

Please, I have exactly the same problem! I kindly ask you to look at:
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1215236/can-t-load-environment-and-u-boot-from-nand-by-spl---bug-suspected

As for your questions:
1. Yes, it has, but HW people say that it is disabled by default. The chip is MT29F2G08ABAEAWP-IT
2. uboot@nand info
Device 0: nand0, sector size 128 KiB
Page size 2048 b
OOB size 64 b
Erase size 131072 b
subpagesize 512 b
options 0x4000400c
bbt options 0x00008000
3. I still didn't tried to boot the kernel as I am struggling with this problem for 2 weeks already. But I don't think there will be problems, as everything works fine in u-boot from NAND and UART as it may be seen from my post (link in the head).

PLEASE, HELP!!!!!!!!!!!!!!!

0 Leon Pollak over 2 years ago

Intellectual 960 points

Dear OneLoop,

I have the exactly same problem and will be very thankful to you for the help. It looks like you have solved it.

I am ready to pay your for the solution.

Please, reply!

Processors

Processors forum

PROCESSOR-SDK-AM335X: "omap-elm: uncorrectable ECC errors" in U-Boot SPL