need help with NAND ECC

Arno:

Currently the OMAP3 PSP 04.02.00.07 has an issue with the u-boot and kernel not using the

same offset in NAND spare area for ECC parity data.  We are actively working to resolve this issue.

See this post for more details:

http://e2e.ti.com/support/dsp/sitara_arm174_microprocessors/f/416/p/116753/449341.aspx#449341

Regards,

Michael T

PS: Please mark this post as answered via the Verify Answer button below if you think it answers your question.  Thanks!

How can I switch back to Software ECC in kernel?

I found that this is set in omap2.c:
       pdata->ecc_opt          = OMAP_ECC_HAMMING_CODE_HW;
Unfortunatly setting it back to
       pdata->ecc_opt          = OMAP_ECC_HAMMING_CODE_DEFAULT;
(which is the setting for SW-ECC) doesn't prevent me from getting messages like:

uncorrectable error :
uncorrectable error :
mtd->read(0xd18 bytes from 0x7eae8) returned ECC error

So what else has to be set? I couldn't found it. Of course in this case I also used a uboot with SW-ECC handling (my previous u-boot).

What do you think is the difference in performance of SW vs HW-ECC while reading files from flash? Noticable?

- Arno

Arno:

See ECC user's guide.  For kernel also need to change board-flash.c

http://processors.wiki.ti.com/index.php/AM37x_EVM_Software_Developer%27s_Guide#Error_Correction_.28ECC.29_User.27s_Guide

Regards,

Michael T

Thanks, this are good hints. But I have still 3 questions:

a)
If I want to change uboot out of linux, do I have to use everywhere SW-ECC? As uboot and kernel use (as I understand this) different HW-ECC (type 1 or 2)

b)
When I do "save" in uboot-environment - what scheme is used then? As far as I could see I can't influence it.  But from my understanding I have to. The uboot-tool fw_setconfig uses the configuration of the linux kernel, and this can be according to the link you send me:

• "OMAP_ECC_HAMMING_CODE_HW"
  
• "OMAP_ECC_BCH4_CODE_HW"
  
• "OMAP_ECC_BCH8_CODE_HW"

And how to set SW-ECC in kernel? This is not described in this document.

c:
In the ECC description for kernel ECC-BCH4/8 is described as HW-ECC, but they have SW in its suffixes.  Is this a typo ?

Usage:
nandecc [hw 1/hw 2/sw/bch4_sw/bch8_sw] - Switch between NAND hardware for kernel/FS layout (hw 1), hardware for xloader/uboot layout (hw 2), 1-bit software (sw), 4-bit software (bch4_sw), or 8-bit software (bch8_sw) ecc algorithm

Thanks, this are good hints. But I have still 3 questions:

a)
If I want to change uboot out of linux, do I have to use everywhere SW-ECC? As uboot and kernel use (as I understand this) different HW-ECC (type 1 or 2)

b)
When I do "save" in uboot-environment - what scheme is used then? As far as I could see I can't influence it.  But from my understanding I have to. The uboot-tool fw_setconfig uses the configuration of the linux kernel, and this can be according to the link you send me:

• "OMAP_ECC_HAMMING_CODE_HW"
  
• "OMAP_ECC_BCH4_CODE_HW"
  
• "OMAP_ECC_BCH8_CODE_HW"

And how to set SW-ECC in kernel? This is not described in this document.

c:
In the ECC description for kernel ECC-BCH4/8 is described as HW-ECC, but they have SW in its suffixes.  Is this a typo ?

Usage:
nandecc [hw 1/hw 2/sw/bch4_sw/bch8_sw] - Switch between NAND hardware for kernel/FS layout (hw 1), hardware for xloader/uboot layout (hw 2), 1-bit software (sw), 4-bit software (bch4_sw), or 8-bit software (bch8_sw) ecc algorithm

Hi Amo ,

This has two problems ,

First of all the u-boot nand environment should be matched with that of kernel's.

For this you can do following changes in your u-boot code :

File : include/configs/omap3_evm.h

- #define SMNAND_ENV_OFFSET               0x260000 /* environment starts here */

+#define SMNAND_ENV_OFFSET               0x1C0000 /* environment starts here */

Secondly,

When nand is flashed from linux and read from u-boot :

This is basically hw_ecc layout mismatch between u-boot and kernel.

To solve this make following changes in u-boot :

--- a/arch/arm/include/asm/arch-omap3/omap_gpmc.h

+++ b/arch/arm/include/asm/arch-omap3/omap_gpmc.h

@@ -58,13 +58,12 @@

 }

 /* NAND device layout in synch with the kernel */

 #define GPMC_NAND_HW_ECC_LAYOUT_KERNEL {\

-       .eccbytes = 12,\

-       .eccpos = {\

-                  40, 41, 42, 43, 44, 45, 46, 47,\

-                  48, 49, 50, 51},\

+ .eccbytes = 12,\

+ .eccpos = {52,53, 54, 55, 56, 57, 58,\

+ 59, 60, 61, 62, 63},\

        .oobfree = {\

                {.offset = 2,\

-                .length = 38} } \

+                .length = 50} } \

 }

 #endif

 Also make following changes in kernel : 

--- a/arch/arm/mach-omap2/gpmc.c

+++ b/arch/arm/mach-omap2/gpmc.c

@@ -858,7 +858,7 @@ int gpmc_enable_hwecc(int ecc_type, int cs, int mode,

                        bch_mod = 1;

                        bch_wrapmode = 0x04;

                } else

-                       eccsize1 = ((ecc_size >> 1) - 1) << 22;

+                       eccsize1 = ((ecc_size >> 1) - 1);

                break;

        case GPMC_ECC_READSYN:

@@ -874,7 +874,9 @@ int gpmc_enable_hwecc(int ecc_type, int cs, int mode,

                        bch_mod = 1;

                        bch_wrapmode = 0x06;

                } else

-                       eccsize1 = ((ecc_size >> 1) - 1) << 22;

+                       //eccsize1 = ((ecc_size >> 1) - 1) << 22;

+                       eccsize1 = ((ecc_size >> 1) - 1);

+                       

                break;

        default:

Thanks Hrishikesh, this really helps. I could now use HW-ECC and modify uboot environment out of linux.

What I found really strange: Before this changes you describe I could write the mtd partitions from uboot with nandecc hw 1 and nandecc hw 2 - and it boots right. And I even checked that OOB bytes has been different.

Thanks
-Arno

Welcome,

Forgot to mention Abhilash KV <abhilash.kv@ti.com> who had fixed eccsize left shifting problem in kernel.

Amo,

Can you please give me more info on the lines : " Before this changes you describe I could write the mtd partitions from uboot with nandecc hw 1 and nandecc hw 2 - and it boots right. And I even checked that OOB bytes has been different." so that I could help you.

Thanks,

Hrishikesh

Just to explain. I did use xloader, uboot, kernel from http://arago-project.org/git/projects/. That versions that have been tagged PSP04.02.00.07 (so somewhat 3 month old). My board is slightly different to the EVM board, but I take this as base for doing changes for my board. But I keep everything as close as possible to the original code.

" Before this changes you describe I could write the mtd partitions from uboot with nandecc hw 1 and nandecc hw 2 - and it boots right. And I even checked that OOB bytes has been different."

This means: Usually (during development) I write my nand-mtd-partitions (jffs2) via uboot. But: I could write my root partition using 'nandecc hw 1' or 'nandecc hw 2'. Both gave me a different OOB layout, but nevertheless both will boot !?! Only If I try write the partion with 'nandecc sw' - this fails. Again, this was before adding the shift patch. But I was really confused by this behaviour.

Unfortunatly I am not finished with this ECC topic. I can't find a match so that x-loader can start uboot AND I can write uboot out of linux. I need this functionality, to update the system out of linux. As previous everything except X-loader used SW-ECC, it was not a problem. I compiled x-loader for using HW-ECC and SW-ECC. I expect, this is just effecting the loading of uboot itself. So it seems I did not get it to compile X-loader in that way, so that it can load a uboot, written by linux.

From what I believe is, that uboot has to be written by "hw 2" to become readable for x-loader. But it will be written with "hw 1" out of linux. I still have to verify this.

Thanks and best regards
-Arno

For the "u-boot" image written out of linux-kernel you need to first compile the kernel configured for OMAP_ECC_HAMMING_CODE_HW_ROMCODE instead of OMAP_ECC_HAMMING_CODE_HW . This is as good as configuring kernel to write using hw2 .

More info on where to change the macros : 

http://processors.wiki.ti.com/index.php/AM35x-OMAP35x-PSP_04.02.00.07_UserGuide#Selecting_NAND_ECC_scheme_for_Linux_Kernel

Thanks,

Hrishikesh

It seem's to be a nightmare.

This is the requirement

• uboot can read / save uboot environment
• uboot can read / write all available partitions (xloader, uboot, env, kernel, linux-part)
• kernel can read linux partitions
• partitions should be readable without a problem after modifying them in linux
• linux should write uboot, envrionent and all partitions.

For this I need a complete chain with same ECC scheme (except xloader). As far as I understand the documents you refer, it is not possible to configure x-loader in "hw 1" mode. It has just  "1 bit HW ECC" or "1 bit SW ECC", while uboot offers 2 kind of "1 bit HW-ECC".  So the only chance is either SW-ECC (as I did in previous versions) or hw 2, which is originally just for x-loader and uboot. (Is there a reason for this?) What is the disadvantage of SW-ECC? It's maybe slower, but it is noticable?

The other problem I was faced off after getting uboot and kernel to make compatible (with removing the shift, as recommended before) was, that I could boot fine, and also write partitions with with nandwrite, but a change in the filesystem by (example touch file) gives a lot of ECC errors in next boot. To get everything working like before become a big issue. That was the most confusing issue and really drives me nut.

Nevertheless, thanks for your hints. I start to understand this a bit more ...

- arno

Your JFFS2 image must be built without -p option. refer : http://processors.wiki.ti.com/index.php/AM35x-OMAP35x-PSP_04.02.00.07_UserGuide#Creating_JFFS2_filesystem 

Also , note that : The image size should be upward aligned to NAND page size which is 2KB (i.e. 0x800). For example, if the image size is 0x19B8004 the size to be passed to the NAND write command should be 0x19B8800. 

This could possibly solve the issue of ECC errors on next boot.

Thanks,

Hrishikesh

Dear Hrishikesh, thanks for your patience with me.  I will try this next.

Nevertheless I have the problem, that xloader does not support "hw 1" and kernel does not support "hw 2".

Currently I want switch back to SW-ECC, but it seems to be impossible to configure the kernel to use SW-ECC. Also the documents you refer do not tell about this, just about the different HW-ECC. Imho it's ok to add new features or change defaults in kernel. But it becomes hard for not so insiders to keep the system running if previous standard options just dissapear.

Does nobody has been successful in using SW-ECC in 2.6.37 ?

Sorry-doublepost - see below

Now I am sure: SW ECC is not working in uboot environment.

I did tests, modifying single bits in environment. Only HW-ECC can correct this. SW-ECC (1 bit, 4 bit BCH) failed and give CRC error. So definitly the error correction is not working for uboot-environment. Other parts seems not to be suffered. This is a big problem for reliability!

U-Boot 2010.06 (Oct 20 2011 - 10:20:28)

AM/DM37xx-GP ES 2.0
OMAP3 Visor board + LPDDR/NAND
I2C:   ready
DRAM:  128 MiB
NAND:  4 BIT SW ECC selected
128 MiB
err_loc=2692
*** Warning - bad CRC or NAND, using default environment

I used the latest PSP 4.02.00.07 on a AM3703 / Omap3503.

Best regards

Arno

Hello Arno,

I am not using PSP kernels, I am using the kernel from rowboat (Android on omap3) project on a custom board. However that kernel is close to what you might have in PSP: it is supported by TI engineers and contains numerous patches from TI. I got almost the same list of problems when I tried to use SW ECC on 2.6.37: 'uncorrectable error' on jffs2 (when writing jffs2 rootfs from u-boot using SW ECC) and other problems including ubifs problems as well.

I did some investigation and comparison between rowboat 2.6.37 kernel and upstream 2.6.37 and 3.1 kernels. The comparison results were as follows: upstream 3.1 kernel worked well with SW ECC, rowboat kernel and upstream 2.6.37 didn't work well with the same set of symptoms ('uncorrectable erro' and so on).

I did a closer look and it turned out that on 2.6.37 kernels (both rowboat and upstream) SW ECC didn't work well with prefetch functionality. In the case of upstream 2.6.37 kernel you can control it using config: Device Drivers -> MTD support -> NAND device support -> NAND Flash device on OMAP2 and OMAP3 ->   GPMC prefetch support for NAND device. SW ECC worked just fine after I disabled prefetch feature.

In rowboat 2.6.37 kernel there is no such switch in configuration. Probably the same is true for PSP kernels. But prefetch enablement can be controlled from platform files using 'xfer_type' field of field of 'omap_nand_platform_data' structure. So if you are using custom nand init function in your board file, you can do the following:

static struct omap_nand_platform_data your_custom_nand_data = {
        .ecc_opt        = OMAP_ECC_HAMMING_CODE_DEFAULT,  /* SW ECC */
        .xfer_type      = NAND_OMAP_POLLED, /* disable prefetch */
           ...
};    

If your board file uses 'board_nand_init' function from arch/arm/mach-omap2/board-flash.c, then you can modify 'board_nand_data' structure in that file accordingly. 

And, as you have already noted, you have to remove the line 'pdata->ecc_opt = OMAP_ECC_HAMMING_CODE_HW;' from drivers/mtd/nand/omap2.c file.

These changes fixed SW ECC for me: I could write partitions from u-boot and boot from those partitions (using jffs2 or ubifs) without problems. So it looks like something is wrong with prefetch feature in 2.6.37 kernel, even in upstream.  Better way would be to fix prefetch rather than to disable it, though I haven't yet done it. But upstream 3.1 kernel works fine with prefetch + SW ECC, so prefetch fixes can be backported from that kernel.

Thanks,

Sergey

Hello Arno,

Finally I took a look at the difference between 2.6.37 and 3.1 kernels. That difference is not that big, so I could easily spot the fix for sw ecc and prefetch: see commit c5d8c0cae4af7d78823d32fcd1c458ee1a1b5489  in upstream linux tree on git.kernel.org. I attached that patch here: 2438.0001-mtd-omap-fix-subpage-ecc-issue-with-prefetch.txt

With that patch applied I had no problems using SW ECC and prefetch on my rowboat-2.6.37 kernel. Here is my nand driver settings:

static struct omap_nand_platform_data some_board_nand_data = {
        .devsize        = NAND_BUSWIDTH_16,
        .ecc_opt        = OMAP_ECC_HAMMING_CODE_DEFAULT,        /* SW ECC */
        .xfer_type      = NAND_OMAP_PREFETCH_POLLED,            /* enable prefetch */
        ....

}

Thanks,

Sergey

Thanks Sergey, great work! Have been busy with uboot trouble (that SW ECC doesn't work in enviroment while booting). Finally I could solve this, described in another thread. Regarding kernel - right now I am busy with other stuff, so I have keep my old but running 2.6.33 kernel, so I can't confirm your patch in next days but will keep this in mind.

Thanks

Arno

Hi Sergey

I am having customised AM3505 board and using xloader, u-boot of previous release (PSP 03.00.01.06) and linux kernel of PSP_04.02.00.07 release.

But as per following link, there is no support of SOFTWARE ECC in PSP_04.02.00.07 kernel but older version of xloader, u-boot expect SOFTWARE ECC in kernel.

http://processors.wiki.ti.com/index.php/AM35x-OMAP35x-PSP_04.02.00.07_UserGuide#Read_This_First

For that I applied your patch and changed following code in drivers/mtd/nand/omap2.c to explicitly set Software ECC.

pdata->ecc_opt          = OMAP_ECC_HAMMING_CODE_DEFAULT;

pdata->xfer_type        = NAND_OMAP_PREFETCH_POLLED;

But while creating ubifs file sytem getting ECC error (logs are shown bellow):

root@am3517-evm:/home/ubifs# ubiformat /dev/mtd5 -s 2048 -O 2048 -y
ubiformat: mtd5 (nand), size 419430400 bytes (400.0 MiB), 3200 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 3199 -- 100 % complete  
ubiformat: 3198 eraseblocks have valid erase counter, mean value is 0
ubiformat: 1 eraseblocks are supposedly empty
ubiformat: 1 bad eraseblocks found, numbers: 3196
ubiformat: formatting eraseblock 3199 -- 100 % complete  
root@am3517-evm:/home/ubifs# ubiattach /dev/ubi_ctrl -m 5 -O 2048
[  104.620910] UBI: attaching mtd5 to ubi0
[  104.625244] UBI: physical eraseblock size:   131072 bytes (128 KiB)
[  104.631835] UBI: logical eraseblock size:    126976 bytes
[  104.637573] UBI: smallest flash I/O unit:    2048
[  104.642883] UBI: sub-page size:              512
[  104.647796] UBI: VID header offset:          2048 (aligned 2048)
[  104.654083] UBI: data offset:                4096
[  104.665924] uncorrectable error :
[  104.669372] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  104.680725] uncorrectable error :
[  104.684143] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 1:2048, read 512 bytes
[  105.543395] UBI: max. sequence number:       0
[  105.580505] UBI: attached mtd5 to ubi0
[  105.584442] UBI: MTD device name:            "factory-filesystem"
[  105.591247] UBI: MTD device size:            400 MiB
[  105.596527] UBI: number of good PEBs:        3199
[  105.601531] UBI: number of bad PEBs:         1
[  105.606170] UBI: number of corrupted PEBs:   0
[  105.610900] UBI: max. allowed volumes:       128
[  105.615722] UBI: wear-leveling threshold:    4096
[  105.620727] UBI: number of internal volumes: 1
[  105.625457] UBI: number of user volumes:     0
[  105.630096] UBI: available PEBs:             3164
[  105.635375] UBI: total number of reserved PEBs: 35
[  105.640472] UBI: number of PEBs reserved for bad PEB handling: 31
[  105.646850] UBI: max/mean erase counter: 1/0
[  105.651397] UBI: image sequence number:  797487910
[  105.662292] UBI: background thread "ubi_bgt0d" started, PID 1890
[  105.688751] uncorrectable error :
[  105.692169] uncorrectable error :
[  105.695953] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
UBI device number 0, total 3199 LEBs (406196224 bytes, 387.4 MiB), available 3164 LEBs (401752064 bytes, 383.1 M[  105.740631] uncorrectable error : iB), LEB size 12
[  105.745147] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
6976 bytes (124.[  105.758056] UBI: run torture test for PEB 3198
0 KiB)
root@am3517-evm:/home/ubifs# [  105.914215] UBI: PEB 3198 passed torture test, do not mark it a bad
[  105.923583] uncorrectable error :
[  105.927032] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  105.951599] uncorrectable error :
[  105.955169] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  105.970184] UBI: run torture test for PEB 3198
[  106.107543] UBI: PEB 3198 passed torture test, do not mark it a bad
[  106.116363] uncorrectable error :
[  106.119964] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  106.144561] uncorrectable error :
[  106.148376] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  106.159790] UBI: run torture test for PEB 3198
[  106.300689] UBI: PEB 3198 passed torture test, do not mark it a bad
[  106.309570] uncorrectable error :
[  106.313201] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  106.336059] uncorrectable error :
[  106.339508] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  106.350952] UBI: run torture test for PEB 3198
[  106.493041] UBI: PEB 3198 passed torture test, do not mark it a bad
[  106.501861] uncorrectable error :
[  106.505310] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  106.529937] uncorrectable error :
[  106.533508] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  106.548889] UBI: run torture test for PEB 3198
[  106.686614] UBI: PEB 3198 passed torture test, do not mark it a bad
[  106.695892] uncorrectable error :
[  106.699340] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  106.724517] uncorrectable error :
[  106.728332] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  106.743286] UBI: run torture test for PEB 3198
[  106.880767] UBI: PEB 3198 passed torture test, do not mark it a bad
[  106.889617] uncorrectable error :
[  106.893188] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  106.917846] uncorrectable error :
[  106.921447] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  106.932891] UBI: run torture test for PEB 3198
[  107.072967] UBI: PEB 3198 passed torture test, do not mark it a bad
[  107.081756] uncorrectable error :
[  107.085327] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  107.109497] uncorrectable error :
[  107.112945] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  107.124664] UBI: run torture test for PEB 3198
[  107.265716] UBI: PEB 3198 passed torture test, do not mark it a bad
[  107.274810] uncorrectable error :
[  107.278472] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  107.303741] uncorrectable error :
[  107.307312] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  107.322265] UBI: run torture test for PEB 3198
[  107.459197] UBI: PEB 3198 passed torture test, do not mark it a bad
[  107.468170] uncorrectable error :
[  107.471588] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 0:2048, read 512 bytes
[  107.496276] uncorrectable error :
[  107.499847] UBI error: ubi_io_read: error -74 (ECC error) while reading 512 bytes from PEB 3198:2048, read 512 bytes
[  107.514770] UBI: run torture test for PEB 3198