• State
  • Locked Locked
  • Replies 16 replies
  • Subscribers 101 subscribers
  • Views 3471 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

NAND ECC error addresses above 512

German Baranov
Intellectual 660 points

Hi!

My question is about using values, provided by DM365 EMIF in ECC error address registers.

In all this discussion, the number of ECC errors per 512 block, occured while interfacing to NAND flash, is less then 4, i.e. errors are "correctable". (usually 1 error)

The address values of errors can be up to 519, while amount of data we are processing is 512.

UBL program in case when error address is above 512 declares failure. (device_nand.c)

My small experience with getting errors with addresses above 512 shows, that in this case error bits are located in "syndrom" written to OOB.

Is that always so?, and

Is that really sign of failure for Reed-solomon code, as UBL says, or we can just use data and keep in mind that OOB contains flipped bit?

 

Regards,

German Baranov

  • 0
    Guru 31670 points

    Hi,

    Can you send the logs of the steps that you are trying, when you see this error? For whichever sector that fails, could you please do a "nand dump 0xoffset" of the offset and send?

  • 0 in reply to Renjith Thomas
    Intellectual 660 points

    Hi!

    We have DM365 based system which is set to boot from SD-card.

    NAND flash in the system is connected to EMIF; interface is 8-bit wide.  We use 4-bit ECC correction.

    SD-card contains sdboot tool, which starts after power on and displays menu on console.

    We select “install UBL” from sdboot menu. (All further experiments described here were about writing UBL to NAND to keep picture simple).

    For sdboot without inserted debug messages we, with high probability, will get “Page write verify failed!”  error message.

    Here I have to say that we have two types of NAND chips, one type produces very low amount of errors in general, and with this chip we haven’t seen discussed situation with UBL at all. Another type generates much higher error rate and causes the errors, we are discussing, when is installed on the same PCB as good IC. The reason of this error rate is unknown to me, I could just have broken, damaged chips or something else, the question is not about NAND device, but about the case, with certain address of error; this case is easier to obtain when error rate is high.

    Back to our case. Now we put debug messages into ubl/sdboot code, namely to DEVICE_NAND_ECC_correct function, to display on console following variables in case of failure:

    1.  corrState

     

    #define DEVICE_EMIF_NANDFSR_ECC_STATE_MASK              (0x00000F00)

    #define DEVICE_EMIF_NANDFSR_ECC_STATE_SHIFT             (8)               

    (from device.h)

    corrState = (AEMIF->NANDFSR & DEVICE_EMIF_NANDFSR_ECC_STATE_MASK) >> DEVICE_EMIF_NANDFSR_ECC_STATE_SHIFT;

    (from device_nand.c)

     

    This variable provides us with value of bits 8-11 (shifted to 0-3 positions) of EMIF NANDFSR, this bit field is called ECC_STATE and values 0x2 and 0x3 correspond to "Error correction complete", that is errors are detected, but number of errors is low and errors are corrected.

     

    2.  AEMIF->NANDERRADD1

     

    This is value of NANDERRADD1 register of EMIF ECC, which gives us number for calculation of offset for damaged data.

    We also used to display NumE, that is number of errors, but it practically always was 1 or 2, so we stopped observing it, as well as we limited observation of address to NANDERRADD1, and not NANDERRADD2.

     

    corrState and NANDERRADD1 are displayed in case of any ECC detected error, namely when

     corrState != 0.

     

    In case of E_FAIL obtained from DEVICE_NAND_ECC_correct function, which leads to

    “Page write verify failed!”  error message, we have dumping of spare_bytes to console, provided by code inserted to NAND_readPage function.

    Four left columns are spare bytes read from NAND by NAND_readPage function.

    Four right columns are spare bytes prepared and written previously to this page by NAND_writePage

    Let me just remind that first six bytes in every 16-byte piece in spare bytes area are set to 0xFF, while 10 remaining represent code for error correction/detection.

    Below is screen dump from running this version of sdboot with debug messages:

    * Flashing UBL

    Reading into SDRAM...

    Flashing into NAND...

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000018A

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000108

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001FD

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000BF

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000C2

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000045

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000104

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000E3

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000110

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x01580034

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000003A

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000142

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000002F

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000119

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000091

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000F2

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000E6

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000159

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000105

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000044

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x01750013

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000125

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000097

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000148

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00AE0021

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001CF

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000095

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000129

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000DB

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x015900F8

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000123

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001A1

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000087

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000002

    Ecc Err

    0x000000FF 0x000000FF 0x000000FF 0x000000FF       0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000056 0x000000BD     0x000000FF 0x000000FF 0x00000056 0x000000BD

    0x00000030 0x00000027 0x00000046 0x000000CD    0x00000030 0x00000027 0x00000046 0x000000CD

    0x000000FE 0x000000C5 0x000000E0 0x000000C5    0x000000FE 0x000000C5 0x000000E0 0x000000C5

    0x000000FF 0x000000FF 0x000000FF 0x000000FF       0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000     0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000010 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF      0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000     0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF      0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000    0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF    0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000    0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF    0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000    0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x0000000000000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF    0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000    0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x000000FF 0x000000FF 0x000000FF 0x000000FF    0x000000FF 0x000000FF 0x000000FF 0x000000FF

    0x000000FF 0x000000FF 0x00000000 0x00000000    0x000000FF 0x000000FF 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    0x00000000 0x00000000 0x00000000 0x00000000    0x00000000 0x00000000 0x00000000 0x00000000

    Page write verify failed!

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001AD

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000BB

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000A5

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000036

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x016E0140

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000DC

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000010C

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000019A

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000013

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000145

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001FD

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000017A

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x01CC019E

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000078

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000012D

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000024

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000139

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000007F

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000000EF

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x0000015F

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x000001CA

    corrState = 0x00000003 AEMIF->NANDERRADD1 = 0x00000158

      

    As we see, the error rate, while reading NAND back after programming, is high (every row represents an error per 512 byte piece) but most of the errors are successfully handled by EMIF ECC; corrState value of 0x3 confirms that errors are considered to be correctable. The only problem occurs when NANDERRADD1 is so low that offset value calculated as

    addOffset = 519 - (AEMIF->NANDERRADD1 & (0x000003FF));

     is above 511 (Bold font). In this case, DEVICE_NAND_ECC_correct function just returns E_FAIL, which in turn forces NAND_readPage to return E_FAIL, and LOCAL_NANDWriteHeaderAndData (from installer.c)  generates “Page write verify failed!” 

    Let’s look at the dump of spare bytes area. We can easily find flipped bit (underlined values), but now we have come to the question: is it so dangerous to declare a failure?

     

    Couple more comments. UBL is written to NAND 8 times and occupies 8 flash blocks if there are no errors. In case of uncorrectable error, installer.c code discards block with erroneous page and moves to the next block. That’s why activity continues after failure.

    Regards,

    German

  • 0 in reply to German Baranov
    Guru 31670 points

    Give me some time to go through. I'll get back to you shortly.

  • 0 in reply to Renjith Thomas
    Guru 31670 points

    German,

    Did you go through the NAND timing configurations. Simplest way to do is, set the maximum timing in A1CR/A2CR register. It will work for all NANDs and you can verify whether the issue is with timing or not. If it works, then you've to tune the A1CR register for the optimal timing.

    I'm assuming that both the NANDs that you've tried is of same page size. If possible, could you please let me know the NAND part numbers?

  • 0 in reply to Renjith Thomas
    Intellectual 660 points

    Hi Renjith,

    Yes, I set maximum timing in A1CR, but I hadn't noted significant difference (I neglected A2CR since I thought that address range for ubl writing doesn't reach second CS space).

    Before we go to NAND comparison, let me note that the subject I'm trying to clear is ECC processing, rather than NAND chip drawbacks. I understand, that "practical" solution would be to use "correct" NAND chip and discard "incorrect" one. To my understanding, there is no mistake in TI hardware (DM365 EMIF) functioning: it DOES detect error in NAND data. But I need help in understanding why the error is processed this way and not another. For instance, if NAND has erroneus bit in "spare bytes" area (which is quite possible) what value will be generated to NANDERRADD register?

    Back to NAND chips. One chip is 2 KByte page Samsung K9HBG08U1M, another is 4KByte page Micron MT29F32G08CBACA.

    Regards,

    German

  • 0 in reply to German Baranov
    Guru 31670 points

    German,

    The NAND that you are using is MLC NAND where as the Samsung one is SLC. Since MLC is more prone to errors, you need a better ECC algorithm for that, which can correct upto 16-bits per 512bytes of data. According to Micron datasheet minimum required is 24-bit ECC per 1080 bytes of data. Since you are using 4-bit per 512bytes I think this is the cause of this behaviour. Since DM36x doesn't support BCH8, and BCH16 implementations, I really doubt whether its possible to support MLC NAND.

    Where did you connect your NAND to? Is it CS0 or CS1? 

  • 0 in reply to Renjith Thomas
    Intellectual 660 points

    Hi Renijth,

    I think you are absolutely right about the reason of NAND chips behaviour: one type generates more errors then another (though I'm not sure that Samsung K9H chip is SLC, I believe letter H in third position stands for MLC technology, but anyway, its requirement is 4-bit per 512bytes), and better ECC algorithm is required to use mentioned above Micron memory.

    Can we spend some time discussing following imaginary situation?:  We are using reliable SLC NAND flash, to be more specific we can say that page is 2K + 64 spare. Once in a while error occurs in the NAND, but it is succesfully corrected by DM365 hardware/sowftare.  Even less frequently error occurs in spare bytes. Will ECC generate an error in NANDFSR in this case? What will be the value in NANDERRADD register?

    As for your question, my NAND is connected to CS0.

    Thank you

  • 0 in reply to German Baranov
    Guru 31670 points

    German,

    I'm not really sure about the error handling in spare area itself. Let me check this and get back to you.

  • 0 in reply to Renjith Thomas
    Guru 10570 points

    Hello,
    Is there update?
    I am facing same phenomenon.
    Best regards, RY

  • 0
    Intellectual 255 points

    Hi German Baranov,

    I met the same problem with you when using DM368 with MT29F1G08ABADA nand flash.

    When UBL boots uboot, in DEVICE_NAND_ECC_correct() (device_nand.c in UBL) function:

        switch( numE )
        {
          case 3:     // Four errors
            addOffset = 519 - ( (AEMIF->NANDERRADD2 & (0x03FF0000))>>16 );
           // if (addOffset > 511) return E_FAIL;
            if (addOffset < 512) {
       corrValue = (AEMIF->NANDERRVAL2 & (0x03FF0000))>>16;
       data[addOffset] ^= (Uint8)corrValue;
      } else {
       //maybe error in OOB area
       //do nothing now
      }
            // Fall through to case 2
          case 2:     // Three errors
            addOffset = 519 - (AEMIF->NANDERRADD2 & (0x000003FF));
            //if (addOffset > 511) return E_FAIL;
            if (addOffset < 512) {
          corrValue = AEMIF->NANDERRVAL2 & (0x000003FF);
          data[addOffset] ^= (Uint8)corrValue;
            } else {
       //do nothing
       // error in oob
      }
            // Fall through to case 1
          case 1:     // Two errors
            addOffset = 519 - ( (AEMIF->NANDERRADD1 & (0x03FF0000))>>16 );
            //if (addOffset > 511) return E_FAIL;
            if (addOffset < 512) {
       corrValue = (AEMIF->NANDERRVAL1 & (0x03FF0000))>>16;
       data[addOffset] ^= (Uint8)corrValue;
      } else {
       //do nothing
       // error in oob
      }  
            // Fall through to case 0
          case 0:     // One error
            addOffset = 519 - (AEMIF->NANDERRADD1 & (0x000003FF));
           // if (addOffset > 511) return E_FAIL;
           if (addOffset < 512) {
             corrValue = AEMIF->NANDERRVAL1 & (0x3FF);
             data[addOffset] ^= (Uint8)corrValue;
           } else {
       //do nothing
       // error in oob
      }
            break;
        }


    I change the code which marked in red, then UBL can boots uboot successfully.

    And I also see the code operates nand flash in uboot (cpu/arm926ejs/davinci/nand.c, nand_davinci_4bit_compare_ecc function ):

     /* Read the error address, error value and correct */
     for (i = 0; i < numErrors; i++) {
      if (i > 1) {
       errorAddress =
           ((emif_addr->NANDERRADD2 >>
             (16 * (i & 1))) & 0x3FF);
       errorAddress = ((512 + 7) - errorAddress);
       errorValue =
           ((emif_addr->NANDERRVAL2 >>
             (16 * (i & 1))) & 0xFF);
      } else {
       errorAddress =
           ((emif_addr->NANDERRADD1 >>
             (16 * (i & 1))) & 0x3FF);
       errorAddress = ((512 + 7) - errorAddress);
       errorValue =
           ((emif_addr->NANDERRVAL1 >>
             (16 * (i & 1))) & 0xFF);
      }

       // printf("errorAddress: %d, errorValue: 0x%0x, wrong data: 0x%02x, correct data: 0x%02x\n",
       //   errorAddress, errorValue, page_data[errorAddress], page_data[errorAddress]^errorValue);
       
      /* xor the corrupt data with error value */
      if (errorAddress < 512)
       page_data[errorAddress] ^= errorValue;

       // printf("corrected data, page_data[%d]: 0x%02x\n", errorAddress, page_data[errorAddress]);
     }

    if errorAddress is greater then 511, the code doesn't treat it as an error, it just skip it.

    So I think if the error happens in OOB area, the errorAddress is greater then 511.

    Xiaoguang Zhao

     

  • 0 in reply to Xiaoguang Zhao
    Intellectual 660 points

    Thank you for your participation in the discussion.

    Sorry, it took me several days to reply.

    Then in 2012 I went the same way - I just ignored errors with addresses above 511, which made practical solution - I'm happy with the work of the devices till these days.

    Unfortunately,  a part of question remained unanswered, which makes knowledge incomplete, namely:

    If error address is above 511, can we apply same error correction algorithm to the error possibly located in OOB area?

    Regards

  • 0 in reply to German Baranov
    Guru 31670 points

    Hi All,

    IMO its possible to calculate the ECC of the spare area bytes, but the problem is that where will you store the checksum of the spare area again? We can think of storing this again in another page or spare area, but again the same question arises. But if we need so much reliability then NAND itself is not an option. We should think of using NOR flash here, as NAND is the result of a compromise on cost/reliability when compared to NOR.

  • 0 in reply to Renjith Thomas
    Intellectual 660 points

    Hello!

    As far as I understand, there is no need in additional checksum calculation - Reed-Solomon code, used for 4-bit ECC calculation, can detect/correct error in any position of codeword (including OOB area). Even more, we see, that DM36x hardware performing ECC calculation does detect error in checksum area.

    The questions discussed here are: 

    1. We (persons participating here) are sure that in the case of error in checksum area, error address, returned by ECC hardware will be above 511 - we are just looking for TI confirmation about that (I couldn't find that in technical doc).

    2. We think that ubl dramatic reaction to errors with addresses above 511 (boot failure) is a mistake. These errors just tell us about wrong bits in OOB area, while main code is OK and boot process can be continued. The simplest way to continue is to ignore errors above 511 - confirmed by practice (and by similar behavior of Linux code). It would be nice to here from TI about possibility of error correction in this case - can we apply same procedure as for addresses below 511? (As I've mentioned above - Reed-Solomon code in theory allows that, but we are not completely aware about TI's hardware functioning in this case). 

    Best regards,

    German Baranov

  • 0 in reply to German Baranov
    Intellectual 255 points

    Hi German Baranov,

    I totally agree with you. I just ignored the errors above 511 in my ubl code.

    Another question, since the nand flash I used needs 4-bit ecc, but in linux kernel code:

    static struct nand_davinci_platform_data nand_data = {
    .options = 0,
    .ecc_mode = NAND_ECC_SOFT,
    .cle_mask = 0x10,
    .ale_mask = 0x08,
    .bbt_td = &nand_davinci_bbt_main_descr,
    .bbt_md = &nand_davinci_bbt_mirror_descr,
    .parts = nand_partitions,
    .nr_parts = ARRAY_SIZE(nand_partitions),
    };

    It use soft ecc for filesystem. when I change ecc_mode to NAND_ECC_HW_SYNDROME, try to burn the filesystem using nfs, then mount the partition (mtdblock3) again, I found some files and directories are lost.

    Do you meet this problem before? I use 2.6.18 linux kernel.

    Thanks.

    Xiaoguang

     

  • 0 in reply to Renjith Thomas
    Intellectual 255 points

    Hi Renjith, 

    Do you mean that nand flash is not a realiable storage solution in industrial product design?

    If we choose a nand flash that only requires 1-bit ecc,  does it can improve the realiability of our production?

    Thanks.

    Xiaoguang

  • 0 in reply to Xiaoguang Zhao
    Guru 31670 points

    Xiaoguang,

    If you are having a very critical application (like Automotive) NOR Flash is most preferred and NAND flash is used for media data storage or for storing less critical data. 

    Choosing NAND with 1-bit ECC requirement and using higher ECC algorithm will definitely improve the reliability.