VFS: Cannot open root device "ubi0:rootfs" or unknown-block(0,0)

Kevin,

I never used ubifs myself so far. However our 3P Linutronix ran a benchmark on Beagleboard (OMAP3530) successfully.

They had to apply the patch mentioned here: http://www.mail-archive.com/linux-omap@vger.kernel.org/msg40851.html They also mentioned some changes due to NAND partitioning but I assume this was in their benchmark code.

Not sure if that is related or not.

Regards.

Kevin,

This "Bad CRC error is most probably because of the buggy mtd-utils(version 1.3) package that has shipped with your distribution.

I recommend download and use the latest stable mtd-utils version 1.4 as mentioned here http://processors.wiki.ti.com/index.php/TI-Android-FroYo-DevKit-V2.2_UserGuide#Creating_ubifs_images

While making UBI image use the following parameters:

$ chmod -R 777 temp/
$ sudo mkfs.ubifs -r temp/ -m 2048 -e 12696 -c 3991 -o ubifs.img
$ sudo ubinize -o ubi.img -m 2048 -p 128KiB -s 512 ubinize.cfg

In ubinize.cfg you can set vol_size=450MiB

Regards,
Amit Pundir

Hi Amit Pundir,

Yes, it seems mkfs.ubifs's bug, when I update mkfs.ubifs from 1.3 to 1.4, the kernel can boot the ubi.img image now.

Something different from your suggestion, I changed logical erase block size to 129024, I mkfs.ubifs as follows,

$ sudo mkfs.ubifs -r temp/ -m 2048 -e 129024 -c 3991 -o ubifs.img

then,

the system starts up, but not completely, some java processes can't be found in the process list.

and lots of errors show up on the console such as

UBIFS error (pid 802): make_reservation: cannot reserve 424 bytes in jhead 1, error -30
UBIFS error (pid 802): ubifs_create: cannot create regular file, error -30

I attached some log files,

Processlist :7217.ProcessList.txt

Kernel log:5023.KernelLog.txt

Thanks,

Kevin

Hi Kevin,

Do you get the same error if you build your image with erase block size = 124KiB i.e. 126976?

Regards,
Amit Pundir

Hi Kevin,

I my first reply I made a typo and wrongly put 12696 instead of 126976(124KiB). Kindly try with this eraseblock size. It should work. Sorry for the confusion.

Regards,
Amit Pundir

Hi Amit Pundir,

I tried to build the image with erase block size =  126976,

that is,

$ sudo mkfs.ubifs -r temp/ -m 2048 -e 126976 -c 3991 -o ubifs.img
$ sudo ubinize -o ubi.img -m 2048 -p 128KiB -s 512 ubinize.cfg

but it stops while kernel starts booting ubi.img.

Kernel log is attached for your review.  2543.boot-log-e-126976.txt

Thanks,

Kevin

Hi,

After fastbooting the ubi.img to NAND,  I changed the bootargs in u-boot command as follows,

#setenv nandargs 'setenv bootargs console=${console} ubi.mtd=4 rootfstype=ubifs root=ubi0:rootfs rw init=/init'

but I found that the system is a read-only file system, I can't copy file to it through adb push xxx /system/bin,

I can't mkdir in / directory either, what is problem?

Any suggestions is welcome.

Thanks,

Kevin

Hi Kevin,

Logical eraseblock size is indeed 129024 and not 126976 as printed in the following error message.
Error: "validate_sb: LEB size mismatch: 126976 in superblock, 129024 real"

Also I see some messages regarding insufficient number of reserved PEBs. So try this command line:

$ sudo mkfs.ubifs -r temp/ -m 2048 -e 129024 -c 3900 -o ubifs.img

Kindly attach kernel log as well as logcat output for debugging.

Regards,
Amit Pundir

Hi Amit Pundir,

I create ubi.img as following command line,

$ sudo mkfs.ubifs -r temp/ -m 2048 -e 129024 -c 3900 -o ubifs.img
$ sudo ubinize -o ubi.img -m 2048 -p 128KiB -s 512 ubinize.cfg

and with  vol_size=450MiB in ubinize.cfg.

however, the result is the same.

Errors is like this,

UBI error: ubi_io_write: error -5 while writing 2048 bytes to PEB 509:57344, written 0 bytes
UBI warning: ubi_eba_write_leb: failed to write data to PEB 509
UBI: recover PEB 509, move data to PEB 2277
UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 2277:512, written 0 bytes
UBI warning: recover_peb: failed to write to PEB 2277
UBI: try again
UBI: recover PEB 509, move data to PEB 2278
UBI: run torture test for PEB 2277
UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 2278:512, written 0 bytes
UBI warning: recover_peb: failed to write to PEB 2278
UBI: try again
UBI: recover PEB 509, move data to PEB 2279
UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 2279:512, written 0 bytes
UBI warning: recover_peb: failed to write to PEB 2279
UBI: try again
UBI: recover PEB 509, move data to PEB 2280
UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 2280:512, written 0 bytes
UBI warning: recover_peb: failed to write to PEB 2280
UBI warning: ubi_ro_mode: switch to read-only mode
UBIFS error (pid 675): ubifs_wbuf_write_nolock: cannot write 3305 bytes to LEB 509:53248, error -5
UBIFS warning (pid 675): ubifs_ro_mode: switched to read-only mode, error -5
UBIFS error (pid 675): do_writepage: cannot write page 14 of inode 773, error -5
UBI error: ubi_io_write: read-only mode
UBI error: erase_worker: failed to erase PEB 2277, error -30
UBI error: do_work: work failed with error code -30
UBI error: ubi_thread: ubi_bgt0d: work failed with error code -30
UBIFS error (pid 801): make_reservation: cannot reserve 392 bytes in jhead 1, error -30
UBIFS error (pid 801): ubifs_mkdir: cannot create directory, error -30
UBIFS error (pid 803): make_reservation: cannot reserve 424 bytes in jhead 1, error -30
UBIFS error (pid 803): ubifs_create: cannot create regular file, error -30

It seems NAND I/O error happened, and the system switched to read-only mode.

My nand indeed has three bad blocks originally (after nand scrub),

AM3517_EVM # nand bad

Device 0 bad blocks:
  0c080000
  14780000
  15a60000

but after fastboot, it has more bad blocks,

AM3517_EVM # nand bad

Device 0 bad blocks:
  00000000
  00020000
  00040000
  00060000
  04520000
  05300000
  0a440000
  0b220000
  0ba20000
  0c080000
  14780000
  15140000
  152a0000
  15a60000
  160a0000
  168a0000
  17580000

To enable my Samsung NAND work in xloader and uboot, I made some changes to them.

Logs are attached.

Kernel Log: 7840.kernel-log.txt

DDMS Log: 8233.ddms-log.txt

Thanks,

Kevin

Hi Kevin,

From the logs it seems that the ubifs image is not properly or completely flashed.

Is it fine with you if you can attach your u-boot changes? Just to make sure that nothing related to Fastboot functionality has changed.

Note that HW ECC was not supported in the release which you are using. For fastboot we changed the partition names and some Nand write parameters. Make sure you are taking care of that as well in your u-boot. Nand partition layout in u-boot should be consistent with the layout you make in your kernel.

Also take care of the GPMC timings as mentioned by Vaibhav in one of your tickets http://e2e.ti.com/support/embedded/f/509/p/112477/398674.aspx#398674

I would also suggest you to validate your NAND with a regular linux using jffs2 or ubifs file-system using mtd-utils instead of jumping directly to Android. Since you won't be having enough NAND debugging tools on Android to rely upon.

Regards,
Amit Pundir

Hi Amit Pundir,

The changes I have made to x-loader:

1.

@include/configs/am3517_evm.h
@line 38
--------from----------
#define K9F1G08R0A_MFR        0xec  /* Samsung */
#define K9F1G08R0A_ID        0xa1  /* part # */
---------to-----------
#define K9F1G08R0A_MFR        0xec  /* Samsung */
#define K9F1G08R0A_ID        0xdc  /* part # */

2.
@x-loader/include/configs/am3517evm.h
@line 114
--------from----------
#define NAND_16BIT
---------to-----------
//#define NAND_16BIT

The changes I have made to u-boot:

1.

@include/configs/am3517_evm.h
@line 302
--------from----------
 #define GPMC_NAND_ECC_LP_x16_LAYOUT    1
---------to-----------
 #define GPMC_NAND_ECC_LP_x8_LAYOUT 1

2.
@include/asm/arch-omap3/mem.h
@line 235
--------from---------
#define M_NAND_GPMC_CONFIG1    0x00001800
---------to----------
#ifdef NAND_16BIT
# define M_NAND_GPMC_CONFIG1 0x00001800
#else
# define M_NAND_GPMC_CONFIG1 0x00000800
#endif

3.
@arch/arm/cpu/arm_cortexa8/omap3/sys_info.c
@line 252
--------from---------
u32 get_gpmc0_width(void)
{
    return WIDTH_16BIT;
}
---------to----------
u32 get_gpmc0_width(void)
{
    return WIDTH_8BIT;
}

Do these changes affect the fastboot's working?

Thanks,

Kevin

Hi Amit Pundir,

I tried mtd-utils in the system running on MMC, the procedure and logs are posted on Thu, May 26 2011 2:46 AM at

http://e2e.ti.com/support/embedded/f/509/p/111277/400322.aspx#400322

Hope it helps.

Thanks,

Kevin

Hi Kevin,

I have noticed that in some of the board specific files in kernel we set the default NAND bus width to be 16bit i.e. NAND_BUSWIDTH_16.

Are you making sure that you are using 8bit options and corresponding operations in driver(s)?

Regards,
Amit Pundir

Hi Amit Pundir,

I checked the board specific files, and found default NAND bus width was set in two files,

@kernel/arch/arm/mach-omap2/board-omap3beagle.c
static struct omap_nand_platform_data omap3beagle_nand_data = {                   
        .options        = NAND_BUSWIDTH_16,                                       
        .parts          = omap3beagle_nand_partitions,                            
        .nr_parts       = ARRAY_SIZE(omap3beagle_nand_partitions),                
        .dma_channel    = -1,           /* disable DMA in OMAP NAND driver */     
        .nand_setup     = NULL,                                                   
        .dev_ready      = NULL,                                                   
}; 

@kernel/arch/arm/mach-omap2/board-omap3touchbook.c
static struct omap_nand_platform_data omap3touchbook_nand_data = {
        .options        = NAND_BUSWIDTH_16,                                      
        .parts          = omap3touchbook_nand_partitions,                        
        .nr_parts       = ARRAY_SIZE(omap3touchbook_nand_partitions),            
        .dma_channel    = -1,           /* disable DMA in OMAP NAND driver */    
        .nand_setup     = NULL,                                                  
        .dev_ready      = NULL,                                                  
};

but these two files was not compiled.

And the same struct for am3517evm is

@kernel/arch/arm/mach-omap2/board-am3517evm.c
static struct omap_nand_platform_data am3517evm_nand_data = {               
       .parts          = am3517evm_nand_partitions,                         
       .nr_parts       = ARRAY_SIZE(am3517evm_nand_partitions),             
       .nand_setup     = NULL,                                              
       .dma_channel    = -1,           /* disable DMA in OMAP NAND driver */
       .dev_ready      = NULL,                                              
};

no options field is specified, and in omap2 NAND driver,

@kernel/drivers/mtd/nand/omap2.c

@line 1028

/* DIP switches on some boards change between 8 and 16 bit            
         * bus widths for flash.  Try the other width if the first try fails. 
         */                                                                   
                                                                              
        if (nand_scan(&info->mtd, 1)) {                                       
                info->nand.options ^= NAND_BUSWIDTH_16;                       
                if (nand_scan(&info->mtd, 1)) {                               
                        err = -ENXIO;                                         
                        goto out_release_mem_region;                          
                }                                                             
        }

it seems the nand scan will try 8-bit width, if it fails, then will try 16-bit width. Can the driver realize the bus width by scanning the nand chip info by itself.

Moreover, I referred to the section 9.1.5.14 of the AM3517 TRM for more information for NAND device basic programming model, and follow its guide, I did the following changes.

@kernel/arch/arm/plat-omap/include/plat/gpmc.h
@line 45
--------from---------
#define GPMC_CONFIG1_DEVICESIZE_16      GPMC_CONFIG1_DEVICESIZE(1)
---------to----------
#define GPMC_CONFIG1_DEVICESIZE_16      GPMC_CONFIG1_DEVICESIZE(0)

I am not sure whether other places should be changed.

I also test with ubiformat when booting from MMC, it works fine on board with Micron NAND 512MiB 1,8V 16-bit, fails on board with Samsung NAND 512MiB 3,3V 8-bit.

and some errors was printed in kernel when ubiformat doing read/write operations. such as

ubiformat: error!: cannot write eraseblock 383
           error 5 (Input/output error)
ubiformat: mark it as bad? Continue (yes/no) yes
ubiformat: marking block 383 bad
ubiformat: flashing eraseblock 478 -- 100 % complete  ubiformat: error!: eof reached; 131072 bytes remaining

Another interesting thing is,

when I test with regular linux,I fastboot rootfs.jffs2 image to NAND, it can work, but ubi.img can't.. however, the android  works on ubifs or yaffs2 instead of jffs2.

Thanks,

Kevin

Hi Amit Pundir,

In my opinion, I think NAND drivers is OK.

The layers of the UBI, is Hardware Driver -> MTD -> UBI -> UBIFS, since rootfs.jffs2 works ok,  the layer of Hardware Drivers -> MTD should be ok.

In the kernel log,

UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 3935:0, written 0 bytes
UBI error: erase_worker: failed to erase PEB 3935, error -5
UBI: mark PEB 3935 as bad
UBI error: ubi_io_mark_bad: cannot mark PEB 3935 bad, error -5
UBI warning: ubi_ro_mode: switch to read-only mode
UBI error: do_work: work failed with error code -5
UBI error: ubi_thread: ubi_bgt0d: work failed with error code -5
omaplfb: OMAPLFB_Init: OMAPLFBInit failed
UBIFS error (pid 845): ubifs_wbuf_write_nolock: cannot write 3778 bytes to LEB 477:0, error -30
UBIFS warning (pid 845): ubifs_ro_mode: switched to read-only mode, error -30
UBIFS error (pid 845): do_writepage: cannot write page 0 of inode 770, error -30
UBIFS error (pid 782): make_reservation: cannot reserve 424 bytes in jhead 1, error -30
UBIFS error (pid 782): ubifs_create: cannot create regular file, error -30

UBI error printed firstly, and then the UBIFS errors, so the problem may be in the layer MTD -> UBI, this problem leads to another problem in the above layer   UBI -> UBIFS,

Thanks,

Kevin

Hi Kevin,

Good to know that jffs2 works for you. So I assume NAND mtd layer to be working fine.

Few suggestions/tips I found for UBIFS, in case you have not tried them yet:

1. This is what is on UBIFS FAQ:

"
I get "ubi_io_write: error -5 while writing 512 bytes to PEB 5:512"

If you have a 2048 bytes per NAND page device, and have CONFIG_MTD_NAND_VERIFY_WRITE enabled in your kernel, you will need to turn it off. The code does not currently (as of 2.6.26) perform verification of sub-page writes correctly. As UBI is one of the few users of sub-page writes, not much else seems to be affected by this bug.
"

2. This error is reported for corrupt ubi images as well. Kindly check that the md5sum of ubi image you are creating and flashing are same.

3. You can also get this error if the NAND is being accessed simultaneously by any other controller on your platform. For example, this error is being reported on platforms which have both NOR and NAND served by the same controller.

4. Some have also reported this issue specific to 2.6.32 kernel http://www.jukie.net/bart/blog/ubifs-on-sheeva . You might be hitting those un-discovered/reported corner cases. I have seen patches being pushed in 2.6.32-longterm kernel for ubifs lately. It would be nice if you diff/sync the UBIFS driver to the latest.

Regards,
Amit Pundir

Hi Amit Pundir,

In init.rc file, I comment off zygote script,

#service zygote /system/bin/app_process -Xzygote /system/bin --zygote --start-system-server
#    socket zygote stream 666
#    onrestart write /sys/android_power/request_state wake
#    onrestart write /sys/power/state on
#    onrestart restart media

then create the ubi.img,  this time the errors disappear, because zygote didn't startup and android processes didn't start up either,

Except that, I tried some of your suggestions,

1.  I tested  the kernel with CONFIG_MTD_NAND_VERIFY_WRITE option disabled all the time.

2. I am flashing the same image as I have created, for they are of the same md5sum.

3. I substitute kernel/drivers/mtd/ubi and kernel/fs/ubifs from linux-2.6.32.41-longterm  for that in ti android kernel.

however, after trying the above steps, the errors are the same.

I am not sure whether both NOR and NAND are served by the same conntroller, I don't know how to recognize the controller declaration in sources.

The above is what I have tried these days.

Thanks,

Kevin

Hi Kevin,

Are you saying that you are able to mount Android rootfs if you disable all the Android services e.g. ServiceManager, Zygote, MediaServer etc? That is strange.

If that is the case have you tried mounting a regular Linux file-system now instead of Android file-system?

Regards,
Amit Pundir

Hi Amit Pundir,

I have tried with a regular Linux file-system, no such errors in kernel log, it works ok.

I have tried with AM3517EVM with Micron NAND 512MiB 1,8V 16-bit, it works ok, all Android services start up correctly.

I have tried with CustomBoard with Samsung NAND 512MiB 3,3V 8-bit, it doesn't start up correctly, and shows errors we discussed before.

Is it a regular problem or an interesting problem, I think someone else would meet the same problem as me, but I haven't met one.

Another thing,

when booting from MMC, copy  the same ubi.img through ubiformat, on am3517evm, it works OK, but on my custom board, it can't.

Thanks,

Kevin

Kevin,

This is indeed a strange problem.

On Android UBI image you are getting Write errors. So I want to know if you just mounted the UBI image of normal Linux file system or also tried some I/O operations? Frankly it should fail as well. But if you are able to do write operations on Linux UBI image then I would suggest you to start Android processes turn by turn to isolate the service/process which is giving you crash logs on logcat and dmesg. In init.rc first start "servicemanager" if it doesn't crash then go on starting "vold", "netd", "zygote" etc turn by turn.

Regards,
Amit Pundir

Hi Amit Pundir,

WITH NORMAL LINUX:

I tried with a normal linux provided with am3517evm,

First, I use init=/init in u-boot argument, but in fact it should be init=/linuxrc, I forget to change it for the first time, and many ubi errors and ubifs errors display on the console,

Second, I use init=/linuxrc in u-boot argument, neither ubi errors nor ubifs errors appear, but after kernel log

......

VFS: Mounted root (ubifs filesystem) on device 0:12.
Freeing init memory: 172K
Warning: unable to open an initial console.

no response even after clicking Enter Key.

Third, I use init=/init again, no response after start up.

Fourth, just reboot again, command # will show on the console, and no ubi or ubifs errors in the log.

After login in,  I can do mkdir/rmdir operations.

WITH ANDROID:

When servicemanager and other services were comment off, then no ubi errors shows in the kernel log, I can do mkdir/rmdir operations.

When I just start servicemanager in init.rc, then only one ubi error show up., I can do mkdir/rmdir operations.

UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 735:0, written 0 bytes
UBI error: erase_worker: failed to erase PEB 735, error -5
UBI: mark PEB 735 as bad
UBI: 38 PEBs left in the reserve

when I start servicemanager, vold, netd, rild, and etc, but not zygote, then only one ubi error shows up, maybe first time startup., and nexttime startup no ubi errors, and  I can do mkdir/rmdir operations.

when I start zygote in init.rc, then the ubi errors will show up repeatly., I can't do mkdir/rmdir operations anymore.

UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 2846:0, written 0 bytes
UBI error: erase_worker: failed to erase PEB 2846, error -5
UBI: mark PEB 2846 as bad
UBI error: ubi_io_mark_bad: cannot mark PEB 2846 bad, error -5
UBI warning: ubi_ro_mode: switch to read-only mode
UBI error: do_work: work failed with error code -5
UBI error: ubi_thread: ubi_bgt0d: work failed with error code -5
UBIFS error (pid 861): ubifs_wbuf_write_nolock: cannot write 3778 bytes to LEB 477:0, error -30
UBIFS warning (pid 861): ubifs_ro_mode: switched to read-only mode, error -30
UBIFS error (pid 861): do_writepage: cannot write page 0 of inode 764, error -30
UBIFS error (pid 805): make_reservation: cannot reserve 424 bytes in jhead 1, error -30
UBIFS error (pid 805): ubifs_create: cannot create regular file, error -30

............

Thanks,

Kevin

Hi Kevin,

I'm running out of ideas for you at the moment. I also placed your query on Android Porting list but no success in that.

Luckily I managed to get a development board with Samsung 8bit NAND. I have not started working on it yet. But I think once I'm able reproduce this problem on that board I would be in much better shape to help you out.

Meanwhile I suggest you raise this query in Android-Porting, UBIFS/MTD or other similar concerned mailing forums.

Regards,
Amit Pundir

Hi Amit Pundir,

I did a new test, I just write the result down here to help analyse if needed.

I only fastboot xloader, uboot, uImage to NAND, no ubi.img, and in the boot log, there are still some ubi errors,

......

UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 339:0, written 0 bytes
UBI error: erase_worker: failed to erase PEB 339, error -5
UBI: reserve more 1 PEBs
UBI: mark PEB 339 as bad
UBI: 40 PEBs left in the reserve

......

UBI error: ubi_io_write: error -5 while writing 512 bytes to PEB 551:0, written 0 bytes
UBI error: erase_worker: failed to erase PEB 551, error -5
UBI: reserve more 1 PEBs
UBI: mark PEB 551 as bad
UBI: 40 PEBs left in the reserve

......

Thanks

Kevin

Hi Kevin,

Are you erasing/clearing the UBI partition you created/flashed earlier?

$ sudo fastboot erase <partition_name>

Regards,
Amit Pundir

Hi Amit Pundir,

I cleared all the partitions by "#nand scrub" in u-boot command, as I want to start flashing from scratch.

Then, I reboot to u-boot command from MMC, and fastboot x-loader, u-boot, uImage to NAND.

Thanks,

Kevin

Could You tell me where exactly can i enabled  MTD_NAND_OMAP_PREFETCH and MTD_NAND_OMAP_PREFETCH_DMA. I try found it in ach/arm/config/(in source from rowboat android gingerbreak) but they aren't there

There is a .config named file in the root directory of the Kernel, if not, please "$make am3517_evm_android_defconfig" in that directory first.

Then, you can "$make menuconfig", and look for "Device Drivers -> Memory Technology Device (MTD) support -> NAND Device Support -> NAND Flash device on OMAP2 and OMAP3", select both "GPMC prefetch support for NAND Flash device" and "DMA mode" under "NAND Flash device on OMAP2 and OMAP3" item, then Save & Exit, "$make CROSS_COMPILE=arm-eabi-  uImage", That is all.