DM365 SD Card Issues

I've found the same problem (montavista kernel, DVSDK2.10).

If a program is writing to SD card and the card is removed, then the program locks on write. It cannot be killed in any way, the filesystem cannot be umounted and any access to the filesystem locks the program accessing it.

I hope someone will suggest a solution for this.

Can anyone confirm if this problem has been solved on 3.10?

Thanks!

I alse want to know!

Thanks!

Hello,

Plugging out MMC/SD card without unmounting is not supported since there could be in-flight transfers which can result in corrupted data.

I do not believe this is a limitation with TI devices, but a general MMC/SD limitation. In fact removing any removable storage (like USB drive) without unmounting would not be recommended.

Thanks,

Sekhar

Hi Sekhar,

while I do strongly agree that removing a device while writing on it is a really bad idea, there are 2 points:

1) System stability should not be affected by this (data loss may be acceptable). Instead I've seen that the system is affected in several functionalities:

- The umount of the filesystem hangs, blocking any process trying to do it. Even "lazy" or "forced" umount do not work.

- Any other umount on other filesystem blocks, the same for some system commands (i.e. sync).

- Since reboot requires to sync all devices, the system hangs on reboot.

- The SD slot becomes unusable, since the file system cannot be umounted. So inserting another SD/MMC is useless.

- Processes accessing the unmounted filesystem block and cannot be killer anyway.

- The serial device "/dev/ttySx" does not work anymore.

2) People using embedded devices and easily removable storages often happen to show the bad habit of unplugging power or removing units even if they are told not to do it. They expect the device to survive and still be usable after the bad event.

So, my concern it not for the data (there are even filesystem strong enough to survive an hot removal) but the problem is the system.

Regards,

Marco

Hi Marco,

Agreed that plugging out MMC/SD card without unmount should  not cause system stability issues.

Are you able to see this issue with DVSDK 3.10?

Also, can you please try this by keeping CONFIG_HOTPLUG disabled in Linux kernel configuration case the removal is triggering a userspace process which is trying do do an unmount causing instability.

Thanks,

Sekhar

Hi Sekhar,

1) I am still evaluating the impact of switching to 3.10 in our products, so at the moment I haven't tried it on 3.10. Honeslty I hope that the problem is fixed in 3.10's kernel. It is a lot newer...

2) I confirm that hotplug is enabled and mdev is on, also if the card is removed, a "lazy" umount is performed. That's how we want it to work. And in fact, if nobody's writing on the card, it works well. If I remove hotplug, the kernel won't be able to know if a new SD must be mounted. Switching the media under the file system is a good way to corrupt a working file system. So, this does not solve the problem. If the SD is removed, it must be remounted if a new one is inserted. Since umount does not work, I can't insert a new SD. This simply leads to a different failure. I'm not sure if I've showed the point.

This problem is impacting our ability to provide DM365 products that operate with SD in a reliable way. Even if we specify that the SD must not be removed until unmounted (very unelegant, even if true) there will always be someone trying it and complaining that the device locked up. Sadly people get bad habits from the PC world and expect embedded devices to "just work", like a toaster or a microwave oven...

Sekhar Nori said:

I confirm that hotplug is enabled and mdev is on, also if the card is removed, a "lazy" umount is performed. That's how we want it to work

Why do you need the system to umount after the card is removed? Why not just remove the mount point and have hotplug mount again at re-insertion?

umounting a removed card wont be of much use.

[/quote]

What do you mean with "remove the mount point"? Suppose an SD is inserted... I need to mount the block device file system to a directory.

If the SD is removed, the file system still appears as in use, and the mount point if still mapped to that device/file system. The mount point by itself cannot be "deleted" if there is a file system mounted on it. If there is any command to suggest...

Umounting a removed card means that the system does not map that filesystem anymore.

Can you suggest an example on how to "remove the mount point" without unmounting the file system on it? If then an SD is inserted and removed several time, won't this leave the system with a lot of mounted file systems that will never be used anymore?

 

Hi Marco,

Yes, I do see not doing an unmount over time will create lot of unused mounted filesystems.

Have you tried mount using '-o sync' option too? That should reduce the probability of instability by flushing the writes to disk along with the write call.

Thanks,

Sekhar

Hi Sekhar,

no I have not tried on DM365 specifically, but on other DaVinci I've noticed that the throughput is severly reduced with this option. I'll try it, at least I am curious to know if the umount problem is solved.

Hello,

As a notice, I've tryed the "sync" option. It does not solve the problem.

Hello,

I also need to know this solution.

If somebody find how to modify it , please let us know.

Thanks.

I am currently working on this issue / similar issue on the DM355 on the 2.6.32 kernel. I have it working with quite a few changes and am hoping to localize it to the davinci mmc driver if possible. If any one else is working on this, let me know!

I'm also seeing this same issue, but with USB Thumb Drives. Removing the drive without first unmounting causes a deadlock in the udev mount.sh (unmount) script. I'd be curious what changes you are making to the mmc driver and how it might apply to this USB/thumb problem.

I  could not solve the issue with just MMC driver changes. I had to tweek some things in the upper layers of the MMC block layer to get the umount to work. By any chance are you using the arrago kernel?

Yes. Using the 2.6.32 Arago kernel, and 3...19 Dvsdk.

What did you find was doing the locking? Something waiting for a hardware response, or some kind of thread contention?

The locking looked like was happening because of a del_gendisk. The lock up happens when sync_inodes_sb (fs/fs-writeback.c) is called for a 2nd time. It is called once during del_gendisk and then again drring umount. bdi_sync_writeback is called and will queue uninterruptible work that will never reach the device driver. I think something was changed in the Arago kernel that is causing this as I have not seen this issue in other devices using mainline / ubuntu kernel. The way I got around this was having the kernel call umount before del_gendisk, sync_inodes_sb is not called a 2nd time when this is the case. This will remove the mount and any open file descriptors will receive errors allowing them to close.

Sounds like a reasonable fix. Do you have a patch you could post that I could try out?

Give this a shot, let me know how it goes.

Nice! Patch worked great. Looks pretty clean to me as well. I can now plug in and out thumb drives without locking the system...

Though, of course anything being written to the drive at the moment of being yanked out is gone, but at least plugging the drive back in mounts it and there are no zombie processes anymore.

Thanks! Hopefully this or something that fixes this can be pushed into the main arago kernel.

Looks like there is still some lock possibility. If the drive is removed during a write, it can also lock. Although, at least less likely than before, still something is seriously wrong with file handling in this kernel.

This is more of a work around at the moment, the underlining issue still exists somewhere. I hope the patch helps for now until it is found and fixed!

That is what I feared... If no one else fixes this, I will provide a patch when I get back to it. For now, the work around is sufficient  for demo purposes.

Thanks again!

I've managed to work around the issue entirely. At first I was looking at a specific patch set that was in the mainline .35 and up kernels. however, by migrating to the latest Arago .34 kernel, the problem no longer happens. Both USB and SD cards work fine with unexpected removals. At least, no user space process locks up, and udev is able to remount the device next time it is inserted.

Thats great news, thank you! I will be doing the same.

I don't  use  Arago kernel, And my kernel is  Linux ADST-E1-009000878890 2.6.18_pro500-davinci_evm-arm_v5t_le #1 Fri Dec 31 13:53:25 CST 2010 armv5tejl GNU/Linux.

i meet  this issue too,  mmc  del_gendisk will  locks during a write

could you  do me a favor ? 

step

1.   mount /dev/mmcblk0p1  /mnt/sd

1.   remove  sd card when  writting some data to /mmt/sd

2.  umounting  /mnt/sd    locks

That is the same issue I ran into. If you check the patch notes I posted you can make the same modifications I made for a work around.

hi,  Dmitri Zakharevski , thank your reply !

I try to patch linux kernel  via  the patch notes you posted,  maybe my kernel is 2.6.18_pro500-davinci_evm-arm_v5t_le, and your patch is to  Arago kernel,  so fail to patch check.c

patching file fs/namespace.c
Hunk #1 succeeded at 176 (offset -307 lines).
Hunk #2 succeeded at 851 with fuzz 2 (offset -173 lines).
patching file fs/partitions/check.c
Hunk #1 FAILED at 20.
Hunk #2 FAILED at 651.
2 out of 2 hunks FAILED -- saving rejects to file fs/partitions/check.c.rej
patching file include/linux/fs.h
Hunk #1 succeeded at 915 (offset -409 lines).
patching file include/linux/mount.h
Hunk #1 succeeded at 95 with fuzz 2 (offset -30 lines).

//my del_gendisk source code

void del_gendisk(struct gendisk *disk)
{
 int p;

 /* invalidate stuff */
 for (p = disk->minors - 1; p > 0; p--) {
  invalidate_partition(disk, p);
  delete_partition(disk, p);
 }
 invalidate_partition(disk, 0);
 disk->capacity = 0;
 disk->flags &= ~GENHD_FL_UP;
 unlink_gendisk(disk);
 disk_stat_set_all(disk, 0);
 disk->stamp = 0;

 kobject_uevent(&disk->kobj, KOBJ_REMOVE);
 if (disk->holder_dir)
  kobject_unregister(disk->holder_dir);
 if (disk->slave_dir)
  kobject_unregister(disk->slave_dir);
 if (disk->driverfs_dev) {
  char *disk_name = make_block_name(disk);
  sysfs_remove_link(&disk->kobj, "device");
  if (disk_name) {
   sysfs_remove_link(&disk->driverfs_dev->kobj, disk_name);
   kfree(disk_name);
  }
  put_device(disk->driverfs_dev);
  disk->driverfs_dev = NULL;
 }
 sysfs_remove_link(&disk->kobj, "subsystem");
 kobject_del(&disk->kobj);
}

I  merge your mount.txt  into  my  kernel,  because, your patch is not to my kernel, so some functons can not  compiled  and linked,  

My  kenel is 2.6.18_pro500-davinci_evm-arm_v5t_le,   i  am not familiar with kernel source code, could you give me a patch for me,.

thanks!

//red  parts  linking  error

+                sb = bdget_disk(disk, part->partno)->bd_super;
+               
+                if (!sb) {
+                        continue;
+                }
+               
+                d_root = sb->s_mountpoint;
+                mnt = dentry_path(d_root, buf, sizeof(buf));
+               
+                if(IS_ERR(mnt)) {
+                        goto del_partition;
+                }
+               
+                user_path(mnt, &path);       
+               
+                if (path.dentry != path.mnt->mnt_root)
+                        goto dput_and_out;
+                if (!capable(CAP_SYS_ADMIN))
+                        goto dput_and_out;
+               
+                do_umount(path.mnt, MNT_DETACH);
+               
+        dput_and_out:
+               
+                dput(path.dentry);
+                mntput_no_expire(path.mnt);
+       
+        del_partition:

It would be nice, if sd/fat FS would support option intr of NFS:

intr

If an NFS file operation has a major timeout and it is hard mounted, then allow signals to interupt the file operation and cause it to return EINTR to the calling program. The default is to not allow file operations to be interrupted.

i try to umount sd card before del_gendisk,  but the issue still appear, del_gendisk locks

static void mmc_blk_remove(struct mmc_card *card)
{
 struct mmc_blk_data *md = mmc_get_drvdata(card);

 if (md) {
  int devidx;

  sys_umount("/mnt/sd", 0 );
        printk("%s %d\n", __FUNCTION__, __LINE__ );
  /* Stop new requests from getting into the queue */
  del_gendisk(md->disk);
        printk("%s %d\n", __FUNCTION__, __LINE__ );
  /* Then flush out any already in there */
  mmc_cleanup_queue(&md->queue);

  devidx = md->disk->first_minor >> MMC_SHIFT;
  __clear_bit(devidx, dev_use);

  mmc_blk_put(md);
 }
 mmc_set_drvdata(card, NULL);
}

I try to mount sd/vfat with option intr, but it fails

it worked fine for me, too.

But I Think if sb == NULL you should go to del_partition ?!

 	while ((part = disk_part_iter_next(&piter))) {
+                sb = bdget_disk(disk, part->partno)->bd_super;
+                
+                if (!sb) {
+                        goto del_partition;
+                }

Otherwise you'll get something like: "sda: p1 could not be added: 17"  if  plug and unplug and replug a usb-storage-device
it without mounting it after first plug ;)

hello Marco:

could you teach me how to let mdev on busybox, i also use dm365(dm368) , ...thank you!