This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

System freezes when SD card is removed in the middle of writes

Hi,

While writing to an SD card, sometimes the system freezes when the SD card is removed in the middle of write operation.

(Application is wring to a file on SD card)

<------ SD card is removed

[   49.067382] mmcblk0: error -84 transferring data, sector 20296, nr 1, cmd response 0x900, card status 0x0 

<------ System freezes here

This symptom happens with Beagle Bone with Sitara SDK 06.00.00.00.

Does anybody know how to fix this problem?

  • Hi,

    I will ask the SW team to look at this.

  • Hi,

    you can try to debug code of this module <>/drivers/mmc/card/block.c, there on line 976 appears the error message.
    I have tested similar scenario on BeagleBone Black with SDK6 kernel and have no system halts.
    Try it with SDK7.

    BR,
    Georgi
  • Hi Georgi,

    With SDK6, system lockup happens rarely (I would say it occurs once out of ten times). Most of the time, write() system call returns EIO(5): "Input/output error" so that application is able to exit gracefully.

    Actually, SDK7 behaves worse than SDK6. The kernel does not even return. The serial console still accepts and echoes back my keyboard input so I don't think the kernel is completely locked up, but it doesn't return from the write() system call so application can do nothing.

    To reproduce this problem, I use the following sample code.

    #include <stdio.h>
    #include <stdlib.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    
    main()
    {
    	int fd;
    	char buf[1024];
    
    	system("mkdir /media/mmcblk0p1/000000");
    
    	fd = open("/media/mmcblk0p1/000000/AAAA", O_RDWR | O_CREAT |O_APPEND|O_NONBLOCK|O_SYNC);
    	while(1) {
    		// initialize the file descriptor sets
    		fd_set writeSet;
    		FD_ZERO(&writeSet);
    		FD_SET(fd, &writeSet);
    
    		// 1s timeout. We need to set the timeout in every loop!
    		struct timeval timeOut;
    		timeOut.tv_sec = 1;
    		timeOut.tv_usec = 0;
    
    		// wait for the event
    		int32_t status = select(fd+1, NULL, &writeSet, NULL, &timeOut);
    		if (status <= 0)
    		{
    			printf("retStatus = SD_MEDIA_ERROR;\n");
    			break;
    		}
    
    		if (FD_ISSET(fd, &writeSet))
    		{
    			int32_t ret = write(fd, buf, sizeof(buf));
    			if (ret < 0)
    			{
    				printf("retStatus = SD_MEDIA_ERROR;\n");
    				usleep(10000);
    				break;
    			}
    		}
    	}
    	close(fd);
    }

  • I just gave it a try with SDK8, but the same problem exists.
  • Hi,

    what board are you working on? what are the symptoms of system freeze, except console hangs, do you use lcd?

    Gerogi
  • I use BeagleBone with pre-built MLO, u-boot, uImage and file system plus my simple test application (source code is on my previous post).  

    I don't use LCD.

    SDK6:

    When the system hangs, the console completely freezes and ping stops responding too.  

    SDK7 & 8:

    When the system hangs, the console still accepts input and echoes back.  Ping responds too.  But application doesn't return.

  • Hi,

    Looks like your application hangs, because of use of break; try it with exit();

    BR,
    Georgi
  • I gave it a try with exit(), but the problem still happens.

    Actually, when the problem occurs, the application process doesn't even get terminated by CTRL+C.  Therefore, I don't think this is an application hang.  If it was application hang, I would have been able to terminate the process by pressing CTRL+C.    

    Here is the updated code with exit().

    #include <stdio.h>
    #include <stdlib.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    
    main()
    {
    	int fd;
    	char buf[1024];
    
    	system("mkdir /media/mmcblk0p1/000000");
    
    	fd = open("/media/mmcblk0p1/000000/AAAA", O_RDWR | O_CREAT |O_APPEND|O_NONBLOCK|O_SYNC);
    	while(1) {
    		// initialize the file descriptor sets
    		fd_set writeSet;
    		FD_ZERO(&writeSet);
    		FD_SET(fd, &writeSet);
    
    		// 1s timeout. We need to set the timeout in every loop!
    		struct timeval timeOut;
    		timeOut.tv_sec = 1;
    		timeOut.tv_usec = 0;
    
    		// wait for the event
    		int32_t status = select(fd+1, NULL, &writeSet, NULL, &timeOut);
    		if (status <= 0)
    		{
    			printf("retStatus = SD_MEDIA_ERROR;\n");
    			/*break;*/
    			exit(1);
    		}
    
    		if (FD_ISSET(fd, &writeSet))
    		{
    			int32_t ret = write(fd, buf, sizeof(buf));
    			if (ret < 0)
    			{
    				printf("retStatus = SD_MEDIA_ERROR;\n");
    				usleep(10000);
    				/*break;*/
    				exit(1);
    			}
    		}
    	}
    	close(fd);
    	exit(0);
    }

  • Can you, please, share your log .
    I am trying to reproduce it, but my system does not hang.

    Georgi
  • Attached please find the log.  The problem happened immediately after SD card is removed.

    SDK7.txt
    U-Boot SPL 2014.07-g7e537bf (Feb 10 2015 - 15:24:43)
    reading args
    spl_load_image_fat_os: error reading image args, err - -1
    reading u-boot.img
    reading u-boot.img
    
    
    U-Boot 2014.07-g7e537bf (Feb 10 2015 - 15:24:43)
    
    I2C:   ready
    DRAM:  256 MiB
    NAND:  0 MiB
    MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
    reading uboot.env
    
    ** Unable to read "uboot.env" from mmc0:1 **
    Using default environment
    
    Net:   <ethaddr> not set. Validating first E-fuse MAC
    cpsw, usb_ether
    Hit any key to stop autoboot:  0
    switch to partitions #0, OK
    mmc0 is current device
    SD/MMC found on device 0
    reading uEnv.txt
    175 bytes read in 4 ms (42 KiB/s)
    Loaded environment from uEnv.txt
    Importing environment from mmc ...
    Running uenvcmd ...
    Booting from network ...
    cpsw Waiting for PHY auto negotiation to complete. done
    link up on port 0, speed 100, full duplex
    BOOTP broadcast 1
    DHCP client bound to address 172.16.40.203
    link up on port 0, speed 100, full duplex
    Using cpsw device
    TFTP from server 172.16.40.43; our IP address is 172.16.40.203
    Filename 'chantaek/zImage-am335x-evm.bin'.
    Load address: 0x82000000
    Loading: #################################################################
             #################################################################
             #################################################################
             #################################################################
             #####################
             2.3 MiB/s
    done
    Bytes transferred = 4117616 (3ed470 hex)
    link up on port 0, speed 100, full duplex
    Using cpsw device
    TFTP from server 172.16.40.43; our IP address is 172.16.40.203
    Filename 'chantaek/am335x-bone.dtb'.
    Load address: 0x88000000
    Loading: ###
             2.1 MiB/s
    done
    Bytes transferred = 33206 (81b6 hex)
    Kernel image @ 0x82000000 [ 0x000000 - 0x3ed470 ]
    ## Flattened Device Tree blob at 88000000
       Booting using the fdt blob at 0x88000000
       Loading Device Tree to 8e714000, end 8e71f1b5 ... OK
    
    Starting kernel ...
    
    [    0.000000] Booting Linux on physical CPU 0x0
    [    0.000000] Linux version 3.12.10-ti2013.12.01 (jenkins@sdit-build02) (gcc version 4.7.3 20130226 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) ) #1 Sun Mar 30 20:55:31 CDT 2014
    [    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d
    [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
    [    0.000000] Machine: Generic AM33XX (Flattened Device Tree), model: TI AM335x BeagleBone
    [    0.000000] cma: CMA: reserved 24 MiB at 8c800000
    [    0.000000] Memory policy: ECC disabled, Data cache writeback
    [    0.000000] CPU: All CPU(s) started in SVC mode.
    [    0.000000] AM335X ES1.0 (sgx neon )
    [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 64256
    [    0.000000] Kernel command line: console=ttyO0,115200n8 root=/dev/nfs nfsroot=172.16.40.43:/home/chantaek/Projects/IB/Embedded/beagleFS_07.00,nolock rw ip=dhcp
    [    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
    [    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
    [    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
    [    0.000000] Memory: 223456K/259072K available (5555K kernel code, 559K rwdata, 1876K rodata, 345K init, 225K bss, 35616K reserved, 0K highmem)
    [    0.000000] Virtual kernel memory layout:
    [    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    [    0.000000]     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    [    0.000000]     vmalloc : 0xd0800000 - 0xff000000   ( 744 MB)
    [    0.000000]     lowmem  : 0xc0000000 - 0xd0000000   ( 256 MB)
    [    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    [    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
    [    0.000000]       .text : 0xc0008000 - 0xc074a04c   (7433 kB)
    [    0.000000]       .init : 0xc074b000 - 0xc07a15f0   ( 346 kB)
    [    0.000000]       .data : 0xc07a2000 - 0xc082ddd8   ( 560 kB)
    [    0.000000]        .bss : 0xc082ddd8 - 0xc0866400   ( 226 kB)
    [    0.000000] NR_IRQS:16 nr_irqs:16 16
    [    0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
    [    0.000000] Total of 128 interrupts on 1 active controller
    [    0.000000] OMAP clockevent source: timer2 at 24000000 Hz
    [    0.000000] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
    [    0.000000] OMAP clocksource: timer1 at 24000000 Hz
    [    0.000000] Console: colour dummy device 80x30
    [    0.000337] Calibrating delay loop... 479.23 BogoMIPS (lpj=2396160)
    [    0.119756] pid_max: default: 32768 minimum: 301
    [    0.119857] Security Framework initialized
    [    0.119909] Mount-cache hash table entries: 512
    [    0.127400] CPU: Testing write buffer coherency: ok
    [    0.127827] Setting up static identity map for 0xc0572e68 - 0xc0572ed8
    [    0.128728] devtmpfs: initialized
    [    0.130690] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
    [    0.192676] omap_hwmod: debugss: _wait_target_disable failed
    [    0.193405] pinctrl core: initialized pinctrl subsystem
    [    0.194283] regulator-dummy: no parameters
    [    0.196787] NET: Registered protocol family 16
    [    0.199051] DMA: preallocated 256 KiB pool for atomic coherent allocations
    [    0.201831] cpuidle: using governor ladder
    [    0.201849] cpuidle: using governor menu
    [    0.210554] platform mpu.1: FIXME: clock-name 'fck' DOES NOT exist in dt!
    [    0.211650] platform 49000000.edma: FIXME: clock-name 'fck' DOES NOT exist in dt!
    [    0.212886] OMAP GPIO hardware version 0.1
    [    0.220836] platform 56000000.sgx: FIXME: clock-name 'fck' DOES NOT exist in dt!
    [    0.222453] DSS not supported on this SoC
    [    0.222473] No ATAGs?
    [    0.222484] hw-breakpoint: debug architecture 0x4 unsupported.
    [    0.246510] bio: create slab <bio-0> at 0
    [    0.261355] edma-dma-engine edma-dma-engine.0: TI EDMA DMA engine driver
    [    0.262190] vmmcsd_fixed: 3300 mV
    [    0.264955] vgaarb: loaded
    [    0.266255] SCSI subsystem initialized
    [    0.267589] usbcore: registered new interface driver usbfs
    [    0.267773] usbcore: registered new interface driver hub
    [    0.267974] usbcore: registered new device driver usb
    [    0.268928] omap_i2c 44e0b000.i2c: could not find pctldev for node /pinmux@44e10800/pinmux_i2c0_pins, deferring probe
    [    0.268956] platform 44e0b000.i2c: Driver omap_i2c requests probe deferral
    [    0.269158] media: Linux media interface: v0.10
    [    0.269334] Linux video capture interface: v2.00
    [    0.269600] pps_core: LinuxPPS API ver. 1 registered
    [    0.269610] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    [    0.269808] PTP clock support registered
    [    0.272428] Switched to clocksource timer1
    [    0.291250] NET: Registered protocol family 2
    [    0.291935] TCP established hash table entries: 2048 (order: 2, 16384 bytes)
    [    0.291989] TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
    [    0.292020] TCP: Hash tables configured (established 2048 bind 2048)
    [    0.292097] TCP: reno registered
    [    0.292112] UDP hash table entries: 256 (order: 0, 4096 bytes)
    [    0.292142] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
    [    0.292352] NET: Registered protocol family 1
    [    0.292827] RPC: Registered named UNIX socket transport module.
    [    0.292840] RPC: Registered udp transport module.
    [    0.292847] RPC: Registered tcp transport module.
    [    0.292854] RPC: Registered tcp NFSv4.1 backchannel transport module.
    [    0.293774] NetWinder Floating Point Emulator V0.97 (double precision)
    [    0.294379] PM: Loading am335x-pm-firmware.bin
    [    0.450726] VFS: Disk quotas dquot_6.5.2
    [    0.450807] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
    [    0.451419] NFS: Registering the id_resolver key type
    [    0.451519] Key type id_resolver registered
    [    0.451529] Key type id_legacy registered
    [    0.451577] jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
    [    0.451765] msgmni has been set to 484
    [    0.453503] NET: Registered protocol family 38
    [    0.453549] io scheduler noop registered
    [    0.453558] io scheduler deadline registered
    [    0.453593] io scheduler cfq registered (default)
    [    0.455540] pinctrl-single 44e10800.pinmux: 142 pins at pa f9e10800 size 568
    [    0.461760] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    [    0.464639] 44e09000.serial: ttyO0 at MMIO 0x44e09000 (irq = 88, base_baud = 3000000) is a OMAP UART0
    [    1.066420] console [ttyO0] enabled
    [    1.071499] omap_rng 48310000.rng: OMAP Random Number Generator ver. 20
    [    1.088740] brd: module loaded
    [    1.097327] loop: module loaded
    [    1.103899] mtdoops: mtd device (mtddev=name/number) must be supplied
    [    1.114197] usbcore: registered new interface driver asix
    [    1.120132] usbcore: registered new interface driver ax88179_178a
    [    1.126751] usbcore: registered new interface driver cdc_ether
    [    1.133099] usbcore: registered new interface driver r815x
    [    1.139096] usbcore: registered new interface driver smsc95xx
    [    1.145318] usbcore: registered new interface driver net1080
    [    1.151474] usbcore: registered new interface driver cdc_subset
    [    1.157876] usbcore: registered new interface driver zaurus
    [    1.164029] usbcore: registered new interface driver cdc_ncm
    [    1.170502] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    [    1.177436] ehci-pci: EHCI PCI platform driver
    [    1.182336] ehci-omap: OMAP-EHCI Host Controller driver
    [    1.188729] usbcore: registered new interface driver cdc_wdm
    [    1.194949] usbcore: registered new interface driver usb-storage
    [    1.202320] mousedev: PS/2 mouse device common for all mice
    [    1.210997] omap_rtc 44e3e000.rtc: rtc core: registered 44e3e000.rtc as rtc0
    [    1.218583] 44e3e000.rtc: already running
    [    1.223549] i2c /dev entries driver
    [    1.227746] Driver for 1-wire Dallas network protocol.
    [    1.235426] omap_wdt: OMAP Watchdog Timer Rev 0x01: initial timeout 60 sec
    [    1.244487] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
    [    1.251684] edma-dma-engine edma-dma-engine.0: allocated channel for 0:24
    [    1.259040] omap_hsmmc 48060000.mmc: unable to get vmmc regulator -517
    [    1.265995] edma-dma-engine edma-dma-engine.0: freeing channel for 24
    [    1.272825] edma-dma-engine edma-dma-engine.0: freeing channel for 25
    [    1.279736] platform 48060000.mmc: Driver omap_hsmmc requests probe deferral
    [    1.289333] ledtrig-cpu: registered to indicate activity on CPUs
    [    1.296091] edma-dma-engine edma-dma-engine.0: allocated channel for 0:36
    [    1.303358] omap-sham 53100000.sham: hw accel on OMAP rev 4.3
    [    1.312859] omap_hwmod: gpio1: _wait_target_disable failed
    [    1.319607] omap-aes 53500000.aes: OMAP AES hw accel rev: 3.2
    [    1.325816] edma-dma-engine edma-dma-engine.0: allocated channel for 0:5
    [    1.332935] edma-dma-engine edma-dma-engine.0: allocated channel for 0:6
    [    1.341320] usbcore: registered new interface driver usbhid
    [    1.347245] usbhid: USB HID core driver
    [    1.353483] oprofile: no performance counters
    [    1.358497] oprofile: using timer interrupt.
    [    1.363409] TCP: cubic registered
    [    1.366914] Initializing XFRM netlink socket
    [    1.371442] NET: Registered protocol family 17
    [    1.376218] NET: Registered protocol family 15
    [    1.380961] 8021q: 802.1Q VLAN Support v1.8
    [    1.385483] Key type dns_resolver registered
    [    1.390639] cpu cpu0: cpu0 regulator not ready, retry
    [    1.396108] platform cpufreq-cpu0.0: Driver cpufreq-cpu0 requests probe deferral
    [    1.404483] ThumbEE CPU extension supported.
    [    1.413889] DCDC1: at 1800 mV
    [    1.418100] vdd_mpu: 925 <--> 1375 mV at 1275 mV
    [    1.424034] vdd_core: 925 <--> 1150 mV at 1125 mV
    [    1.429988] LDO1: at 1800 mV
    [    1.434029] LDO2: at 3300 mV
    [    1.438068] LDO3: 1800 <--> 3300 mV at 3300 mV
    [    1.443764] LDO4: at 3300 mV
    [    1.447617] tps65217 0-0024: TPS65217 ID 0xf version 1.1
    [    1.453297] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz
    [    1.460072] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
    [    1.467317] edma-dma-engine edma-dma-engine.0: allocated channel for 0:24
    [    1.559999] mmc0: host does not support reading read-only switch. assuming write-enable.
    [    1.571088] mmc0: new high speed SDHC card at address 1234
    [    1.576927] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6
    [    1.583384] davinci_mdio 4a101000.mdio: detected phy mask fffffffe
    [    1.590552] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [    1.596924] libphy: 4a101000.mdio: probed
    [    1.601226] davinci_mdio 4a101000.mdio: phy[0]: device 4a101000.mdio:00, driver SMSC LAN8710/LAN8720
    [    1.611026]  mmcblk0: p1 p2
    [    1.615622] Detected MACID = 50:56:63:c6:42:a2
    [    1.622747] omap_rtc 44e3e000.rtc: setting system clock to 2015-02-14 12:17:58 UTC (1423916278)
    [    1.635411] net eth0: initializing cpsw version 1.12 (0)
    [    1.643319] net eth0: phy found : id is : 0x7c0f1
    [    1.653848] 8021q: adding VLAN 0 to HW filter on device eth0
    [    5.802943] libphy: 4a101000.mdio:00 - Link is Up - 100/Full
    [    5.832457] Sending DHCP requests ., OK
    [    5.852920] IP-Config: Got DHCP answer from 172.16.40.13, my address is 172.16.40.203
    [    5.861680] IP-Config: Complete:
    [    5.865132]      device=eth0, hwaddr=50:56:63:c6:42:a2, ipaddr=172.16.40.203, mask=255.255.254.0, gw=172.16.40.1
    [    5.875898]      host=172.16.40.203, domain=dynatele.com, nis-domain=(none)
    [    5.883265]      bootserver=0.0.0.0, rootserver=172.16.40.43, rootpath=
    [    5.890068]      nameserver0=172.16.40.13, nameserver1=172.16.40.12
    [    5.905914] VFS: Mounted root (nfs filesystem) on device 0:13.
    [    5.912733] devtmpfs: mounted
    [    5.916487] Freeing unused kernel memory: 344K (c074b000 - c07a1000)
    INIT: version 2.88 booting
    Error opening /dev/fb0: No such file or directory
    Starting udev
    [    6.632731] udevd[788]: starting version 182
    [    7.463244] PM: CM3 Firmware Version = 0x186
    [   10.807338] musb-hdrc musb-hdrc.0.auto: Enabled SW babble control
    [   10.878379] musb-hdrc musb-hdrc.0.auto: MUSB HDRC host driver
    [   10.941515] musb-hdrc musb-hdrc.0.auto: new USB bus registered, assigned bus number 1
    [   11.001528] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
    [   11.008814] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
    [   11.016466] usb usb1: Product: MUSB HDRC host driver
    [   11.021714] usb usb1: Manufacturer: Linux 3.12.10-ti2013.12.01 musb-hcd
    [   11.028719] usb usb1: SerialNumber: musb-hdrc.0.auto
    [   11.188417] hub 1-0:1.0: USB hub found
    [   11.202555] hub 1-0:1.0: 1 port detected
    [   11.223929] musb-hdrc musb-hdrc.1.auto: Enabled SW babble control
    [   11.232078] musb-hdrc musb-hdrc.1.auto: MUSB HDRC host driver
    [   11.272121] musb-hdrc musb-hdrc.1.auto: new USB bus registered, assigned bus number 2
    [   11.296824] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
    [   11.304182] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
    [   11.311821] usb usb2: Product: MUSB HDRC host driver
    [   11.317104] usb usb2: Manufacturer: Linux 3.12.10-ti2013.12.01 musb-hcd
    [   11.324112] usb usb2: SerialNumber: musb-hdrc.1.auto
    [   11.345885] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    [   11.358421] hub 2-0:1.0: USB hub found
    [   11.365806] hub 2-0:1.0: 1 port detected
    [   11.401426] EXT4-fs (mmcblk0p2): recovery complete
    [   11.414352] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    [   11.853881] cryptodev: driver 1.6 loaded.
    Starting Bootlog daemon: bootlogd: cannot allocate pseudo tty: No such file or directory
    bootlogd.
    ALSA: Restoring mixer settings...
    /usr/sbin/alsactl: load_state:1696: No soundcards found...
    NOT configuring network interfaces: / is an NFS mount
    INIT: Entering runlevel: 5
    Starting system message bus: dbus.
    UIM SYSFS Node Not Found
    Starting Dropbear SSH server: dropbear.
    Starting telnet daemon.
    Starting syslogd/klogd: done
    Starting thttpd.
    Starting Lighttpd Web Server: lighttpd.
    2015-02-14 12:18:10: (log.c.166) server started
    /
    Starting Matrix GUI application.
    ***************************************************************
    ***************************************************************
    NOTICE: This file system contains the followin GPLv3 packages:
            binutils-symlinks
            binutils
            gdbserver
    
    If you do not wish to distribute GPLv3 components please remove
    the above packages prior to distribution.  This can be done using
    the opkg remove command.  i.e.:
        opkg remove <package>
    Where <package> is the name printed in the list above
    
    NOTE: If the package is a dependency of another package you
          will be notified of the dependent packages.  You should
          use the --force-removal-of-dependent-packages option to
          also remove the dependent packages as well
    ***************************************************************
    ***************************************************************
    Stopping Bootlog daemon: bootlogd.
    
     _____                    _____           _         _
    |  _  |___ ___ ___ ___   |  _  |___ ___  |_|___ ___| |_
    |     |  _| .'| . | . |  |   __|  _| . | | | -_|  _|  _|
    |__|__|_| |__,|_  |___|  |__|  |_| |___|_| |___|___|_|
                  |___|                    |___|
    
    Arago Project http://arago-project.org am335x-evm ttyO0
    
    Arago 2013.12 am335x-evm ttyO0
    
    am335x-evm login: root
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# uname -a
    Linux am335x-evm 3.12.10-ti2013.12.01 #1 Sun Mar 30 20:55:31 CDT 2014 armv7l GNU/Linux
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# df
    Filesystem           1K-blocks      Used Available Use% Mounted on
    172.16.40.43:/home/chantaek/Projects/IB/Embedded/beagleFS_07.00
                         666321600 593385336  42230096  93% /
    devtmpfs                111728         4    111724   0% /dev
    tmpfs                    16384       184     16200   1% /var/volatile
    tmpfs                   124188         0    124188   0% /dev/shm
    tmpfs                    16384         0     16384   0% /media/ram
    /dev/mmcblk0p1           70561       739     69823   1% /var/volatile/run/media/mmcblk0p1
    /dev/mmcblk0p2          903800    620172    236884  72% /var/volatile/run/media/mmcblk0p2
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# ./sdtest
    (It runs quiet well if SD card is not removed.)
    (<------ I removed the SD card.)
    ^C^C
    
    
    ^C^C^C^C^\^\^\^\^\^\
    
    
    
    

  • Here is another log.   Used the same SDK7.

    root@am335x-evm:~# ./sdtest
    [   29.116171] mmcblk0: error -110 requesting status
    [   29.147811] end_request: I/O error, dev mmcblk0, sector 2084
    [   29.153870] Buffer I/O error on device mmcblk0p1, logical block 36
    [   29.160401] lost page write due to I/O error on mmcblk0p1
    retStatus = SD_MEDIA_ERROR;
    root@am335x-evm:~# [   29.272537] mmc0: card 1234 removed
    
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# [   34.741381] mmc0: host does not support reading read-only switch. assuming write-enable.
    [   34.753272] mmc0: new high speed SDHC card at address 1234
    [   34.763885] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [   34.776578]  mmcblk0: p1 p2
    [   35.337942] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    [   35.373994] EXT4-fs (mmcblk0p2): recovery complete
    [   35.379104] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    
    root@am335x-evm:~#
    root@am335x-evm:~# ./sdtest
    [   44.129006] mmcblk0: error -110 sending status command, retrying
    [   44.161369] mmcblk0: error -110 sending status command, retrying
    [   44.193740] mmcblk0: error -110 sending status command, aborting
    [   44.304296] mmc0: card 1234 removed
    retStatus = SD_MEDIA_ERROR;
    root@am335x-evm:~#
    root@am335x-evm:~# [   47.870999] mmc0: host does not support reading read-only switch. assuming write-enable.
    [   47.882801] mmc0: new high speed SDHC card at address 1234
    [   47.890033] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [   47.902155]  mmcblk0: p1 p2
    [   48.496509] EXT4-fs (mmcblk0p2): recovery complete
    [   48.501612] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    [   48.516641] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    
    root@am335x-evm:~#
    root@am335x-evm:~# ./sdtest
    [   56.450929] mmcblk0: error -84 transferring data, sector 5038, nr 2, cmd response 0x900, card status 0xc00
    [   56.487343] end_request: I/O error, dev mmcblk0, sector 5038
    [   56.493355] Buffer I/O error on device mmcblk0p1, logical block 2990
    [   56.500131] lost page write due to I/O error on mmcblk0p1
    [   56.505864] end_request: I/O error, dev mmcblk0, sector 5039
    [   56.511870] Buffer I/O error on device mmcblk0p1, logical block 2991
    [   56.518606] lost page write due to I/O error on mmcblk0p1
    [   56.628492] mmc0: card 1234 removed
    retStatus = SD_MEDIA_ERROR;
    root@am335x-evm:~# [   57.981304] mmc0: host does not support reading read-only switch. assuming write-enable.
    [   57.993153] mmc0: new high speed SDHC card at address 1234
    [   58.003622] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [   58.016734]  mmcblk0: p1 p2
    [   58.700423] EXT4-fs (mmcblk0p2): recovery complete
    [   58.706205] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    [   58.715124] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    
    root@am335x-evm:~# ./sdtest
    [   63.771130] mmcblk0: error -110 sending status command, retrying
    [   63.778149] mmcblk0: error -110 sending status command, retrying
    [   63.811381] mmcblk0: error -110 sending status command, aborting
    [   63.895925] mmc0: card 1234 removed
    retStatus = SD_MEDIA_ERROR;
    root@am335x-evm:~# [   65.981242] mmc0: host does not support reading read-only switch. assuming write-enable.
    [   65.993013] mmc0: new high speed SDHC card at address 1234
    [   66.003660] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [   66.015413]  mmcblk0: p1 p2
    [   66.583776] EXT4-fs (mmcblk0p2): recovery complete
    [   66.588954] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    [   66.599960] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    
    root@am335x-evm:~# ./sdtest
    [   73.319710] mmcblk0: error -84 transferring data, sector 5144, nr 2, cmd response 0x900, card status 0xc00
    [   73.357298] end_request: I/O error, dev mmcblk0, sector 5144
    [   73.363321] Buffer I/O error on device mmcblk0p1, logical block 3096
    [   73.370043] lost page write due to I/O error on mmcblk0p1
    [   73.375767] end_request: I/O error, dev mmcblk0, sector 5145
    [   73.381745] Buffer I/O error on device mmcblk0p1, logical block 3097
    [   73.388469] lost page write due to I/O error on mmcblk0p1
    [   73.447109] mmcblk0: error -110 sending status command, retrying
    [   73.479538] mmcblk0: error -110 sending status command, retrying
    [   73.512143] mmcblk0: error -110 sending status command, aborting
    [   73.518535] end_request: I/O error, dev mmcblk0, sector 3189
    [   73.524532] Buffer I/O error on device mmcblk0p1, logical block 1141
    [   73.531246] lost page write due to I/O error on mmcblk0p1
    [   73.562990] mmcblk0: error -110 sending status command, retrying
    [   73.595295] mmcblk0: error -110 sending status command, retrying
    [   73.627589] mmcblk0: error -110 sending status command, aborting
    [   73.633951] end_request: I/O error, dev mmcblk0, sector 2086
    [   73.639932] Buffer I/O error on device mmcblk0p1, logical block 38
    [   73.646475] lost page write due to I/O error on mmcblk0p1
    [   73.756204] mmc0: card 1234 removed
    retStatus = SD_MEDIA_ERROR;
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# [   78.970804] mmc0: host does not support reading read-only switch. assuming write-enable.
    [   78.982894] mmc0: new high speed SDHC card at address 1234
    [   78.993625] mmcblk0: mmc0:1234 SA04G 3.63 GiB
    [   79.006104]  mmcblk0: p1 p2
    [   79.581892] EXT4-fs (mmcblk0p2): recovery complete
    [   79.587902] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    [   79.601367] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    
    root@am335x-evm:~#
    root@am335x-evm:~#
    root@am335x-evm:~# ./sdtest
    
    ^C^C^C
    
    ^C^C
    
    ^C^C^C^C^C^C^C^\^\^\^\^Z^Z^Z^Z^Z
    

  • Managed to reproduce the issue with sdk7. While serial console terminal stop to response, was able to open ssh session through ethernet.

    If you watch the process states, you'll see that your application's process falls into D state - "uninterruptible sleep". By design, write() and read() system calls are uninterruptible, and could not be terminated if are in waiting mode, except system restart.

    see:
    stackoverflow.com/.../what-is-an-uninterruptable-process
    lwn.net/.../288056

    BR,
    Georgi
  • Thank you for sharing the information.

    Would it be possible to make it get out of waiting mode when SD card is removed in the middle of the operation?

    Actually, our product is based on AM335x SDK6. With SDK6 sometimes it completely locks up when SD card is removed in the middle of write operations. Can this problem be fixed?
  • Hi,

     If you try to ftrace the execution of your app, you'll see that system freezes when it syncing data from memory to SD-card in:
    mm/filemap.c -> filemap_fdatawait_range ()
    there is a for() cycle
     in this for cycle there is a function wait_on_write() where system freezes.

     There is a way to get out of waiting, if you kill the process (take in mind that data, that was not written to the SD card will be lose).

    To make the process killeable you can, for exemple in ./include/linux/pagemap.h

    on line 410 (in sdk6) in wait_on_page_writeback(struct page *page) function
    replace place of wait_on_page_bit() with wait_on_page_bit_killable()

    Tried with sdk 7 and it works. Ctrl+C kills your app.

    BR,
    Georgi

  • Thank you for suggesting that.
    It would be nice if write() can return -EIO or -ETIMEDOUT when card is removed in the middle of the write operation.
    I think either MMC core or MMC host layer should detect and return an error so we can keep the kernel core function as is.
  • I found the following e-mail thread.
    thread.gmane.org/.../1192438

    Could that be a right solution that solves this problem?
  • For me it does not help.
    The system still freezes.
  • Would this be a known issue with MMC/SD interface?
    Is it TI specific or common across all MMC/SD interfaces?

    I don't see this kind of problem when writing to USB Memory Stick. Even if USB Memory Stick is removed during file copy, the system wouldn't freeze.
  • Hi,

    Looks like it is linux kernel common keeping.
    System freezes on write sync operation in I/O scheduler.

    For newer kernels the behavior is same - tried it.
    Kernel's writing strategy to USB Memory stick and mmc are different, thats why there is no freezes on USB stick remove.
    For information you may refer to
    free-electrons.com/.../block_drivers.pdf

    Think that the problem can be resolved by modifying the mmc sdcard module.