This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

console hangs periodically for 1-2 seconds

Guru 20755 points

Hello,

I have some strange behavior with a custom board running linux on DM8148.

The serial console halts for 2-3 seconds periodically every ~20 seconds  and then everything is back. I see this every behaviour repeating every few seconds. I wonder if anyone is familiar with this and can direct me to the problem.

Thanks,

Ran

  • Hi Ran,

    I haven't experienced this issue.

    You can try to use different serial s/w such as minicom, "using cat /dev/ttyXX" for getting debug serial port log.

    Are you getting the same behavior for EVM board ?

    Have you tried it on any other host machine ?

  • Hi Titusrathinaraj Stalin,

    Thank you very much for the suggestions.
    I will do this testings when I'll get to lab in few days and get back with results.
    But from what I seen this far:

    1. I'm currently using only teraterm, and never seen such behaviour with EVM, or other projects. Probably it is not a host issue, because it happens only with the custom board, and only after getting to linux prompt. But there are some slights differences in the way I boot with EVM (sdcard or nfs) with regards to custom (complete boot from nand)

    2. when this happens, the entered chars in console will eventually appear after this ~1-2 second hang.

    3. when I stop in u-boot console, there is no such issue.

    4. It "feels" as if the linux is busy in something else, and that's why I don't get the echo for the enters chars in teraterm, but maybe it is something else (serial connection problem). I can't be sure yet.

    I will do more testing and get back with more information on this.

    Thanks,

    Ran

     

  • Hi Titusrathinaraj Stalin,

    I have made additional testings in this is what I see:

    1.  I see also this behavior in EVM when rootfs is in nand, with nfs I don't see this (also in custom board, I don't have it with nfs). Though, this behavior is more visible with custom board (hangs more)

    2. I see this especially when I press a single character in the keyboard (without removing the pressing), right after getting the "login:" text in serial console, then after some characters are echoed it hangs for a second or 2 and continues.

    3. I also toggle leds periodically for software keep alive, and seen that whenever this freeze happens, the leds stop toggle for this freeze time (1-2 second)

    4. It is not a serial interface freeze issue, but a system freeze. (this also happens when telnet is used)


    Thanks,

    Ran

  • Hi Ran,

    Try to see the kernel process when you see the hang.

    Use "top" command to check, which process could cause the hang.

    I think, the NAND driver could cause the problem since we didn't get hang when we are in NFS.

  • Hi Titusrathinaraj Stalin

    Thanks for suggestion!

    Yes, it seems that it is related to nand, jffs.

    I wonder why it takes so much cpu. It makes the linux bad for real-time if it takes so much cpu from application...

    PID PPID USER STAT VSZ %VSZ %CPU COMMAND
    582 2 root RWN 0 0% 81% [jffs2_gcd_mtd5]
    510 1 root R 2260 3% 2% ./bin/remote_debug_client.out 0xbff000
    698 696 root R 2472 3% 0% top
    696 1 root S 2856 4% 0% -sh
    660 1 messageb S 2604 4% 0% /usr/bin/dbus-daemon --system
    681 1 nobody S 2212 3% 0% /usr/sbin/thttpd -d /srv/www -p 8080
    113 1 root S < 2192 3% 0% /sbin/udevd -d
    679 113 root S < 2188 3% 0% /sbin/udevd -d
    680 113 root S < 2188 3% 0% /sbin/udevd -d
    671 1 root S 2148 3% 0% /sbin/syslogd -n -O /var/log/messages
    673 1 root S 2148 3% 0% /sbin/klogd -n
    663 1 root S 2148 3% 0% /usr/sbin/telnetd
    668 1 root S 2148 3% 0% /usr/sbin/inetd
    697 1 root S 1880 3% 0% /sbin/getty 38400 tty1
    1 0 root S 1632 2% 0% init [5]
    7 2 root SW 0 0% 0% [irq/72-serial i]
    24 2 root SW 0 0% 0% [kworker/0:1]
    14 2 root SW 0 0% 0% [sync_supers]
    2 0 root SW 0 0% 0% [kthreadd]
    40 2 root SW 0 0% 0% [scsi_eh_0]

    Regards,

    Ran

  • Hi Ran,

    What is your size of your NAND flash that used for "mtd5" filesystem storage ?

    You can also try some other filesystem such as UBIFS,YAFFS2 etc.,

    What kernel version are you using ?

    Whether the linux kernel released by TI or 3rd party of open source or custom ?

    1.  I see also this behavior in EVM when rootfs is in nand, with nfs I don't see this (also in custom board, I don't have it with nfs). Though, this behavior is more visible with custom board (hangs more)


    I can't help if you face the same behavior in EVM because I don't aware about your DM814x but I can move to correct DM814x forum to get better answers if you are using TI SDK.

  • Hi Titusrathinaraj Stalin,

    We have DDR 512M, size of mtd is as following:

    ;;0x000000000000-0x000000020000 : "U-Boot-min"
    ;;0x000000020000-0x000000260000 : "U-Boot"
    ;;0x000000260000-0x000000280000 : "U-Boot Env"
    ;;0x000000280000-0x000000580000 : "U-Boot Logo"
    ;;0x000000580000-0x0000009c0000 : "Kernel"
    ;;0x0000009c0000-0x00000d1e0000 : "File System"
    ;;0x00000d1e0000-0x000010000000 : "Reserved"

    I create jffs rootfile system with:

    mkfs.jffs2 -n -r rfs_814x -e 128 -o rootfs.jffs2

    and program to nand by doing:

    mw.b 0x81000000 0xff 0x3000000; tftp 0x81000000 rootfs.jffs; nand erase 0x9c0000 0x3000000; nandecc hw 2; nand write.i 0x81000000 0x9c0000 0x3000000

    After first program, it works well, but in the next reset, I get errors in boot , untill eventually it does not find the rootfs.

    I understand from this thread and others that jffs is not recommended, and should be replaced with ubifs (though we worked well with jffs with other TI's chip such as  omap35x.

    Thanks,

    Ran

  • Hi,

    moving to ubifs solved the freeze and error issue.

    Thanks,

    Ran

  • Hi Ran,

    Sounds good.

    Thanks for your update.