DVSDK 4.01 DM365 Kernel Hangs after load

Hi Bill,

Thanks for flagging this issue.

I don't think I have seen the kernel hang at boot before. If you are booting from SD card, did you create a new bootable SD card as directed in the Software Developers' Guide? If you are booting from Nand, did you erase your flash prior to flashing the new bootloader? Maybe there were residuals from your previous installation. If you are using TFTP to download the kernel, make sure that when you run setup.sh that you select to boot using the new uImage-dm365-evm.bin file and not the old kernel.

Also could you give us more details regarding "the demos still didn't run"? When you booted with the filesystem on the SD card the Matrix application launcher should have come up with the demos. Did the Matrix not launch? Or were you unable to click on some of the demos? From the log, it looks like you have successfully reached the login prompt.

Best regards,

Vincent

Vincent.

Thanks for the quick response.

TO your questions about creating SD flash. .Yes I used the mksdboot.sh in the bin directory..

When I said the demo's didn't boot this was with TFTF/NFS bootargs. (using setup.sh or setup-uboot-evm.sh)

Does the Demo's not run under NFS by default? If not there might be no problem.

Since it NOW works (20+ reboots), I am willing accept the "residual" answer while I don' t see how that could be..

With regard to SD card boot and the demos.. That a longer answer.

This is where this thread might need to be separated and created a new post. please advise.

Last week we started working with DVSDK 4.00 (previous to this release). We were VERY interested in using SD flash in our final product so I was researching how well this works. I ran through the GSG and had no trouble with booting with a SD 2GIG flash card. System booted from SD flash, launched UBL, UBOOT and the kernel and mounted the SD root filesystem. The demos would launch and the menu would show up on the Component out. Hey I even got the mouse to work!

In the process of working with the demos, my EVM board stopped working. (hardware failure). So i switch to another DM365 EVM board. Reconfigured uboot with the setup.sh scripts and plugged in the SAME SD flash.

The EVM would boot but the kernel could not mount the rootfilesytem (Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)).

However, If I boot the board with a NFS rootfilesystem, I coud mount the filesystem from the command line! ($mount /dev/mmcblk0p2 /media/sd)

Today we decided to try the new DVSDK4.1

Now I have simliar results!

mmc0: new high speed SD card at address 0002
mmcblk0: mmc0:0002 00000 1.86 GiB (ro)
 mmcblk0: p1 p2
VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2)
Please append a correct "root=" boot option; here are the available partitions:
1f00           15360 mtdblock0 (driver?)
1f01            1024 mtdblock1 (driver?)
1f02            4096 mtdblock2 (driver?)
1f03          524288 mtdblock3 (driver?)
1f04         1552384 mtdblock4 (driver?)
b300         1955840 mmcblk0 driver: mmcblk
  b301           80325 mmcblk0p1
  b302         1710922 mmcblk0p2
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)
Backtrace:
[<c00305f4>] (dump_backtrace+0x0/0x114) from [<c031b0b8>] (dump_stack+0x18/0x1c)
 r7:00008000 r6:c2013000 r5:c0025808 r4:c044b9b0
[<c031b0a0>] (dump_stack+0x0/0x1c) from [<c031b110>] (panic+0x54/0x12c)
[<c031b0bc>] (panic+0x0/0x12c) from [<c0008fbc>] (mount_block_root+0x1e0/0x220)
 r3:00000001 r2:c2027e98 r1:c2027f60 r0:c03b4056
[<c0008ddc>] (mount_block_root+0x0/0x220) from [<c00090c0>] (mount_root+0xc4/0xfc)
 r8:00000000 r7:00000000 r6:00000000 r5:c0025808 r4:0b300002
[<c0008ffc>] (mount_root+0x0/0xfc) from [<c0009268>] (prepare_namespace+0x170/0x1c8)
 r5:c0025808 r4:c044b4c0
[<c00090f8>] (prepare_namespace+0x0/0x1c8) from [<c00084bc>] (kernel_init+0xe4/0x118)
 r5:00000000 r4:c044b280
[<c00083d8>] (kernel_init+0x0/0x118) from [<c00450c4>] (do_exit+0x0/0x668)
 r5:00000000 r4:00000000

I have tried 4 different type of 2 gig flash with simliar results.

I have seen some reference to this could be related to 2.6.32 but I can't tell. I am currently playing with Rootwait=x to see if that helps. So far it didn't help.

I have reinstalled both the DVSDK 4.00 and 4.01 complete from scratch and followed the GSG guide to the Letter. 

The key to me is that the kernel can not mount the SD flash at boot but can after wards when using a NFS rootfilesystem.

I would appreciate any help and idea you might have.

Hi Bill,

The NFS filesystem differs from the one on the SD card in the sense that it does not start the Matrix Application Launcher at boot up. This is because we anticipate users to use NFS mainly for development purposes and it'd be a hindrance for the developer to have the demos come up every time he reboots the board. So it sounds like your NFS filesystem is fine.

Regarding the SD card filesystem, make sure you pick SD card as the option for both kernel and filesystem when running through the setup.sh script. Also remember to setup your DVSDK variable to point to the new DVSDK installation directory before running setup.sh and creating the SD card. Can you verify that you have these bootargs when booting from SD:

console=ttyS0,115200n8 rw mem=54M video=davincifb:vid0=OFF:vid1=OFF:osd0=720x576
x16,4050K dm365_imp.oper_mode=0 davinci_capture.device_type=4 vpfe_capture.cont_
bufsize=6291456 davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mode=
480P-60 root=/dev/mmcblk0p2 rootwait ip=off

You can view them from the uboot prompt by running printenv.

Also, if you have a third EVM lying around, it may be good to see if it produces a different behavior, since your trouble seems to have started after moving to the 2nd EVM (what revision of the board is it by the way?).

Best regards,

Vincent

Vincent,

To you questions

1) DVSDK environment var is set correctly

user@ubuntu:~/ti-dvsdk_dm365-evm_4_01_00_09$ echo $DVSDK
/home/user/ti-dvsdk_dm365-evm_4_01_00_09

2) Setup options. Yes I used option 2 in both kernel and filesystem

3) bootargs

I will attach a log of uboot environment and boot sequence but here is the command line the linux kernel booted with while in SD mode

Machine: DaVinci DM365 EVM
Memory policy: ECC disabled, Data cache writeback
DaVinci dm365_rev1.2 variant 0x8
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 13716
Kernel command line: console=ttyS0,115200n8 rw mem=54M video=davincifb:vid0=OFF:vid1=O
FF:osd0=720x576x16,4050K dm365_imp.oper_mode=0 davinci_capture.device_type=4 vpfe_capt
ure.cont_bufsize=6291456 davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mo
de=480P-60 root=/dev/mmcblk0p2 rootwait ip=off

4) new board ordered from TI yesterday..

My questions for you.

1) You have not addressed why DVSDK4.1 new uboot has this problem. Again this just started happening with new uboot.

U-Boot 2010.12-rc2 (Jan 12 2011 - 19:44:22)

Cores: ARM 297 MHz
DDR:   243 MHz
I2C:   ready
DRAM:  128 MiB
NAND:  2048 MiB
MMC:   davinci: 0, davinci: 1
Bad block table found at page 524224, version 0x01
Bad block table found at page 1048512, version 0x01
Bad block table found at page 524160, version 0x01
Bad block table found at page 1048448, version 0x01
nand_bbt: ECC error while reading bad block table
nand_read_bbt: Bad block at 0x000000800000
nand_read_bbt: Bad block at 0x000000820000
nand_read_bbt: Bad block at 0x000000840000
nand_read_bbt: Bad block at 0x000000860000
nand_read_bbt: Bad block at 0x000000880000
nand_read_bbt: Bad block at 0x0000008a0000

(200+ lines deleted )

nand_read_bbt: Bad block at 0x000050a60000
nand_read_bbt: Bad block at 0x0000603e0000
nand_read_bbt: Bad block at 0x000063740000
nand_read_bbt: Bad block at 0x000064700000
nand_read_bbt: Bad block at 0x000073560000
nand_read_bbt: Bad block at 0x000079680000
nand_read_bbt: Bad block at 0x00007a420000
nand_read_bbt: Bad block at 0x00007b320000
Net:   Ethernet PHY: GENERIC @ 0x00
DaVinci-EMAC
Hit any key to stop autoboot:  0

2) Why would file system be able to mount from command line yet not from kernel boot?

NOTE: While i was capturing logs. the system "froze" again right after load of kernel. This HAS NEVER happened in the 6+ months of using this board with DVSDK2.0 tools (ubl/uboot/kernel...) and just happened starting with 4.1.   I strongly believe this new u-boot has some problems.

SEE ATTACHED FILE for boot logs.

PART 1) NFS boot . SD card mounts both partitions automatically

PART 2) SD boot. Kernel CAN NOT mount ROOTFS

Can you please forward this to Kernel and Uboot team for some assistance?

Thanks

Bill

Vincent,

UPDATE:

I reverted to our older kernel 2.6.18

root@dm365-evm:~# uname -r
2.6.18_pro500-davinci_evm-arm_v5t_le

and it mounted the SD file system without a problem. It even launched and displayed the "Matrix Application  Launcher"

kjournald starting.  Commit interval 5 seconds
EXT3 FS on mmcblk0p2, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem).

I found some reference on the web said this was introduced in between 2.6.24 and 2.6.32

Can you check with your team for a solution?

Thanks

Hi Bill,

We have notified our PSP team about this. In the meantime, I have downloaded the SDK but was unable to reproduce the kernel hang using my setup. My board seems to boot fine and I am able to run the demos. Fyi, I am using a SanDisk 8 GB Class 2 SD card. I see the nand_read_bbt messages as well, but they don't seem to impact the kernel boot.

Could you post the following binaries from your DVSDK installation (you may want to verify their md5sum at the same time. I have listed mine on the left):

b58bdd4439039d13e2da3684c6eebbd9  psp/board-utilities/ubl/UBL_DM36x_SDMMC.bin

5cd601c050c083372be8d07581897785  psp/prebuilt-images/u-boot.bin

a6c58319893a1d31d34dc4d5d0be5f61  filesystem/dvsdk-dm365-evm-rootfs.tar.gz

d12ea4b679165e3aa31b6b65b5656568  psp/prebuilt-images/uImage-dm365-evm.bin

Also, could you provide the revision of the board and silicon you are using. The board revision is printed on the board in a corner, underneath the words "DM36x EVM". The revision of the silicon can be found if you follow the instructions here: http://www.ti.com/litv/pdf/sprz294d.

Best regards,

Vincent

Vincent,

Sorry for the delay.

All the checksums match.

We did purchase another EVM board (REV F) and it works correctly (IE the Rootfilesystem  mounted during kernel boot)

The only difference from REV E and REV F is moving from

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

TMS320DM365BZCE ->  TMS320DM365ZCE30

I would think this is just a partnumber change but maybe not.

We are trying to make a design decision to either use SD flash with the DM365 or stick with NAND flash by Wednesday. Any help answering why there would be a difference would be much appreciated.

Thanks

PS. Having the bootloader spewi 200+ error lines on each boot is a concern esp when no explained. The u-boot that came preloaded on the NEW spectrum digital board did not do this either until I updated uboot with the version provided with DVSDK4.01

Bill

Hi Bill,

I was using a Rev E with TMS320DM365BZCE myself and experienced no issue. It sounds like your Rev E board may just happened to be defective.

I agree that the 200+ warning messges are a concern when seeing them out-of-box. We'll ask our PSP team to continue to investigate for the root cause.

Thanks and best regards,

Vincent

Hi Bill,

There can be many reasons for bad block error messages:

1. NAND chip on your EVM may have these bad blocks. The earlier u-boot on the EVM which was pre-flashed, may not have this print enabled or bad block table support was not present in that version of U-Boot.

2. ECC layout followed by newly flashed U-Boot may be different from the one which was pre-flashed. This also causes such errors to occur. Erase the NAND completely and then try flashing the NAND with the new U-Boot image.

On the DM368 EVM I have (DM368ZCE), when I flashed the 2010.12-rc2 u-boot binary, I saw around 20 such messages.

Regards, Sudhakar