• State
  • Locked Locked
  • Replies 12 replies
  • Subscribers 98 subscribers
  • Views 1441 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

[DM8168] How to analyze the exception message of dvrrdk

Eric Mao
Genius 4330 points

Hi all

    when run dvrrdk program , print exception message.

[m3vpss ] Unhandled Exception:
[m3vpss ] Exception occurred in ThreadType_Task
[m3vpss ] handle: 0x3b0b55f0.
[m3vpss ] stack base: 0x3b1b0080.
[m3vpss ] stack size: 0x2800.
[m3vpss ] R0 = 0x00000000 R8 = 0x00000000
[m3vpss ] R1 = 0xdead0000 R9 = 0x00000001
[m3vpss ] R2 = 0x4810d000 R10 = 0x00000000
[m3vpss ] R3 = 0x3b12646c R11 = 0x3ac8bd4c
[m3vpss ] R4 = 0x0000c4a4 R12 = 0x004331ed
[m3vpss ] R5 = 0x0000b4a4 SP(R13) = 0x3b1b27a0
[m3vpss ] R6 = 0x3b125af8 LR(R14) = 0x99deecdb
[m3vpss ] R7 = 0x0387f1c9 PC(R15) = 0x99deece2
[m3vpss ] PSR = 0x41000000
[m3vpss ] ICSR = 0x00400803
[m3vpss ] MMFSR = 0x00
[m3vpss ] BFSR = 0x00
[m3vpss ] UFSR = 0x0001
[m3vpss ] HFSR = 0x40000000
[m3vpss ] DFSR = 0x00000000
[m3vpss ] MMAR = 0xe000ed34
[m3vpss ] BFAR = 0xe000ed38
[m3vpss ] AFSR = 0x00000000
[m3vpss ] Terminating Execution...
[m3vpss ] TaskName = NULL
[m3video] Unhandled Exception:
[m3video] Exception occurred in ThreadType_HWI
[m3video] handle: 0x99020ec0.
[m3video] stack base: 0x39cb2484.
[m3video] stack size: 0x4000.
[m3video] R0 = 0x99020ec0 R8 = 0xffffffff
[m3video] R1 = 0x98fcf695 R9 = 0x0000000f
[m3video] R2 = 0x00000001 R10 = 0x00000000
[m3video] R3 = 0x990209ec R11 = 0xffffffff
[m3video] R4 = 0x99020ec0 R12 = 0x39caebd8
[m3video] R5 = 0x98f8eda0 SP(R13) = 0x39cb6398
[m3video] R6 = 0x00000000 LR(R14) = 0x98fc3f47
[m3video] R7 = 0xffffffff PC(R15) = 0x98fcf696
[m3video] PSR = 0x8100000f
[m3video] ICSR = 0x00400003
[m3video] MMFSR = 0x00
[m3video] BFSR = 0x00
[m3video] UFSR = 0x0001
[m3video] HFSR = 0x40000000
[m3video] DFSR = 0x00000000
[m3video] MMAR = 0xe000ed34
[m3video] BFAR = 0xe000ed38
[m3video] AFSR = 0x00000000
[m3video] Terminating Execution...
[m3video] TaskName = NULL
Unable to handle kernel paging request at virtual address ffff7648
pgd = cde20000
[ffff7648] *pgd=8fefe021, *pte=00000000, *ppte=00000000
Internal error: Oops: 17 [#1]
last sysfs file: /sys/devices/virtual/TI81XX_HDMI/TI81XX_HDMI/uevent
Modules linked in: gpiorw ti81xxhdmi ti81xxfb vpss osa_kermod syslink
CPU: 0 Not tainted (2.6.37 #2)
PC is at hrtimer_interrupt+0x14/0x2d4
LR is at omap2_gp_timer_interrupt+0x24/0x30
pc : [<c008f32c>] lr : [<c0058d6c>] psr: 40000193
sp : cd091ef8 ip : cd091f50 fp : cd091f4c
r10: beb75c20 r9 : cd090000 r8 : 00000000
r7 : 00000043 r6 : ffff75e0 r5 : 00000000 r4 : c0510c98
r3 : c008f318 r2 : 00000002 r1 : fa02e000 r0 : c0510c40
Flags: nZcv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 8de20019 DAC: 00000015
Process test_rtp_send (pid: 668, stack limit = 0xcd0902e8)
Stack: (0xcd091ef8 to 0xcd092000)
1ee0: 00000040 c055cf08
1f00: cd091f44 cd091f10 c0333134 c026f51c cd091f3c 006992c5 c00a1b74 00000001
1f20: c057ee50 c0510c98 00000000 00000000 00000043 00000000 cd090000 beb75c20
1f40: cd091f5c cd091f50 c0058d6c c008f324 cd091f7c cd091f60 c00a1448 c0058d54
1f60: c0529328 00000043 40a9ddf0 00000000 cd091f94 cd091f80 c00a2e38 c00a1428
1f80: 00000043 00000000 cd091fac cd091f98 c0041078 c00a2d98 ffffffff fa200000
1fa0: 00000000 cd091fb0 c03b657c c004100c 0022ace8 0022accc 40a9dda4 40a9dda4
1fc0: 00000000 00000000 40a9ddf0 00000000 00000000 400554c0 beb75c20 40a9ddc4
1fe0: 000d8290 40a9dd98 0009aa28 400eeaec 20000010 ffffffff 00000000 00000000
Backtrace:
[<c008f318>] (hrtimer_interrupt+0x0/0x2d4) from [<c0058d6c>] (omap2_gp_timer_interrupt+0x24/0x30)
[<c0058d48>] (omap2_gp_timer_interrupt+0x0/0x30) from [<c00a1448>] (handle_IRQ_event+0x2c/0xe8)
[<c00a141c>] (handle_IRQ_event+0x0/0xe8) from [<c00a2e38>] (handle_level_irq+0xac/0x11c)
r7:00000000 r6:40a9ddf0 r5:00000043 r4:c0529328
[<c00a2d8c>] (handle_level_irq+0x0/0x11c) from [<c0041078>] (asm_do_IRQ+0x78/0x9c)
r5:00000000 r4:00000043
[<c0041000>] (asm_do_IRQ+0x0/0x9c) from [<c03b657c>] (__irq_usr+0x3c/0xa0)
Exception stack(0xcd091fb0 to 0xcd091ff8)
1fa0: 0022ace8 0022accc 40a9dda4 40a9dda4
1fc0: 00000000 00000000 40a9ddf0 00000000 00000000 400554c0 beb75c20 40a9ddc4
1fe0: 000d8290 40a9dd98 0009aa28 400eeaec 20000010 ffffffff
r5:fa200000 r4:ffffffff
Code: e92ddff0 e24cb004 e24dd02c e59f6298 (e5963068)
---[ end trace d664d2953c312b4a ]---
Kernel panic - not syncing: Fatal exception in interrupt
Backtrace:
[<c004efac>] (dump_backtrace+0x0/0x110) from [<c03b438c>] (dump_stack+0x18/0x1c)
r6:cd091d5f r5:c008f32e r4:c055e4d8 r3:60000113
[<c03b4374>] (dump_stack+0x0/0x1c) from [<c03b43ec>] (panic+0x5c/0x178)
[<c03b4390>] (panic+0x0/0x178) from [<c004f40c>] (die+0x29c/0x2ec)
r3:00010000 r2:cd091d28 r1:c055e920 r0:c048249a
r7:00000000
[<c004f170>] (die+0x0/0x2ec) from [<c00520f0>] (__do_kernel_fault+0x6c/0x8c)
[<c0052084>] (__do_kernel_fault+0x0/0x8c) from [<c03b8238>] (do_page_fault+0x1e8/0x204)
r8:00000017 r7:cdd44180 r6:cd18e2c0 r5:ffff7648 r4:cd091eb0
r3:cd091eb0
[<c03b8050>] (do_page_fault+0x0/0x204) from [<c00411f4>] (do_DataAbort+0x38/0xa0)
[<c00411bc>] (do_DataAbort+0x0/0xa0) from [<c03b632c>] (__dabt_svc+0x4c/0x60)
Exception stack(0xcd091eb0 to 0xcd091ef8)
1ea0: c0510c40 fa02e000 00000002 c008f318
1ec0: c0510c98 00000000 ffff75e0 00000043 00000000 cd090000 beb75c20 cd091f4c
1ee0: cd091f50 cd091ef8 c0058d6c c008f32c 40000193 ffffffff
r8:00000000 r7:00000043 r6:ffff75e0 r5:cd091ee4 r4:ffffffff
[<c008f318>] (hrtimer_interrupt+0x0/0x2d4) from [<c0058d6c>] (omap2_gp_timer_interrupt+0x24/0x30)
[<c0058d48>] (omap2_gp_timer_interrupt+0x0/0x30) from [<c00a1448>] (handle_IRQ_event+0x2c/0xe8)
[<c00a141c>] (handle_IRQ_event+0x0/0xe8) from [<c00a2e38>] (handle_level_irq+0xac/0x11c)
r7:00000000 r6:40a9ddf0 r5:00000043 r4:c0529328
[<c00a2d8c>] (handle_level_irq+0x0/0x11c) from [<c0041078>] (asm_do_IRQ+0x78/0x9c)
r5:00000000 r4:00000043
[<c0041000>] (asm_do_IRQ+0x0/0x9c) from [<c03b657c>] (__irq_usr+0x3c/0xa0)
Exception stack(0xcd091fb0 to 0xcd091ff8)
1fa0: 0022ace8 0022accc 40a9dda4 40a9dda4
1fc0: 00000000 00000000 40a9ddf0 00000000 00000000 400554c0 beb75c20 40a9ddc4
1fe0: 000d8290 40a9dd98 0009aa28 400eeaec 20000010 ffffffff
r5:fa200000 r4:ffffffff

  

  • 0
    TI__Guru 59700 points

    Refer http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/t/270638.aspx which has details on crash dump analysis. In your case both A8 and M3 are simultaneously crashing. This indicates DDR is unstable on your board.

    Check you have done DDR byte wise sw leevlign and you have run the DDR stress test

    http://processors.wiki.ti.com/index.php/DM816x_C6A816x_AM389x_DDR3_Init

    http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/717/7725.DDRStressTest.zip

    Check your board is compliant to Design guidelines

    http://processors.wiki.ti.com/index.php/DM816x_Design_Resources (check DDR section)

  • 0 in reply to Badri Narayanan
    Genius 4330 points

    still have some question:

     1, Crash_Dump_Analysis introduce the format of dump file.  where can i find the  target type of other process, as A8 and M3?

      2. Is the  memory entry the same as MEMORY list in the *.cmd file?

      3. in my dvr_rdk_m3video_release_1024M_256M.xem3.map file, has a MEMORY VIDEO_M3_CODE_MEM, the start address is 0x98d1f000 and length is 0x00300000. How the write this Memory Entry.

      4. I use ccsv5.4, when create target configuration file.  the connect choose Data Snapshot Viewer. Board or Device choose DM8168 or EVMDM8168, can't find M3 core.

  • 0 in reply to Eric Mao
    TI__Guru 59700 points

    Eric Mao said:
    1, Crash_Dump_Analysis introduce the format of dump file.  where can i find the  target type of other process, as A8 and M3?

    Target type for cortex M3 is 14. Refer

    /dvr_rdk/demos/mcfw_api_demos/mcfw_demo/demo_crash_dump_analyzer_format.c

    /dvr_rdk/demos/mcfw_api_demos/mcfw_demo/demo_crash_dump_analyzer_format.h

    for details on how to format and print the exception info in form that can be analyzed by crash dump analyzer. The DVRRDK demo already print the CrashDump analysis text file. You just have to give it to crash dump analyzer for analysis.

    It is not possible to analyze A8 kernel crash with CCS crash dump analyzer. It is meant for M3/c674 crash analysis.

    Eric Mao said:
      2. Is the  memory entry the same as MEMORY list in the *.cmd file?

    No . Memory entry is the stack dump. Refer the exception event handling in DVRRDK mcfw demo.c to understand.

    Eric Mao said:
    3. in my dvr_rdk_m3video_release_1024M_256M.xem3.map file, has a MEMORY VIDEO_M3_CODE_MEM, the start address is 0x98d1f000 and length is 0x00300000. How the write this Memory Entry.

    You should dump memory range you require similar to how it is done in DVRRDK exception reporting.Or you could copy memory using mem_rdwr utility.

    Eric Mao said:
      4. I use ccsv5.4, when create target configuration file.  the connect choose Data Snapshot Viewer. Board or Device choose DM8168 or EVMDM8168, can't find M3 core.

    Use TI816x as target not DM8168/EVM8168.

  • 0 in reply to Badri Narayanan
    TI__Guru 59700 points

    Attached is sample DataShanpshotAnalyzer.xml and associated crash dump analysis script and sample crash dump for reference

    .7563.CrashDumpAnalyzerM3.zip

  • 0 in reply to Badri Narayanan
    Genius 4330 points

    the rdk4.1 didn't generate dump file.  It seems the slave core don't send exception notify to a8.

    I met another kernel error. 

    Unable to handle kernel paging request at virtual address 3dd40000
    pgd = cdc18000
    [3dd40000] *pgd=00000000
    Internal error: Oops: 5 [#1]
    last sysfs file: /sys/bus/platform/drivers/TI81XX_HDMI/uevent
    Modules linked in: gpiorw ti81xxhdmi ti81xxfb vpss osa_kermod syslink
    CPU: 0 Not tainted (2.6.37 #1)
    PC is at __kmalloc_track_caller+0x6c/0xc8
    LR is at __kmalloc_track_caller+0x44/0xc8
    pc : [<c00ca050>] lr : [<c00ca028>] psr: 20000093
    sp : cc299bd0 ip : cc299bd0 fp : cc299bf4
    r10: 00000000 r9 : cc299c38 r8 : c032774c
    r7 : 3dd40000 r6 : a0000013 r5 : 000004d0 r4 : ce402400
    r3 : c079a060 r2 : 00000000 r1 : 000004d0 r0 : ce402400
    Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
    Control: 10c5387d Table: 8dc18019 DAC: 00000015
    Process test_enc (pid: 722, stack limit = 0xcc2982e8)
    Stack: (0xcc299bd0 to 0xcc29a000)
    9bc0: c079a018 cd27f180 ce402100 00000680
    9be0: 000004d0 c032774c cc299c1c cc299bf8 c032ba60 c00c9ff0 cde30200 7fffffff
    9c00: 00000000 00000000 000004d0 cc298000 cc299c6c cc299c20 c032774c c032ba14
    9c20: 000005c1 cde3025c 00000000 cc29f094 cc299c64 cc299c40 c0352be0 c0336070
    9c40: 00000008 cde30200 00000000 cc299d84 cd237500 0000058e 00000008 00000014
    9c60: cc299c84 cc299c70 c03279b4 c03276a8 cc299ccc cc299c80 cc299cfc cc299c88
    9c80: c03538a8 c032799c cc299cc4 cc299c98 000005a2 000005c8 cc299f58 c0351d84
    9ca0: 0000058e 00000000 00000010 000005dc 00000000 00000000 00000000 000005dc
    9cc0: cde30274 0000058e cc299cfc cc299cd8 00000001 cde30200 cc299f3c be01a8c0
    9ce0: 0a01a8c0 00000000 00002e16 00000586 cc299db4 cc299d00 c037081c c035342c
    9d00: 00000008 cc299d74 cc299d84 00000000 be01a8c0 00000000 c00a1cc8 c00a1bdc
    9d20: 00000004 ce563350 00000000 00000000 0a01a8c0 00000000 3b9aca00 00000000
    9d40: 389fd980 00000000 ce56334c ce6598c0 00000002 00000000 fffffffe 00000000
    9d60: cc299d94 cc299d70 c03b83fc c00469ec ffffffff 0a01a8c0 00000000 00000000
    9d80: cd0aeb00 00000000 cc299dac cc299dd8 00000586 cc299f3c cde30200 00000000
    9da0: 00000000 00000000 cc299dd4 cc299db8 c0377fe4 c0370390 ce09e780 00000586
    9dc0: cc299f3c 00000000 cc299eb4 cc299dd8 c03245f4 c0377f88 d0b37000 cc298000
    9de0: 00000000 00000001 ffffffff 00000000 00000000 00000000 00000000 00000000
    9e00: ce6598c0 c0098754 00000000 00000000 00000000 00000000 cc299e50 d3fe7000
    9e20: cc299e4c cc299e30 00000031 cc299e2c cc299e2c cc299e34 cc299e34 ce6598c0
    9e40: 00000000 0024b000 cde7ca80 00000790 00000000 00000000 00000000 00000586
    9e60: ce09e780 cc299e70 00000000 cc299f3c bf0285cc bf01b99c 00000001 00000000
    9e80: c0581590 cc299f64 cc299f60 53c43dac cc299eb4 53c41f64 00000586 ce09e780
    9ea0: 00000000 cc298000 cc299f8c cc299eb8 c03258a4 c0324558 0024b790 00000000
    9ec0: cc299f54 cc299ed0 c009a550 c0098ce8 ffffffff 00000000 c026f844 c00a1c60
    9ee0: c0527610 ce09ad00 00000011 53c42404 cc299f74 cc299f00 c00d957c bf04d834
    9f00: cc299f44 cc299f10 c03331fc c026f5e4 cc299f3c 001fe498 c00a1b74 00000001
    9f20: cc299f3c cc299f30 c0058d90 c006dbb4 cc299f64 cc299f40 c0092dd4 00000000
    9f40: 00000000 cc299f58 00000001 00000000 00000000 00000000 53c41f64 00000586
    9f60: 00000001 fffffff7 00000004 0000000c 53c43dac 00000121 c004b568 00000000
    9f80: cc299fa4 cc299f90 c03258e8 c03257f8 00000000 00000000 00000000 cc299fa8
    9fa0: c004b3c0 c03258d4 00000004 0000000c 0000001e 53c41f64 00000586 00000000
    9fc0: 00000004 0000000c 53c43dac 00000121 490f8104 0000045c 00000001 00000586
    9fe0: 00000000 53c41f48 4013a77c 4013b5b4 80000010 0000001e 1a000058 e59d3038
    Backtrace:
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c032774c r7:000004d0 r6:00000680 r5:ce402100 r4:cd27f180
    r3:c079a018
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c032774c>] (sock_alloc_send_pskb+0xb0/0x2f4)
    [<c032769c>] (sock_alloc_send_pskb+0x0/0x2f4) from [<c03279b4>] (sock_alloc_send_skb+0x24/0x2c)
    [<c0327990>] (sock_alloc_send_skb+0x0/0x2c) from [<c03538a8>] (ip_append_data+0x488/0x984)
    [<c0353420>] (ip_append_data+0x0/0x984) from [<c037081c>] (udp_sendmsg+0x498/0x5c0)
    [<c0370384>] (udp_sendmsg+0x0/0x5c0) from [<c0377fe4>] (inet_sendmsg+0x68/0x74)
    [<c0377f7c>] (inet_sendmsg+0x0/0x74) from [<c03245f4>] (sock_sendmsg+0xa8/0xc4)
    r7:00000000 r6:cc299f3c r5:00000586 r4:ce09e780
    [<c032454c>] (sock_sendmsg+0x0/0xc4) from [<c03258a4>] (sys_sendto+0xb8/0xdc)
    r9:cc298000 r8:00000000 r6:ce09e780 r5:00000586 r4:53c41f64
    [<c03257ec>] (sys_sendto+0x0/0xdc) from [<c03258e8>] (sys_send+0x20/0x28)
    [<c03258c8>] (sys_send+0x0/0x28) from [<c004b3c0>] (ret_fast_syscall+0x0/0x30)
    Code: e5943000 e5937000 e3570000 15942010 (17972002)
    ---[ end trace aefa0ff79f71a291 ]---
    Unable to handle kernel NULL pointer dereference at virtual address 00000230
    pgd = cdc0c000
    [00000230] *pgd=8de7e031, *pte=00000000, *ppte=00000000
    Internal error: Oops: 17 [#2]
    last sysfs file: /sys/bus/platform/drivers/TI81XX_HDMI/uevent
    Modules linked in: gpiorw ti81xxhdmi ti81xxfb vpss osa_kermod syslink
    CPU: 0 Tainted: G D (2.6.37 #1)
    PC is at __kmalloc_track_caller+0x6c/0xc8
    LR is at __kmalloc_track_caller+0x44/0xc8
    pc : [<c00ca050>] lr : [<c00ca028>] psr: 20000093
    sp : cdc739e0 ip : cdc739e0 fp : cdc73a04
    r10: 00000001 r9 : cdc72000 r8 : c035a9e4
    r7 : 00000230 r6 : a0000013 r5 : 00000020 r4 : ce402400
    r3 : c079a060 r2 : 00000000 r1 : 00000020 r0 : ce402400
    Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
    Control: 10c5387d Table: 8dc0c019 DAC: 00000017
    Process syslogd (pid: 581, stack limit = 0xcdc722e8)
    Stack: (0xcdc739e0 to 0xcdc74000)
    39e0: c079a108 cde7c780 ce404580 00000700 00000020 c035a9e4 cdc73a2c cdc73a08
    3a00: c032ba60 c00c9ff0 cdc24480 000005a8 cdc73a94 cdc24480 00000000 cdf0f804
    3a20: cdc73a4c cdc73a30 c035a9e4 c032ba14 00009f28 cdc24480 00000000 cdc73a94
    3a40: cdc73ac4 cdc73a50 c035ac4c c035a9bc cdc73a8c 00000000 00000000 00000000
    3a60: cdc73a7c 00000000 cdc73c08 0000007c 000005a8 00000000 000005a8 cdc244f4
    3a80: 00000000 00004040 c0377fe4 c035aab8 00000000 000005a8 cdc73bb4 cdc73ae8
    3aa0: 0000007c cdc73bec cdc24480 00000000 00000000 00000000 cdc73ae4 cdc73ac8
    3ac0: c0377fe4 c035aab8 ce004c00 0000007c cdc73bec cdc280ec cdc73bc4 cdc73ae8
    3ae0: c03245f4 c0377f88 c0527610 3b9ac9ff 00000000 00000001 ffffffff 00000000
    3b00: 00000000 00000000 00000000 00000000 cd162dc0 00000020 00000000 00000000
    3b20: cdc73b54 cdc73b30 cdc73b60 c01cc5dc 00000000 3b51a375 00000017 ce428580
    3b40: 00000000 00000082 00000000 cd162dc0 cd162df0 c05258e8 cd162dc0 cd162df0
    3b60: cdc73b94 cdc73b70 c0070af4 0000007c ce004c00 c05258e8 00000000 cdc73bec
    3b80: 00000000 00000082 cdc73bac cdc73b98 c006f5ac c0070a88 ce428580 00000001
    3ba0: cdc73bcc bf000000 00000000 cdf0f804 00004040 ce07a998 cdc73bdc cdc73bc8
    3bc0: c0324658 c0324558 00000001 ce004c00 cdc73c34 cdc73be0 c0398530 c032461c
    3be0: 0000007c cdc4001c cd162dc0 00000000 00000000 cdc73c08 00000001 00000000
    3c00: 00000000 00004040 cdf0f804 0000007c 00000001 cdc280ec 00000000 ce004c00
    3c20: 0000007c cd16b880 cdc73c74 cdc73c38 c03985c0 c03984ac 00000000 00000000
    3c40: 00000024 00000009 cdc73ca8 cdc280e8 ce7db000 cdc280ec c0597778 cdc28178
    3c60: ce07a998 cd16b880 cdc73ca4 cdc73c78 c0398854 c0398544 00000000 00000000
    3c80: ce7db000 cdc280e8 cd16b880 cdc2815c cdc28178 00000000 cdc73cd4 cdc73ca8
    3ca0: c0396cb8 c0398808 2ffa804e 000052d1 cd16b880 cd16b880 cdc280e8 cdc280e8
    3cc0: 00000000 00000000 cdc73cfc cdc73cd8 c03940d8 c0396bc8 ce07a8a8 cd16b880
    3ce0: cd16b8b8 cd16b880 cd16b8b8 c0597778 cdc73d24 cdc73d00 c039af48 c0393ef4
    3d00: cd16b880 00000000 cdc73d64 c059777c cdc72000 ce7e3009 cdc73d3c cdc73d28
    3d20: c039b178 c039aec4 00000002 cd16b880 cdc73d5c cdc73d40 c03951d0 c039b124
    3d40: c03f17c4 00000000 cdc73dc0 ce705500 cdc73d9c cdc73d60 c0395320 c03950d0
    3d60: ce079ad8 00000000 ce705500 cdc73dc0 c03f17c4 00000000 00000000 00000000
    3d80: c0158d50 cdc72000 cdc73dc0 ce705500 cdc73dbc cdc73da0 c0158968 c03952d0
    3da0: 00000000 ce7bfcc0 ce7bb600 ce07a8a8 cdc73dec cdc73dc0 c0159ad0 c0158944
    3dc0: c0532268 ce07a8a8 ce7bfcc0 00000000 c014bb74 ce07a998 ce7bfcc0 ce7bb600
    3de0: cdc73e14 cdc73df0 c014bca4 c0159a74 00000000 cdc73ec8 ce07a998 ce07aa3c
    3e00: ce07a8a0 00000010 cdc73e34 cdc73e18 c014be00 c014bc00 c01544b4 cdc73ec8
    3e20: ce07a998 ce07ed00 cdc73e4c cdc73e38 c01544d4 c014bdbc cdc73ec8 00000000
    3e40: cdc73e9c cdc73e50 c00d5d70 c01544c0 00000000 0000006c cdc73f20 0026c8ca
    3e60: 00000003 ce7e3005 ce7046c0 ce07ed00 c00d4a4c cdc73ec8 00020d41 0000002a
    3e80: 00000000 cdc72000 00020d42 0000002a cdc73f5c cdc73ea0 c00d7200 c00d5a20
    3ea0: 00000000 00000000 ffffff9c ce7e3000 cdc73edc 000001b6 ffffffff 00000000
    3ec0: 00000000 00000000 ce7046c0 ce074d80 cd162dc0 cdc73ee0 00000000 ce7046c0
    3ee0: ce074700 00000014 00000004 00000001 000000ff 00000000 000000ff 00000000
    3f00: 00000000 00000000 00000000 cc015500 00000000 cdc59308 cdc59300 00000000
    3f20: 00020d41 00000004 cdc73f5c cdc73f38 c00e09c4 00000004 00020d41 ce7e3000
    3f40: 00000000 000001b6 cdc72000 ffffff9c cdc73f94 cdc73f60 c00cb3c0 c00d712c
    3f60: 00000000 cdc73f70 c00ca1cc be9eaefb 51088c07 0009f45c 00000005 c004b568
    3f80: cdc72000 00000000 cdc73fa4 cdc73f98 c00cb478 c00cb368 00000000 cdc73fa8
    3fa0: c004b3c0 c00cb45c be9eaefb 51088c07 be9eaefb 00020d41 000001b6 00000000
    3fc0: be9eaefb 51088c07 0009f45c 00000005 000979d0 00095780 0000006d be9eab6c
    3fe0: 000953cc be9eaaa8 0003fa88 40219af4 60000010 be9eaefb 00000000 00000000
    Backtrace:
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c035a9e4 r7:00000020 r6:00000700 r5:ce404580 r4:cde7c780
    r3:c079a108
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c035a9e4>] (sk_stream_alloc_skb+0x34/0xfc)
    [<c035a9b0>] (sk_stream_alloc_skb+0x0/0xfc) from [<c035ac4c>] (tcp_sendmsg+0x1a0/0xaac)
    r6:cdc73a94 r5:00000000 r4:cdc24480 r3:00009f28
    [<c035aaac>] (tcp_sendmsg+0x0/0xaac) from [<c0377fe4>] (inet_sendmsg+0x68/0x74)
    [<c0377f7c>] (inet_sendmsg+0x0/0x74) from [<c03245f4>] (sock_sendmsg+0xa8/0xc4)
    r7:cdc280ec r6:cdc73bec r5:0000007c r4:ce004c00
    [<c032454c>] (sock_sendmsg+0x0/0xc4) from [<c0324658>] (kernel_sendmsg+0x48/0x80)
    r9:ce07a998 r8:00004040 r6:cdf0f804 r5:00000000 r4:bf000000
    [<c0324610>] (kernel_sendmsg+0x0/0x80) from [<c0398530>] (xs_send_kvec+0x90/0x98)
    r4:ce004c00 r3:00000001
    [<c03984a0>] (xs_send_kvec+0x0/0x98) from [<c03985c0>] (xs_sendpages+0x88/0x1f0)
    [<c0398538>] (xs_sendpages+0x0/0x1f0) from [<c0398854>] (xs_tcp_send_request+0x58/0x16c)
    [<c03987fc>] (xs_tcp_send_request+0x0/0x16c) from [<c0396cb8>] (xprt_transmit+0xfc/0x1e0)
    [<c0396bbc>] (xprt_transmit+0x0/0x1e0) from [<c03940d8>] (call_transmit+0x1f0/0x264)
    r8:00000000 r7:00000000 r6:cdc280e8 r5:cdc280e8 r4:cd16b880
    [<c0393ee8>] (call_transmit+0x0/0x264) from [<c039af48>] (__rpc_execute+0x90/0x230)
    r6:c0597778 r5:cd16b8b8 r4:cd16b880
    [<c039aeb8>] (__rpc_execute+0x0/0x230) from [<c039b178>] (rpc_execute+0x60/0x68)
    [<c039b118>] (rpc_execute+0x0/0x68) from [<c03951d0>] (rpc_run_task+0x10c/0x118)
    r4:cd16b880 r3:00000002
    [<c03950c4>] (rpc_run_task+0x0/0x118) from [<c0395320>] (rpc_call_sync+0x5c/0x84)
    r6:ce705500 r5:cdc73dc0 r4:00000000 r3:c03f17c4
    [<c03952c4>] (rpc_call_sync+0x0/0x84) from [<c0158968>] (nfs3_rpc_wrapper.clone.10+0x30/0x70)
    r6:ce705500 r5:cdc73dc0 r4:cdc72000
    [<c0158938>] (nfs3_rpc_wrapper.clone.10+0x0/0x70) from [<c0159ad0>] (nfs3_proc_getattr+0x68/0xa4)
    r6:ce07a8a8 r5:ce7bb600 r4:ce7bfcc0 r3:00000000
    [<c0159a68>] (nfs3_proc_getattr+0x0/0xa4) from [<c014bca4>] (__nfs_revalidate_inode+0xb0/0x1bc)
    r6:ce7bb600 r5:ce7bfcc0 r4:ce07a998
    [<c014bbf4>] (__nfs_revalidate_inode+0x0/0x1bc) from [<c014be00>] (nfs_revalidate_mapping+0x50/0xf4)
    r7:00000010 r6:ce07a8a0 r5:ce07aa3c r4:ce07a998
    [<c014bdb0>] (nfs_revalidate_mapping+0x0/0xf4) from [<c01544d4>] (nfs_follow_link+0x20/0x78)
    r6:ce07ed00 r5:ce07a998 r4:cdc73ec8 r3:c01544b4
    [<c01544b4>] (nfs_follow_link+0x0/0x78) from [<c00d5d70>] (link_path_walk+0x35c/0xa48)
    r5:00000000 r4:cdc73ec8
    [<c00d5a14>] (link_path_walk+0x0/0xa48) from [<c00d7200>] (do_filp_open+0xe0/0x4ac)
    [<c00d7120>] (do_filp_open+0x0/0x4ac) from [<c00cb3c0>] (do_sys_open+0x64/0xf4)
    [<c00cb35c>] (do_sys_open+0x0/0xf4) from [<c00cb478>] (sys_open+0x28/0x2c)
    [<c00cb450>] (sys_open+0x0/0x2c) from [<c004b3c0>] (ret_fast_syscall+0x0/0x30)
    Code: e5943000 e5937000 e3570000 15942010 (17972002)
    Unable to handle kernel NULL pointer dereference at virtual address 00000230
    pgd = cdc0c000
    [00000230] *pgd=8de7e031, *pte=00000000, *ppte=00000000
    Internal error: Oops: 17 [#3]
    last sysfs file: /sys/bus/platform/drivers/TI81XX_HDMI/uevent
    Modules linked in: gpiorw ti81xxhdmi ti81xxfb vpss osa_kermod syslink
    CPU: 0 Tainted: G D (2.6.37 #1)
    PC is at __kmalloc_track_caller+0x6c/0xc8
    LR is at __kmalloc_track_caller+0x44/0xc8
    pc : [<c00ca050>] lr : [<c00ca028>] psr: 20000193
    sp : cdc735e8 ip : cdc735e8 fp : cdc7360c
    r10: 00000000 r9 : 00000000 r8 : c032c098
    r7 : 00000230 r6 : a0000113 r5 : 00000020 r4 : ce402400
    r3 : c079a060 r2 : 00000000 r1 : 00000020 r0 : ce402400
    Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
    Control: 10c5387d Table: 8dc0c019 DAC: 00000017
    Process syslogd (pid: 581, stack limit = 0xcdc722e8)
    Stack: (0xcdc735e8 to 0xcdc74000)
    35e0: c079a018 ce7bfd80 ce402100 000006c0 00000020 c032c098
    3600: cdc73634 cdc73610 c032ba60 c00c9ff0 ce7be600 ce563340 076258f8 ce563000
    3620: ce563340 0000003c cdc73644 cdc73638 c032c098 c032ba14 cdc7365c cdc73648
    3640: c026d128 c032c084 ce7be600 ce7be600 cdc7368c cdc73660 c026f970 c026d11c
    3660: c03bcfa0 a0000193 00000000 60000113 ce702c80 ce706700 00000000 ce7be600
    3680: cdc736b4 cdc73690 c0271100 c026f8fc 00000000 c000003c 4a1020c0 1aa8a0c0
    36a0: ce563000 00000001 cdc736d4 cdc736b8 c0271200 c0271084 ce702c80 00000000
    36c0: 00000040 ce563340 cdc736f4 cdc736d8 c0271244 c0271110 00010000 ce56334c
    36e0: 00000001 00000040 cdc73724 cdc736f8 c026f68c c0271220 cdc73724 00000001
    3700: ce56334c c055cf00 00000001 0000012c 00000040 c055cf08 cdc7375c cdc73728
    3720: c03331fc c026f5e4 00000002 001fe50c cdc73730 00000001 c057ee50 cdc72000
    3740: 00000100 c057ee00 00000000 00000009 cdc7379c cdc73760 c0079a0c c033319c
    3760: c0058d6c c008f324 cdc73794 00000003 c0529328 00000043 00000000 cdc73847
    3780: 00000000 00000017 00000000 c00ca050 cdc737ac cdc737a0 c0079dfc c0079994
    37a0: cdc737c4 cdc737b0 c004107c c0079dc0 ffffffff fa200000 cdc73884 cdc737c8
    37c0: c03b6434 c004100c 00000000 c055e4d8 00000080 cdc72000 00000001 c00ca052
    37e0: cdc73847 00000000 00000017 00000000 c00ca050 cdc73884 cdc73810 cdc73810
    3800: c004f3d0 c0074c2c 60000113 ffffffff cdc722e8 0000000b cde7e800 00000000
    3820: 65c73998 33343935 20303030 33393565 30303037 35336520 30303037 35312030
    3840: 30323439 28203031 37393731 32303032 c0002029 c03b45e0 c04d6852 00000230
    3860: 00000017 cdd15300 cdc73998 00000000 00010000 cdd15334 cdc738ac cdc73888
    3880: c00520f0 c004f17c cdc73998 cdc73998 00000230 cd162dc0 cdd15300 00000017
    38a0: cdc738e4 cdc738b0 c03b82f8 c0052090 c03331fc c026f5e4 cdc738e4 00000017
    38c0: 00000007 00000230 c050e560 cdc73998 20000193 00000001 cdc73994 cdc738e8
    38e0: c00411f4 c03b811c c0079a78 c00798d0 00000000 00000000 cdc73924 00000003
    3900: c0528af4 0000002a 00000000 000005a8 00000080 cde7c798 00000000 000005a8
    3920: cdc7393c cdc73930 c0079e08 c0072064 cdc73954 cdc73940 c004107c cdc24668
    3940: cdc73974 cdc73950 c007f3a8 c007eab8 c03b6434 a0000013 cdc24480 cdc24480
    3960: cde7c780 000005a8 cdc7398c cdc73978 c0327424 ffffffff cdc739cc a0000013
    3980: 00000230 c035a9e4 cdc73a04 cdc73998 c03b63ec c00411c8 ce402400 00000020
    39a0: 00000000 c079a060 ce402400 00000020 a0000013 00000230 c035a9e4 cdc72000
    39c0: 00000001 cdc73a04 cdc739e0 cdc739e0 c00ca028 c00ca050 20000093 ffffffff
    39e0: c079a108 cde7c780 ce404580 00000700 00000020 c035a9e4 cdc73a2c cdc73a08
    3a00: c032ba60 c00c9ff0 cdc24480 000005a8 cdc73a94 cdc24480 00000000 cdf0f804
    3a20: cdc73a4c cdc73a30 c035a9e4 c032ba14 00009f28 cdc24480 00000000 cdc73a94
    3a40: cdc73ac4 cdc73a50 c035ac4c c035a9bc cdc73a8c 00000000 00000000 00000000
    3a60: cdc73a7c 00000000 cdc73c08 0000007c 000005a8 00000000 000005a8 cdc244f4
    3a80: 00000000 00004040 c0377fe4 c035aab8 00000000 000005a8 cdc73bb4 cdc73ae8
    3aa0: 0000007c cdc73bec cdc24480 00000000 00000000 00000000 cdc73ae4 cdc73ac8
    3ac0: c0377fe4 c035aab8 ce004c00 0000007c cdc73bec cdc280ec cdc73bc4 cdc73ae8
    3ae0: c03245f4 c0377f88 c0527610 3b9ac9ff 00000000 00000001 ffffffff 00000000
    3b00: 00000000 00000000 00000000 00000000 cd162dc0 00000020 00000000 00000000
    3b20: cdc73b54 cdc73b30 cdc73b60 c01cc5dc 00000000 3b51a375 00000017 ce428580
    3b40: 00000000 00000082 00000000 cd162dc0 cd162df0 c05258e8 cd162dc0 cd162df0
    3b60: cdc73b94 cdc73b70 c0070af4 0000007c ce004c00 c05258e8 00000000 cdc73bec
    3b80: 00000000 00000082 cdc73bac cdc73b98 c006f5ac c0070a88 ce428580 00000001
    3ba0: cdc73bcc bf000000 00000000 cdf0f804 00004040 ce07a998 cdc73bdc cdc73bc8
    3bc0: c0324658 c0324558 00000001 ce004c00 cdc73c34 cdc73be0 c0398530 c032461c
    3be0: 0000007c cdc4001c cd162dc0 00000000 00000000 cdc73c08 00000001 00000000
    3c00: 00000000 00004040 cdf0f804 0000007c 00000001 cdc280ec 00000000 ce004c00
    3c20: 0000007c cd16b880 cdc73c74 cdc73c38 c03985c0 c03984ac 00000000 00000000
    3c40: 00000024 00000009 cdc73ca8 cdc280e8 ce7db000 cdc280ec c0597778 cdc28178
    3c60: ce07a998 cd16b880 cdc73ca4 cdc73c78 c0398854 c0398544 00000000 00000000
    3c80: ce7db000 cdc280e8 cd16b880 cdc2815c cdc28178 00000000 cdc73cd4 cdc73ca8
    3ca0: c0396cb8 c0398808 2ffa804e 000052d1 cd16b880 cd16b880 cdc280e8 cdc280e8
    3cc0: 00000000 00000000 cdc73cfc cdc73cd8 c03940d8 c0396bc8 ce07a8a8 cd16b880
    3ce0: cd16b8b8 cd16b880 cd16b8b8 c0597778 cdc73d24 cdc73d00 c039af48 c0393ef4
    3d00: cd16b880 00000000 cdc73d64 c059777c cdc72000 ce7e3009 cdc73d3c cdc73d28
    3d20: c039b178 c039aec4 00000002 cd16b880 cdc73d5c cdc73d40 c03951d0 c039b124
    3d40: c03f17c4 00000000 cdc73dc0 ce705500 cdc73d9c cdc73d60 c0395320 c03950d0
    3d60: ce079ad8 00000000 ce705500 cdc73dc0 c03f17c4 00000000 00000000 00000000
    3d80: c0158d50 cdc72000 cdc73dc0 ce705500 cdc73dbc cdc73da0 c0158968 c03952d0
    3da0: 00000000 ce7bfcc0 ce7bb600 ce07a8a8 cdc73dec cdc73dc0 c0159ad0 c0158944
    3dc0: c0532268 ce07a8a8 ce7bfcc0 00000000 c014bb74 ce07a998 ce7bfcc0 ce7bb600
    3de0: cdc73e14 cdc73df0 c014bca4 c0159a74 00000000 cdc73ec8 ce07a998 ce07aa3c
    3e00: ce07a8a0 00000010 cdc73e34 cdc73e18 c014be00 c014bc00 c01544b4 cdc73ec8
    3e20: ce07a998 ce07ed00 cdc73e4c cdc73e38 c01544d4 c014bdbc cdc73ec8 00000000
    3e40: cdc73e9c cdc73e50 c00d5d70 c01544c0 00000000 0000006c cdc73f20 0026c8ca
    3e60: 00000003 ce7e3005 ce7046c0 ce07ed00 c00d4a4c cdc73ec8 00020d41 0000002a
    3e80: 00000000 cdc72000 00020d42 0000002a cdc73f5c cdc73ea0 c00d7200 c00d5a20
    3ea0: 00000000 00000000 ffffff9c ce7e3000 cdc73edc 000001b6 ffffffff 00000000
    3ec0: 00000000 00000000 ce7046c0 ce074d80 cd162dc0 cdc73ee0 00000000 ce7046c0
    3ee0: ce074700 00000014 00000004 00000001 000000ff 00000000 000000ff 00000000
    3f00: 00000000 00000000 00000000 cc015500 00000000 cdc59308 cdc59300 00000000
    3f20: 00020d41 00000004 cdc73f5c cdc73f38 c00e09c4 00000004 00020d41 ce7e3000
    3f40: 00000000 000001b6 cdc72000 ffffff9c cdc73f94 cdc73f60 c00cb3c0 c00d712c
    3f60: 00000000 cdc73f70 c00ca1cc be9eaefb 51088c07 0009f45c 00000005 c004b568
    3f80: cdc72000 00000000 cdc73fa4 cdc73f98 c00cb478 c00cb368 00000000 cdc73fa8
    3fa0: c004b3c0 c00cb45c be9eaefb 51088c07 be9eaefb 00020d41 000001b6 00000000
    3fc0: be9eaefb 51088c07 0009f45c 00000005 000979d0 00095780 0000006d be9eab6c
    3fe0: 000953cc be9eaaa8 0003fa88 40219af4 60000010 be9eaefb 00000000 00000000
    Backtrace:
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c032c098 r7:00000020 r6:000006c0 r5:ce402100 r4:ce7bfd80
    r3:c079a018
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c032c098>] (dev_alloc_skb+0x20/0x44)
    [<c032c078>] (dev_alloc_skb+0x0/0x44) from [<c026d128>] (emac_rx_alloc+0x18/0x5c)
    [<c026d110>] (emac_rx_alloc+0x0/0x5c) from [<c026f970>] (emac_rx_handler+0x80/0x104)
    r5:ce7be600 r4:ce7be600
    [<c026f8f0>] (emac_rx_handler+0x0/0x104) from [<c0271100>] (__cpdma_chan_free+0x88/0x8c)
    r8:ce7be600 r7:00000000 r6:ce706700 r5:ce702c80 r4:60000113
    [<c0271078>] (__cpdma_chan_free+0x0/0x8c) from [<c0271200>] (__cpdma_chan_process+0xfc/0x110)
    [<c0271104>] (__cpdma_chan_process+0x0/0x110) from [<c0271244>] (cpdma_chan_process+0x30/0x54)
    r7:ce563340 r6:00000040 r5:00000000 r4:ce702c80
    [<c0271214>] (cpdma_chan_process+0x0/0x54) from [<c026f68c>] (emac_poll+0xb4/0x2a4)
    r6:00000040 r5:00000001 r4:ce56334c r3:00010000
    [<c026f5d8>] (emac_poll+0x0/0x2a4) from [<c03331fc>] (net_rx_action+0x6c/0x15c)
    [<c0333190>] (net_rx_action+0x0/0x15c) from [<c0079a0c>] (__do_softirq+0x84/0x114)
    [<c0079988>] (__do_softirq+0x0/0x114) from [<c0079dfc>] (irq_exit+0x48/0x98)
    [<c0079db4>] (irq_exit+0x0/0x98) from [<c004107c>] (asm_do_IRQ+0x7c/0x9c)
    [<c0041000>] (asm_do_IRQ+0x0/0x9c) from [<c03b6434>] (__irq_svc+0x34/0xa0)
    Exception stack(0xcdc737c8 to 0xcdc73810)
    37c0: 00000000 c055e4d8 00000080 cdc72000 00000001 c00ca052
    37e0: cdc73847 00000000 00000017 00000000 c00ca050 cdc73884 cdc73810 cdc73810
    3800: c004f3d0 c0074c2c 60000113 ffffffff
    r5:fa200000 r4:ffffffff
    [<c004f170>] (die+0x0/0x2ec) from [<c00520f0>] (__do_kernel_fault+0x6c/0x8c)
    [<c0052084>] (__do_kernel_fault+0x0/0x8c) from [<c03b82f8>] (do_page_fault+0x1e8/0x204)
    r8:00000017 r7:cdd15300 r6:cd162dc0 r5:00000230 r4:cdc73998
    r3:cdc73998
    [<c03b8110>] (do_page_fault+0x0/0x204) from [<c00411f4>] (do_DataAbort+0x38/0xa0)
    [<c00411bc>] (do_DataAbort+0x0/0xa0) from [<c03b63ec>] (__dabt_svc+0x4c/0x60)
    Exception stack(0xcdc73998 to 0xcdc739e0)
    3980: ce402400 00000020
    39a0: 00000000 c079a060 ce402400 00000020 a0000013 00000230 c035a9e4 cdc72000
    39c0: 00000001 cdc73a04 cdc739e0 cdc739e0 c00ca028 c00ca050 20000093 ffffffff
    r8:c035a9e4 r7:00000230 r6:a0000013 r5:cdc739cc r4:ffffffff
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c035a9e4 r7:00000020 r6:00000700 r5:ce404580 r4:cde7c780
    r3:c079a108
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c035a9e4>] (sk_stream_alloc_skb+0x34/0xfc)
    [<c035a9b0>] (sk_stream_alloc_skb+0x0/0xfc) from [<c035ac4c>] (tcp_sendmsg+0x1a0/0xaac)
    r6:cdc73a94 r5:00000000 r4:cdc24480 r3:00009f28
    [<c035aaac>] (tcp_sendmsg+0x0/0xaac) from [<c0377fe4>] (inet_sendmsg+0x68/0x74)
    [<c0377f7c>] (inet_sendmsg+0x0/0x74) from [<c03245f4>] (sock_sendmsg+0xa8/0xc4)
    r7:cdc280ec r6:cdc73bec r5:0000007c r4:ce004c00
    [<c032454c>] (sock_sendmsg+0x0/0xc4) from [<c0324658>] (kernel_sendmsg+0x48/0x80)
    r9:ce07a998 r8:00004040 r6:cdf0f804 r5:00000000 r4:bf000000
    [<c0324610>] (kernel_sendmsg+0x0/0x80) from [<c0398530>] (xs_send_kvec+0x90/0x98)
    r4:ce004c00 r3:00000001
    [<c03984a0>] (xs_send_kvec+0x0/0x98) from [<c03985c0>] (xs_sendpages+0x88/0x1f0)
    [<c0398538>] (xs_sendpages+0x0/0x1f0) from [<c0398854>] (xs_tcp_send_request+0x58/0x16c)
    [<c03987fc>] (xs_tcp_send_request+0x0/0x16c) from [<c0396cb8>] (xprt_transmit+0xfc/0x1e0)
    [<c0396bbc>] (xprt_transmit+0x0/0x1e0) from [<c03940d8>] (call_transmit+0x1f0/0x264)
    r8:00000000 r7:00000000 r6:cdc280e8 r5:cdc280e8 r4:cd16b880
    [<c0393ee8>] (call_transmit+0x0/0x264) from [<c039af48>] (__rpc_execute+0x90/0x230)
    r6:c0597778 r5:cd16b8b8 r4:cd16b880
    [<c039aeb8>] (__rpc_execute+0x0/0x230) from [<c039b178>] (rpc_execute+0x60/0x68)
    [<c039b118>] (rpc_execute+0x0/0x68) from [<c03951d0>] (rpc_run_task+0x10c/0x118)
    r4:cd16b880 r3:00000002
    [<c03950c4>] (rpc_run_task+0x0/0x118) from [<c0395320>] (rpc_call_sync+0x5c/0x84)
    r6:ce705500 r5:cdc73dc0 r4:00000000 r3:c03f17c4
    [<c03952c4>] (rpc_call_sync+0x0/0x84) from [<c0158968>] (nfs3_rpc_wrapper.clone.10+0x30/0x70)
    r6:ce705500 r5:cdc73dc0 r4:cdc72000
    [<c0158938>] (nfs3_rpc_wrapper.clone.10+0x0/0x70) from [<c0159ad0>] (nfs3_proc_getattr+0x68/0xa4)
    r6:ce07a8a8 r5:ce7bb600 r4:ce7bfcc0 r3:00000000
    [<c0159a68>] (nfs3_proc_getattr+0x0/0xa4) from [<c014bca4>] (__nfs_revalidate_inode+0xb0/0x1bc)
    r6:ce7bb600 r5:ce7bfcc0 r4:ce07a998
    [<c014bbf4>] (__nfs_revalidate_inode+0x0/0x1bc) from [<c014be00>] (nfs_revalidate_mapping+0x50/0xf4)
    r7:00000010 r6:ce07a8a0 r5:ce07aa3c r4:ce07a998
    [<c014bdb0>] (nfs_revalidate_mapping+0x0/0xf4) from [<c01544d4>] (nfs_follow_link+0x20/0x78)
    r6:ce07ed00 r5:ce07a998 r4:cdc73ec8 r3:c01544b4
    [<c01544b4>] (nfs_follow_link+0x0/0x78) from [<c00d5d70>] (link_path_walk+0x35c/0xa48)
    r5:00000000 r4:cdc73ec8
    [<c00d5a14>] (link_path_walk+0x0/0xa48) from [<c00d7200>] (do_filp_open+0xe0/0x4ac)
    [<c00d7120>] (do_filp_open+0x0/0x4ac) from [<c00cb3c0>] (do_sys_open+0x64/0xf4)
    [<c00cb35c>] (do_sys_open+0x0/0xf4) from [<c00cb478>] (sys_open+0x28/0x2c)
    [<c00cb450>] (sys_open+0x0/0x2c) from [<c004b3c0>] (ret_fast_syscall+0x0/0x30)
    Code: e5943000 e5937000 e3570000 15942010 (17972002)
    ---[ end trace aefa0ff79f71a292 ]---
    Kernel panic - not syncing: Fatal exception in interrupt
    Backtrace:
    [<c004efac>] (dump_backtrace+0x0/0x110) from [<c03b4454>] (dump_stack+0x18/0x1c)
    r6:cdc7344f r5:c00ca052 r4:c055e4d8 r3:60000113
    [<c03b443c>] (dump_stack+0x0/0x1c) from [<c03b44b4>] (panic+0x5c/0x178)
    [<c03b4458>] (panic+0x0/0x178) from [<c004f40c>] (die+0x29c/0x2ec)
    r3:00000100 r2:cdc73418 r1:c055e920 r0:c04825aa
    r7:00000000
    [<c004f170>] (die+0x0/0x2ec) from [<c00520f0>] (__do_kernel_fault+0x6c/0x8c)
    [<c0052084>] (__do_kernel_fault+0x0/0x8c) from [<c03b82f8>] (do_page_fault+0x1e8/0x204)
    r8:00000017 r7:cdd15300 r6:cd162dc0 r5:00000230 r4:cdc735a0
    r3:cdc735a0
    [<c03b8110>] (do_page_fault+0x0/0x204) from [<c00411f4>] (do_DataAbort+0x38/0xa0)
    [<c00411bc>] (do_DataAbort+0x0/0xa0) from [<c03b63ec>] (__dabt_svc+0x4c/0x60)
    Exception stack(0xcdc735a0 to 0xcdc735e8)
    35a0: ce402400 00000020 00000000 c079a060 ce402400 00000020 a0000113 00000230
    35c0: c032c098 00000000 00000000 cdc7360c cdc735e8 cdc735e8 c00ca028 c00ca050
    35e0: 20000193 ffffffff
    r8:c032c098 r7:00000230 r6:a0000113 r5:cdc735d4 r4:ffffffff
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c032c098 r7:00000020 r6:000006c0 r5:ce402100 r4:ce7bfd80
    r3:c079a018
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c032c098>] (dev_alloc_skb+0x20/0x44)
    [<c032c078>] (dev_alloc_skb+0x0/0x44) from [<c026d128>] (emac_rx_alloc+0x18/0x5c)
    [<c026d110>] (emac_rx_alloc+0x0/0x5c) from [<c026f970>] (emac_rx_handler+0x80/0x104)
    r5:ce7be600 r4:ce7be600
    [<c026f8f0>] (emac_rx_handler+0x0/0x104) from [<c0271100>] (__cpdma_chan_free+0x88/0x8c)
    r8:ce7be600 r7:00000000 r6:ce706700 r5:ce702c80 r4:60000113
    [<c0271078>] (__cpdma_chan_free+0x0/0x8c) from [<c0271200>] (__cpdma_chan_process+0xfc/0x110)
    [<c0271104>] (__cpdma_chan_process+0x0/0x110) from [<c0271244>] (cpdma_chan_process+0x30/0x54)
    r7:ce563340 r6:00000040 r5:00000000 r4:ce702c80
    [<c0271214>] (cpdma_chan_process+0x0/0x54) from [<c026f68c>] (emac_poll+0xb4/0x2a4)
    r6:00000040 r5:00000001 r4:ce56334c r3:00010000
    [<c026f5d8>] (emac_poll+0x0/0x2a4) from [<c03331fc>] (net_rx_action+0x6c/0x15c)
    [<c0333190>] (net_rx_action+0x0/0x15c) from [<c0079a0c>] (__do_softirq+0x84/0x114)
    [<c0079988>] (__do_softirq+0x0/0x114) from [<c0079dfc>] (irq_exit+0x48/0x98)
    [<c0079db4>] (irq_exit+0x0/0x98) from [<c004107c>] (asm_do_IRQ+0x7c/0x9c)
    [<c0041000>] (asm_do_IRQ+0x0/0x9c) from [<c03b6434>] (__irq_svc+0x34/0xa0)
    Exception stack(0xcdc737c8 to 0xcdc73810)
    37c0: 00000000 c055e4d8 00000080 cdc72000 00000001 c00ca052
    37e0: cdc73847 00000000 00000017 00000000 c00ca050 cdc73884 cdc73810 cdc73810
    3800: c004f3d0 c0074c2c 60000113 ffffffff
    r5:fa200000 r4:ffffffff
    [<c004f170>] (die+0x0/0x2ec) from [<c00520f0>] (__do_kernel_fault+0x6c/0x8c)
    [<c0052084>] (__do_kernel_fault+0x0/0x8c) from [<c03b82f8>] (do_page_fault+0x1e8/0x204)
    r8:00000017 r7:cdd15300 r6:cd162dc0 r5:00000230 r4:cdc73998
    r3:cdc73998
    [<c03b8110>] (do_page_fault+0x0/0x204) from [<c00411f4>] (do_DataAbort+0x38/0xa0)
    [<c00411bc>] (do_DataAbort+0x0/0xa0) from [<c03b63ec>] (__dabt_svc+0x4c/0x60)
    Exception stack(0xcdc73998 to 0xcdc739e0)
    3980: ce402400 00000020
    39a0: 00000000 c079a060 ce402400 00000020 a0000013 00000230 c035a9e4 cdc72000
    39c0: 00000001 cdc73a04 cdc739e0 cdc739e0 c00ca028 c00ca050 20000093 ffffffff
    r8:c035a9e4 r7:00000230 r6:a0000013 r5:cdc739cc r4:ffffffff
    [<c00c9fe4>] (__kmalloc_track_caller+0x0/0xc8) from [<c032ba60>] (__alloc_skb+0x58/0xe8)
    r8:c035a9e4 r7:00000020 r6:00000700 r5:ce404580 r4:cde7c780
    r3:c079a108
    [<c032ba08>] (__alloc_skb+0x0/0xe8) from [<c035a9e4>] (sk_stream_alloc_skb+0x34/0xfc)
    [<c035a9b0>] (sk_stream_alloc_skb+0x0/0xfc) from [<c035ac4c>] (tcp_sendmsg+0x1a0/0xaac)
    r6:cdc73a94 r5:00000000 r4:cdc24480 r3:00009f28
    [<c035aaac>] (tcp_sendmsg+0x0/0xaac) from [<c0377fe4>] (inet_sendmsg+0x68/0x74)
    [<c0377f7c>] (inet_sendmsg+0x0/0x74) from [<c03245f4>] (sock_sendmsg+0xa8/0xc4)
    r7:cdc280ec r6:cdc73bec r5:0000007c r4:ce004c00
    [<c032454c>] (sock_sendmsg+0x0/0xc4) from [<c0324658>] (kernel_sendmsg+0x48/0x80)
    r9:ce07a998 r8:00004040 r6:cdf0f804 r5:00000000 r4:bf000000
    [<c0324610>] (kernel_sendmsg+0x0/0x80) from [<c0398530>] (xs_send_kvec+0x90/0x98)
    r4:ce004c00 r3:00000001
    [<c03984a0>] (xs_send_kvec+0x0/0x98) from [<c03985c0>] (xs_sendpages+0x88/0x1f0)
    [<c0398538>] (xs_sendpages+0x0/0x1f0) from [<c0398854>] (xs_tcp_send_request+0x58/0x16c)
    [<c03987fc>] (xs_tcp_send_request+0x0/0x16c) from [<c0396cb8>] (xprt_transmit+0xfc/0x1e0)
    [<c0396bbc>] (xprt_transmit+0x0/0x1e0) from [<c03940d8>] (call_transmit+0x1f0/0x264)
    r8:00000000 r7:00000000 r6:cdc280e8 r5:cdc280e8 r4:cd16b880
    [<c0393ee8>] (call_transmit+0x0/0x264) from [<c039af48>] (__rpc_execute+0x90/0x230)
    r6:c0597778 r5:cd16b8b8 r4:cd16b880
    [<c039aeb8>] (__rpc_execute+0x0/0x230) from [<c039b178>] (rpc_execute+0x60/0x68)
    [<c039b118>] (rpc_execute+0x0/0x68) from [<c03951d0>] (rpc_run_task+0x10c/0x118)
    r4:cd16b880 r3:00000002
    [<c03950c4>] (rpc_run_task+0x0/0x118) from [<c0395320>] (rpc_call_sync+0x5c/0x84)
    r6:ce705500 r5:cdc73dc0 r4:00000000 r3:c03f17c4
    [<c03952c4>] (rpc_call_sync+0x0/0x84) from [<c0158968>] (nfs3_rpc_wrapper.clone.10+0x30/0x70)
    r6:ce705500 r5:cdc73dc0 r4:cdc72000
    [<c0158938>] (nfs3_rpc_wrapper.clone.10+0x0/0x70) from [<c0159ad0>] (nfs3_proc_getattr+0x68/0xa4)
    r6:ce07a8a8 r5:ce7bb600 r4:ce7bfcc0 r3:00000000
    [<c0159a68>] (nfs3_proc_getattr+0x0/0xa4) from [<c014bca4>] (__nfs_revalidate_inode+0xb0/0x1bc)
    r6:ce7bb600 r5:ce7bfcc0 r4:ce07a998
    [<c014bbf4>] (__nfs_revalidate_inode+0x0/0x1bc) from [<c014be00>] (nfs_revalidate_mapping+0x50/0xf4)
    r7:00000010 r6:ce07a8a0 r5:ce07aa3c r4:ce07a998
    [<c014bdb0>] (nfs_revalidate_mapping+0x0/0xf4) from [<c01544d4>] (nfs_follow_link+0x20/0x78)
    r6:ce07ed00 r5:ce07a998 r4:cdc73ec8 r3:c01544b4
    [<c01544b4>] (nfs_follow_link+0x0/0x78) from [<c00d5d70>] (link_path_walk+0x35c/0xa48)
    r5:00000000 r4:cdc73ec8
    [<c00d5a14>] (link_path_walk+0x0/0xa48) from [<c00d7200>] (do_filp_open+0xe0/0x4ac)
    [<c00d7120>] (do_filp_open+0x0/0x4ac) from [<c00cb3c0>] (do_sys_open+0x64/0xf4)
    [<c00cb35c>] (do_sys_open+0x0/0xf4) from [<c00cb478>] (sys_open+0x28/0x2c)
    [<c00cb450>] (sys_open+0x0/0x2c) from [<c004b3c0>] (ret_fast_syscall+0x0/0x30)

     

  • 0 in reply to Eric Mao
    TI__Guru 59700 points

    Enable exception reporting by modifying

    /dvr_rdk/mcfw/src_bios6/utils/src/utils_execp_trace.c

    utils_exception_send_notify_to_host

    Uncomment:

    /* System_ipcSendNotify(SYSTEM_HOST_LINK_ID_EXCEPTION_NOTIFY_BASE + curProcId); */

    As I mentioned most likely DDR is unstable on your board causing random A8 and M3 crash. This doesn't look like sw issue.

  • 0 in reply to Badri Narayanan
    Genius 4330 points

    Hi Badri Narayanan

    I uncomment the /* System_ipcSendNotify(SYSTEM_HOST_LINK_ID_EXCEPTION_NOTIFY_BASE + curProcId); */ then rdk generate dump file.

    In order to learn how to use the AnalyzeCrashDumpM3, i add a new capture link cmd to generate a stack error exception . 

    After a send the cmd to captureLink, the console print:

    14501:!!!SLAVE CORE [VPSS-M3] DOWN!!!
    SystemLink_copySlaveCoreExceptionContext:120
    SystemLink_copySlaveCoreExceptionContext:127
    SystemLink_handleSlaveCoreException:149


    14506:!!!SLAVE CORE DOWN!!!.EXCEPTION INFO DUMP

    !!HW EXCEPTION ACTIVE (0/1): [1]

    !!EXCEPTION CORE NAME : [VPSS-M3]

    !!EXCEPTION TASK NAME : [CAPTURE ]

    !!EXCEPTION LOCATION : []

    !!EXCEPTION INFO : [H/W EXCEPTION]
    [m3vpss ] Unhandled Exception:
    [m3vpss ] Exception occurred in ThreadType_Task
    [m3vpss ] handle: 0x3b0b5a18.
    [m3vpss ] stack base: 0x3b14b980.
    [m3vpss ] stack size: 0x8000.
    [m3vpss ] R0 = 0x3b157910 R8 = 0x3a85b0b0
    [m3vpss ] R1 = 0x00000000 R9 = 0x00000000
    [m3vpss ] R2 = 0x12345678 R10 = 0x3a85b0ec
    [m3vpss ] R3 = 0x12345678 R11 = 0x00000000
    [m3vpss ] R4 = 0x00000000 R12 = 0x99e0e20d
    [m3vpss ] R5 = 0x3a85b0b0 SP(R13) = 0x3b153918
    [m3vpss ] R6 = 0x00000000 LR(R14) = 0x0043823d
    [m3vpss ] R7 = 0x00000000 PC(R15) = 0x12345678
    [m3vpss ] PSR = 0x60000000
    [m3vpss ] ICSR = 0x00400803
    [m3vpss ] MMFSR = 0x00
    [m3vpss ] BFSR = 0x01
    [m3vpss ] UFSR = 0x0000
    [m3vpss ] HFSR = 0x40000000
    [m3vpss ] DFSR = 0x00000000
    [m3vpss ] MMAR = 0xe000ed34
    [m3vpss ] BFAR = 0xe000ed38
    [m3vpss ] AFSR = 0x00000000
    [m3vpss ] Terminating Execution...
    [m3vpss ] TaskName = CAPTURE

    !!EXCEPTION CCS CRASH DUMP FORMAT FILE STORED @ ./CCS_CRASH_DUMP_VPSS-M3.txt
    SystemLink_handleSlaveCoreException:154

    Then i use the 7563.CrashDumpAnalyzerM3 you post to analyze the CCS_CRASH_DUMP_VPSS-M3.txt and dvr_rdk_fw_m3vpss_1024M_256M,xem3, but this tools can't work correctly.  the attachment include all files.

    6763.exceptionAna.zip

  • 0 in reply to Eric Mao
    Genius 4330 points

    The kernel crash problem i will test the ddr with DDRStressTest.

     

  • 0 in reply to Eric Mao
    TI__Guru 59700 points

    You are using the .xem3 where symbols are stripped. Use the original xem3 with debug symbols present under

    /dvr_rdk/build/dvr_rdk/bin/ti816x-evm

    There are two CCS_CRASH_DUMP_VPSS-M3.txt.Which one should be used.

  • 0 in reply to Badri Narayanan
    Genius 4330 points

    please use exceptionAna\excep1\CCS_CRASH_DUMP_VPSS-M3.txt.

    the build profile of m3vpss firmwire is release.  

      there is no different between the file under dvr_rdk/build/dvr_rdk/bin/ti816x-evm and the file i attach.

  • 0 in reply to Eric Mao
    TI__Guru 59700 points

    The size of .xem3 with debug symbols will be 14MB. The .xem3 present in the package you shared is 2 MB. You have obviously uploaded the .xem3 with all debug symbols stripped. Upload the correct file.

  • 0 in reply to Badri Narayanan
    Genius 4330 points

    6330.error.zip

    I add a exception in CAPTURE Link.  The console can get correct exception taskname. why this tools can't .