We're seeing a crash of the kernel due to skb_push() failing a check in Linux kernel 3.2. This seems to happen because of some cross-over from d_can packets on can0 interface when using dhclient application.
I've tried to do a workaround in dhclient application by modifying next_iface4() function in common/discover.c of dhcp-4.2.3-P1, basically not to add can0 interface to the list of interfaces. This seemed to reduce number of crashes but hasn't eliminated them.
[ 3605.118530] dhclient[2040]: Bound to *:546
[ 3665.425997] packet_rcv_spkt: header( (null)) < head(cf3d8a00).
[ 3665.445946] packet_rcv_spkt: data (cf3d8a40)
[ 3665.517431] skb_under_panic: text:c03cb6a8 len:-818050480 put:-818050496 head:cf3d8a00 data: (null) tail:0xcf3d8a50 end:0xcf3d8b40 dev:can0
[ 3665.556441] ------------[ cut here ]------------
[ 3665.570671] kernel BUG at net/core/skbuff.c:147!
[ 3665.584899] Internal error: Oops - undefined instruction: 0 [#1]
[ 3665.603417] Modules linked in:
[ 3665.612826] CPU: 0 Not tainted (3.2.0-ts-armv7l #479)
[ 3665.629489] PC is at skb_push+0x74/0x80
[ 3665.641312] LR is at console_unlock+0x16c/0x1f8
[ 3665.655276] pc : [<c031d954>] lr : [<c003a2ec>] psr: 600f0113
[ 3665.655284] sp : cfa09c78 ip : cfa09b88 fp : cfa09ca4
[ 3665.690697] r10: 00000000 r9 : c0d0e6d8 r8 : 00000c00
[ 3665.706802] r7 : ced42800 r6 : ce9b4c00 r5 : cf3d8a50 r4 : c052a93c
[ 3665.726931] r3 : c0d2c254 r2 : c0d2c254 r1 : 00000000 r0 : 00000096
[ 3665.747063] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 3665.769071] Control: 10c5387d Table: 8e9b0019 DAC: 00000015
[ 3665.786785] Process dhclient (pid: 2053, stack limit = 0xcfa082e8)
[ 3665.805840] Stack: (0xcfa09c78 to 0xcfa0a000)
[ 3665.819266] 9c60: cf3d8a00 00000000
[ 3665.844498] 9c80: cf3d8a50 cf3d8b40 ced42800 cfa09ca8 cea1a600 ced42800 cfa09cc4 cfa09ca8
[ 3665.869731] 9ca0: c03cb6a8 c031d8ec ced42800 cea1af00 c0d10db8 c0d0e6d0 cfa09d0c cfa09cc8
[ 3665.894966] 9cc0: c0324ff4 c03cb5d0 00000003 00000020 00730003 cea1af00 20302e30 c0d10dd8
[ 3665.920198] 9ce0: cfa09d0c cea1af00 00000003 00000020 00730003 0000b408 20302e30 cea1af00
[ 3665.945430] 9d00: cfa09d2c cfa09d10 c032525c c0324d08 1f892d5c 0d234ce7 ced42800 ced42800
[ 3665.970661] 9d20: cfa09d74 cfa09d30 c0266864 c0325220 00000020 ced42c24 20302e30 00000000
[ 3665.995893] 9d40: 00000041 cf3d8a40 cfa09dcc c0266494 ced42c24 00000020 0000012c c0d5a780
[ 3666.021123] 9d60: c0d5a780 c0d2d2b0 cfa09dac cfa09d78 c0327428 c02664a0 c0d5a788 000522a0
[ 3666.046356] 9d80: cfa09dc4 00000001 0000000c c0d6c594 c0d6c584 cfa08000 00000003 00000100
[ 3666.071586] 9da0: cfa09dec cfa09db0 c003ed4c c0327390 cfa08000 00000000 cfa09ddc 0000000a
[ 3666.096818] 9dc0: c006e4fc cfa08000 00000034 00000000 cfa09e74 c00141a8 cfa08000 00000000
[ 3666.122049] 9de0: cfa09e04 cfa09df0 c003f190 c003ecb8 c00702f4 c0d3e038 cfa09e24 cfa09e08
[ 3666.147280] 9e00: c0014ee0 c003f114 00000034 fa200000 cfa09e40 ffffffff cfa09e3c cfa09e28
[ 3666.172512] 9e20: c00086c8 c0014eb0 c032489c 600f0013 cfa09e94 cfa09e40 c0013c40 c0008684
[ 3666.197743] 9e40: ce9b4ec0 ce9b4ec0 00000300 00000300 ce9b4c00 00000000 beb4db70 0000011a
[ 3666.222974] 9e60: c00141a8 cfa08000 00000000 cfa09e94 cfa09e98 cfa09e88 c03ca0e0 c032489c
[ 3666.248207] 9e80: 600f0013 ffffffff cfa09eb4 cfa09e98 c03ca0e0 c0324830 c0318d9c 00000001
[ 3666.273437] 9ea0: ce801800 ce9b4c00 cfa09ed4 cfa09eb8 c03ccbd4 c03ca02c cfa09ed4 00000300
[ 3666.298668] 9ec0: ce9b4c00 00000010 cfa09efc cfa09ed8 c03ccc70 c03ccaf0 68746510 00000030
[ 3666.323900] 9ee0: beb4db70 c0317290 ce998940 cf455e40 cfa09fa4 cfa09f00 c03174dc c03ccc24
[ 3666.349130] 9f00: 00000000 00000000 74650011 00003068 00000000 00000000 cfa09f64 cfa09f28
[ 3666.374360] 9f20: c0316348 c00a10e8 cfa09f64 cf8105c0 cf479698 00000000 00000000 c04f3e28
[ 3666.399591] 9f40: 00e9b5a8 0000000a 00000000 00000004 00000119 c00141a8 cfa09f84 cfa09f68
[ 3666.424821] 9f60: c03163ec c009e3f0 cfa09f8c ce998940 cfa5ba00 0000000a cfa09fa4 cfa09f88
[ 3666.450053] 9f80: c03172f8 c03163cc cfa08000 00e9b5a8 00000006 00000004 00000000 cfa09fa8
[ 3666.475284] 9fa0: c0014000 c0317484 00e9b5a8 00000006 00000006 beb4db70 00000010 beb4db7f
[ 3666.500515] 9fc0: 00e9b5a8 00000006 00000004 0000011a 00000000 00000004 00000010 00000001
[ 3666.525746] 9fe0: 0012e8ac beb4db6c 0002a124 4018984c 600f0010 00000006 00000000 00000000
[ 3666.550968] Backtrace:
[ 3666.558521] [<c031d8e0>] (skb_push+0x0/0x80) from [<c03cb6a8>] (packet_rcv_spkt+0xe4/0x1ac)
[ 3666.584281] r5:ced42800 r4:cea1a600
[ 3666.595314] [<c03cb5c4>] (packet_rcv_spkt+0x0/0x1ac) from [<c0324ff4>] (__netif_receive_skb+0x2f8/0x3dc)
[ 3666.624560] r6:c0d0e6d0 r5:c0d10db8 r4:cea1af00 r3:ced42800
[ 3666.642040] [<c0324cfc>] (__netif_receive_skb+0x0/0x3dc) from [<c032525c>] (netif_receive_skb+0x48/0x6c)
[ 3666.671304] [<c0325214>] (netif_receive_skb+0x0/0x6c) from [<c0266864>] (d_can_poll+0x3d0/0x720)
[ 3666.698405] r4:ced42800
[ 3666.706208] [<c0266494>] (d_can_poll+0x0/0x720) from [<c0327428>] (net_rx_action+0xa4/0x188)
[ 3666.732251] [<c0327384>] (net_rx_action+0x0/0x188) from [<c003ed4c>] (__do_softirq+0xa0/0x13c)
[ 3666.758826] [<c003ecac>] (__do_softirq+0x0/0x13c) from [<c003f190>] (irq_exit+0x88/0x90)
[ 3666.783806] [<c003f108>] (irq_exit+0x0/0x90) from [<c0014ee0>] (handle_IRQ+0x3c/0x8c)
[ 3666.807956] r4:c0d3e038 r3:c00702f4
[ 3666.818981] [<c0014ea4>] (handle_IRQ+0x0/0x8c) from [<c00086c8>] (omap3_intc_handle_irq+0x50/0x54)
[ 3666.846618] r6:ffffffff r5:cfa09e40 r4:fa200000 r3:00000034
[ 3666.864095] [<c0008678>] (omap3_intc_handle_irq+0x0/0x54) from [<c0013c40>] (__irq_svc+0x40/0x60)
[ 3666.891464] Exception stack(0xcfa09e40 to 0xcfa09e88)
[ 3666.907039] 9e40: ce9b4ec0 ce9b4ec0 00000300 00000300 ce9b4c00 00000000 beb4db70 0000011a
[ 3666.932271] 9e60: c00141a8 cfa08000 00000000 cfa09e94 cfa09e98 cfa09e88 c03ca0e0 c032489c
[ 3666.957495] 9e80: 600f0013 ffffffff
[ 3666.968232] r5:600f0013 r4:c032489c
[ 3666.979258] [<c0324824>] (__dev_remove_pack+0x0/0x98) from [<c03ca0e0>] (__unregister_prot_hook+0xc0/0xc8)
[ 3667.009052] [<c03ca020>] (__unregister_prot_hook+0x0/0xc8) from [<c03ccbd4>] (packet_do_bind+0xf0/0x134)
[ 3667.038298] r5:ce9b4c00 r4:ce801800
[ 3667.049327] [<c03ccae4>] (packet_do_bind+0x0/0x134) from [<c03ccc70>] (packet_bind_spkt+0x58/0x64)
[ 3667.076963] r5:00000010 r4:ce9b4c00
[ 3667.087988] [<c03ccc18>] (packet_bind_spkt+0x0/0x64) from [<c03174dc>] (sys_bind+0x64/0x8c)
[ 3667.113747] r4:cf455e40
[ 3667.121549] [<c0317478>] (sys_bind+0x0/0x8c) from [<c0014000>] (ret_fast_syscall+0x0/0x30)
[ 3667.147040] r6:00000004 r5:00000006 r4:00e9b5a8
[ 3667.161289] Code: 01a0c004 e59f000c e58dc010 eb037e77 (e7f001f2)
[ 3667.180179] ---[ end trace b93eb1e0b7c40a12 ]---
[ 3667.194430] Kernel panic - not syncing: Fatal exception in interrupt
[ 3667.214036] Backtrace:
[ 3667.221594] [<c00175e4>] (dump_backtrace+0x0/0x10c) from [<c03fcedc>] (dump_stack+0x18/0x1c)
[ 3667.247637] r6:00000000 r5:cfa08000 r4:c0d5bc08 r3:c0d2c254
[ 3667.265138] [<c03fcec4>] (dump_stack+0x0/0x1c) from [<c03fd20c>] (panic+0x64/0x18c)
[ 3667.288775] [<c03fd1a8>] (panic+0x0/0x18c) from [<c00178cc>] (die+0x128/0x264)
[ 3667.311064] r3:00000100 r2:00000000 r1:0000c806 r0:c04e2810
[ 3667.328563] r7:cfa09b2f
[ 3667.336379] [<c00177a4>] (die+0x0/0x264) from [<c0017a28>] (arm_notify_die+0x20/0x58)
[ 3667.360554] [<c0017a08>] (arm_notify_die+0x0/0x58) from [<c00084d8>] (do_undefinstr+0x104/0x110)
[ 3667.387681] [<c00083d4>] (do_undefinstr+0x0/0x110) from [<c0013ca8>] (__und_svc+0x48/0x60)
[ 3667.413189] Exception stack(0xcfa09c30 to 0xcfa09c78)
[ 3667.428776] 9c20: 00000096 00000000 c0d2c254 c0d2c254
[ 3667.454024] 9c40: c052a93c cf3d8a50 ce9b4c00 ced42800 00000c00 c0d0e6d8 00000000 cfa09ca4
[ 3667.479268] 9c60: cfa09b88 cfa09c78 c003a2ec c031d954 600f0113 ffffffff
[ 3667.499678] r8:e7100000 r7:00000000 r6:c0013f14 r5:600f0113 r4:c031d958
[ 3667.520409] [<c031d8e0>] (skb_push+0x0/0x80) from [<c03cb6a8>] (packet_rcv_spkt+0xe4/0x1ac)
[ 3667.546185] r5:ced42800 r4:cea1a600
[ 3667.557228] [<c03cb5c4>] (packet_rcv_spkt+0x0/0x1ac) from [<c0324ff4>] (__netif_receive_skb+0x2f8/0x3dc)
[ 3667.586490] r6:c0d0e6d0 r5:c0d10db8 r4:cea1af00 r3:ced42800
[ 3667.603993] [<c0324cfc>] (__netif_receive_skb+0x0/0x3dc) from [<c032525c>] (netif_receive_skb+0x48/0x6c)
[ 3667.633267] [<c0325214>] (netif_receive_skb+0x0/0x6c) from [<c0266864>] (d_can_poll+0x3d0/0x720)
[ 3667.660383] r4:ced42800
[ 3667.668200] [<c0266494>] (d_can_poll+0x0/0x720) from [<c0327428>] (net_rx_action+0xa4/0x188)
[ 3667.694255] [<c0327384>] (net_rx_action+0x0/0x188) from [<c003ed4c>] (__do_softirq+0xa0/0x13c)
[ 3667.720845] [<c003ecac>] (__do_softirq+0x0/0x13c) from [<c003f190>] (irq_exit+0x88/0x90)
[ 3667.745828] [<c003f108>] (irq_exit+0x0/0x90) from [<c0014ee0>] (handle_IRQ+0x3c/0x8c)
[ 3667.769994] r4:c0d3e038 r3:c00702f4
[ 3667.781035] [<c0014ea4>] (handle_IRQ+0x0/0x8c) from [<c00086c8>] (omap3_intc_handle_irq+0x50/0x54)
[ 3667.808686] r6:ffffffff r5:cfa09e40 r4:fa200000 r3:00000034
[ 3667.826187] [<c0008678>] (omap3_intc_handle_irq+0x0/0x54) from [<c0013c40>] (__irq_svc+0x40/0x60)
[ 3667.853572] Exception stack(0xcfa09e40 to 0xcfa09e88)
[ 3667.869161] 9e40: ce9b4ec0 ce9b4ec0 00000300 00000300 ce9b4c00 00000000 beb4db70 0000011a
[ 3667.894407] 9e60: c00141a8 cfa08000 00000000 cfa09e94 cfa09e98 cfa09e88 c03ca0e0 c032489c
[ 3667.919647] 9e80: 600f0013 ffffffff
[ 3667.930396] r5:600f0013 r4:c032489c
[ 3667.941438] [<c0324824>] (__dev_remove_pack+0x0/0x98) from [<c03ca0e0>] (__unregister_prot_hook+0xc0/0xc8)
[ 3667.971248] [<c03ca020>] (__unregister_prot_hook+0x0/0xc8) from [<c03ccbd4>] (packet_do_bind+0xf0/0x134)
[ 3668.000511] r5:ce9b4c00 r4:ce801800
[ 3668.011567] [<c03ccae4>] (packet_do_bind+0x0/0x134) from [<c03ccc70>] (packet_bind_spkt+0x58/0x64)
[ 3668.039219] r5:00000010 r4:ce9b4c00
[ 3668.050260] [<c03ccc18>] (packet_bind_spkt+0x0/0x64) from [<c03174dc>] (sys_bind+0x64/0x8c)
[ 3668.076033] r4:cf455e40
[ 3668.083848] [<c0317478>] (sys_bind+0x0/0x8c) from [<c0014000>] (ret_fast_syscall+0x0/0x30)
[ 3668.109353] r6:00000004 r5:00000006 r4:00e9b5a8
We haven't had this issue on other platforms, but we also didn't use the socket can interface (they used pcan). Wondering if anyone else has seen this issue or may know of a work-around.
Saw a similar issue on http://comments.gmane.org/gmane.linux.can/6078 but their workaround still didn't work for us. Even though I tried adding the skb_reset_mac_header(), something is setting the mac_header back to null....