This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM3352: Kernel crash with USB 4G modem

Part Number: AM3352


Tool/software: Linux

I use ti-processor-sdk-linux-am335x-evm-05.03.00.07-Linux-x86-Install.bin to install kernel , and add 4g modules (Quectel EC20) on board . According to Quectel 's USB_Driver_User_Guide , I success install driver and I can find ttyUSB0~ttyUSB3 in /dev . I also success use PPP to install ppp0(as follow) . while after about 1 minute the system can't work as  follow, I want to know where I went wrong. thankyou!

root@am335x-evm:~# ifconfig

ppp0 Link encap:Point-to-Point Protocol
inet addr:10.21.56.202 P-t-P:10.64.64.64 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:5 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:62 (62.0 B) TX bytes:349 (349.0 B)

[ 103.097341] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[ 103.105936] pgd = db0f0000
[ 103.108819] [00000000] *pgd=00000000
[ 103.112639] Internal error: Oops: 805 [#1] PREEMPT ARM
[ 103.118105] Modules linked in:
[ 103.121383] CPU: 0 PID: 95 Comm: sd-resolve Tainted: G W 4.14.79-ge669d52447 #91
[ 103.130445] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 103.136917] task: dc630000 task.stack: db0ce000
[ 103.141775] PC is at usb_anchor_urb+0x7c/0x104
[ 103.146508] LR is at irq_work_queue+0xe8/0xfc
[ 103.151139] pc : [<c0718074>] lr : [<c01aafb4>] psr: 600e0093
[ 103.157796] sp : db0cf860 ip : db0cf730 fp : db0cf87c
[ 103.163345] r10: 00000044 r9 : c0e16700 r8 : db0ce000
[ 103.168892] r7 : 00000044 r6 : 200e0093 r5 : dc57b044 r4 : dc5de380
[ 103.175824] r3 : dc5de39c r2 : 00000000 r1 : c0e10aa0 r0 : c0c5cef4
[ 103.182760] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
[ 103.190432] Control: 10c5387d Table: 9b0f0019 DAC: 00000051
[ 103.196537] Process sd-resolve (pid: 95, stack limit = 0xdb0ce210)
[ 103.203097] Stack: (0xdb0cf860 to 0xdb0d0000)
[ 103.207747] f860: 00000000 dc57b020 dc5de380 00000044 db0cf8d4 db0cf880 c0744ec8 c0718004
[ 103.216451] f880: db0cf89c db0cf890 00000044 dc57b000 400e0013 01080020 dc5d1000 dc5da340
[ 103.225147] f8a0: db3900b0 dc57b040 00000040 c0744d2c db36280e 00000040 00000001 c0a24f78
[ 103.233846] f8c0: db390000 00000001 db0cf8ec db0cf8d8 c073ed08 c0744d38 00000044 db36280e
[ 103.242542] f8e0: db0cf93c db0cf8f0 c06f7c70 c073ecd8 db0ce000 c0e3de90 db7ad400 c0e16700
[ 103.251243] f900: 00000001 ffffc021 00000000 00000000 00000000 db390000 db1396c0 db6a0518
[ 103.259935] f920: 00000000 01080020 00400080 c0e03f14 db0cf954 db0cf940 c06f7ea8 c06f78e0
[ 103.268637] f940: db390094 db6a04e0 db0cf9c4 db0cf958 c06f24a0 c06f7e60 c08b112c c08b07e8
[ 103.277335] f960: c016b1a8 c012dff8 dc64f600 c0e0fbd0 0002eabf 00000000 0002eabf dc052438
[ 103.286034] f980: dc052430 db0fee40 c0e0501c c08b0e1c db1396c0 dc00e800 db0cfa0c db6a04e0
[ 103.294736] f9a0: 00000021 00000021 db1396c0 01080020 00400080 c0e03f14 db0cfa14 db0cf9c8
[ 103.303433] f9c0: c06f4370 c06f2430 db1396c0 00000000 c0c5a3b0 c0c5a3cc 00000000 c0c5a37c
[ 103.312131] f9e0: db0cfa2c db0cf9f0 c0870de8 db6a04e0 db1396c0 c0a92330 d71cd600 c0e03b0c
[ 103.320830] fa00: 00000000 c0e03f14 db0cfa2c db0cfa18 c06f65f0 c06f3f8c db1396c0 db6a0000
[ 103.329529] fa20: db0cfa4c db0cfa30 c06f6790 c06f65b4 db1396c0 00000000 db6a0000 d71cd600
[ 103.338230] fa40: db0cfa84 db0cfa50 c08714d8 c06f6658 db0cfa8c c0e16700 c0871420 db2e4f00
[ 103.346928] fa60: db1396c0 d71cd600 db6a0000 00000001 00000000 00000000 db0cfab4 db0cfa88
[ 103.355621] fa80: c0899850 c087145c 00000000 00000010 db1396c0 00000136 00000000 db2e4f00
[ 103.364324] faa0: 00000000 0000003e db0cfb14 db0cfab8 c0871a44 c0899664 db2e4f48 00000001
[ 103.373022] fac0: 0000003e 00000000 000000f8 00000000 fffffff4 00000000 db0cfafc db0cfae8
[ 103.381723] fae0: c08556e4 c0854924 014000c0 db6a0000 db1396c0 db2e4400 00000010 000005dc
[ 103.390424] fb00: 00000011 0000002a db0cfb24 db0cfb18 c0871e40 c0871634 db0cfb34 db0cfb28
[ 103.399124] fb20: c087e2b4 c0871e38 db0cfb64 db0cfb38 c08b4188 c087e2ac dc630504 e17ee6b7
[ 103.407822] fb40: db0cfb6c db1396c0 db1396c0 dc1cd600 c0e55410 000005dc db0cfb9c db0cfb68
[ 103.416522] fb60: c08b5a50 c08b3fb8 c08abcb8 c08abb64 db1396c0 db1396c0 c0e55410 db6a0000
[ 103.425216] fb80: dc1cd600 dc1cd830 00000011 0000002a db0cfbe4 db0cfba0 c08b6578 c08b5914
[ 103.433911] fba0: c08b5bd0 c08b3f38 c08b4dd4 00000000 db0cfbf4 dc1cd600 dc1cd830 c08b56dc
[ 103.442610] fbc0: db0cfeb4 0000002a db1396c0 c0e55410 dc1cd600 0000003b db0cfc04 db0cfbe8
[ 103.451311] fbe0: c08b5cb0 c08b64e4 db1396c0 c0e55410 dc1cd600 0000003b db0cfc1c db0cfc08
[ 103.460005] fc00: c08b6e60 c08b5c7c db1396c0 db362824 db0cfc44 db0cfc20 c08e1504 c08b6e50
[ 103.468704] fc20: dc1cd600 db1396c0 00000022 e17ee6b7 00000002 dc1cd830 db0cfd4c db0cfc48
[ 103.477403] fc40: c08e3c70 c08e1400 0000002a 00000008 db0cfc84 db0cfc80 00044040 c085cbc0
[ 103.486099] fc60: 4a68400a 00000000 00003500 c08b56dc 00000000 00000000 00000001 00000011
[ 103.494798] fc80: 00000000 db396f48 db0c0000 db0cfc98 c04098ec e17ee6b7 00000000 00000000
[ 103.503496] fca0: ffff0000 c04098c0 ffffffff db0cfd0c db0cfda8 00000008 ffffe000 db0cfda8
[ 103.512197] fcc0: 00000008 00000000 00000001 c0221584 00000008 db0cfebc 00000000 db0cfd84
[ 103.520896] fce0: 00000001 00000001 00004000 db9ec600 db0cfd34 db0cfd00 c03eb45c c0221504
[ 103.529591] fd00: db0cfda8 db0cfd0c 00000000 db0cfda8 db8abf68 db0cfebc 00000000 db0cfd84
[ 103.538292] fd20: 00000051 dc1cd600 00044000 00000000 db9ec600 00000002 db0cfd88 00000000
[ 103.546991] fd40: db0cfd6c db0cfd50 c08edbe0 c08e37d0 00000001 00000000 00000000 db0cfeb4
[ 103.555688] fd60: db0cfd7c db0cfd70 c0852cdc c08edbb0 db0cfe94 db0cfd80 c08530cc c0852ccc
[ 103.564383] fd80: c0e56e70 00000000 00000000 dc1b6a1c 00000000 c0ac2f90 db0cfe20 00000001
[ 103.573079] fda0: ffffffff dc1cd830 b6acc1f0 00000022 dc1cd830 db6a0000 db333000 db0cfe20
[ 103.581774] fdc0: db0cfe1c db0cfdd0 c08aef64 c08ac524 00000000 dc1cd830 db6a0000 dc7c6780
[ 103.590470] fde0: 00000000 00000000 db333000 00000000 c08aee68 dc1cd830 c0e55410 00000000
[ 103.599172] fe00: db0cff08 dc1cd830 dc7c6780 c0e55410 db0cff08 dc1cd830 00000000 dc7c6780
[ 103.607867] fe20: db0cfe54 db0cfe30 db139240 0000000d db0cfe5c db0cfe40 c0240e94 c01734cc
[ 103.616566] fe40: db0cfeb0 db0cfeac 00000002 00000176 db0cfe6c db0cfe60 c0240f08 c0240e04
[ 103.625265] fe60: db0cfe7c db0cfe70 c0240f74 00000000 b6acc168 00044000 db0ce000 00000002
[ 103.633970] fe80: 00004000 db9ec600 db0cff94 db0cfe98 c08540c0 c0853024 db0cfee4 00000080
[ 103.642667] fea0: c0856bd0 00000001 dc1cd600 00000001 00000000 00000000 00000000 00000001
[ 103.651365] fec0: 00000000 00000000 db0cfdb0 00000000 db0cfefc 00000000 00000000 00044040
[ 103.660061] fee0: 00000000 c08edb18 db9ec600 c08edb18 db0cffa4 db0cff00 c0853a64 c0222184
[ 103.668762] ff00: 00000000 00000001 35000002 e17ee6b7 db0cff54 db0cff20 c0180990 c0119804
[ 103.677461] ff20: ffffffff 00000000 2f9a4780 00c5e32e db0cff80 00000000 00000000 0000004e
[ 103.686153] ff40: c0107de4 00000001 b6acc134 00000000 00000000 c0107de4 db0ce000 000000a8
[ 103.694852] ff60: db0cffa4 ffffffff c0236ca4 0000000d 00004000 00000002 00000176 c0107de4
[ 103.703553] ff80: db0ce000 00000176 db0cffa4 db0cff98 c085418c c0854048 00000000 db0cffa8
[ 103.712251] ffa0: c0107be0 c0854188 0000000d 00004000 0000000d b6acc168 00000002 00004000
[ 103.720946] ffc0: 0000000d 00004000 00000002 00000176 00000000 00000000 32243650 5ca804e2
[ 103.729644] ffe0: 00000176 b6acc068 b6cc6319 b6c487e6 800e0030 0000000d 00000000 00000000
[ 103.738321] Backtrace:
[ 103.740974] [<c0717ff8>] (usb_anchor_urb) from [<c0744ec8>] (usb_wwan_write+0x19c/0x2a4)
[ 103.749579] r7:00000044 r6:dc5de380 r5:dc57b020 r4:00000000
[ 103.755603] [<c0744d2c>] (usb_wwan_write) from [<c073ed08>] (serial_write+0x3c/0x68)
[ 103.763839] r10:00000001 r9:db390000 r8:c0a24f78 r7:00000001 r6:00000040 r5:db36280e
[ 103.772146] r4:c0744d2c
[ 103.774873] [<c073eccc>] (serial_write) from [<c06f7c70>] (ppp_async_push+0x39c/0x580)
[ 103.783275] r5:db36280e r4:00000044
[ 103.787091] [<c06f78d4>] (ppp_async_push) from [<c06f7ea8>] (ppp_async_send+0x54/0x5c)
[ 103.795514] r10:c0e03f14 r9:00400080 r8:01080020 r7:00000000 r6:db6a0518 r5:db1396c0
[ 103.803829] r4:db390000
[ 103.806542] [<c06f7e54>] (ppp_async_send) from [<c06f24a0>] (ppp_push+0x7c/0x69c)
[ 103.814493] r5:db6a04e0 r4:db390094
[ 103.818310] [<c06f2424>] (ppp_push) from [<c06f4370>] (__ppp_xmit_process+0x3f0/0x554)
[ 103.826730] r10:c0e03f14 r9:00400080 r8:01080020 r7:db1396c0 r6:00000021 r5:00000021
[ 103.835039] r4:db6a04e0
[ 103.837749] [<c06f3f80>] (__ppp_xmit_process) from [<c06f65f0>] (ppp_xmit_process+0x48/0xa4)
[ 103.846727] r10:c0e03f14 r9:00000000 r8:c0e03b0c r7:d71cd600 r6:c0a92330 r5:db1396c0
[ 103.855037] r4:db6a04e0
[ 103.857751] [<c06f65a8>] (ppp_xmit_process) from [<c06f6790>] (ppp_start_xmit+0x144/0x1ac)
[ 103.866529] r5:db6a0000 r4:db1396c0
[ 103.870365] [<c06f664c>] (ppp_start_xmit) from [<c08714d8>] (dev_hard_start_xmit+0x88/0x114)
[ 103.879330] r7:d71cd600 r6:db6a0000 r5:00000000 r4:db1396c0
[ 103.885375] [<c0871450>] (dev_hard_start_xmit) from [<c0899850>] (sch_direct_xmit+0x1f8/0x228)
[ 103.894533] r10:00000000 r9:00000000 r8:00000001 r7:db6a0000 r6:d71cd600 r5:db1396c0
[ 103.902848] r4:db2e4f00
[ 103.905563] [<c0899658>] (sch_direct_xmit) from [<c0871a44>] (__dev_queue_xmit+0x41c/0x804)
[ 103.914438] r8:0000003e r7:00000000 r6:db2e4f00 r5:00000000 r4:00000136
[ 103.921576] [<c0871628>] (__dev_queue_xmit) from [<c0871e40>] (dev_queue_xmit+0x14/0x18)
[ 103.930180] r10:0000002a r9:00000011 r8:000005dc r7:00000010 r6:db2e4400 r5:db1396c0
[ 103.938490] r4:db6a0000
[ 103.941216] [<c0871e2c>] (dev_queue_xmit) from [<c087e2b4>] (neigh_direct_output+0x14/0x18)
[ 103.950106] [<c087e2a0>] (neigh_direct_output) from [<c08b4188>] (ip_finish_output2+0x1dc/0x378)
[ 103.959450] [<c08b3fac>] (ip_finish_output2) from [<c08b5a50>] (ip_finish_output+0x148/0x200)
[ 103.968507] r8:000005dc r7:c0e55410 r6:dc1cd600 r5:db1396c0 r4:db1396c0
[ 103.975636] [<c08b5908>] (ip_finish_output) from [<c08b6578>] (ip_output+0xa0/0x120)
[ 103.983868] r10:0000002a r9:00000011 r8:dc1cd830 r7:dc1cd600 r6:db6a0000 r5:c0e55410
[ 103.992180] r4:db1396c0
[ 103.994887] [<c08b64d8>] (ip_output) from [<c08b5cb0>] (ip_local_out+0x40/0x44)
[ 104.002659] r7:0000003b r6:dc1cd600 r5:c0e55410 r4:db1396c0
[ 104.008678] [<c08b5c70>] (ip_local_out) from [<c08b6e60>] (ip_send_skb+0x1c/0x7c)
[ 104.016632] r7:0000003b r6:dc1cd600 r5:c0e55410 r4:db1396c0
[ 104.022656] [<c08b6e44>] (ip_send_skb) from [<c08e1504>] (udp_send_skb+0x110/0x24c)
[ 104.030792] r5:db362824 r4:db1396c0
[ 104.034607] [<c08e13f4>] (udp_send_skb) from [<c08e3c70>] (udp_sendmsg+0x4ac/0x844)
[ 104.042750] r9:dc1cd830 r8:00000002 r7:e17ee6b7 r6:00000022 r5:db1396c0 r4:dc1cd600
[ 104.050987] [<c08e37c4>] (udp_sendmsg) from [<c08edbe0>] (inet_sendmsg+0x3c/0x70)
[ 104.058950] r10:00000000 r9:db0cfd88 r8:00000002 r7:db9ec600 r6:00000000 r5:00044000
[ 104.067259] r4:dc1cd600
[ 104.069986] [<c08edba4>] (inet_sendmsg) from [<c0852cdc>] (sock_sendmsg+0x1c/0x2c)
[ 104.078021] r4:db0cfeb4
[ 104.080735] [<c0852cc0>] (sock_sendmsg) from [<c08530cc>] (___sys_sendmsg+0xb4/0x20c)
[ 104.089070] [<c0853018>] (___sys_sendmsg) from [<c08540c0>] (__sys_sendmmsg+0x84/0x140)
[ 104.097583] r10:db9ec600 r9:00004000 r8:00000002 r7:db0ce000 r6:00044000 r5:b6acc168
[ 104.105898] r4:00000000
[ 104.108612] [<c085403c>] (__sys_sendmmsg) from [<c085418c>] (SyS_sendmmsg+0x10/0x14)
[ 104.116847] r10:00000176 r9:db0ce000 r8:c0107de4 r7:00000176 r6:00000002 r5:00004000
[ 104.125157] r4:0000000d
[ 104.127884] [<c085417c>] (SyS_sendmmsg) from [<c0107be0>] (ret_fast_syscall+0x0/0x54)
[ 104.136213] Code: e5853004 e34c00c5 e584501c e5842020 (e5823000)
[ 104.142706] ---[ end trace 60393f08805d93d6 ]---
[ 104.147616] Kernel panic - not syncing: Fatal exception in interrupt
[ 104.154379] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
[ 108.110491] musb-hdrc musb-hdrc.1: VBUS_ERROR in a_suspend (88, <AValid), retry #0, port1 00000507
[ 108.120069] musb-hdrc musb-hdrc.0: VBUS_ERROR in a_suspend (80, <SessEnd), retry #0, port1 00000507

  • Hello,

    Thanks for your post. We'll need your further help to isolate whether this is the modem's driver or some interaction with the kernel. This series should provide good information to debug this further and see where the problem is arising:

    https://training.ti.com/debugging-embedded-linux-systems-training-series

    Please take a look and let us know what you find. Thank you.

  • Thanks for your reply, but I am sorry for I can't watch the video as I am in China.

    Could you give me some ways to watch it?

    According to related blog, I use arm-linux-gnueabihf-gdb to locate the problem, and I find the error is here, but I don't know how to modify it:

  • Hi,

    The following kernel backtrace in the log you provided shows the issue is likely between kernel wwan driver and the vendor's modem driver. it is out of the support scope on this forum. Please ask the modem vendor or the Linux community for support. I also recommend to test the latest community kernel, for example, v5.1.12, to see if the issue is solved.

    user6041157 said:
    [ 103.738321] Backtrace:
    [ 103.740974] [<c0717ff8>] (usb_anchor_urb) from [<c0744ec8>] (usb_wwan_write+0x19c/0x2a4)
    [ 103.749579] r7:00000044 r6:dc5de380 r5:dc57b020 r4:00000000
    [ 103.755603] [<c0744d2c>] (usb_wwan_write) from [<c073ed08>] (serial_write+0x3c/0x68)
    [ 103.763839] r10:00000001 r9:db390000 r8:c0a24f78 r7:00000001 r6:00000040 r5:db36280e
    [ 103.772146] r4:c0744d2c
    [ 103.774873] [<c073eccc>] (serial_write) from [<c06f7c70>] (ppp_async_push+0x39c/0x580)
    [ 103.783275] r5:db36280e r4:00000044

  • Thanks for your reply, I have made some new progress. I step by step test instead of running the script and I found that the above problem occurred in /etc/resolv.conf. The 4g module and PPP can work normally. When I add intranet IP (10.12.13.***) the kernel no change but no internet access, but when I add extranet IP (211.136.17.107).which the 4g module automatic generated. the kernel will crash. such as follow, I think may some problem about DNS servers. could you give me some suggestion?

  • Hi,

    Sorry, I don't any additional comments other than those in my previous response.