• State TI Thinks Resolved
  • Locked Locked
  • Replies 7 replies
  • Answers 1 answer
  • Subscribers 101 subscribers
  • Views 1185 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM3505: USB modem issue

Jemish Patel37
Intellectual 490 points
Part Number: AM3505

Tool/software: Linux

Hi All,

Processor part number : AM3505

Kernel : 2.6.37

  • We are using AM3505 based product from long time.  We are using different types of USB modems in our product for streaming purpose.
  • From last couple months we are experiencing strange issue. While streaming continuously from  the  device it get following crash. 
[ 8604.721832] NETDEV WATCHDOG: wwan0 (qmi_wwan): transmit queue 0 timed out
[ 8604.730346] Modules linked in: act_mirred sch_ingress cls_u32 sch_prio xt_mark g_mass_storage ifb xt_conntrack wl12xx_sdio wl12xx mac80211 cfg80211 compat qmi_wwan cdc_wdm tun rndis_host cdc_ether dieid(P) cdc_acm sierra qcserial option usb_wwan usbserial ppp_async ppp_synctty ppp_generic slhc nfsd exportfs [last unloaded: g_mass_storage]
[ 8604.809692] [<c007eecc>] (unwind_backtrace+0x0/0xec) from [<c009cf00>] (warn_slowpath_common+0x4c/0x64)
[ 8604.820922] [<c009cf00>] (warn_slowpath_common+0x4c/0x64) from [<c009cf98>] (warn_slowpath_fmt+0x2c/0x3c)
[ 8604.832336] [<c009cf98>] (warn_slowpath_fmt+0x2c/0x3c) from [<c037eb08>] (dev_watchdog+0x144/0x22c)
[ 8604.843200] [<c037eb08>] (dev_watchdog+0x144/0x22c) from [<c00a6194>] (run_timer_softirq+0x13c/0x1d8)
[ 8604.854248] [<c00a6194>] (run_timer_softirq+0x13c/0x1d8) from [<c00a1cd4>] (__do_softirq+0x74/0xfc)
[ 8604.865112] [<c00a1cd4>] (__do_softirq+0x74/0xfc) from [<c00a1d9c>] (irq_exit+0x40/0x8c)
[ 8604.875000] [<c00a1d9c>] (irq_exit+0x40/0x8c) from [<c006f078>] (asm_do_IRQ+0x78/0x98)
[ 8604.884674] [<c006f078>] (asm_do_IRQ+0x78/0x98) from [<c043c774>] (__irq_svc+0x34/0x80)
[ 8604.894439] Exception stack(0xc0591f80 to 0xc0591fc8)
[ 8604.901123] 1f80: 00000000 40000013 40000013 00003505 c0590000 c05d0900 c002e36c c059401c
[ 8604.911071] 1fa0: 8002c954 411fc087 0000001f 00000000 c05a3b60 c0591fc8 c0087128 c0087738
[ 8604.921051] 1fc0: 60000013 ffffffff
[ 8604.926116] [<c043c774>] (__irq_svc+0x34/0x80) from [<c0087738>] (omap3_pm_idle+0x48/0x4c)
[ 8604.936157] [<c0087738>] (omap3_pm_idle+0x48/0x4c) from [<c007abf0>] (cpu_idle+0x48/0x88)
[ 8604.946105] [<c007abf0>] (cpu_idle+0x48/0x88) from [<c0008914>] (start_kernel+0x24c/0x2a0)
[ 8604.956146] [<c0008914>] (start_kernel+0x24c/0x2a0) from [<80008034>] (0x80008034)
[ 8604.965454] ---[ end trace 3d7e4faaa8ec4b9d ]---
Sometimes... Following crash but method to reproduce is same.

2019-06-03T16:24:33.452203+00:00 corecard kernel: [ 9453.719726] NETDEV WATCHDOG: eth1 (cdc_ether): transmit queue 0 timed out

2019-06-03T16:24:33.452234+00:00 corecard kernel: [ 9453.726867] Modules linked in: g_mass_storage act_mirred sch_ingress cls_u32 sch_prio xt_mark ifb xt_conntrack wl12xx_sdio wl12xx mac80211 cfg80211 compat qmi_wwan cdc_wdm tun rndis_host cdc_ether dieid(P) cdc_acm sierra qcserial option usb_wwan usbserial ppp_async ppp_synctty ppp_generic slhc nfsd exportfs [last unloaded: g_mass_storage]

2019-06-03T16:24:33.452264+00:00 corecard kernel: [ 9453.758575] [<c007eecc>] (unwind_backtrace+0x0/0xec) from [<c009ced4>] (warn_slowpath_common+0x4c/0x64)

2019-06-03T16:24:33.452264+00:00 corecard kernel: [ 9453.768463] [<c009ced4>] (warn_slowpath_common+0x4c/0x64) from [<c009cf6c>] (warn_slowpath_fmt+0x2c/0x3c)

2019-06-03T16:24:33.452295+00:00 corecard kernel: [ 9453.778533] [<c009cf6c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c037e728>] (dev_watchdog+0x144/0x22c)

2019-06-03T16:24:33.452325+00:00 corecard kernel: [ 9453.788085] [<c037e728>] (dev_watchdog+0x144/0x22c) from [<c00a6168>] (run_timer_softirq+0x13c/0x1d8)

2019-06-03T16:24:33.452325+00:00 corecard kernel: [ 9453.797790] [<c00a6168>] (run_timer_softirq+0x13c/0x1d8) from [<c00a1ca8>] (__do_softirq+0x74/0xfc)

2019-06-03T16:24:33.452356+00:00 corecard kernel: [ 9453.807312] [<c00a1ca8>] (__do_softirq+0x74/0xfc) from [<c00a1d70>] (irq_exit+0x40/0x8c)

2019-06-03T16:24:33.452386+00:00 corecard kernel: [ 9453.815826] [<c00a1d70>] (irq_exit+0x40/0x8c) from [<c006f078>] (asm_do_IRQ+0x78/0x98)

2019-06-03T16:24:33.452417+00:00 corecard kernel: [ 9453.824188] [<c006f078>] (asm_do_IRQ+0x78/0x98) from [<c043c034>] (__irq_svc+0x34/0x80)

2019-06-03T16:24:33.452417+00:00 corecard kernel: [ 9453.832580] Exception stack(0xcefa5ac0 to 0xcefa5b08)

2019-06-03T16:24:33.452447+00:00 corecard kernel: [ 9453.837890] 5ac0: 00002bd7 ffffffff 000008e8 00000000 0003f420 cf558480 000000bc cab02a80

2019-06-03T16:24:33.452447+00:00 corecard kernel: [ 9453.846496] 5ae0: cf4f3210 00000000 cf4f3010 00000000 0003f420 cefa5b08 c028fd60 c0202700

2019-06-03T16:24:33.452478+00:00 corecard kernel: [ 9453.855102] 5b00: 20000113 ffffffff

2019-06-03T16:24:33.452508+00:00 corecard kernel: [ 9453.858795] [<c043c034>] (__irq_svc+0x34/0x80) from [<c0202700>] (__delay+0x0/0xc)

2019-06-03T16:24:33.452508+00:00 corecard kernel: [ 9453.866760] ---[ end trace 63ce703603e351f1 ]---

  • Only solution is to reboot board to recover and again put device on streaming.
  • This issue is not very frequently reproducible. It takes sometimes 2 to 3 hours sometimes complete day.
  • One observation from our side is that we use Audio playback application while streaming this issue took less time to reproduce.

Any suggestion or pointers to proceed for this issue will be appreciated.

Thanks,

Jemish

  • 0
    TI__Guru*** 125266 points

    Hi Jemish,

    The kernell v2.6.37 is quite old, and no longer supported on the forums. The log indicates that the watchdog detects a qmi_wwan driver failure, so you might want to debug the new modem driver to see why the transmit queue timeout happens.

  • 0 in reply to Bin Liu
    Intellectual 490 points

    Hi Bin,

    Thanks for you quick response!

    I know this very old kernel. But this product is market from very long time so can't change kernel right now.

    We are using external qmi_wwan driver only and we tried to took driver from kernel 4.9.11 also. But issue still happens.

    The good observation is that when we do Audio playback (receive audio stream from network-->decode-->playback ) simultaneously with streaming ( Audio + Video ) then this issue reproduced easily.

    Is it possible to recover from situation?

    Thanks,

    Jemish

  • 0 in reply to Jemish Patel37
    TI__Guru*** 125266 points

    Jemish,

    Jemish Patel37 said:
    We are using external qmi_wwan driver only and we tried to took driver from kernel 4.9.11 also. But issue still happens.

    No, you cannot simply take a single driver from one kernel and drop it in another kernel. It likely won't work.

    Jemish Patel37 said:
    Is it possible to recover from situation?

    You'd better get support from elsewhere, maybe from the modem vendor. None of the modem driver or qmi_wwan driver is developed by TI, it is out of the support scope on this forum.

  • 0 in reply to Bin Liu
    Intellectual 490 points

    Hi Bin,

    Thanks for your suggestion we surely contact modem vendor for this issue.

    But from SOC point of view due you have any direction from observation I mentioned

    "The good observation is that when we do Audio playback (receive audio stream from network-->decode-->playback ) simultaneously with streaming ( Audio + Video ) then this issue reproduced easily."

    can it be because of SOC bandwidth limitation? If there is such limitation at least it should not crash we should have mechanism to recover it. I am trying to understand relation between these two.

    Any pointers will be appreciated!!

    Thanks,

    Jemish

     

  • 0 in reply to Jemish Patel37
    TI__Guru*** 125266 points

    Jemish,

    Jemish Patel37 said:
    can it be because of SOC bandwidth limitation? If there is such limitation at least it should not crash

    That is true, regardless kernel shouldn't crash. It seems to be a problem between wwan driver and the modem driver which you would have to debug to find the root cause. Further support from TI is out of the support scope on this forum.

  • 0 in reply to Bin Liu
    Intellectual 490 points

    Hi Bin,

    Thanks for your support !  We will debug it. You can close this thread.

    Thanks,

    Jemish

  • 0 in reply to Jemish Patel37
    TI__Guru*** 125266 points

    Jemish, alright, I am closing it for now.