This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Kernel freeze when disconecting an external modem

Hi,

We are using kernel 3.12 with AM335 with an external CDC modem. When we disconnect the external modem and the modem serial port it's open, the system freeze.

It's quite easy to reproduce. When we connect the modem, we receive:

[   42.805381] usb 1-1.2: new high-speed USB device number 3 using musb-hdrc
[   42.917657] usb 1-1.2: New USB device found, idVendor=12d1, idProduct=1573
[   42.924989] usb 1-1.2: New USB device strings: Mfr=5, Product=4, SerialNumber=0
[   42.932825] usb 1-1.2: Product: HUAWEI Mobile
[   42.937497] usb 1-1.2: Manufacturer: Huawei Technologies
[   42.960929] cdc_ether 1-1.2:2.0 eth2: register 'cdc_ether' at usb-musb-hdrc.0.auto-1.2, CDC Ethernet Device, 00:a0:c6:41:96:70
[   42.998747] option 1-1.2:2.2: GSM modem (1-port) converter detected
[   43.057934] usb 1-1.2: GSM modem (1-port) converter now attached to ttyUSB0
[   43.072686] option 1-1.2:2.3: GSM modem (1-port) converter detected
[   43.107534] usb 1-1.2: GSM modem (1-port) converter now attached to ttyUSB1
[   43.125200] option 1-1.2:2.4: GSM modem (1-port) converter detected
[   43.151597] usb 1-1.2: GSM modem (1-port) converter now attached to ttyUSB2
[   43.161256] option 1-1.2:2.5: GSM modem (1-port) converter detected
[   43.187725] usb 1-1.2: GSM modem (1-port) converter now attached to ttyUSB3
[   43.197438] option 1-1.2:2.6: GSM modem (1-port) converter detected
[   43.216046] usb 1-1.2: GSM modem (1-port) converter now attached to ttyUSB4

And when we disconnect it:

[   60.215538] option1 ttyUSB0: option_instat_callback: error -71
[   60.222198] option1 ttyUSB0: option_instat_callback: error -71
[   60.228844] option1 ttyUSB0: option_instat_callback: error -71
[   60.235471] option1 ttyUSB0: option_instat_callback: error -71
[   60.242072] option1 ttyUSB0: option_instat_callback: error -71
[   60.248370] usb 1-1.2: USB disconnect, device number 3
[   60.253880] option1 ttyUSB0: option_instat_callback: error -71
[   60.260530] cdc_ether 1-1.2:2.0 eth2: unregister 'cdc_ether' usb-musb-hdrc.0.auto-1.2, CDC Ethernet Device
[   60.299235] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
[   60.310243] option 1-1.2:2.2: device disconnected
[   60.331838] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
[   60.347580] option 1-1.2:2.3: device disconnected
[   60.370205] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
[   60.397797] option 1-1.2:2.4: device disconnected
[   60.410902] option1 ttyUSB3: GSM modem (1-port) converter now disconnected from ttyUSB3
[   60.436020] option 1-1.2:2.5: device disconnected
[   60.455055] option1 ttyUSB4: GSM modem (1-port) converter now disconnected from ttyUSB4
[   60.466782] option 1-1.2:2.6: device disconnected

But if the /dev/ttyUSB0 it's open, for example:

microcom -s 115200 /dev/ttyUSB0

When we disconnect the modem, the system freeze. After ten minutes it restarts because we enabled the watchdog.

We tried to enable the CONFIG_LOCKUP_DETECTOR and CONFIG_DETECT_HUNG_TASK, but we didn't receive anything.


What's causing this issue? How we can debug it?

  • I tried the same thing but using the kernel 3.2, and it works without any problem.

  • Hi,

    We applied all the patches you suggested [1] in order to fix another problem related with the CDC and DMA [2], but it doesn't fix the issue. Do you have any other suggestion?

    Regards


    [1] http://processors.wiki.ti.com/index.php/Sitara_Linux_SDK_MUSB_Issues

    [2] http://e2e.ti.com/support/embedded/linux/f/354/t/357716.aspx?pi196120=2

  • Hi,

    I tried the kernel 3.12.33 and the issue it's fixed, but not in the 3.12.30.

    When it's planed to support this last release?

    Regards

  • Hi again,


    I was wrong, the issue it's only present in TI kernel, not in kernel vanilla.

    The issue can partially fixed applying the next patch:

    diff --git a/drivers/usb/musb/musb_host.c b/drivers/usb/musb/musb_host.c
    index 5c48d4d..2a81937 100644
    --- a/drivers/usb/musb/musb_host.c
    +++ b/drivers/usb/musb/musb_host.c
    @@ -2608,7 +2608,7 @@ static const struct hc_driver musb_hc_driver = {
            .description            = "musb-hcd",
            .product_desc           = "MUSB HDRC host driver",
            .hcd_priv_size          = sizeof(struct musb *),
    -       .flags                  = HCD_USB2 | HCD_MEMORY | HCD_BH,
    +       .flags                  = HCD_USB2 | HCD_MEMORY,
     
            /* not using irq handler or reset hooks from usbcore, since
             * those must be shared with peripheral code for OTG configs

    But this patch only fixes partially the issue. Some times when I remove the USB, I get:

    [   43.786387] option1 ttyUSB0: option_instat_callback: error -71
    [   43.792884] option1 ttyUSB0: option_instat_callback: error -71
    [   43.799404] option1 ttyUSB0: option_instat_callback: error -71
    [   43.806018] option1 ttyUSB0: option_instat_callback: error -71
    [   43.812641] option1 ttyUSB0: option_instat_callback: error -71
    [   43.819130] option1 ttyUSB0: option_instat_callback: error -71
    [   43.825636] option1 ttyUSB0: option_instat_callback: error -71
    [   43.832132] option1 ttyUSB0: option_instat_callback: error -71
    [   43.838631] option1 ttyUSB0: option_instat_callback: error -71
    [   43.845134] option1 ttyUSB0: option_instat_callback: error -71
    [   43.851645] option1 ttyUSB0: option_instat_callback: error -71
    [   43.858018] option1 ttyUSB0: usb_wwan_indat_callback: resubmit read urb failed. (-19)
    [   43.866422] option1 ttyUSB0: usb_wwan_indat_callback: resubmit read urb failed. (-19)
    [   43.874766] option1 ttyUSB0: option_instat_callback: error -71
    [   43.881046] option1 ttyUSB0: usb_wwan_indat_callback: resubmit read urb failed. (-19)
    [   43.889408] option1 ttyUSB0: usb_wwan_indat_callback: resubmit read urb failed. (-19)
    [   43.897785] usb 1-1.2: USB disconnect, device number 3
    [   43.909204] cdc_ether 1-1.2:2.0 eth2: unregister 'cdc_ether' usb-musb-hdrc.0.auto-1.2, CDC Ethernet Device
    [   43.943544] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
    [   43.961786] option 1-1.2:2.2: device disconnected
    [   43.982132] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
    [   44.003499] option 1-1.2:2.3: device disconnected
    [   44.025621] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
    [   44.047196] option 1-1.2:2.4: device disconnected
    [   44.065840] option1 ttyUSB3: GSM modem (1-port) converter now disconnected from ttyUSB3
    [   44.091614] option 1-1.2:2.5: device disconnected
    [   44.110980] option1 ttyUSB4: GSM modem (1-port) converter now disconnected from ttyUSB4

    But sometimes (about 50% of the times) the kernel enters in a endless loop and I get:

    [   95.903182] option1 ttyUSB0: option_instat_callback: error -71
    [   95.909518] option1 ttyUSB0: option_instat_callback: error -71
    [   95.915881] option1 ttyUSB0: option_instat_callback: error -71
    [   95.922176] option1 ttyUSB0: option_instat_callback: error -71
    [   95.928547] option1 ttyUSB0: option_instat_callback: error -71
    ...

    If I use the vanilla kernel 3.12.33 I never get stuck in the endless loop, but I don't know which part solves the problem.

    Regards

    diff --git a/drivers/usb/musb/musb_host.c b/drivers/usb/musb/musb_host.c
    index 5c48d4d..2a81937 100644
    --- a/drivers/usb/musb/musb_host.c
    +++ b/drivers/usb/musb/musb_host.c
    @@ -2608,7 +2608,7 @@ static const struct hc_driver musb_hc_driver = {
            .description            = "musb-hcd",
            .product_desc           = "MUSB HDRC host driver",
            .hcd_priv_size          = sizeof(struct musb *),
    -       .flags                  = HCD_USB2 | HCD_MEMORY | HCD_BH,
    +       .flags                  = HCD_USB2 | HCD_MEMORY,
     
            /* not using irq handler or reset hooks from usbcore, since
             * those must be shared with peripheral code for OTG configs
  • Update: this issue is closed by moving to TI 3.14 kernel (tag: ti2014.10.01).

  • The issue isn't solved by moving to TI 3.14.

    Seems this issue happens if there is a HUB USB. And the kernel works again if you disconnect the HUB USB.

  • Joan,

    Sorry for the confusion. I recently had another report from a customer which has exact the same issue as yours. Then it turned out the issue disappeared after migrated to TI 3.14 kernel. I thought that report and this thread are from the same customer, so I updated this thread with the closure. I did confirm with the customer if this thread is from them, but there was miscommunication which made me think both are the same case.

    Anyway, my apologies. Can you please try the followings if you haven't done so?

    - Test on TI EVMs other than your board? Since the issue only exists with a hub, I am wondering if it is signal integrity related, since a hub with an extra usb cable changes the bus electrical.

    - Test on your board with USB CPPI DMA disabled.

  • Please first try with different usb hubs, preferebly with vender's hub EVMs, I am wondering if the hub you use has problem which does not detect the modem removal event correctly. The log shows no disconnect message and the modem keeps re-trying transfers.
  • Hi Bin Liu,

    We don't have an EVM, we are working with an embedded device from Variscite, so we can't test it. Also the USB hub it's integrated with the embedded device, so we can't detach it. The USB hub is:

    [    2.918838] usb 1-1: new high-speed USB device number 2 using musb-hdrc
    [    3.089452] usb 1-1: New USB device found, idVendor=0424, idProduct=2514
    [    3.096530] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
    [    3.122631] hub 1-1:1.0: USB hub found
    [    3.137841] hub 1-1:1.0: 4 ports detected

    Using the USB 2, the modem works perfectly, until we attach an USB hub, then we can reproduce the same issue.

    Regardless of this thinks, Variscite have sent a modem to TI, and TI reproduced the issue, and now they are trying to solve the issue.

  • Hi,

    the USB (SMSC/Microchip USB2514) hub you are using has many configuration options, including an option called "Downstream non-removable ports".

    If the non-removable option is selected for downstream ports, the disconnect will be filtered by the hub causing the problem.

    The non-removable option can be set by strap option or by register access to the hub (see USB2514 datasheet). I would suggest to check whether this option is enabled unintentionally

    tom

  • I don't think it's related to this option because:

    - Why it works using kernel 3.2?
    - Why it only fails if the /dev/ttyUSBx it's open?
  • Joan,

    Please try to add 'usbcore.autosuspend=-1' into u-boot bootargs to see if the issue still happens.

    The another report of the similar issue I mentioned was indeed came from Variscite, but Variscite claimed the issue only exists in TI 3.12 kernel, but not in TI 3.14. Could you please double check with Variscite?

    Yes, I received the modem from Variscite, and was able to replicate the issue with 3.12, but not 3.14.

     

  • Adding usbcore.autosuspend=-1 the issue still happens.

    It's strange because the issue happens with 3.12 and 3.14. I will check with Variscite.