LE910 USB Enumeration Problem

More information: I am convinced our problem root cause is the USB babble interrupt discussed in this forum post:
e2e.ti.com/.../484475.

If we boot without the eth0 connected, we get a USB babble event. If we boot with eth0 connected, run the firmware so that the le910 is connected, then pull the eth0 plug, then we get a USB babble event. The root cause is the cause of the USB babble event.

I looked into the drivers/musb_core.c driver last night (very roughly) and it appears
that the babble message comes from receiving a babble interrupt from the AM335x
on-board USB peripheral.

If that is the case, how can any patch or kernel version fix this problem? It appears
what we need to do to fix our problem is understand what causes the AM335x USB
peripheral to generate this interrupt, e.g., does it monitor power variations, DM/DP
voltages, etc., or what?

If so, does anyone at TI know the conditions which cause the AM335x USB peripheral
to emit a babble interrupt?

Hi Randy,

You've referenced another thread about USB babble interrupt: , on which I am still working.

Also I see you use debian jessie/4.4 linux, with which I am not very well familiar, so I will reference the official TI SDK (kernel 4.1.13).

Randy Yates said:

If so, does anyone at TI know the conditions which cause the AM335x USB peripheral 
to emit a babble interrupt?

 

See this thread: https://e2e.ti.com/support/arm/sitara_arm/f/791/t/308549 According to usb specs : "Babble: Unexpected bus activity that persists beyond a specified point in a (micro)frame.". In my experience I've seen reports for babble interrupt when usb is "overly busy", upon hotplugging, when a usb hub with more than one device doing I/O, etc. 

The workarounds, I've seen to have positive effect, are as stated in the thread you refer to earlier: https://e2e.ti.com/support/embedded/linux/f/354/t/484475 

This is what I can comment on this problem. I hope it is of use to you. 

Best Regards, 
Yordan

Matt,

Thank you, thank you, thank you!!! That fixed it! No more babble interrupts!

--Randy

Update: That did not really solve the problem. It mitigated the problem greatly. Root cause will be posted next.

USB Babble Interrupt Resolution

1 Introduction

1.1 Hardware Overview

Our board is a custom board based heavily on the beagle bone black. It
uses the same or similar major components (an AM3352 instead of the
BBB AM3358/9) and adds certain other components which are specific to
our application. One such component is the Telit LE910 cellular modem
module.

Our application does not require any other USB interfaces besides that
going to the on-board LE910, thus we removed all USB connectors and
connected the DM, DP, and VBUS signals directly between the AM3352 and
the LE910.

It is crucial to note that our application also required a buck-boost
converter ahead of the TPS65217C PMIC which generated the 5VDC into
the PMIC. Our hardware engineer chose a TI TPS63060 for this purpose.

For breadboard development, we included the wired ethernet interface.

1.2 Software Overview

The software build is a customized version of RCN's bb-kernel linux
kernel. We made some changes to uboot as well as the device tree to
support our custom hardware. Otherwise it is a fairly standard
debian/linux build.

Our custom application is using the libqmi library for interfacing
with the LE910. We also use qmicli for testing.

2 Problem Description

2.1 Top Level

The USB would never properly enumerate the LE910 when the ethernet was
un- plugged! During systemd initialization, the musb driver would
initialize and attempt to enumerate the devices on the bus, which
would result in an immediate babble interrupt:

[13314.062686] musb-hdrc musb-hdrc.1.auto: Babble

With the ethernet plugged in, the LE910 would enumerate just fine.

2.2 Troubleshooting Details

Looking at the DM, DP, and VBUS signals with a 100 MHz scope, we found
that the noise on these lines (and at almost every place else on the
board) was very high, having large busts at 500 mVpp or greater at a
variety of intervals depending input voltage. A typical interval was
1/270 kHz (about 3.7 microseconds).

We also found that pulling the ethernet plug caused the magnitude of
the noise to increase, perhaps about 20 percent.

The TPS63060 has a power-save mode which increases efficiency at low
output currents. We had this mode enabled by tying pin 4 low.
According to the datasheet, the device switches into power-save mode
when the current is about 100 mA.

We also noticed (via the power supply) that about 120 mA of current
was being drawn with the ethernet plugged in and around 90 mA of
current was being drawn with the ethernet unplugged.

2.3 Workaround Attempts

Thinking the problem may be a signal integrity issue on the dm/dp
lines, we slowed down the USB bus to full-speed (12 MHz) from
high-speed (480 MHz) via a device tree modification:

&usb1 { maximum-speed = "full-speed"; };

This had the effect of suppressing the babble interrupt, but
introduced a new prob- lem: the LE910 module would unenumerate from
the USB bus after a variable amount of time, from about 5 minutes to 3
hours. Every time this happened, would would consistently get a flush
problem message in kern.log:

Feb 29 16:00:53 arm kernel: [ 288.655415] musb-hdrc musb-hdrc.1.auto:
Could not flush host TX10 fifo: c* *sr: 2003

3 Root Cause

We modified the board to tie pin 4 high, thereby disabling power save
mode on the TPS63060. The large noise spikes completely disappeared,
both with and without the ethernet plugged in.

Since, according to the datasheet, the device switches into power-save
mode when the current is about 100 mA, we deduced that without the
ethernet plugged in, the current was near enough to the
straight/power-save threshold built into the TPS63060 to cause more
severe switching between modes, thus more noise, and that this noise
was (understandably) producing an error in the physical layer
operation of the USB.

Operating the unit with the ethernet plugged in added about 30 mA of
current to the load, thus moving the TPS63060 far enough away from the
mode switch threshold to produce less noise, and keep the noise small
enough that the USB physical layer did experienced errors much, much
less frequently.

Also note that it is theorized that running the LE910 at high-speed,
12 MHz mode was on the threshold of too slow (at times) for the data
transfer occuring between the processor and the LE910, thus
overflowing a TX fifo in the musb driver and causing a disconnect at a
random time.