TUSB2077 Device Disconnect Issue

I managed to capture a Log using USBlyzer, the log shoud show the following:

1.- Proper Connect and Disconnect of the device

2.- Failed disconnect of the device and the Hub restarting itself. 

7144.Log.zip

Hello Gustavo,

Your schematic looks correct.

It is less likely that the Hub could cause a blue screen, some older USB 1.1 devices are no compliant, if you system was updated recently it is probably that the USB driver now have issues with that device.

Can you try connecting the problematic device directly to the USB Host or through another hub, you should see the blue screen as well.

The PortDis output is also controlled by the USB Host, if the TUSB2077 receives from the Host the instruction to disable the port.

Your USB Log shows the only TI hub with PID=2046, this could be because you have the TUSB2077 in bus power mode although your schematic show it in self-power mode.

Regards.

I also noticed that the Hub says it is bus-powered but I actually have it configured as self-powered. Any ideas on why that is? 

I manage to capture some more logs of the anomaly:

This log is from the Hub itself:

2161.Log 2.zip

This log should show data from the Host, the Hub and the Device. It has a lot of cycles where the device was removed correctly, the last cycle is the failed one. 

4137.Log 3.zip

There is also still no explanation why the TI Hub goes into suspend mode when the device is removed. 

I will try this device connecting it directly to the Host. 

I would like to give you a little more information:

After the failure occurs, and the Hub went into Suspend mode it will automatically come back. But it would come back withouth enumerating all of the devices hooked up to it. If I do a hardware Reset on the Hub I get a BSoD. 

I have completed 50 hotsawp cycles with the device connected directly to the Host and no BSoD occurred, nor any other USB Reset sequence. 

Hello Gustavo,

The hub does not put itself in Suspend, it is sent by the Host.

Please check your voltage rail, the Reset# signal and XI input when you perform the hot-plug, I've see hubs going to suspend or de-asserting PortDis when something wrong happens on the above signals.

Can you give more details of the USB peripheral causing issues, also operating system details.

Regards.

Elias,

We are running Windows XP Embedded SP3. The peripheral is a Thermal Ticket Reader. 

When I connect it directly to the host, hot plugging works without problems, so I think the hub built directly into the PC has more tolerance to whatever this ticket reader is doing. 

I placed the ESD protection device right underneath the connector, and this same device has a surge suppressor for VBUS. 

I have also checked the Reset line and nothing strage is happening, I see no drop. I haven't checked XI though, I will do that. 

If you can provide me an email address, I can share my Altium project file so you can take a look at my PCB Layout. 

Elias, are you still following this thread?

I checked the Reset# line under the scope and nothing odd happens during the fault. The XTAL on the other hand stops oscillating, so I think the IC goes into suspend mode, but I do not understand why the host would do that. 

In windows XP I have disallowed the OS from sending USB Hubs into suspend mode, but something else is happening and I cannot interpret those logs. 

Hello Gustavo,

If the Hub is going into suspend there are some thing to look at:

1) Too much EMI could cause the oscillator to stop, check your power rails and ground planes.

2) If the USB Host sees more than 3ms of bus inactivity, then it will send the Hub to suspend, do you see activity on DP/DM?

3) If the downstream devices pull more than 500 mA of current, the hub goes in to SUSPEND state since it is violation of USB specs.

4)  if the reset timing isn’t accurate or if the host doesn’t load the hub class driver correctly, the hub goes in to SUSPEND state and shows us as an “Unknown device” in device manager. 

5) If the hub goes into suspend, but the Host/driver doesn't have the remote wake-up enabled, then the Host won't be ale to signal a resume.

Regards

Thanks for your reply, Elias:

1) Too much EMI could cause the oscillator to stop, check your power rails and ground planes.

I've been several times to the EMI lab with this board and we haven't had issues with EMI yet. 

2) If the USB Host sees more than 3ms of bus inactivity, then it will send the Hub to suspend, do you see activity on DP/DM?

Yes, there is plenty of activity on DP/DM. 

3) If the downstream devices pull more than 500 mA of current, the hub goes in to SUSPEND state since it is violation of USB specs.

I am using the TPS2044 to limit the current to the device, but still I measured the current and it is drawing less than 50mA. But if the condition arose where a device consumes more than 500mA, shouldn't shut down the port, instead of going into suspend mode? Isn't that the purpose of the TPS2044?

4)  if the reset timing isn’t accurate or if the host doesn’t load the hub class driver correctly, the hub goes in to SUSPEND state and shows us as an “Unknown device” in device manager. 

Since the device was enumerated correctly, and it was working properly before I disconnected it, I guess we can assume that the class driver was loaded correctly. 

5) If the hub goes into suspend, but the Host/driver doesn't have the remote wake-up enabled, then the Host won't be ale to signal a resume.

The Hub does come back after the fault. It remains in suspend mode about 2 seconds and then it turns on automatically. 

If I do not disconnect the hub everything works fine. But it is only when disconnecting it that it will send the hub into this odd suspend/reset state. It happens about 10% of the disconnections. 

Additionally, there are 3 other devices connected to the hub, one of them is permanently connected (FTDI Virctual COM port), so there is plenty of activity going around, hence it shouldn't go into suspend mode for lack of activity. 

I installed Windows 7 and all its patches today, thinking that it would be something with Windows XP just being too old. I am able to duplicate the problem on Win 7 too, albeit the hub recovers much more quickly than on Windows XP. So the host OS is involved in a certain way.

Are you able to interpret the logs?

I captured 2 more logs with a different tool. I see major differences between eachother, but I am sure Elias can interpret them much better than I can (I can't). 

OK Unplug:

https://www.dropbox.com/s/ofd7ifpn1yazxbh/OK%20Unplug.html

Fail Unplug:

https://www.dropbox.com/s/nob9owkifjo3w3v/NG%20Unplug.html

I captured a few more logs:

From the Device,

OK: https://www.dropbox.com/s/nsm5i2qye02opqw/OMR%20OK%20Disconnect.html

No Good: https://www.dropbox.com/s/9tzctxlzkea4mvl/OMR%20NG%20Disconnect.html

They seem to be about the same.

Now from the root-hub:

OK: https://www.dropbox.com/s/1fo8imue0es89p4/Root%20Hub%20OK.html

No Good: https://www.dropbox.com/s/d3vwnsj15byisom/Root%20Hub%20NG.html

I noticed that the "No good" file has more of this call: *SYNC_RESET_PIPE_AND_CLEAR_STALL*

I captured 2 more logs, but this time looking at all three involved devices:

The Host (Root Hub inside PC) The TI 7-port Hub The Device

The OK log shows 41 lines of captured data, whereas the "No Good" log shows 524 lines of data (in part because the hub rebooted and it is enumerating everything again). But that *SYNC_RESET_PIPE_AND_CLEAR_STALL* keeps popping back up.

OK: https://www.dropbox.com/s/0d2yj8opigygh0i/Full%20Log%20Clean%20-%20OK.html

No Good: https://www.dropbox.com/s/njf6t140scfppo0/Full%20Log%20Clean%20-%20NG.html 

I reuploaded the No Good file with a much better capture. 

Hello Gustavo,

We are reviewing the logs.

Have you tried using a different Hub? 

Is your problematic device High speed capable? If so, can you use a High speed hub to see if the issue occurs?

I am thinking about the downstream device's driver not being able to handle the Address changes involved in a Hub topology in which case the issue should e present with many other hubs.

Regards.

Hi Elias,

Thanks for your reply and for reviewing the logs. This is enumeration information on the device side:

It is a Fullspeed device. I tried on a High-speed of-the-shelve hub and it did not power down. I notice from the above screenshot that the devices uses some TI DSP device, I am not sure which one, and it is using the usbio.sys generic driver from Thesycon. I am not sure if Thesycon has had issues with their driver and your hub, I emailed them but they refused to give support as I am not the licensee of the usbio.sys driver. 

Can you try to disable this thesycon driver and let windows to use the default USB driver?

Are you able to ship on of your systems to us for debug?

Regards.

Based on what little I understand of the log this is what little I can interpret. Please tell me if I am wrong:

One the disconnection occurs, the host tries to perm a reset on the device, as it cannot find it:

Because that was unsuccesful (there is no device to reply), it performs something called QUERY device relations:

I am not sure if that "STATUS_NOT_SUPPORTED" is because it again cannot find the device, or if something is wrong there. 

It continues to try and do something, and now performs an ABORT_PIPE:

I can see from lines 18 and 19 that it has raised the SURPRISE_REMOVAL Flag, and resets the port. The weird thing is, before that, in line #21 and #22 it queries device relation again on USBPDO-6, I believe this is the HUB, or something within the HUB according to USBTrace it is running under usbhub.sys driver and it is \Device\00000063, and the RESET_PORT goes to this "device". You mentioned that the addresses within a HUB are changed, but I am not sure I understand. When this RESET_PORT is sent to \Device\USBPDO-6 instead of \Device\USBPDO-7 where my device is, is this resetting the HUB instead of the individual port? 

Hello Gustavo,

One of the USB logs shows the DSP to be not WHQL certified, can you try on a different system which doesn't have this DSP?

Also, please look at the link below, the post dated on  Dec 14 2012 12:43 PM

Do you have this driver installed on your system?

Can you give me the full specs of the downstream device to see if I can get one of those?

Regards.

http://e2e.ti.com/support/data_converters/precision_data_converters/f/73/p/233750/884222.aspx

Elias,

The device is a made to order Thermal Ticket Reader. I don't think you will find it anywhere in the market. I don't have a ticket reader that doesn't use that DSP. 

The driver I am using with my device is attached:

2804.usbio driver.zip

This is Thesycon's generic USB driver "usbio.sys", provided to us by the people that made the Ticker Reader. We have been using this device for 10 years, albeit this is the first time we connect it to a downstream usb HUB instead of connecting it directly to the host's root hub. 

Elias,

I have been capturing some more logs, this time with a different tool and capturing the full stack, and this is what I found:

On the good disconnection, we see the last bulk transfer using the correct IN endpoint for the reader 0x82:

[img]http://i.imgur.com/TpNLPdZ.png[/img]

whilst on the bad disconnection, we see this strange C:I:E values FF:FF:8F

[img]http://i.imgur.com/VdbCrox.png[/img]

I am going to venture and say this actually crashes the USB Hub, hence we don't see a Get Port Status on the Hub, and later on we see the host trying to clear a stall on the hub:

[img]http://i.imgur.com/tnqxA2m.png[/img]

But according to the USB spec, the hub should be able to handle loss of data, erros and everything caused by the removal of the device.

Hello Gustavo,

Please see below comment from a college:

"one thing I noticed was that on the failing disconnection vs. passing disconnection – there is a difference in the buffer size reported.  The passing one says 64 bytes – which is the right size for a full speed device and the failing one says 1024 bytes which is the size for a high speed device.  I couldn’t tell where that transfer was supposed to go, but the TUSB2077 is a full speed hub so it would definitely fail on any high speed traffic."

Regards.

Elias,

I noticed that too, but I was not aware that 1024 corresponded to a high speed device. I have no high-speed peripherals in my product, so I do not know where that was coming from or going to. 

This morning I got myself a Ellisys Hardware Analyzer, which I think will give us much better insight on what traffic is moving around before, during and after the disconnection. I will share that log with you as soon as I have it. I am currently installing the software on my PC. 

You had mentioned that if you could have one prototype for your testing. I believe that would be a great idea. Are you still up for that idea? 

Elias,

Please see attached Hardware Logs:

https://www.dropbox.com/s/guakbe2dxemf3hu/Hardware%20Analyzer%20Logs.zip

0001.ufo = Good disconnection

0002.ufo = bad disconnection

I used Ellysis Visual USB Analyzer to capture this logs. 

There are no other devices connected on the hub, so all traffic is between the host, the hub and the device. It looks like the hub could be crashing after the disconnection. 

Things that I have noticed:

On the "good" disconnection, the Incomplete IN transaction has an interface value of "0", whilst on the "bad" disconnection, the Incomplete IN Transaction has no "interface" value. Does this mean anything to you?

On the good disconnection I can see 6 "Incomplete IN Transactions attempts", and this is repeatable with all "good" disconnections. While on the "bad" disconnection I can only see one attempt, and then immediately go into the reboot process.

As per your request, I will make one post with all the information I have:

The problem:

The error occurs upon disconnection of the device; when disconnecting the device the HUB will go into SUSPEND mode for about 5 seconds and return to normal state, this happens regardless of the fact that I have disabled Selective Suspend in the OS as per Microsoft's recommendation. The error has about 10% occurrence rate.

The Device

The device is a Ticket Reader + Thermal Printer. It is a USB 1.1 Full speed device. It uses the usbio.sys from Thesycon and it has been in use by our company for about 10 years. It does not have a problem when being disconnected from the Host's Root Hub (PCI). The device has 2 stages: Bootloader mode and Firmware Uplaoded mode. The failure only occurs when the firmware has been loaded, although both states use the same device driver. 

Enumeration Descriptors without Firmware:

https://www.dropbox.com/s/jlftvic7xg8g5rw/Enumeration%20No%20Firmware.html

Enumeration Descriptors with Firmware:

https://www.dropbox.com/s/r7hfi30puuy0ncn/OMR%20Reader%20Enumeration%2014.html

The logs

This my very small understanding of what is going on: 

USBPDO-12 = The Device (it is using the Thesycon usbio.sys driver)
USBPDO-10 = The Hub (it uses the usbhub.sys driver)
USBPDO-4 = The Root Hub
0000006d = The Hub

According to one of your previous posts, and confirmed on Section 5 of the USB Spec 2.0, the first two Bulk Transfers you see are OK, and they are 64 bytes because it is a Full Speed device. Then I can see some Get Port Status on Port 4, which is where the device is located. The reply is "00 01 01 00" which basically means that the port is not in powered-off state and that the current connect status has changed. 

Full log is here:

https://www.dropbox.com/s/zms8bt20dcmnf3x/OK%20Disconnect%20Firmware%20Loaded.html

Now onto the bad disconnect:

Now, again based on one of your replies, and confirmed on the USB Spec the first two lines show a Bulk Transfer of 1024 bytes, this is NOT correct for a Full Speed device. I see a Get port status on Port 4 of the hub (where the device use to be), but there is no reply. After that the host gets a port status (2), which is a status on the Hub itself. To which the hub replys: "01 01" -> A device is present and not in powered-off state, and then a "02 00" which means "Disabled because of a port error condition". This looks to show that the HUB has crashed. 

Full log is here:

https://www.dropbox.com/s/pqm73w9dfquv9dt/NG%20Disconnect%20Firmware%20Loaded.html

--------------------------------------------------------------------------------------------------------------------------------------------------

From a Hardware Analyzer point of view I can see nothing of interest, other than the last IN transaction being identical except before the device can return NAK, the bus segment resets. So the problem appears to be upstream from the bus segment. Maybe the hub driver chokes on getting an URB requesting a 1024-byte transaction. But on a hardware transaction level, they are exactly the same. Please see screenshots:

See the log here:

https://www.dropbox.com/s/guakbe2dxemf3hu/Hardware%20Analyzer%20Logs.zip

Based on this latest information that I have provided, could you offer your expert opinion as to what could be the root cause of the problem? Is the HUB crashing because of this 1024 byte IN transaction token packet request? 

Hello Gustavo,

Is this still an issue? Are you able to ship one of your platform for testing?

Regards.