USB gadget enumeration failing

Not sure if I can help. Some background questions.

Which processor? AM1808? Is this a custom board? If a custom board, are the USB HW exactly the same as EVM.

What is the Host OS? If Windows, is there anything in the Device Manager? Maybe try using UVCView or Thesycon USB descriptor viewers. Some Windows verisons require a INF. See linux file at Documentation//usb/gadget_serial.txt.

Try a different cable. Some designs are very sensitive to the cable impedance.

Yes it's an AM1808 on a custom board, though the USB hardware is the same as the EVM. We are just using a USB-B connector with the USB0_ID left floating to indicate device mode.

I've tried a couple of different machines as the host, either running windows 8 or XP,
I found the linux-cdc-acm.inf file in the documentation and also went and found the usbser.sys windows needs for the serial gadget too, but windows just says the driver file isn't for the attached hardware, I assume as the device isn't completing enumeration to be able to see the Vid and Pid identifiers.
It get's listed as an known device in the device manager.

I've tried a couple of different cables but with the same result.

The output of Thesycon USB is below.

Information for device USB\Vendor_0000_Product_0000:

Connection Information:
------------------------------
Connection status: Device failed enumeration
Device actual bus speed: Low
Device is hub: No
Device adress: 0x0000
Current configuration value: 0x00
Number of open pipes: 0

String Descriptor Hex Dump
--------------------------------
Index LANGID Hex Dump
bLength bDescriptorType bString

------------------------------

Whole unknown Descriptor as hex dump:

------------------------------

Connection path for device:
Intel(R) ICH8 Family USB2 Enhanced Host Controller - 2836
Root Hub
Generic USB Hub
USB\Vendor_0000_Product_0000

Brought to you by TDD v1.81.0, Jul 2 2013, 09:51:54

I would guess that with a VID and PID of 0, that the enumeration does not get very far or Windows has already torn it down. You are using your own VID/PID, so you have to change the INF file to your VID/PID. Your PC may have "remember" that the last connect attempt failed and no more connection attempts are made. You might have to "uninstall" the bad device from Device Manager before trying again. You might have to use a USB sniffer.

I've managed to get a bit further and it seems to be a problem with the high speed / full speed issue, I've not got it 100% reliable yet, but with the following combination of switches to the musb_hdrc controller windows finally reconfigured it as a gadget serial port. (The below forces it between high speed/ full speed and forces a reconnect after being suspended)

# echo "I" > /proc/driver/musb_hdrc
# echo "i" > /proc/driver/musb_hdrc
# echo "c" > /proc/driver/musb_hdrc
# echo "I" > /proc/driver/musb_hdrc
# echo "S" > /proc/driver/musb_hdrc
# echo "i" > /proc/driver/musb_hdrc
# echo "F" > /proc/driver/musb_hdrc

after which the following debug appears, which I've cut short, at the end windows sees the serial_gadget correctly as a comm port.

# USB IRQ 00090000
musb_interrupt 1614: ** IRQ peripheral usb0009 tx0000 rx0000
musb_stage0_irq 496: <== Power=c0, DevCtl=99, int_usb=0x9
musb_stage0_irq 688: SUSPEND (b_peripheral) devctl 99 power c0
musb_stage0_irq 705: IN OTG STATE B PERIPH ISN
musb_g_suspend 1958: devctl 99
USB IRQ 00040000
musb_interrupt 1614: ** IRQ peripheral usb0004 tx0000 rx0000
musb_stage0_irq 496: <== Power=c0, DevCtl=99, int_usb=0x4
musb_stage0_irq 873: BUS RESET as b_peripheral
musb_g_reset 2048: <== B-Device addr=0 driver 'g_serial'
musb_g_disconnect 1994: devctl 99
USB IRQ 00010000
musb_interrupt 1614: ** IRQ peripheral usb0001 tx0000 rx0000
musb_stage0_irq 496: <== Power=c0, DevCtl=99, int_usb=0x1
musb_stage0_irq 688: SUSPEND (b_peripheral) devctl 99 power c0
musb_stage0_irq 705: IN OTG STATE B PERIPH ISN
musb_g_suspend 1958: devctl 99
USB IRQ 00040000
musb_interrupt 1614: ** IRQ peripheral usb0004 tx0000 rx0000
musb_stage0_irq 496: <== Power=c0, DevCtl=99, int_usb=0x4
musb_stage0_irq 873: BUS RESET as b_peripheral
musb_g_reset 2048: <== B-Device addr=0 driver 'g_serial'
musb_g_disconnect 1994: devctl 99
USB IRQ 00080001
musb_interrupt 1614: ** IRQ peripheral usb0008 tx0001 rx0000
musb_stage0_irq 496: <== Power=c0, DevCtl=99, int_usb=0x8
musb_g_ep0_irq 682: csr 0001, count 8, myaddr 0, ep0stage setup
musb_read_fifo 311: RX ep0 fifo fee00420 count 8 buf c0417e00
musb_read_setup 616: SETUP req80.06 v0100 i0000 l64
musb_g_ep0_irq 864: handled 0, csr 0001, ep0stage in
musb_g_ep0_queue 970: queue to ep0 (OUT/RX), length=18
musb_write_fifo 272: TX ep0 fifo fee00420 count 18 buf c79d8000
musb_g_giveback 144: ep0 done request c6c4fd40, 18/18
USB IRQ 00080001

Anyone seen something similar and know how to make a correct fix?