USB RNDIS boot fails with AM3352 on custom board

Hi Didier,

Can you provide more information about your Host PC configuration? Have you seen this: http://processors.wiki.ti.com/index.php/AM335x_U-Boot_User's_Guide#CPSW_Ethernet_2 ? USB boot does not differ from Ethernet boot.

Thank you to pay attention to my problem.

Yes I saw this document.

My dhcpd.conf file contains :

subnet 192.168.3.0 netmask 255.255.255.0 {
  range dynamic-bootp 192.168.3.20 192.168.3.99;
  option broadcast-address 192.168.3.255;
#  next-server 192.168.3.10;
  if substring (option vendor-class-identifier, 0, 10) = "AM335x ROM" {
#1er boot loader :
    filename "u-boot-usb-spl.bin";
  } elsif substring (option vendor-class-identifier, 0, 17) = "AM335x U-Boot SPL" {
#2eme boot loader :
    filename "u-boot-usb-nfs.img";
  } else {
#image linux :
    filename "uImage";
  }
}

Once they are compiled I renamed the u_boot files as u-boot-usb-spl.bin and u-boot-usb-nfs.img and place them Inside /tftpboot/

These king of settings works correctly with the ethernet boot on the am335x EVM demo board.

When I sniff the traffic on the usb0 port on the host machine, I see bootp request and reply + tftp transfer for the SPL boot. But once the SPL boot loader is started, I don't see any other bootp or tftp traffic on the usb0 interface.

I suppose that the SPL u-boot fails before sending its bootp request on the USB RNDIS connection.

What does mean "musb-hdrc: peripheral reset irq lost!" ? Is it an error or just an expected warning in this context ?

What does mean "ERROR: The remote end did not respond in time." ? Is the "remote end" being the host PC interface or only the am3352 board usb port ?

BR

Didier

For some reason the host did not respond in time. If you look at drivers/usb/gadget/ether.c you will see

         if (getenv("cdc_connect_timeout"))
                 timeout = simple_strtoul(getenv("cdc_connect_timeout"),
                                                 NULL, 10) * CONFIG_SYS_HZ;
         ts = get_timer(0);
         while (!l_ethdev.network_started) {
                 /* Handle control-c and timeouts */
                 if (ctrlc() || (get_timer(ts) > timeout)) {
                         error("The remote end did not respond in time.");
                         goto fail;
                 }
                 usb_gadget_handle_interrupts();
         }

In your u-boot environment, can you add the environment variable cdc_connect_timeout and set it to a value greater than 10? Maybe try 15 or 20?

Steve K.

Thank you Steve.

I already applied the patch : 0001-u-boot-Change-default-cdc_connect_timeout-to-15s.patch

Therefore the timeout is now 15s. The error come later but is still here.

Since my last post, I discovered that the network interface (named "usb0") on the host PC which is associated with the USB RNDIS connection of my board, is active when the u-boot SPL start, but when it fails, the usb0 interface disappears.

From that observation I tried to detect where and when, in the u-boot source code, something "break" that connection.

I'm now sure that the interface vanish when musb_init_controller() in drivers/usb/musb-new/musb_core.c is called (by musb_register() in musb_uboot.c, called by arch_misc_init() in board.c, called by spl_board_init() in boot-common.c, called by board_init_r() in spl.c ...)

When musb_init_controller() is called, the following compilation symbols are defined :

CONFIG_AM335X_USB0
CONFIG_MUSB_GADGET

I suppose that something is wrong in the configuration when musb_init_controller() is called. This bad configuration then disconnects the usb0 interface on the host PC.

Can you confirm that the usb should be configured as "gadget" and not "host" for using USB RNDIS ?

I will try to get more data from Inside musb_init_controller about, for example, values of arguments passed to it.

Unfortunatly I haven't got any JTAG connection, therefore I have to add display instructions (like puts or printf) to Watch what is happening Inside functions. This process is not very fast and convinient.

BR

Didier

Didier,

Is the host a WinXP? If so, please try the following patch.

diff --git a/drivers/usb/musb-new/musb_dsps.c b/drivers/usb/musb-new/musb_dsps.c
index 9a03917..197ddb5 100644
--- a/drivers/usb/musb-new/musb_dsps.c
+++ b/drivers/usb/musb-new/musb_dsps.c
@@ -415,6 +415,7 @@ static irqreturn_t dsps_interrupt(int irq, void *hci)
         return ret;
  }

+#define RNDIS_DELAY_MS (5 * 1000)
  static int dsps_musb_init(struct musb *musb)
  {
  #ifndef __UBOOT__
@@ -459,6 +460,9 @@ static int dsps_musb_init(struct musb *musb)
         /* Reset the musb */
         dsps_writel(reg_base, wrp->control, (1 << wrp->reset));

+       /* HACK: delay musb start to work with WinXP RNDIS driver */
+       mdelay(RNDIS_DELAY_MS);
+
         /* Start the on-chip PHY and its PLL. */
         if (data->set_phy_power)
                 data->set_phy_power(1);

Bin,

My host machine is an ubuntu 12.04 LTS running as a virtual machine on a Windows 7 physical host (using virtual box as virtualization software).

Even if this is not Windows XP, I tried your patch to see, but it didn't change anything, except that the failure come later (due to the extra delay) but it doesn't work better.

Since my previous post, I detected that the network interface in my host, related to the usb port of my custom board, called "usb0", disappears when the instruction "dsps_writel(reg_base, wrp->control, (1 << wrp->reset));" (in the same area as the patch you suggested), is called.

When I compare register settings before (when usb0 still exists) and after the board initialisation (made by spl_board_init() execution inside the function board_init_r in <u-boot>/common/spl/spl.c), I can see these contents :

Before :

register 0x47401000 (offset 0x0) = 0x4ea20800
register 0x47401014 (offset 0x14) = 0x0
register 0x47401018 (offset 0x18) = 0x0
register 0x47401020 (offset 0x20) = 0x0
register 0x47401028 (offset 0x28) = 0x4000b
register 0x4740102c (offset 0x2c) = 0xd
register 0x47401030 (offset 0x30) = 0x0
register 0x47401034 (offset 0x34) = 0x0
register 0x47401038 (offset 0x38) = 0x0
register 0x4740103c (offset 0x3c) = 0x0
register 0x47401040 (offset 0x40) = 0x0
register 0x47401044 (offset 0x44) = 0x0
register 0x47401070 (offset 0x70) = 0x0
register 0x47401074 (offset 0x74) = 0x0
register 0x47401080 (offset 0x80) = 0x0
register 0x47401084 (offset 0x84) = 0x0
register 0x47401088 (offset 0x88) = 0x0
register 0x4740108c (offset 0x8c) = 0x0
register 0x47401090 (offset 0x90) = 0x0
register 0x47401094 (offset 0x94) = 0x0
register 0x47401098 (offset 0x98) = 0x0
register 0x4740109c (offset 0x9c) = 0x0
register 0x474010a0 (offset 0xa0) = 0x0
register 0x474010a4 (offset 0xa4) = 0x0
register 0x474010a8 (offset 0xa8) = 0x0
register 0x474010ac (offset 0xac) = 0x0
register 0x474010b0 (offset 0xb0) = 0x0
register 0x474010b4 (offset 0xb4) = 0x0
register 0x474010b8 (offset 0xb8) = 0x0
register 0x474010d0 (offset 0xd0) = 0x0
register 0x474010d4 (offset 0xd4) = 0x280de80
register 0x474010d8 (offset 0xd8) = 0x0
register 0x474010e0 (offset 0xe0) = 0x200002
register 0x474010e4 (offset 0xe4) = 0x10c20104
register 0x474010e8 (offset 0xe8) = 0x100

After :

register 0x47401000 (offset 0x0) = 0x4ea20800
register 0x47401014 (offset 0x14) = 0x0
register 0x47401018 (offset 0x18) = 0x0
register 0x47401020 (offset 0x20) = 0x0
register 0x47401028 (offset 0x28) = 0x0
register 0x4740102c (offset 0x2c) = 0x0
register 0x47401030 (offset 0x30) = 0x0
register 0x47401034 (offset 0x34) = 0x0
register 0x47401038 (offset 0x38) = 0x0
register 0x4740103c (offset 0x3c) = 0x0
register 0x47401040 (offset 0x40) = 0x0
register 0x47401044 (offset 0x44) = 0x0
register 0x47401070 (offset 0x70) = 0x0
register 0x47401074 (offset 0x74) = 0x0
register 0x47401080 (offset 0x80) = 0x0
register 0x47401084 (offset 0x84) = 0x0
register 0x47401088 (offset 0x88) = 0x0
register 0x4740108c (offset 0x8c) = 0x0
register 0x47401090 (offset 0x90) = 0x0
register 0x47401094 (offset 0x94) = 0x0
register 0x47401098 (offset 0x98) = 0x0
register 0x4740109c (offset 0x9c) = 0x0
register 0x474010a0 (offset 0xa0) = 0x0
register 0x474010a4 (offset 0xa4) = 0x0
register 0x474010a8 (offset 0xa8) = 0x0
register 0x474010ac (offset 0xac) = 0x0
register 0x474010b0 (offset 0xb0) = 0x0
register 0x474010b4 (offset 0xb4) = 0x0
register 0x474010b8 (offset 0xb8) = 0x0
register 0x474010d0 (offset 0xd0) = 0x0
register 0x474010d4 (offset 0xd4) = 0x280de80
register 0x474010d8 (offset 0xd8) = 0x0
register 0x474010e0 (offset 0xe0) = 0x2
register 0x474010e4 (offset 0xe4) = 0x14860104
register 0x474010e8 (offset 0xe8) = 0x100

Few registers changed, but if I restore their content after the initialization with :

 __raw_writel(0x4000b, 0x47401028);
 __raw_writel(0xd, 0x4740102c);
 __raw_writel(0x200002, 0x474010e0);
 __raw_writel(0x10c20104, 0x474010e4);

This doesn't restore the usb0 interface of my host, so the bug is still here.

BR.

Didier.

Didier,

We have seen the similar problem with Windows XP during usbboot validation, and found the root causes is that while the execution transitioning from ROM to SPL, the SPL will reset the USB0 controller immediately and start over, but Windows XP RNDIS/USB stack is not fast enough to teardown the usb0 interface from ROM and re-enumerate the usb0 interface from SPL, so the usb0 disappeared on the host.

The patch I posted above delays the usb controller reset in SPL, so it gives the host some time to teardown the ROM usb0 interface.

Even we did not see this issue with Windows 7 host, but since you run Linux virtual machine within Windows, I guess it takes longer time for the Linux to see the SPL usb0 interface.

I think you have two options, 1 - my patch adds 5 seconds delay, you can try to increase it; 2 - I don't think we have validated the usbboot feature with hosts in virtual machine, so you can try to use a dedicated Linux host.

Thank you for this answer.

It seems very close to what I see here.

I will try to apply again the patch with larger delay. I hope it will work, otherwise I will have to find a another computer to install linux on and copying all files I made on it. This could take some times...

Thank you.

BR

Didier.

I tried with 15s, then with 30s, the usb0 interface is never back !

It doesn't work.

When I start my custom board, it takes less than 5s for the host (linux virtual machine) to detect the new usb0 interface (coming from the ROM code execution).

The stange thing is that when looking at the Windows 7 (physical PC) device manager, a "virtual box USB" appears when my custom board starts and disappears few seconds later. But it never appears again, except if I restart the custom board. Therefore, I wonder if it's just a question of delay.

Does this correlates with the problem you saw on Windows XP machines ?

BR

Didier.

Didier,

didier cadiou said:

The stange thing is that when looking at the Windows 7 (physical PC) device manager, a "virtual box USB" appears when my custom board starts and disappears few seconds later.

Can you tell this "Virtual box USB" is ROM or SPL usb0 interface?

If it is SPL usb0, then this is not the same issue as what we observed (then my patch will not fix it). If so, please debug in the Virtual Linux if it sees the usb0 as while, and then use for example Wireshark to debug why Linux does not response to dhcp/bootp request.

Bin Liu said:

Can you tell this "Virtual box USB" is ROM or SPL usb0 interface?

Ok, I think it is easy to tell. Use my patch to add a delay to bring up usb in SPL. Then if

- the usb interface shows up on host when power on the board - this is the usb0 from ROM;

- then the board boots to SPL, the usb interface disappeared for a 5 seconds, which is the delay from the patch;

- then the usb interface shows up on host again, this is the usb0 from SPL;

- then after a few seconds the usb interface disappeared for ever, this is because the host does not respond to the DHCP/BOOTP request.

If so, this is most likely a host configuration issue, you can debug it with for example Wireshark. 

When watching traffic on usb0 interface using wireshark, i don't see any bootp request coming from u-boot SPL.

So I suppose it's not a host configuration problem.

To get more data, I'm trying now to use Windows servers instead of linux, but it doesn't work :

The bootp server (opendhcp) responds to the custom board bootp request and a new IP address is allotted (192.168.3.22), but the tftp server (opentftp) doesn't answer anything.

I tested the tftp server from another PC in the local network and it works (the asked file is received).

I tried to watch traffic on the Windows usb0 interface, but wireshark didn't list this interface, as if it doesn't exist. However the interface exists in the connexion manager ( with an "AM335x USB" description and 192.168.3.20 IP address).

I'm wondering if the USB RNDIS driver I use on Windows is the one I'm supposed to use. I choosed it "manually" : Microsoft Corporation, "remote NDIS compatible device". It is not specific to AM335x.

BR

Didier

Didier,

have you tried to use an AM335x EVM instead to validate you host setup?

When you built SPL/u-boot did you 'make am335x_evm_usbspl' 

Steve K.

Yes I use the 'am335x_evm_usbspl' target config.

Additionnal information : To install the Windows driver I followed the process described at http://developer.toradex.com/knowledge-base/how-to-install-microsoft-rndis-driver-for-windows-7

I will try a NFS boot with our am335x evm sitara kit.

I found the reason why I can't boot with Windows servers. I forgot to update the "next-server" option Inside the opendhcp.ini file.

As I understood, this option tells the client which is the next server (its IP address) in the boot process, which means where is the TFTP server to continue with. In my case it's the same machine that includes both bootp and tftp servers, so I set this option at 192.168.3.20 (which is the IP address of the usb0 interface on my Windows host).

Now I'm able to reproduce the same booting process as in my initial linux machine. As with this one, the booting process starts (bootp request and reply, SPL image downloading, running of the SPL u-boot) and fails (same error message on serial port as with linux host and no more usb0 inferface on host).

To make this trial, I set the delay in the patch at 30 s.

So now it seems pretty sure that it's not a delay problem.

One strange thing : Once the u-boot SPL si started and fails, opentftp display the error message "C:\servers\tftpboot\u-boot-usb-spl.bin, Timeout" after saying "Client 192.168.3.22:1234 C:\servers\tftpboot\u-boot-usb-spl.bin, 198 Blocks Served".

"u-boot-usb-spl.bin" is the name of the u-boot SPL image I configured, and the board downloaded. Despite the timeout, this image is fully downloaded because it starts and display outputs on serial port. So why the opentftp server said timeout ?

BR

Didier

Didier,

didier cadiou said:

I found the reason why I can't boot with Windows servers. I forgot to update the "next-server" option Inside the opendhcp.ini file.

Good to know you have some progress. I am not familiar with opendhcp. I only used tftpd32 for Windows to validate the ROM usbboot feature. For the whole ROM/SPL usbboot I used isc-dhcp-server and atftpd on Ubuntu.

didier cadiou said:

SPL image downloading, running of the SPL u-boot) and fails (same error message on serial port as with linux host and no more usb0 inferface on host).

To make this trial, I set the delay in the patch at 30 s.

To double check on this, the usb0 interface on host is gone before the 30s delay or after?

didier cadiou said:

One strange thing : Once the u-boot SPL si started and fails, opentftp display the error message "C:\servers\tftpboot\u-boot-usb-spl.bin, Timeout" after saying "Client 192.168.3.22:1234 C:\servers\tftpboot\u-boot-usb-spl.bin, 198 Blocks Served".

Not sure what the timeout is for, but again, I am not familiar with opentftp.

One point with Linux host, the usb0 interface from ROM and SPL are two different onces, so the Linux dhcp server has to be configured to restart every time when a new usb0 is enumerated, so that the dhcp server can listen to it and serve its dhcp/bootp requests.

Didier,

Another note is that in Wireshark you have to use 'Pseudo-device that captures on all interfaces' from the Capture Interface List, so that Wireshark can capture all USB traffic. Please don't just use usb0 interface. (usb0 is even unavailable until power on the board.)

Bin,

Your last post helped me a lot.

I watched with an increased attention the USB peripherals list in the device manager of the physical machine during the complete u-boot SPL execution. I discovered that during a short period of time (15s of the first patch I applied) a new USB peripheral appears in the list, and then disappears (when u-boot SPL displays error messages).

(sorry for the low resolution, the forum doesn't allowed me to insert it in its original size)

Now I know that the new peripheral is here, so it will be far easier to continue.

Thank you.

I will try now to attach this new peripheral to the virtual machine, and then to configure an associated network interface Inside the virtual machine.

I will inform you about progress.

BR

Didier

It works better !

After temporarely modified the delay to have time enough to configure my virtual machine's network connections, the u-boot SPL is now able to run to end and download the final u-boot and start it.

Lines modified to enable an increased delay :

Didier,

Glad you have a great progress! USB association is always a pain in virtual machine :(

"wait_for_user_access Timeout" is from cpsw.c, I thought we are doing USB Ethernet, cpsw shouldn't be used in here...

Can you please post the kernel boot log which shows the failure?

Please find attached the boot log seen on the serial port.

U-Boot SPL 2013.01.01 (May 02 2014 - 13:13:59)
Compiled by E.D.E Labs.
SPAD (ver.1.0)
SPAD DDR3 OK.
s_init done.
musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
musb-hdrc: MHDRC RTL version 2.0
musb-hdrc: setup fifo_mode 4
musb-hdrc: 28/31 max ep, 16384/16384 memory
USB Peripheral mode controller at 47401000 using PIO, IRQ 0
Using default environment

CONFIG_USB_ETHER...
registering USB_ETHER OK.
usb_ether
using musb-hdrc, OUT ep1out IN ep1in STATUS ep2in
MAC d0:ff:50:ec:15:06
HOST MAC de:ad:be:af:00:00
RNDIS ready
musb-hdrc: peripheral reset irq lost!
high speed config #2: 2 mA, Ethernet Gadget, using RNDIS
USB RNDIS network up!
bootp request...
BOOTP broadcast 1
BOOTP broadcast 2
Using usb_ether device
TFTP from server 192.168.3.10; our IP address is 192.168.3.21
Filename 'u-boot-usb-nfs.img'.
Load address: 0x807fffc0
Loading: #################################################################
         #######
         241.2 KiB/s
done
Bytes transferred = 368372 (59ef4 hex)


U-Boot 2013.01.01 (May 02 2014 - 13:13:59)

I2C:   ready
DRAM:  256 MiB
WARNING: Caches not enabled
NAND:  No NAND device found!!!
0 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
*** Warning - readenv() failed, using default environment

musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
musb-hdrc: MHDRC RTL version 2.0
musb-hdrc: setup fifo_mode 4
musb-hdrc: 28/31 max ep, 16384/16384 memory
USB Peripheral mode controller at 47401000 using PIO, IRQ 0
musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
musb-hdrc: MHDRC RTL version 2.0
musb-hdrc: setup fifo_mode 4
musb-hdrc: 28/31 max ep, 16384/16384 memory
USB Host mode controller at 47401800 using PIO, IRQ 0
Net:   CONFIG_DRIVER_TI_CPSW...<ethaddr> not set. Validating first E-fuse MAC
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
...

...
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
PHY reset timed out
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
...

...
wait_for_user_access Timeout
wait_for_user_access Timeout
CONFIG_USB_ETHER...
registering USB_ETHER OK.
cpsw, usb_ether
Hit any key to stop autoboot:  0
Execution bootcmd...
Card did not respond to voltage select!
mmc0(part 0) is current device
Card did not respond to voltage select!
Sans MMC, boot avec run netboot

Booting from E.D.E Labs network ...

wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
BOOTP broadcast 1
wait_for_user_access Timeout
wait_for_user_access Timeout
wait_for_user_access Timeout
...

You are probably right, there is an error message about "CONFIG_DRIVER_TI_CPSW...<ethaddr> not set".

BR

Didier

I made a mistake, the error/warning message appears BEFORE the kernel is started, and before it is loaded.

I've seen I can fixe this issue by undefining CONFIG_DRIVER_TI_CPSW Inside include/configs/am335x_evm.h

Another step has been done, but there is still another error which prevent kernel loading "ERROR: The remote end did not respond in time." in the final u-boot.

BR

Didier

I solved my previous problem by inserting a "sleep 2" command Inside the netboot environnement variable of u-boot, between the tftp command and the tftp command, in the include/configs/am335x_evm.h.

Both commands musn't be too close from eachother, otherwise the network connection is lost.

Now my next problem is to make the kernel booting fully (it hangs before the end, due to a bad NFS path or a problem establishing network connection for it.

I think my initial problem is now solved.

Thank you very much for all your help

BR

Didier