wl1271 timeout on AP and MR initialization

Hi Ken,

It looks to me that you are using an old version of the driver/firmware. The latest version is ol_R5.SP7.01 (Fw: 6.3.10.0.139)

Can you update to the latest and see if it works fine?

To update to the latest driver/firmware run this script: https://github.com/TI-ECS/build-utilites/blob/master/wl12xx_build.sh (please modify gen_tag to ol_R5.SP7.01).

Regards,
Gigi Joseph.

Joseph,

This sounds like a great suggestion but I am having a bit of trouble making it work.  Here's what I did:

(1)  git clone https://github.com/TI-ECS/build-utilites.git

(2)  Configured setup-env for my environment.

(3)  Edited wl12xx_build.sh file, changed line:

 declare -A gen_tag="ol_R5.SP5.01"

to

declare -A gen_tag="ol_R5.SP7.01"

(4)  Used the wl12xx_build.sh script to download, build, and install all utilities/modules except wl12xx_modules (because I have a working driver, just need to update the firmware).  The script would not allow me to install just the firmware without building some or most of the other utilities/modules.

(5)  Looked at the resulting firmware files in $rootfs/lib/firmware/ti-connectivity.  Here is what I found:

 Android.mk
 wl1271-fw-multirole-plt.bin
 LICENCE
 wl1271-nvs.bin
 wl1271-fw-multirole-roc.bin
 wl127x-fw-4-mr.bin
 wl127x-fw-4-plt.bin
 wl127x-fw-4-sr.bin
 wl128x-fw-4-mr.bin
 wl128x-fw-4-plt.bin
 wl128x-fw-multirole-plt.bin
 wl128x-fw-4-sr.bin
 wl128x-fw-multirole-roc.bin
 ini_files/

...But my driver is looking to load wl127x-fw-5-sr.bin and wl127x-fw-5-mr.bin.  Those files do not appear in the firmware directory.

Am I doing something wrong here?

Thanks.

--ken

Hi Ken,

You won't be able to update only the firmware - Its a combination.
One driver works with one firmware version. The latest fw name is "wl127x-fw-4-*.bin".

You would need to update the wl12xx drivers as well.
Your steps are all correct.

Regards,
Gigi Joseph.

Joseph,

Ok, I was afraid you were going to tell me that.  I have tried building the driver ("wl12xx_build.sh wl12xx_modules build") and I have been running into build problems (see attached build log).  I will continue to try to work through the issues to get the modules to build; if you have any suggestions please let me know.

In the meantime, maybe you can help me out on this: Is there any information (documentation, etc.) that can be obtained regarding the following:

(1)  What is the difference between the firmware versions (e.g. you said I am using an old version of firmware, but wl127x-fw-5-xx.bin seems to be more advanced than wl127x-fw-4-xx.bin, but apparently this is a bad assumption.  Is there a guide for this somewhere?

(2)  Is there a guide or some kind of documentation on configuring and tuning the firmware using expert mode?  I know the calibrator utility is somewhat self-documenting, but for instance, what are the bits of the plt tx_bip command used for and what is the best way to configure them?  So far I have only been guided by the very terse information in the ti-utils/README file, trial and error experience, and by a few examples that I was able to find on the net.

Thanks.

--ken

Joseph,

Is there a minimum linux kernel version for this version (ol_R5.SP7.01n) of the driver to build properly?  I thought that the compat-wireless project adapts to the version of the linux kernel that it is configured to build with?  In my case that is 3.8.13.

The reason I ask is that in order to get the driver to build so far I had to patch my kernel in three places.

Now I have run into the following error, which does not seem to be dependent on my kernel, but maybe something in the compat-wireless configuration?  Do you know what this is (CONFIG_COMPAT_MAC80211_RC_DEFAULT) and how to configure for it?

 ...

CC [M]  /home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/work.o
  CC [M]  /home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/iface.o
  CC [M]  /home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/rate.o
/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/rate.c:27:42: error: ‘CONFIG_COMPAT_MAC80211_RC_DEFAULT’ undeclared here (not in a function)
scripts/Makefile.build:307: recipe for target '/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/rate.o' failed
make[3]: *** [/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211/rate.o] Error 1
scripts/Makefile.build:454: recipe for target '/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211' failed
make[2]: *** [/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless/net/mac80211] Error 2
Makefile:1222: recipe for target '_module_/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless' failed
make[1]: *** [_module_/home/ken/work/blelink/BeagleBone_Black/tmp/compat-wireless] Error 2
make[1]: Leaving directory '/home/ken/work/blelink/software/antgateway/cape-bone-antgw-kernel-3.8.13-bone64/bb-kernel-3.8.13-bone64/KERNEL'
Makefile:66: recipe for target 'modules' failed
make: *** [modules] Error 2

Thanks for any help you can provide.

--ken

Hi Ken,

Actually the drivers were not tested on 3.8 kernel. The compat-wireless will not work as is on 3.8 kernel. I'm sorry, I overlooked your first post regarding the kernel version.

Can you please tell me from where you got the previous driver/firmware versions?

Regards,
Gigi Joseph.

Joseph,

The driver was already integrated into the kernel that I started with, custom configured for the BeagleBone platform by Robert Nelson.   It is at:

https://github.com/beagleboard/linux/releases/tag/3.8.13-bone64

I don't know what version the driver is; I will try to determine that.

Meanwhile, is there any documentation available regarding expert mode of calibration (see my earlier post)?

Thanks.

--ken

Joseph,

I looked through the source code of the driver that is integrated into my kernel, and there is no tag or other indication that I have been able find that identifies what the driver version is.  The closest I was able to find regarding this is in file wl12xx.h which includes the following lines:

/* minimum FW required for driver for wl127x */
#define WL127X_CHIP_VER         6
//#define WL127X_IFTYPE_VER     3
#define WL127X_IFTYPE_VER       WLCORE_FW_VER_IGNORE
#define WL127X_MAJOR_VER        10
//#define WL127X_SUBTYPE_VER    2
#define WL127X_SUBTYPE_VER      WLCORE_FW_VER_IGNORE
#define WL127X_MINOR_VER        115

Aside from my questions from my 10/16 post, I now have these questions:

(1)  You said that the driver was not tested on the 3.8 kernel.  Is there a newer version of the kernel that the driver is compatible with?  If not, then what is the next older version of the kernel that the driver can be built against and is known to work with?

(2)  My driver is trying to load the wl127x-fw-5-xx.bin firmware, but apparently it needs to use the wl127x-fw-4-xx.bin firmware, is that correct?  Please explain what is the basic difference between those firmware versions.  Is it possible that I can modify my driver to load the correct firmware and fix this issue?

Thanks.

--ken

Hi Ken,

(1) No - the latest kernel is 3.2
(2) The fw name does not matter - the version does. The one you are using is 6.3.10.0.133 - the latest is 6.3.10.0.139. You fw version is not a multirole fw. 

Can you send me:
(1) Full driver logs - to enable driver logs, please see: http://processors.wiki.ti.com/index.php/WL12xx_NLCP_Driver_Debug
(2) The exact sequence of commands that you execute for (a) AP & (b) Multirole.

Regards,
Gigi Joseph.

Joseph,

The full driver logs for both cases are in the zip file attached to my first post (on 10/15/14).

To start up the multirole scenario, I did the following:

(1)  First I configured the system to operate in station mode.  My /etc/network/interfaces file looks like this:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eth0
#iface eth0 inet dhcp
# Example to keep MAC address between reboots
#hwaddress ether DE:AD:BE:EF:CA:FE

# The secondary network interface
#auto eth1
#iface eth1 inet dhcp

# WiFi interface
allow-hotplug wlan0
iface wlan0 inet manual
    wpa-roam /etc/wpa_supplicant/wpa_roam.conf

iface default inet dhcp
iface home inet dhcp
iface work inet dhcp

# Ethernet/RNDIS gadget (g_ether)
# ... or on host side, usbnet and random hwaddr
# Note on some boards, usb0 is automaticly setup with an init script
iface usb0 inet static
    address 192.168.7.2
    netmask 255.255.255.0
    network 192.168.7.0
    gateway 192.168.7.1

...and I have a /etc/wpa_supplicant/wpa_roam.conf file with appropriate parameters and credentials configured.

(2)  Next I set up the following two files:

----- /etc/hostapd.conf -----

interface=wlan1
driver=nl80211
channel=11
hw_mode=g
preamble=1
dtim_period=2
beacon_int=100
logger_syslog=-1
logger_syslog_level=2
logger_stdout=-1
logger_stdout_level=2
dump_file=/tmp/hostapd.dump
ctrl_interface=/var/run/hostapd
ctrl_interface_group=0
supported_rates=60 90 120 180 240 360 480 540
basic_rates=60 90 120 180 240
ssid=BBB_AP
max_num_sta=5
macaddr_acl=0
auth_algs=3
ieee80211d=0
uapsd_advertisement_enabled=1
wep_rekey_period=0
own_ip_addr=127.0.0.1
wpa_group_rekey=0
wpa_strict_rekey=0
wpa_gmk_rekey=0
wpa_ptk_rekey=0
#ap_table_max_size=255
#ap_table_expiration_time=60
eap_server=1
disassoc_low_ack=1
ap_max_inactivity=4294967295

----- /etc/udhcpd.conf -----

# start      192.168.7.1
# end        192.168.7.1
# interface  usb0
# max_leases 1
# option subnet 255.255.255.252

# The start and end of the IP lease block
start           192.168.2.20
end             192.168.2.254
# The interface that udhcpd will use
interface   wlan1

#Examples
opt     dns     8.8.8.8  8.8.4.4 # public google dns servers
option  subnet  255.255.255.0
opt     router  192.168.2.1
option  lease   864000          # 10 days

(3)  Then I executed the following two commands:

iw phy0 interface add wlan1 type managed
ifconfig wlan1 192.168.2.1 netmask 255.255.255.0 up

The first command causes the desired effect of adding the wlan1 interface.

The second command never returns.

I was able to capture a dmesg log on a second console.

For AP mode, I essentially started with the same configuration, but I made the following changes:

(1)  I disabled station mode by commenting out the lines referring to the wlan0 interface in the /etc/network/interfaces file, and then rebooted.

(2)  In the /etc/hostapd.conf file, I changed this line:

interface=wlan1

to this:

interface=wlan0

(3)  I then executed the following command:

hostapd -B /etc/hostapd.conf -P /var/run/hostapd.pid

This command returns after a few seconds with the following messages:

Configuration file: /etc/hostapd.conf

Could not set interface wlan0 flags: Invalid argument
nl80211 driver initialization failed.
rmdir[ctrl_interface]: No such file or directory


I captured the dmesg log that is in the zip file.

--ken

Hi Ken,

Can you please use the attached scripts for starting STA & AP roles?
The sequence should be:

1 – start AP script
2 – add STA script

Regards,
Gigi Joseph.

http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/307/0247.sta_5F00_add.sh

http://e2e.ti.com/cfs-file.ashx/__key/communityserver-discussions-components-files/307/7028.ap_5F00_start.sh

Joseph,

I'm just now getting back to this problem; I had to attend to some higher priority issues.

I couldn't use your scripts as is because they are for an Android system, but I tried the same sequence of commands. It didn't help.

This is what I think is going on here: The wlan interface is able to boot up and manage to get associated with an AP and then transfer data without any apparent problems. The problem comes if the firmware must be booted up a second time. This occurs if the interface is taken down (e.g. ip link set dev wlan0 down) or restarted, and it even occurs if the firmware must be reloaded due to a role shift (e.g switch from station role to AP role, or from single role to multi-role).

See the dmesg logs (dmesg_wlan0_debug and dmesg_wlan0_down_up) in the attached zipfile. In both cases all debug tracing in the driver is turned on. The dmesg_wlan0_debug log shows the sequence when the driver is booted up the first time; everything works. The dmesg_wlan0_down_up log shows the sequence that occurs when first the 'ip link set wlan0 down' command is issued (at time 597.366796). Then the 'ip link set wlan0 up' command is issued (at time 681.529384). In both up and down cases, the driver fails to (re)boot the firmware; it fails with "timeout waiting for the hardware to complete initialization".

It seems like the firmware is getting loaded properly, and it seems to boot, but apparently it fails to signal initialization complete. Or maybe the firmware is signalling properly, but the driver is in a state in which it fails to get the completion interrupt. I checked the value of the interrupt vector bits that are returned by the firmware, and they have a value of 0x1, which is a hardware watchdog timeout. Maybe the watchdog interrupt is preventing the initialization complete interrupt from registering(?).

I'm convinced that this is the reason that the multirole operation is not working, because the firmware must be reloaded in order to support that mode, and the reload fails. In the case of AP mode, I was able to get that to work if I prevent the device from booting up in station mode first.

Are you aware of any problem with the driver similar to this that has been solved, or from the evidence in the logs (e.g. something missing, out of sequence, etc.) can you maybe determine what might be going wrong here?

Thanks for any clues you can provide.

--ken

dmesg_boot_logs.zip

Joseph,

I am still unable to resolve this issue.

Here is the situation at the moment:

I am using the second method that you have referenced above (registering the set_power functions), because the 1.8V power is not dedicated to the WL1271 device and so it is enabled at boot time and then left enabled from that point on.

The implementation of the driver in my linux source tree uses the configurable function pointers init() and reset() found in struct wl1271_if_operations in platform_data.  These function pointers are now configured by boot-up code to point to simple functions that assert and de-assert respectively the WLAN_ENA signal to the WL1271 device.

So, now the sequence of events at boot-up is as follows:

(1)  1.8 V power is applied to the WL1271 device.

(2)  100 ms after the 1.8 V power is applied, 3.2 V to 1.8 V level shifters are enabled, which enables 1.8 V signal levels to the SDIO and SLEEP_CLK signal pins of the WL1271 device.  Previously these pins are floating.  A 32.768 KHz clock signal is applied to the SLEEP_CLK pin at that time.

(3)  100 ms after the level shifters are enabled, the CD control to the mmc1 interface is asserted true, which causes the mmc1 SDIO port to initialize.

(4)  100 ms after the CD control is asserted true, the WLAN_ENA signal is asserted (driven high).

(5)  At this point, if the function pointers init() and reset() are left NULL so that these functions are disabled, then the WL1271 device completes initialization and the driver operates correctly.  However, if these function pointers had previously been initialized to point to the init() and reset() functions, then the following happens:

(6)  During the WL1271 initialization sequence, the reset() function is called.  This function simply de-asserts the WLAN_ENA signal (drives it low), and then delays for 100 ms.

(7)  Immediately after the reset() function is called, the init() function is called.  This function simply asserts the WLAN_ENA signal (drives it high), and then delays for 70 ms.

(8)  Subsequently, the second call to wl12xx_sdio_raw_write() fails with a timeout error (-110).  Here is a snippet of the dmesg log that includes the failure messages:

[    0.814680] initialize_si5351b() :   device status = 0x11
[    0.814693] initialize_si5351b() :   output enable = 0xcb
[    0.814702] initialize_si5351b() : OEBn pin enable = 0xff
[    0.814710] initialize_si5351b() :    CLK2 control = 0x6e
[    0.814719] initialize_si5351b() :    CLK4 control = 0x4d
[    0.814727] initialize_si5351b() :    CLK5 control = 0x6e
[    0.814735] initialize_si5351b() :    crystal load = 0xd2
[    0.814743] initialize_si5351b() :   fanout enable = 0xd0
[    0.814756] enable_wlan_bt_interface() : enabling v180_supply and v260_supply
[    0.914710] enable_wlan_bt_interface() : enabling level shifters
[    1.014516] enable_wlan() : enabling SDIO interface
[    1.114312] enable_wlan() : enabling WLAN interface
[    1.214124] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6
[    1.214135] davinci_mdio 4a101000.mdio: detected phy mask fffffffe
[    1.214881] of_get_named_gpio_flags exited with status 116
[    1.214893] of_get_named_gpio_flags: can't parse gpios property
[    1.214901] of_get_named_gpio_flags: can't parse gpios property
[    1.214920] omap_hsmmc mmc.19: of_parse_phandle_with_args of 'reset' failed
[    1.222243] omap_hsmmc mmc.19: Failed to get rstctl; not using any
[    1.222560] edma-dma-engine edma-dma-engine.0: allocated channel for 0:33
[    1.222613] edma-dma-engine edma-dma-engine.0: allocated channel for 0:32
[    1.222759] mmc.19 supply vmmc_aux not found, using dummy regulator
[    1.223096] omap_hsmmc mmc.19: pins are not configured from the driver
...
[    1.261071] gpio-rctrl rstctl.4: gpio_rctrl_request eMMC_RSTn
[    1.261102] omap_hsmmc mmc.20: Got rstctl (gpio:#0 name eMMC_RSTn) label:eMMC_RSTn
[    1.261114] gpio-rctrl rstctl.4: gpio_rctrl_deassert eMMC_RSTn
[    1.261541] edma-dma-engine edma-dma-engine.0: allocated channel for 0:3
[    1.261653] edma-dma-engine edma-dma-engine.0: allocated channel for 0:2
[    1.262017] mmc1: card claims to support voltages below the defined range. These will be ignored.
[    1.262113] mmc.20 supply vmmc_aux not found, using dummy regulator
[    1.262185] omap_hsmmc mmc.20: pins are not configured from the driver
[    1.273923] mmc1: queuing unknown CIS tuple 0x91 (3 bytes)
[    1.275283] mmc1: new SDIO card at address 0001
[    1.288692] bone-capemgr bone_capemgr.9: slot #4: Applied #2 overlays.
[    1.288716] bone-capemgr bone_capemgr.9: loader: done slot-4 BB-BONE-EMMC-2G:00A0 (prio 1)
[    1.324627] udevd[100]: starting version 175
[    1.343327] mmc2: BKOPS_EN bit is not set
[    1.347231] mmc2: new high speed MMC card at address 0001
[    1.355073] mmcblk0: mmc2:0001 MMC04G 3.60 GiB
[    1.358996] mmcblk0boot0: mmc2:0001 MMC04G partition 1 2.00 MiB
[    1.360883] mmcblk0boot1: mmc2:0001 MMC04G partition 2 2.00 MiB
[    1.362094]  mmcblk0: p1 p2
[    1.367237]  mmcblk0boot1: unknown partition table
[    1.368757]  mmcblk0boot0: unknown partition table
[    2.898040] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[    3.969213] udevd[271]: starting version 175
[    4.891234] EXT4-fs (mmcblk0p2): re-mounted. Opts: errors=remount-ro
...
[   15.029400] cfg80211: Calling CRDA to update world regulatory domain
[   15.241725] wl12xx_sdio_power_on()
[   15.456593] wl12xx_reset()
[   15.556532] wl12xx_init()
[   15.626412] wlcore: mem_start 00040000 mem_size 00014FC0
[   15.626421] wlcore: reg_start 00300000 reg_size 00008800
[   15.626428] wlcore: mem2_start 00000000 mem2_size 00000000
[   15.626435] wlcore: mem3_start 00000000 mem3_size 00000000
[   15.695035] ------------[ cut here ]------------
[   15.695083] WARNING: at drivers/net/wireless/ti/wlcore/sdio.c:144 wl12xx_sdio_raw_write+0x6c/0xc4 [wlcore_sdio]()
[   15.695091] Modules linked in: wl12xx wlcore mac80211 cfg80211 rfkill g_multi libcomposite wlcore_sdio uio_pruss omap_rng
[   15.695163] [<c001125d>] (unwind_backtrace+0x1/0x98) from [<c002ee73>] (warn_slowpath_common+0x33/0x48)
[   15.695179] [<c002ee73>] (warn_slowpath_common+0x33/0x48) from [<c002ee99>] (warn_slowpath_null+0x11/0x14)
[   15.695196] [<c002ee99>] (warn_slowpath_null+0x11/0x14) from [<bf8081c9>] (wl12xx_sdio_raw_write+0x6c/0xc4 [wlcore_sdio])
[   15.695298] [<bf8081c9>] (wl12xx_sdio_raw_write+0x6c/0xc4 [wlcore_sdio]) from [<bf9491df>] (wlcore_set_partition+0x8e/0x2a0 [wlcore])
[   15.695353] [<bf9491df>] (wlcore_set_partition+0x8e/0x2a0 [wlcore]) from [<bf942329>] (wl12xx_set_power_on+0x48/0xb0 [wlcore])
[   15.695394] [<bf942329>] (wl12xx_set_power_on+0x48/0xb0 [wlcore]) from [<bf9461d9>] (wlcore_nvs_cb+0xe8/0x63c [wlcore])
[   15.695427] [<bf9461d9>] (wlcore_nvs_cb+0xe8/0x63c [wlcore]) from [<c02acbff>] (request_firmware_work_func+0x47/0x78)

[   15.695449] [<c02acbff>] (request_firmware_work_func+0x47/0x78) from [<c0041687>] (process_one_work+0xd7/0x310)
[   15.695466] [<c0041687>] (process_one_work+0xd7/0x310) from [<c0041b13>] (worker_thread+0x107/0x328)
[   15.695486] [<c0041b13>] (worker_thread+0x107/0x328) from [<c004484f>] (kthread+0x6b/0x74)
[   15.695507] [<c004484f>] (kthread+0x6b/0x74) from [<c000c7fd>] (ret_from_fork+0x11/0x34)
[   15.695517] ---[ end trace 07aac9cc0a9b8c49 ]---
[   15.695534] wl1271_sdio mmc1:0001:2: sdio write failed (-110)
[   15.701632] wlcore: ret = -110
[   15.701632]
[   15.701644] wl12xx_debug()
[   15.702664] wl12xx_sdio_power_off()
[   15.714337] wlcore: ERROR couldn't get hw info
...
[   21.168328] net eth0: phy 4a101000.mdio:01 not found on slave 1

I tried several variations on this sequence and also tried different delay values in steps (6) and (7) of the sequence.

I verified that the WLAN_ENA signal is in fact being controlled as described above at the WL1271 WLAN_ENA pin.  I have attached a scope image showing the failure scenario (see attached file wlan_ena.png).

In the scope trace you can see the various signals at the WL1271 device (on the 3.2 V side of the level shifters).  The yellow trace is the WLAN_ENA signal at the pin of the WL1271 device.  The blip on signal CELL_ENA at 80% (400 ms) into the trace is a debug signal that I inserted at the point of failure.  This coincides with the "wl12xx_debug()" message in the log above.  You can see that the WLAN_ENA signal was de-asserted for 100 ms and then re-asserted about 150 ms prior to that point.

I am completely baffled by this behavior, and I cannot understand why the device is behaving this way.  Please let me know if you can suggest any other things to try.  I have a demo of the system scheduled for one week from now, and I really need to get this issue resolved.

Thanks!

--ken

Joseph,

I forgot to mention, another thing I tried is turning off the power to the WL1271 device in the reset() function, and re-enabling the power in the init() function.

That doesn't work either. This makes me believe that maybe there is something else going on here, like maybe some issue with the mmc driver.

I've attached a scope trace showing this scenario below.

The WLAN_BT_ENA signal is the enable for the 3.2 V / 1.8 V level shifters (high = enabled). The V180_V260 signal is the enable for the 1.8 V and 2.6 V regulators (low = enabled), the 1.8 V regulator supplies power to the WL1271 device.

--ken