Customer is using Murata's Type WT module containing TI's WL 1831 chipset. The issue is observed on the field by the customer in 2 modules out of 2500. Please find the attached PDF to find the waveform captured for the power sequence. Also attached the logs with schematics.
Observations:
1. In both working and non-working board WL_EN is made high before clock stabilization.
2. Non working board and the working board almost follows the same power up sequence but the WLAN_IRQ lines are always held high in the non working board.
3. In the customer schematic, I could see the WL_EN and WLAN_IRQ are pulled high.
Questions:
1. If the power sequenced is not right, why only 2 boards are not working and how the remaining boards are working fine?
2. If there is a hardware fault, how the non working boards were discoverable as AP in the first place? (but not able to connect to it)
3. Could you please help to verify the schematic?
Request your help to guide us in the right direction to find the root cause of this issue.
wlcore: PHY firmware version: Rev 8.2.0.0.242 wlcore: firmware booted (Rev 8.9.0.0.79) IPv6: ADDRCONF(NETDEV_UP): wlanap1: link is not ready wlcore: ERROR ELP wakeup timeout! ------------[ cut here ]------------ WARNING: CPU: 0 PID: 403 at /kernel-source/drivers/net/wireless/ti/wlcore/main.c:796 wl12xx_queue_recovery_work.part.9+0x58/0x5c() Modules linked in: evbug CPU: 0 PID: 403 Comm: crda Not tainted 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<80362f54>] (wl12xx_queue_recovery_work.part.9+0x58/0x5c) [<80362f54>] (wl12xx_queue_recovery_work.part.9) from [<8036e75c>] (wl1271_ps_elp_wakeup+0x1f4/0x1fc) [<8036e75c>] (wl1271_ps_elp_wakeup) from [<8036320c>] (wlcore_regdomain_config.part.11+0x28/0x6c) [<8036320c>] (wlcore_regdomain_config.part.11) from [<80590328>] (set_regdom+0x100/0x5a8) [<80590328>] (set_regdom) from [<8059bd50>] (nl80211_set_reg+0x128/0x278) [<8059bd50>] (nl80211_set_reg) from [<804a6100>] (genl_rcv_msg+0x234/0x3a8) [<804a6100>] (genl_rcv_msg) from [<804a54f0>] (netlink_rcv_skb+0xb4/0xd8) [<804a54f0>] (netlink_rcv_skb) from [<804a5ebc>] (genl_rcv+0x24/0x34) [<804a5ebc>] (genl_rcv) from [<804a4e78>] (netlink_unicast+0x154/0x204) [<804a4e78>] (netlink_unicast) from [<804a52d4>] (netlink_sendmsg+0x2e4/0x348) [<804a52d4>] (netlink_sendmsg) from [<804658e0>] (sock_sendmsg+0x14/0x24) [<804658e0>] (sock_sendmsg) from [<804660b0>] (___sys_sendmsg+0x1cc/0x1d4) [<804660b0>] (___sys_sendmsg) from [<80466d5c>] (__sys_sendmsg+0x3c/0x68) [<80466d5c>] (__sys_sendmsg) from [<8000f3c0>] (ret_fast_syscall+0x0/0x3c) ---[ end trace ff73c937079b39c1 ]--- wlcore: Hardware recovery in progress. FW ver: Rev 8.9.0.0.79 cfg80211: Regulatory domain changed to country: IN cfg80211: DFS Master region: JP cfg80211: (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time) ------------[ cut here ]------------ WARNING: CPU: 0 PID: 6 at /kernel-source/drivers/net/wireless/ti/wlcore/io.h:66 wlcore_set_partition+0xa0/0x4c0() cfg80211: (2402000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A) cfg80211: (5170000 KHz - 5250000 KHz @ 80000 KHz, 160000 KHz AUTO), (N/A, 2000 mBm), (N/A) Modules linked in: evbug CPU: 0 PID: 6 Comm: kworker/u2:0 Tainted: G W 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) Workqueue: phy0 wl1271_recovery_work [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<8036aa04>] (wlcore_set_partition+0xa0/0x4c0) [<8036aa04>] (wlcore_set_partition) from [<8036248c>] (wl1271_recovery_work+0x6c/0x328) [<8036248c>] (wl1271_recovery_work) from [<8003ebec>] (process_one_work+0x118/0x3e4) [<8003ebec>] (process_one_work) from [<8003ef04>] (worker_thread+0x4c/0x4f4) [<8003ef04>] (worker_thread) from [<80043e44>] (kthread+0xdc/0xf4) [<80043e44>] (kthread) from [<8000f468>] (ret_from_fork+0x14/0x2c) cfg80211: (5250000 KHz - 5330000 KHz @ 80000 KHz, 160000 KHz AUTO), (N/A, 2000 mBm), (0 s) cfg80211: (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A) ---[ end trace ff73c937079b39c2 ]--- wlcore: down ieee80211 phy0: Hardware restart was requested wlcore: PHY firmware version: Rev 8.2.0.0.242 wlcore: firmware booted (Rev 8.9.0.0.79) IPv6: ADDRCONF(NETDEV_CHANGE): wlanap1: link becomes ready wlcore: ERROR ELP wakeup timeout! ------------[ cut here ]------------ WARNING: CPU: 0 PID: 404 at /kernel-source/drivers/net/wireless/ti/wlcore/main.c:796 wl12xx_queue_recovery_work.part.9+0x58/0x5c() Modules linked in: evbug CPU: 0 PID: 404 Comm: hostapd Tainted: G W 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<80362f54>] (wl12xx_queue_recovery_work.part.9+0x58/0x5c) [<80362f54>] (wl12xx_queue_recovery_work.part.9) from [<8036e75c>] (wl1271_ps_elp_wakeup+0x1f4/0x1fc) [<8036e75c>] (wl1271_ps_elp_wakeup) from [<8035fa30>] (wlcore_op_set_key+0x64/0x11c) [<8035fa30>] (wlcore_op_set_key) from [<805d3b98>] (ieee80211_key_enable_hw_accel+0xfc/0x22c) [<805d3b98>] (ieee80211_key_enable_hw_accel) from [<805d4430>] (ieee80211_key_link+0xe0/0xf8) [<805d4430>] (ieee80211_key_link) from [<805c8620>] (ieee80211_add_key+0xec/0x214) [<805c8620>] (ieee80211_add_key) from [<805a0bbc>] (nl80211_new_key+0x110/0x178) [<805a0bbc>] (nl80211_new_key) from [<804a6100>] (genl_rcv_msg+0x234/0x3a8) [<804a6100>] (genl_rcv_msg) from [<804a54f0>] (netlink_rcv_skb+0xb4/0xd8) [<804a54f0>] (netlink_rcv_skb) from [<804a5ebc>] (genl_rcv+0x24/0x34) [<804a5ebc>] (genl_rcv) from [<804a4e78>] (netlink_unicast+0x154/0x204) [<804a4e78>] (netlink_unicast) from [<804a52d4>] (netlink_sendmsg+0x2e4/0x348) [<804a52d4>] (netlink_sendmsg) from [<804658e0>] (sock_sendmsg+0x14/0x24) [<804658e0>] (sock_sendmsg) from [<804660b0>] (___sys_sendmsg+0x1cc/0x1d4) [<804660b0>] (___sys_sendmsg) from [<80466d5c>] (__sys_sendmsg+0x3c/0x68) [<80466d5c>] (__sys_sendmsg) from [<8000f3c0>] (ret_fast_syscall+0x0/0x3c) ---[ end trace ff73c937079b39c3 ]--- wlcore: Hardware recovery in progress. FW ver: Rev 8.9.0.0.79 wlanap1: failed to set key (1, ff:ff:ff:ff:ff:ff) to hardware (-110) ------------[ cut here ]------------ WARNING: CPU: 0 PID: 6 at /kernel-source/drivers/net/wireless/ti/wlcore/io.h:66 wlcore_set_partition+0xa0/0x4c0() Modules linked in: evbug CPU: 0 PID: 6 Comm: kworker/u2:0 Tainted: G W 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) Workqueue: phy0 wl1271_recovery_work [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<8036aa04>] (wlcore_set_partition+0xa0/0x4c0) [<8036aa04>] (wlcore_set_partition) from [<8036248c>] (wl1271_recovery_work+0x6c/0x328) [<8036248c>] (wl1271_recovery_work) from [<8003ebec>] (process_one_work+0x118/0x3e4) [<8003ebec>] (process_one_work) from [<8003ef04>] (worker_thread+0x4c/0x4f4) [<8003ef04>] (worker_thread) from [<80043e44>] (kthread+0xdc/0xf4) [<80043e44>] (kthread) from [<8000f468>] (ret_from_fork+0x14/0x2c) ---[ end trace ff73c937079b39c4 ]--- wlcore: down ieee80211 phy0: Hardware restart was requested wlcore: PHY firmware version: Rev 8.2.0.0.242 wlcore: firmware booted (Rev 8.9.0.0.79) evbug: Event. Dev: input1, Type: 1, Code: 257, Value: 0 evbug: Event. Dev: input1, Type: 0, Code: 0, Value: 0 evbug: Event. Dev: input1, Type: 1, Code: 257, Value: 1 evbug: Event. Dev: input1, Type: 0, Code: 0, Value: 0 usb 1-1: new high-speed USB device number 4 using ci_hdrc cdc_ether 1-1:1.10 usb0: register 'cdc_ether' at usb-ci_hdrc.1-1, CDC Ethernet Device, fa:96:11:12:13:14 cdc_ether 1-1:1.12 usb1: register 'cdc_ether' at usb-ci_hdrc.1-1, CDC Ethernet Device, fa:96:11:12:13:16 cdc_acm 1-1:1.0: ttyACM0: USB ACM device cdc_acm 1-1:1.2: ttyACM1: USB ACM device cdc_acm 1-1:1.4: ttyACM2: USB ACM device cdc_acm 1-1:1.6: ttyACM3: USB ACM device cdc_acm 1-1:1.8: ttyACM4: USB ACM device usbcore: registered new interface driver cdc_acm cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters wlcore: ERROR ELP wakeup timeout! ------------[ cut here ]------------ WARNING: CPU: 0 PID: 404 at /kernel-source/drivers/net/wireless/ti/wlcore/main.c:796 wl12xx_queue_recovery_work.part.9+0x58/0x5c() Modules linked in: cdc_acm evbug CPU: 0 PID: 404 Comm: hostapd Tainted: G W 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<80362f54>] (wl12xx_queue_recovery_work.part.9+0x58/0x5c) [<80362f54>] (wl12xx_queue_recovery_work.part.9) from [<8036e75c>] (wl1271_ps_elp_wakeup+0x1f4/0x1fc) [<8036e75c>] (wl1271_ps_elp_wakeup) from [<8035fa30>] (wlcore_op_set_key+0x64/0x11c) [<8035fa30>] (wlcore_op_set_key) from [<805d3b98>] (ieee80211_key_enable_hw_accel+0xfc/0x22c) [<805d3b98>] (ieee80211_key_enable_hw_accel) from [<805d4430>] (ieee80211_key_link+0xe0/0xf8) [<805d4430>] (ieee80211_key_link) from [<805c8620>] (ieee80211_add_key+0xec/0x214) [<805c8620>] (ieee80211_add_key) from [<805a0bbc>] (nl80211_new_key+0x110/0x178) [<805a0bbc>] (nl80211_new_key) from [<804a6100>] (genl_rcv_msg+0x234/0x3a8) [<804a6100>] (genl_rcv_msg) from [<804a54f0>] (netlink_rcv_skb+0xb4/0xd8) [<804a54f0>] (netlink_rcv_skb) from [<804a5ebc>] (genl_rcv+0x24/0x34) [<804a5ebc>] (genl_rcv) from [<804a4e78>] (netlink_unicast+0x154/0x204) [<804a4e78>] (netlink_unicast) from [<804a52d4>] (netlink_sendmsg+0x2e4/0x348) [<804a52d4>] (netlink_sendmsg) from [<804658e0>] (sock_sendmsg+0x14/0x24) [<804658e0>] (sock_sendmsg) from [<804660b0>] (___sys_sendmsg+0x1cc/0x1d4) [<804660b0>] (___sys_sendmsg) from [<80466d5c>] (__sys_sendmsg+0x3c/0x68) [<80466d5c>] (__sys_sendmsg) from [<8000f3c0>] (ret_fast_syscall+0x0/0x3c) ---[ end trace ff73c937079b39c5 ]--- wlcore: Hardware recovery in progress. FW ver: Rev 8.9.0.0.79 wlanap1: failed to set key (2, ff:ff:ff:ff:ff:ff) to hardware (-110) ------------[ cut here ]------------ WARNING: CPU: 0 PID: 690 at /kernel-source/drivers/net/wireless/ti/wlcore/io.h:66 wlcore_set_partition+0xa0/0x4c0() Modules linked in: cdc_acm evbug CPU: 0 PID: 690 Comm: kworker/u2:0 Tainted: G W 4.1.15-SML-20200120+g77f6154 #1 Hardware name: Freescale i.MX6 Ultralite (Device Tree) Workqueue: phy0 wl1271_recovery_work [<80015aa0>] (unwind_backtrace) from [<80012664>] (show_stack+0x10/0x14) [<80012664>] (show_stack) from [<805ed99c>] (dump_stack+0x90/0xd0) [<805ed99c>] (dump_stack) from [<8002b95c>] (warn_slowpath_common+0x80/0xb0) [<8002b95c>] (warn_slowpath_common) from [<8002ba28>] (warn_slowpath_null+0x1c/0x24) [<8002ba28>] (warn_slowpath_null) from [<8036aa04>] (wlcore_set_partition+0xa0/0x4c0) [<8036aa04>] (wlcore_set_partition) from [<8036248c>] (wl1271_recovery_work+0x6c/0x328) [<8036248c>] (wl1271_recovery_work) from [<8003ebec>] (process_one_work+0x118/0x3e4) [<8003ebec>] (process_one_work) from [<8003ef04>] (worker_thread+0x4c/0x4f4) [<8003ef04>] (worker_thread) from [<80043e44>] (kthread+0xdc/0xf4) [<80043e44>] (kthread) from [<8000f468>] (ret_from_fork+0x14/0x2c) ---[ end trace ff73c937079b39c6 ]--- wlcore: down ieee80211 phy0: Hardware restart was requested wlcore: PHY firmware version: Rev 8.2.0.0.242 wlcore: firmware booted (Rev 8.9.0.0.79)