Other Parts Discussed in Thread: AM6421, DP83869, DP83869HM
I'm having the same issue as [1] on am64 EVM with my own built image using kernel from [2] with tag tag: 09.00.00.005-rt very close to the kernel in ti-processor-sdk-linux-rt-am64xx-evm-09.00.00.03. I have a a single device tree that sets icssg for DUAL emac mode.
What was the resolution for TI ticket 1158425?
Seems like a race condition when the driver comes up in DUAL emac mode with RX descriptor allocation?
I would attempt to reproduce with AM64EVM and ti-processor-sdk-linux-rt-am64xx-evm-09.00.00.03 image, but I can't get into dual emac mode when using the the command mentioned here [3]
setenv bootcmd 'run findfdt; run envboot; run init_${boot}; run get_kern_${boot}; run get_fdt_${boot}; setenv name_overlays k3-am642-evm-icssg1-dualemac.dtbo; run get_overlay_${boot}; run run_kern'
[2] git.ti.com/.../ti-linux-kernel.git
Here is a simple bash script I use to recreate the problem (tune sleep for how long it takes system to reboot):
#!/bin/bash while [ true ] ; do ping -c 1 -w 1 192.168.0.1 > /dev/null 2>&1 a=$? if [ $a == 0 ] then echo "target reboot" ssh -i ~/key root@192.168.0.1 reboot > /dev/null 2>&1 sleep 30 else echo "Not answering pings!" exit fi done
Here is a kernel crash dump when I take down eth0 when in the "hung" state:
~ # ifconfig eth1 down
[ 600.643551] icssg-prueth icssg1-eth eth1: Link is Down
[ 600.644774] remoteproc remoteproc6: stopped remote processor 3008a000.txpru
[ 600.644795] remoteproc remoteproc5: stopped remote processor 30084000.rtu
[ 600.644806] remoteproc remoteproc4: stopped remote processor 300b4000.pru
[ 600.655987] ----------{}[ cut here ]{}----------
[ 600.656001] k3_knav_desc_pool size 65536 != avail 0
[ 600.656069] WARNING: CPU: 0 PID: 507 at drivers/net/ethernet/ti/k3-cppi-desc-pool.c:35 k3_cppi_desc_pool_destroy+0x90/0x98
[ 600.656104] CPU: 0 PID: 507 Comm: ifconfig Not tainted 5.10.158-rt64 #1
[ 600.656115] Hardware name: Sick SIG300 (DT)
[ 600.656121] pstate: 40000005 (nZcv daif PAN -UAO -TCO BTYPE=-)
[ 600.656129] pc : k3_cppi_desc_pool_destroy+0x90/0x98
[ 600.656139] lr : k3_cppi_desc_pool_destroy+0x90/0x98
[ 600.656148] sp : ffff800014f7ba50
[ 600.656151] x29: ffff800014f7ba50 x28: ffff000004683900
[ 600.656164] x27: ffff0000021b7600 x26: 0000000000000001
[ 600.656171] x25: 00000000000001d0 x24: ffff0000034a6c80
[ 600.656179] x23: ffff00000223c980 x22: 0000000000000001
[ 600.656187] x21: ffff00000223c000 x20: 0000000000010000
[ 600.656194] x19: ffff0000038fdb80 x18: 0000000000000000
[ 600.656202] x17: 0000000000000001 x16: 0000000000000019
[ 600.656209] x15: ffffffffffffffff x14: ffff800010c2ad98
[ 600.656217] x13: ffff800010f7fb20 x12: ffff800010f7fb1e
[ 600.656224] x11: ffff800014f7b9a0 x10: ffff800010e4c910
[ 600.656232] x9 : ffff800014f7ba50 x8 : ffff800010e44ae8
[ 600.656240] x7 : ffff800014f7b8a0 x6 : ffff800010e9cbf0
[ 600.656248] x5 : ffff800014f7b8a0 x4 : 00000000fffff16c
[ 600.656255] x3 : ffff800010e9ee10 x2 : 0000000100000000
[ 600.656262] x1 : 69084cd70e971900 x0 : 0000000000000000
[ 600.656272] Call trace:
[ 600.656276] k3_cppi_desc_pool_destroy+0x90/0x98
[ 600.656286] prueth_cleanup_rx_chns.isra.37+0x1c/0x38
[ 600.656300] emac_ndo_stop+0x218/0x2c8
[ 600.656309] __dev_close_many+0xac/0x130
[ 600.656318] __dev_change_flags+0xd4/0x208
[ 600.656330] dev_change_flags+0x24/0x68
[ 600.656338] devinet_ioctl+0x3f0/0x798
[ 600.656347] inet_ioctl+0x1fc/0x378
[ 600.656355] sock_do_ioctl+0x4c/0x2c8
[ 600.656363] sock_ioctl+0x310/0x4a8
[ 600.656369] __arm64_sys_ioctl+0xa8/0xe8
[ 600.656383] el0_svc_common.constprop.4+0x8c/0x180
[ 600.656396] do_el0_svc+0x24/0x90
[ 600.656404] el0_svc+0x14/0x20
[ 600.656415] el0_sync_handler+0x90/0xb8
[ 600.656424] el0_sync+0x160/0x180
[ 600.656433] -{}[ end trace 0000000000000002 ]{}-
[ 601.881295] ----------{}[ cut here ]{}----------
[ 601.881312] kernel BUG at lib/genalloc.c:254!
[ 602.881539] Internal error: Oops - BUG: 0 1 PREEMPT_RT SMP
[ 602.896223] CPU: 0 PID: 507 Comm: ifconfig Tainted: G W 5.10.158-rt64 #1
[ 602.904212] Hardware name: Sick SIG300 (DT)
[ 602.908383] pstate: 20000005 (nzCv daif PAN -UAO -TCO BTYPE=-)
[ 602.914378] pc : gen_pool_destroy+0xac/0xe0
[ 602.918569] lr : gen_pool_destroy+0xa4/0xe0
[ 602.922744] sp : ffff800014f7ba00
[ 602.926046] x29: ffff800014f7ba00 x28: ffff000004683900
[ 602.931353] x27: ffff0000021b7600 x26: ffff0000038ec000
[ 602.936655] x25: dead000000000100 x24: dead000000000122
[ 602.941956] x23: 0000000000000007 x22: ffff0000038ec040
[ 602.947258] x21: ffff0000038ec040 x20: 0000000000000200
[ 602.952559] x19: ffff8000118d5000 x18: 0000000000000000
[ 602.957860] x17: 0000000000000001 x16: 0000000000000019
[ 602.963161] x15: ffffffffffffffff x14: ffff800010c2ad98
[ 602.968462] x13: ffff800010f7fb20 x12: ffff800010f7fb1e
[ 602.973764] x11: ffff800014f7b9a0 x10: ffff800010e4c910
[ 602.979066] x9 : ffff800014f7ba50 x8 : ffff800010e44ae8
[ 602.984367] x7 : 0000000000000020 x6 : ffffffffffffffff
[ 602.989667] x5 : 0000000000000000 x4 : 0000000000000000
[ 602.994967] x3 : 0000000000000000 x2 : 0000000000000000
[ 603.000268] x1 : 0000000000000000 x0 : 0000000000000000
[ 603.005569] Call trace:
[ 603.008006] gen_pool_destroy+0xac/0xe0
[ 603.011836] k3_cppi_desc_pool_destroy+0x54/0x98
[ 603.016452] prueth_cleanup_rx_chns.isra.37+0x1c/0x38
[ 603.021498] emac_ndo_stop+0x218/0x2c8
[ 603.025240] __dev_close_many+0xac/0x130
[ 603.029157] __dev_change_flags+0xd4/0x208
[ 603.033248] dev_change_flags+0x24/0x68
[ 603.037078] devinet_ioctl+0x3f0/0x798
[ 603.040821] inet_ioctl+0x1fc/0x378
[ 603.044302] sock_do_ioctl+0x4c/0x2c8
[ 603.047958] sock_ioctl+0x310/0x4a8
[ 603.051436] __arm64_sys_ioctl+0xa8/0xe8
[ 603.055357] el0_svc_common.constprop.4+0x8c/0x180
[ 603.060143] do_el0_svc+0x24/0x90
[ 603.063452] el0_svc+0x14/0x20
[ 603.066504] el0_sync_handler+0x90/0xb8
[ 603.070331] el0_sync+0x160/0x180
[ 603.073648] Code: aa1403e1 97ffbe82 eb00029f 54fffd29 (d4210000)
[ 603.079735] -{}[ end trace 0000000000000003 ]{}-
[ 604.079911] printk: enabled sync mode
[ 604.088226] Kernel panic - not syncing:
[ 604.092048] Oops - BUG: Fatal exception
[ 604.092129] Kernel Offset: disabled
[ 604.099425] CPU features: 0x00040002,2000200c
[ 604.103771] Memory Limit: none
[ 604.106818] Rebooting in 60 seconds..