Hello,
1. Problem scenarios
When AM5728 was stable, we encountered a network abnormality on eth1.
----eth1 is assigned an intranet IP; others ping the eth1 IP of AM5728 from the CPU and find that the ping fails.
Positioning situation:
1. Found this error: NETDEV WATCHDOG: eth1 (cpsw): transmit queue 0 timed out
2. At the same time, there is Backtrace, shown in the fourth point below
2. Description of my development environment
Kernel version: 4.4.12-g3639bea54a
SDK version: ti-processor-sdk-linux-am57xx-evm-03.03.00.04
3. My question.
1. Does TI have any special solutions for AM5728? (I saw on your official website that this problem occurs in many of your chips. This should be a common problem of your TI)
2. Recommended a post through your online engineering engineer’s question. This post only provides a workaround. I have a question about this workaround
e2e.ti.com/…/am5726-netdev-watchdog-eth0-cpsw-transmit-queue-0-timed-out
Mentioned in the workaround:
I'm planning to double the descriptor pool size, that currently it is 1024
----What is this pool size and where is it changed? How?
3. In addition to the avoidance solutions mentioned in 2, are there any positive solutions?
4. The detailed error kernel stack information is as follows:
[2023-09-22 14:29:s][ 1100.978464] ------------[ cut here ]------------
[2023-09-22 14:29:s][ 1100.983150] WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:306 dev_watchdog+0x26c/0x278()
[2023-09-22 14:29:s][ 1100.991665] NETDEV WATCHDOG: eth1 (cpsw): transmit queue 0 timed out
[2023-09-22 14:29:s][ 1100.998055] Modules linked in: sha512_generic(O) sha512_arm(O) cbc(O) drbg(O) bsp_product_key(O) bsp_lan_switch(O) bsp_audio_control(O) pcie_rc(O) bsp_automgr_drv(O) bsp_it6801_drv(O) bsp_gpio(O) bsp_adv7604_drv(O) bsp_gs2972_drv(O) bsp_gs2971_drv(O) bsp_mdin380_drv(O) bsp_drvMgr_drv(O) bsp_crosslink(O) snd_soc_simple_card(O) snd_soc_evm(O) mcasp8_codec(O) mcasp7_codec(O) mcasp6_codec(O) mcasp5_codec(O) mcasp4_codec(O) mcasp3_codec(O) mcasp1_codec(O) ad1937(O) snd_soc_davinci_mcasp(O) snd_soc_omap(O) snd_soc_edma(O) snd_soc_core(O) snd_pcm_dmaengine(O) snd_pcm(O) snd_timer(O) snd_rawmidi(O) snd(O) soundcore(O) bsp_netlink(O) stm32_dev(O) bsp_spi_rw(O) bsp_i2c_rw(O) rpmsg_rpc(O) rpmsg_proto(O) virtio_rpmsg_bus(O) cryptodev(O) cmemk(O) pruss_intc(O) dwc3(O) udc_core(O) pru_rproc(O) omap_sham(O) omap_aes_driver(O) pruss(O) omap_wdt(O) ahci_platform(O) libahci_platform(O) libahci(O) libata(O) ti_vpe(O) ti_sc(O) ti_vpdma(O) dwc3_omap(O) rtc_omap(O) omap_des(O) omap_rng(O) rng_core(O) extcon_palmas(O) extcon(O) rtc_palmas(O) rtc_ds1307(O) omap_remoteproc(O) remoteproc(O) virtio(O) virtio_ring(O) sch_fq_codel(O)
[2023-09-22 14:29:s][ 1101.097239] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 4.4.12-g3639bea54a #18
[2023-09-22 14:29:s][ 1101.105538] Hardware name: Generic DRA74X (Flattened Device Tree)
[2023-09-22 14:29:s][ 1101.111654] Backtrace:
[2023-09-22 14:29:s][ 1101.114133] [<c00130a8>] (dump_backtrace) from [<c00132a4>] (show_stack+0x18/0x1c)
[2023-09-22 14:29:s][ 1101.118504] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s][ 1101.121488] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.121495] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.141434] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s][ 1101.148448] r7:c061ff44 r6:200e0113 r5:00000000 r4:c0a1708c
[2023-09-22 14:29:s][ 1101.154167] [<c001328c>] (show_stack) from [<c02e3148>] (dump_stack+0x90/0xa4)
[2023-09-22 14:29:s][ 1101.161422] [<c02e30b8>] (dump_stack) from [<c00366ec>] (warn_slowpath_common+0x88/0xb8)
[2023-09-22 14:29:s][ 1101.167094] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.167100] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.182467] r7:c061ff44 r6:00000132 r5:00000009 r4:ee0a3da8[ 1101.187039] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.194893]
[2023-09-22 14:29:s][ 1101.196570] [<c0036664>] (warn_slowpath_common) from [<c0036754>] (warn_slowpath_fmt+0x38/0x40)
[2023-09-22 14:29:s][ 1101.205302] r8:ee690800 r7:00000001 r6:c09f4100 r5:ee35d800 r4:c0948a24
[2023-09-22 14:29:s][ 1101.212080] [<c0036720>] (warn_slowpath_fmt) from [<c061ff44>] (dev_watchdog+0x26c/0x278)
[2023-09-22 14:29:s][ 1101.212650] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.212655] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.232594] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s][ 1101.240106] r3:ee35d800 r2:c0948a24
[2023-09-22 14:29:s][ 1101.243706] r4:00000000
[2023-09-22 14:29:s][ 1101.246259] [<c061fcd8>] (dev_watchdog) from [<c008a218>] (call_timer_fn+0x30/0x9c)
[2023-09-22 14:29:s][ 1101.253944] r10:c061fcd8 r9:ee35d800 r8:eed243e0[ 1101.258196] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.258201] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.271427] r7:c061fcd8 r6:00000101 r5:eed243c0
[2023-09-22 14:29:s][ 1101.276256] r4:ffffe000[ 1101.278142] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.285516]
[2023-09-22 14:29:s][ 1101.287193] [<c008a1e8>] (call_timer_fn) from [<c008a414>] (run_timer_softirq+0x190/0x208)
[2023-09-22 14:29:s][ 1101.295488] r7:00000200 r6:00000000 r5:eed243c0 r4:ee35da64
[2023-09-22 14:29:s][ 1101.301206] [<c008a284>] (run_timer_softirq) from [<c003a138>] (__do_softirq+0x11c/0x234)
[2023-09-22 14:29:s][ 1101.303818] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.303823] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.322336] r10:40000001[ 1101.323764] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.331685] r9:00000001 r8:c09f4080 r7:00000101 r6:ee0a2000 r5:c09f4084
[2023-09-22 14:29:s][ 1101.338623] r4:00000020
[2023-09-22 14:29:s][ 1101.341175] [<c003a01c>] (__do_softirq) from [<c003a544>] (irq_exit+0xbc/0x11c)
[2023-09-22 14:29:s][ 1101.348512] r10:c09f44f8 r9:c070b444[ 1101.351138] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.351144] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.364946] r8:ee008000 r7:00000000 r6:00000000 r5:00000013
[2023-09-22 14:29:s][ 1101.370833] r4:ffffe000[ 1101.371088] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.380094]
[2023-09-22 14:29:s][ 1101.381774] [<c003a488>] (irq_exit) from [<c007a2a8>] (__handle_domain_irq+0x68/0xbc)
[2023-09-22 14:29:s][ 1101.389635] r5:00000013 r4:c09ed430
[2023-09-22 14:29:s][ 1101.393243] [<c007a240>] (__handle_domain_irq) from [<c000948c>] (gic_handle_irq+0x40/0x7c)
[2023-09-22 14:29:s][ 1101.396767] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.396773] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.414549] r9:c070b444[ 1101.416719] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.423809] r8:fa213000 r7:fa212000 r6:ee0a3f40 r5:fa21200c r4:c09f4894
[2023-09-22 14:29:s][ 1101.430754] [<c000944c>] (gic_handle_irq) from [<c0013d80>] (__irq_svc+0x40/0x74)
[2023-09-22 14:29:s][ 1101.438265] Exception stack(0xee0a3f40 to 0xee0a3f88)
[2023-09-22 14:29:s][ 1101.442397] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.442403] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s][ 1101.456258] 3f40: 00000001 00000000 fe600000 00000000 ee0a2000 c09f44ac 00000000 00000000
[2023-09-22 14:29:s][ 1101.462346] ldq enter SNDRV_PCM_TRIGGER_STOP snd_dmaengine_pcm_trigger 231
[2023-09-22 14:29:s][ 1101.471368] 3f60: ee0a3fb0 c070b444 c09f44f8 ee0a3f9c ee0a3f7c ee0a3f90 c0028228 c0010540
[2023-09-22 14:29:s][ 1101.479577] 3f80: 600e0013 ffffffff
[2023-09-22 14:29:s][ 1101.483076] r9:c070b444 r8:ee0a3fb0 r7:ee0a3f74 r6:ffffffff[ 1101.488009] dmaengine_pcm_prepare_and_submit 168 chan_id=9 dma_addr=0xfe080000 direction=1
[2023-09-22 14:29:s][ 1101.488015] callback=0xbf754460 param=0xed2a6000
[2023-09-22 14:29:s]
[2023-09-22 14:29:s][ 1101.501527] r5:600e0013 r4:c0010540
[2023-09-22 14:29:s][ 1101.505309] [<c0010518>] (arch_cpu_idle) from [<c0070748>] (default_idle_call+0x28/0x34)
Regards,
Gary