This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PROCESSOR-SDK-DRA8X-TDA4X: Device crash after a long run during performing TCP operation in J7 A72 side running linux.

Part Number: PROCESSOR-SDK-DRA8X-TDA4X

SDK Version :  PROCESSOR-SDK-RTOS-AUTOMOTIVE  07_00_00_11

We are performing TCP operation in J7 A72 side running linux. The processor SDK LA is version 7.0. We see this crash after a long run. Is this known issue and if so can you please provide a patch fix for this ?

Logs:

Enter Choice: [ 2900.113697] ------------[ cut here ]------------

[ 2900.118309] kernel BUG at lib/genalloc.c:506!
[ 2900.122651] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[ 2900.128118] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv4 libcrc3
2c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo md5 ecb aes_neon_bs aes_neon_blk des_generic libdes cbc xhci_plat_hcd xh
ci_hcd usbcore rpmsg_char ti_am335x_adc kfifo_buf omap_rng rng_core pru_rproc irq_pruss_intc cdns3 roles udc_core usb_common crct10dif_ce snd_
soc_j721e_evm j721e_cpsw_virt_mac ti_k3_r5_remoteproc ti_am335x_tscadc pruss sa2ul ntb_hw_epf ntb pci_endpoint_test snd_soc_pcm3168a_i2c snd_s
oc_pcm3168a sha512_generic authenc pvrsrvkm(O) ti_k3_dsp_remoteproc cdns3_ti virtio_rpmsg_bus rti_wdt sch_fq_codel rpmsg_kdrv_switch jailhouse
(O) cryptodev(O) ipv6 nf_defrag_ipv6
[ 2900.193000] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O      5.4.40-g66cf445b76 #1
[ 2900.201326] Hardware name: Texas Instruments K3 J721E SoC (DT)
[ 2900.207140] pstate: 00000005 (nzcv daif -PAN -UAO)
[ 2900.211919] pc : gen_pool_free_owner+0xf0/0x100
[ 2900.216433] lr : gen_pool_free_owner+0xa4/0x100
[ 2900.220946] sp : ffff80001000fd30
[ 2900.224247] x29: ffff80001000fd30 x28: 0000000000000040
[ 2900.229541] x27: 0000000000000000 x26: 0000000000000000
[ 2900.234836] x25: 000000000000003f x24: 0000000000000007
[ 2900.240130] x23: ffff00007dde257f x22: 0000000000000000
[ 2900.245424] x21: 0000000000000001 x20: ffff00084422ac00
[ 2900.250718] x19: ffff00007dde2500 x18: 0000000000000000
[ 2900.256012] x17: 0000000000000000 x16: 0000000000000000
[ 2900.261305] x15: 0000000000000000 x14: 0000000000000000
[ 2900.266599] x13: ffff8000125a0000 x12: 0000000000000001
[ 2900.271893] x11: 0000000000000000 x10: 0000000000000001
[ 2900.277187] x9 : 000000000000004a x8 : ffff8000125a0040
[ 2900.282481] x7 : 0000000000000001 x6 : 0000000000000000
[ 2900.287775] x5 : fffffffffffffbff x4 : 0000000000000400
[ 2900.293069] x3 : ffff8000125a0040 x2 : 00000000000007ff                            
[ 2900.298362] x1 : 0000000000000000 x0 : 0000000000000001                            
[ 2900.303656] Call trace:                                                            
[ 2900.306091]  gen_pool_free_owner+0xf0/0x100                                        
[ 2900.310261]  k3_knav_pool_free+0x18/0x20                                            
[ 2900.314171]  am65_cpsw_nuss_xmit_free.isra.0+0x60/0xf8
[ 2900.319292]  am65_cpsw_nuss_tx_poll+0xd4/0x2d0
[ 2900.323721]  net_rx_action+0x114/0x340
[ 2900.327456]  __do_softirq+0x120/0x23c
[ 2900.331103]  irq_exit+0xb8/0xd8
[ 2900.334233]  __handle_domain_irq+0x64/0xb8
[ 2900.338313]  gic_handle_irq+0x5c/0x148
[ 2900.342047]  el1_irq+0xb8/0x180
[ 2900.345175]  arch_cpu_idle+0x10/0x18
[ 2900.348737]  do_idle+0xc0/0x140
[ 2900.351864]  cpu_startup_entry+0x20/0x40
[ 2900.355773]  rest_init+0xd0/0xdc
[ 2900.358989]  arch_call_rest_init+0xc/0x14
[ 2900.362983]  start_kernel+0x40c/0x438
[ 2900.366632] Code: d65f03c0 14000044 17fffff6 d4210000 (d4210000)
[ 2900.372709] ---[ end trace 95bbd4983d8f17e2 ]---
[ 2900.377310] Kernel panic - not syncing: Fatal exception in interrupt
[ 2900.383643] SMP: stopping secondary CPUs
[ 2900.387553] Kernel Offset: disabled
[ 2900.391026] CPU features: 0x0002,20006008
[ 2900.395019] Memory Limit: none
[ 2900.398061] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

Thanks and Regards

Sanny 

  • Hi,

    Please describe the setup in detail.

    1. How many EVM's ?

    2. The communication is between which entities ?

    3. Which command/test are you running ?

    4. Are you running a custom app ? Please post changes

    5. After how much time do you see this issue ?

    Regards

    Vineet

  • Hi Vineet

    Thanks for Reply

    Here is my setup detail :-

    1. How many EVM's ?
    TDA4 EVM and PC. It is a basic TCP transfer

    2. The communication is between which entities ?
    TDA4 EVM and PC Application. The connection is through our office network.

    3. Which command/test are you running ?
    I have my own tcp code to stream the data. We use eth0 and connect to MCU ENET port of the EVM.

    4. Are you running a custom app ? Please post changes
    It is a simple TCP operation. Moreover it is a kernel crash. Do you suspect the application ? I may not be able to share the source.
    But I can tell you that it is integrated into the single cam app. It sends over a thread from a TIOVX node to stream the video data out. Video data you can consider it to be around 2 MP camera resolution

    5. After how much time do you see this issue ?
    It varies. From 30 mins to after 6 hours also.

    Thanks 

    Sanny 

  • Hi,

    Ok, how much bitrate do you get approx ? I will try to simulate with iperf or something similar

    Regards

    Vineet

  • Hi Vineet,

    We have configured 100 Mbps as Ethernet link speed,no compression in the video, video size we can say 10 MB per second, due to decimation factor Other TCP settings are the default kernel settings.

    Thanks and Regards

    Sanny 

     

  • Hi Sanny,

    I have tried to reproduce the issue by running iperf test at 70% load between EVM and PC but I was not able to reproduce the crash. Here are the steps I followed.

    1. I prepared an SD card with SDK 7.0 and booted a J7ES EVM with it
    2. Connected eth0 port (MCU) of EVM to PC and assigned an IP in the same subnet as PC.
    3. Configured the PC as iperf server with the command "iperf -s"
    4. Ran a TCP server on EVM "iperf -c <IP address> -b700m -i10 -t100000
    5. The test ran for a very long time but I didn't see any crash.
    6. The links were configured at 1Gigabit

    You mentioned about running at 100Mbits/s, I am wondering if that's the issue. Can you try to run at 1Gbps and check the same ?

    Regards

    Vineet