[TDA4VM] v4l2 decoder watchdog timeout and kernel panic

Yongsig Do

Other Parts Discussed in Thread: SK-TDA4VM, TDA4VM, PCM3168A, TDA4VH

I made an application which was connected as the below diagram to test the decoding system using v4l2.
My development environments are :
- psdk version : 8.2.0.5
- board : TDA4VM EVM, sk-TDA4VM

The major configurations are as below.
1. Encoder
- input resoultion : 1280
- output resoultion : 960
- input format : NV12
- output foramt : H264
2. Decoder
- input resoultion : 1280
- output resoultion : 960
- input format : H264
- output format : NV12

When I designed the encoder and decoder in the same node, it worked well.
(NV12)-->[-->Encoder--(H264)-->Decoder--(NV12)-->]->
(* '[]' means a node)

But If they are separated into each node and set the pipeline mode like this,
(NV12)-->[-->Encoder--(H264)--]---->[-->Decoder--(NV12)-->]->
(* '[]' means a node)

the system died due to kernel panic when did deque from vxd-dec.
The panic messages are :

img_dec 4300000.video-decoder: vxd_schedule_locked: msgs not empty when the HW is off!
img_dec 4300000.video-decoder: fw decode time for msg_id x0 is not measured
[  318.468355] TID=NULL [GENERAL FIRMWARE PANIC Reason(s): Watch Dog Timeo]
[  318.480914] ------------[ cut here ]------------
[  318.490787] WARNING: CPU: 0 PID: 112 at drivers/media/platform/vxe-vxd/decoder/hw_control.c:901 hwctrl_process_msg+0x174/0x5f0 [vxd_dec]
[  318.503029] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_tables x_tables bridge stp llc overlay xfrm_user xfrm_algo md5 ecb aes_neon_bs aes_neon_blk des_generic libdes cbc usb_storage peak_usb bluetooth ecdh_generic ecc rfkill xhci_plat_hcd xhci_hcd rpmsg_char ti_am335x_adc kfifo_buf cdns3 udc_core roles irq_pruss_intc omap_rng pru_rproc rng_core icss_iep usbcore usb_common crct10dif_ce snd_soc_j721e_evm ti_j721e_cpsw_virt_mac phy_can_transceiver ti_am335x_tscadc ti_k3_r5_remoteproc pruss vxe_enc vxd_dec videobuf2_dma_sg v4l2_mem2mem videobuf2_dma_contig videobuf2_memops pci_endpoint_test videobuf2_v4l2 videobuf2_common pvrsrvkm(O) ti_k3_dsp_remoteproc cdns_dphy sa2ul virtio_rpmsg_bus sha512_generic cdns3_ti authenc snd_soc_pcm3168a_i2c snd_soc_pcm3168a ina2xx m_can_platform m_can can_dev rti_wdt sch_fq_codel rpmsg_kdrv_switch fuse cryptodev(O) ipv6
[  318.587197] CPU: 0 PID: 112 Comm: kworker/0:6 Tainted: G           O      5.10.100-g7a7a3af903 #1
[  318.596052] Hardware name: Texas Instruments K3 J721E SoC (DT)
[  318.601897] Workqueue: events stream_worker [vxd_dec]
[  318.606941] pstate: 40000005 (nZcv daif -PAN -UAO -TCO BTYPE=--)
[  318.612973] pc : hwctrl_process_msg+0x174/0x5f0 [vxd_dec]
[  318.618375] lr : hwctrl_process_msg+0x298/0x5f0 [vxd_dec]
[  318.623755] sp : ffff80001406fa20
[  318.627057] x29: ffff80001406fa20 x28: ffff800011151000
[  318.632357] x27: ffff80001406fd18 x26: ffff000824dc0400
[  318.637657] x25: 0000000000000000 x24: ffff000829854088
[  318.642954] x23: ffff00087fa6e800 x22: 0000000000000000
[  318.648249] x21: 0000000000000040 x20: 00000000000000c0
[  318.653546] x19: ffff000820359a00 x18: 0000000000000000
[  318.658842] x17: 0000000000000000 x16: 0000000000000000
[  318.664138] x15: 0000ffffeeb47140 x14: 0000000000000303
[  318.669434] x13: 0000000000000000 x12: 0000000000000000
[  318.674731] x11: 00000000000000eb x10: 00000000000009a0
[  318.680028] x9 : ffff80001406f610 x8 : ffff00082035a400
[  318.685323] x7 : ffff00087fa6b140 x6 : ffff00087fa6b0c0
[  318.690618] x5 : 0000000000000000 x4 : ffff00087fa61ab0
[  318.695915] x3 : ffff00087fa689b8 x2 : 18b9fdc562cfc500
[  318.701211] x1 : 18b9fdc562cfc500 x0 : 0000000000000000
[  318.706507] Call trace:
[  318.708961]  hwctrl_process_msg+0x174/0x5f0 [vxd_dec]
[  318.714007]  decoder_service_firmware_response+0x58/0x13c0 [vxd_dec]
[  318.720350]  core_fw_response_cb+0x68/0xe0 [vxd_dec]
[  318.725308]  stream_worker+0x64/0xa0 [vxd_dec]
[  318.729744]  process_one_work+0x1a0/0x328
[  318.733739]  worker_thread+0x4c/0x420
[  318.737388]  kthread+0x140/0x160
[  318.740604]  ret_from_fork+0x10/0x34
[  318.744166] ---[ end trace 6d115f7f067e61f4 ]---

It seems like the reason is watchdog timeout in vxd-dec.
Can I turn off the watchdog timer in v4l2?

++)
I've tested encode/decode using separated nodes in debug mode. It worked well.

It died in only release target mode.

Best regards
Yongsig

over 2 years ago

0 Fabiana Jaimes over 2 years ago

TI__Mastermind 19670 points

Hi Yongsig,

Are you using the SK-TDA4VM board and is this an issue that persists when using the latest SDK version?

Thank you,

Fabiana Jaimes

0 Yongsig Do over 2 years ago in reply to Fabiana Jaimes

Expert 1700 points

Hi, Jaimes

I'm using both TDA4VM-EVM and SK-TDA4VM. This issue occurs on both.

Actually system run and then die in the middle. frameCnt when the system crashes is random.

I can't upgrade the version now. Is it any way to resolve it in PSDK8.2?

Best regards

Yongsig

0 Brandon Brnich over 2 years ago in reply to Yongsig Do

TI__Expert 3830 points

Hello Yongsig,

Could you share with me the application you are using to test this scenario? Is it gstreamer, the TI test application, or your own application?

Best,

Brandon

0 Brandon Brnich over 2 years ago in reply to Brandon Brnich

TI__Expert 3830 points

Hi Yongsig,

Are you still having issues with this? I haven't seen any action on this thread so I will close it if there is no response in the next week.

Thanks,

Brandon

0 Yongsig Do over 2 years ago in reply to Brandon Brnich

Expert 1700 points

Yes I still have this. I'm watching this issue again since I resolved another in TDA4VH.

I didn't use gstreamer, I designed application with ioctl. It's the almost same codes as in the TDA4VH.

When I set the number of input and output beffer up to 19, the problem occurs again.

0 Brandon Brnich over 2 years ago in reply to Yongsig Do

TI__Expert 3830 points

I'm interested as to why this problem is not seen in the other use case you described.

What is the difference between the two exactly? I understand you have a combined node passing and when separated they are failing. But how are they separated? What is responsible for copying buffers from encode node to decode node?

0 Yongsig Do over 2 years ago in reply to Brandon Brnich

Expert 1700 points

Hi Brandon.

The difference is whether the decoder and encoder are separate.

If they are separate, there are two nodes. They are pipelined as well.

I think the responsibility for copying buffers from encode is in the pipeline buffer. But I'm not sure if it is the reason.

The time when the kernel panic occurs is always different.

What do you think I should check for this?

I'll appreciate it if you suggest anything.

Best regards

yongsig

0 Brandon Brnich over 2 years ago in reply to Yongsig Do

TI__Expert 3830 points

Hi Yongsig,

Do you have a way for me to recreate this issue using Gstreamer? I have not seen this happen in the driver before where the hardware is being disabled during a decode process. I will explore the call stack to see what could be going on. However, I need a process for recreating this issue.

Thanks,

Brandon

0 Yongsig Do over 2 years ago in reply to Brandon Brnich

Expert 1700 points

Hi Brandon

Actually I wanted to do it.

But I know that there is no example using Gstreamer in PSDK8.2.0.5. Could you give me a very simple example to make a decoder node with Gstreamer in PSDK8.2.0.5 ?

Best regards

Yongsig

0 Brandon Brnich over 2 years ago in reply to Yongsig Do

TI__Expert 3830 points

Hi Yongsig,

There is gstreamer documentation for PSDK8.2 found here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-jacinto7/08_02_00_03/exports/docs/linux/Foundational_Components_Multimedia_D5520_VXE384.html#gstreamer-pipelines.

Just replace the sink option with your desired output location. For ex, if you wanted to write to file output:

gst-launch-1.0 filesrc location=/<path_to_file> ! h264parse ! v4l2h264dec ! video/x-raw,format=NV12 ! filesink location=<output path>

Best,

Brandon

Processors

Processors forum

[TDA4VM] v4l2 decoder watchdog timeout and kernel panic