[TDA4VH] patching to support dynamic sram allocation on wave5 driver.

Hello, Sarabesh

Thanks for your reply.

It looks like the patches are duplicated. So I've applied only 0001-arm64-dts-ti-k3-j784s4-Add-NAVSS-SRAM-node-for-Codec.patch.

And then I did as this process.

1. make linux and make linux-dtbs

2. copy the k3-j784s4-evm.dtb to sdcard (rootfs/boot).

Is there missing steps?

When I did it, wave5 driver did not work.

If the sram is using in TIDL fully, can the encoder and decoder use the sram?

Best regards

Yongsig

Hi Yongsig, 

Yongsig Do said:

2. copy the k3-j784s4-evm.dtb to sdcard (rootfs/boot).

Did you copy this into /root/boot/dtb/ti?

When you run ls /boot does the image and dtb list?

Yongsig Do said:

If the sram is using in TIDL fully, can the encoder and decoder use the sram?

I will consult with our vision-apps expert to see if this is the case.

Thanks,
Sarabesh S

Hi Sarabesh

I copied it into /rootfs/boot/.

There are no sub-directories in boot.

root@j784s4-evm:/boot# pwd
/boot
root@j784s4-evm:/boot# tree
.
|-- Image -> Image-5.10.162-g76b3e88d56
|-- Image-5.10.162-g76b3e88d56
|-- fitImage -> fitImage-5.10.162-g76b3e88d56
|-- fitImage-5.10.162-g76b3e88d56
|-- k3-am69-sk-csi2-ov5640.dtbo
|-- k3-am69-sk-ddr-mem-carveout.dtbo
|-- k3-am69-sk-fpdlink-fusion.dtbo
|-- k3-am69-sk-rpi-cam-imx219.dtbo
|-- k3-am69-sk-rpi-hdr-ehrpwm.dtbo
|-- k3-am69-sk.dtb
|-- k3-j721e-fpdlink-imx390-cm-0-0.dtbo
|-- k3-j721e-fpdlink-imx390-cm-0-1.dtbo
|-- k3-j721e-fpdlink-imx390-cm-0-2.dtbo
|-- k3-j721e-fpdlink-imx390-cm-0-3.dtbo
|-- k3-j721e-fpdlink-imx390-cm-1-0.dtbo
|-- k3-j721e-fpdlink-imx390-cm-1-1.dtbo
|-- k3-j721e-fpdlink-imx390-cm-1-2.dtbo
|-- k3-j721e-fpdlink-imx390-cm-1-3.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-0-0.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-0-1.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-0-2.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-0-3.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-1-0.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-1-1.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-1-2.dtbo
|-- k3-j721e-fpdlink-imx390-rcm-1-3.dtbo
|-- k3-j784s4-edgeai-apps.dtbo
|-- k3-j784s4-evm-csi2-ov5640.dtbo
|-- k3-j784s4-evm.dtb
|-- k3-j784s4-fpdlink-fusion.dtbo
`-- k3-j784s4-vision-apps.dtbo

0 directories, 31 files
root@j784s4-evm:/boot#

Best regards

Yongsig

Hi Yongsig, 

What SDK release are you on? If it is SDK 9.0 or later the destination needs to be /root/boot/dtb/ti/

Best,
Sarabesh S.

Hi Sarabesh.

Sorry, I missed the version information.

I'm using psdkla08.06.00.12.

Best Regards

Yongsig

Hi Yongsig, 

Ok thanks, I will look at getting more information about this and update you soon. 

Regards,
Sarabesh S

Hi Yongsig, 

Yongsig Do said:

If the sram is using in TIDL fully, can the encoder and decoder use the sram?

The device tree patch changes where the VPU is pulling from SRAM to avoid conflicts with TIDL. TIDL uses the MSMC so changing the device tree to use the main_navss_sram should work fine.

Yongsig Do said:

When I did it, wave5 driver did not work.

When you applied the patches you weren't able to run any encode/decode pipelines? Could you gst-inspect-1.0 | grep "v4l2" and see if the driver is getting probed?

If the driver is not getting probed then this is a device tree issue. Back porting from 8.6 SDK should not be difficult since there isn't many changes from 9.0 so the patches should apply pretty cleanly. So it shouldn't be hard to copy over what is different.

Regards,
Sarabesh S.

Hi Sarabesh

Sarabesh Srinivasan said:

The device tree patch changes where the VPU is pulling from SRAM to avoid conflicts with TIDL. TIDL uses the MSMC so changing the device tree to use the main_navss_sram should work fine.

  --> Sounds great!

Sarabesh Srinivasan said:

When you applied the patches you weren't able to run any encode/decode pipelines? Could you gst-inspect-1.0 | grep "v4l2" and see if the driver is getting probed?

root@j784s4-evm:# gst-inspect-1.0 | grep "v4l2"
video4linux2:  v4l2video3h265enc: V4L2 H.265 Encoder
video4linux2:  v4l2video3h264enc: V4L2 H.264 Encoder
video4linux2:  v4l2video2h265dec: V4L2 H265 Decoder
video4linux2:  v4l2video2h264dec: V4L2 H264 Decoder
video4linux2:  v4l2h265enc: V4L2 H.265 Encoder
video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
video4linux2:  v4l2h265dec: V4L2 H265 Decoder
video4linux2:  v4l2h264dec: V4L2 H264 Decoder
video4linux2:  v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2:  v4l2radio: Radio (video4linux2) Tuner
video4linux2:  v4l2sink: Video (video4linux2) Sink
video4linux2:  v4l2src: Video (video4linux2) Source

It looks like the drivers are probed. I'm using v4l2 directly with ioctl actually. and the encoder works well before patching.
I'll run it on PSDK9.1. and then will update until the end of Feb.

And I'm wondering how much faster the decoder is after sram patching when the input resolution is 2560x1920 and the format is HEVC.

Currently, the processing time is about 100ms a frame. (40ms in 1280x960 H264)

Do you know the approximate processing time on the application(e,g vision app) level?

Best regards

Yongsig

Hello Yongsig, 

Yongsig Do said:

It looks like the drivers are probed. I'm using v4l2 directly with ioctl actually. and the encoder works well before patching.
I'll run it on PSDK9.1. and then will update until the end of Feb.

Ok, sounds good.

Yongsig Do said:

Do you know the approximate processing time on the application(e,g vision app) level?

The SRAM patch doesn't focus on making the decoder faster. It does slightly, but the primary reduction is with the amount of bandwidth from DDR read and writes. I can share our performance logs on this if needed.

Thank You,
Sarabesh S.

Hi Sarabesh

It would be good if you could share the logs briefly how much DDR bandwidth was reduced.

Best regards

Yongsig