This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA2PXEVM: Compression of LUT using lz4 algorithm

Part Number: TDA2PXEVM

Hello All,

I am working on TDA2px EVM board and using Vision SDK 3.07. I have Linux based usecase. I need to compress the LUT's (consisting of float values) and store it in a file and then reading-decompressing as well. I need some information regarding lz4 compression. I have gone through srv_caliberation rtos usecase and threads on TI forum regarding lz4 compression. Some points which need clarification are as follows:

1. How much is the compression ratio of the lz4 compression assuming datatype is float numbers?
2. Is LZ4 compression can be done only on DSP? or is it possible on any other core?
3. Is it possible to include lz4 module in rtos based alg_plugin running on DSP?

Best regards,

Salman Nadaf

  • Hi,

    Can you share the E2E link having the lz4 compression discussion?

    Why do you want to compress the LUT? what is the size of this file? where you are going to save this file? Is it in the Linux file system(rootfs).

    Thanks

    Gaviraju

  • Hello Gaviraju,

    Size of uncompressed LUT is about 12MB. Yes, it's saved at /opt/vision_sdk/ and, as of now reading it from A-15 is taking about 600-650 msec. This time is quite high which gets added into boot-up time.

    In order to reduce the boot-up time, if I can save the compressed LUT using another usecase(time does not matter) then reading and decompressing( time matters) should save some time than uncompressed LUT read.

    I have used a shared buffer to access the file content from DSP and A-15. 

    Please see this thread https://e2e.ti.com/support/processors/f/791/p/674030/2517999?tisearch=e2e-sitesearch&keymatch=lz4#2517999

    I have seen that mesh generation tool (pc based) is used to generate the compressed LUTs. However in file \apps\src\rtos\common_srv\chains_common_srv_calib.c compression and decompression example is present. I do not have much clarity about whether I can use the same or not. Hence, raised a question.

    Best regards

    Salman Nadaf

  • Hi,

    The un-compressed file is saved at /opt/vision_sdk/  directory & this can be accessed after the Linux file system up. So the reading of this file how will impact on boot up time? because booting will be completed before the file system up.

    In which algorithm LUT table is going to use?

    In which core the above algorithm is running?

    Can you share a few lines of LUT data saving in a file?

    Thanks

    Gaviraju

  • Hi Gaviraju,

    Let me give you some clarity.

    Currently, my usecase is working by taking (reading at plugin create time) uncompressed LUT stored at /opt/vision_sdk/ from A-15. This LUT buffer is shared with DSP at create time. (Note: This file is saved by another usecase for which the time does not matter. The shared buffer content/ LUT is written by process on DSP and saved in file by A-15) The file size is of 12MB and hence, reading this file is consuming around 600-650msec which I would like to reduce. In order to reduce the read time I want to save the file after compressing it. Due to reduced file size read operation will take less time (assumption).

    In order to achieve this, I wanted to know whether I can use lz4 compression in alg_plugin running on DSP.

    pseudo-code :

    usecase-1:

    1. Generate LUT on the shared buffer. (DSP)

    2. Save the shared buffer content/LUT in a file at /opt/vision_sdk/. (A-15)

    usecase 2

    1. Read the file generated by usecase 1. (A-15)

    2. Share with DSP (required for alg_plugin on DSP)

    Now I would like to add compression and decompression in usecase-1 and usecase-2 respectively. (Both usecases are hlos based.)

    Best Regards,

    Salman

  • Hi,

    I will rephrase the question. 

    I have LUT buffer of 12MB (shared buffer between DSP and A-15). I want to compress this buffer(input buffer) and the resultant output compressed buffer I want to save in a file. Usecase is of hlos and one of its alg_plugin is running on DSP. Can I use lz4 compression in alg_plugin running on DSP? If yes, could you explain how?

    Best Regards,

    Salman

  • Hi,

    • By default, lz4 link is enabled only for the below Algorithm

    Do the same changes for your algorithm.

    File: 

    • To compress the data, you have to send the command to the lz4 link as below.

    • To  decompress the data, you have to send the command to the lz4 link as below.

    For more information, refer apps\src\rtos\common_srv\chains_common_srv_calib.c source file.

    Thanks

    Gaviraju