This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

[TDA4] How to use custom layer at vision_apps?

I'm trying to use vision_apps to demo the deep learning model with custom layer on PC emulation.

I've modified the sample which is named app_tidl. (remove the diplayOutput function) 

I've already done some things as below:

  1. The model with custom layer can import and inference normally.
  2. modified related .mak files :
    1. psdk_rtos_auto_j7_06_02_00_21\tiovx\psdkra_tools_path.mak
      +  TIDL_CUSTOM_PATH ?= $(PSDK_PATH)/tidl_j7_01_00_00_10/ti_dl/custom

    2. psdk_rtos_auto_j7_06_02_00_21\tiovx\kernels_j7\concerto_inc.mak
      +  STATIC_LIBS += tidl_custom
      +  LDIRS+= $(PSDK_PATH)/tidl_j7_01_01_00_10/ti_dl/lib/PC/dsp/algo/$(TARGET_BUILD)

    3. psdk_rtos_auto_j7_06_02_00_21\vision_apps\apps\concerto_x86_64_inc.mak
      +  STATIC_LIBS += tidl_custom
      +  LDIRS+= $(PSDK_PATH)/tidl_j7_01_01_00_10/ti_dl/lib/PC/dsp/algo/$(TARGET_BUILD)

    4. "Required system library libtidl_custom.a"  shows in command window. 

The program is crash in vxProcessGraph function in main.c.

But it can work normally when model remove custom layer.

So my questions are :

  1. Do "Required system library libtidl_custom.a" means the link is succeeded? If not, how to check the result?
  2. How to demo the model with custom layer at vision_apps/apps/dl_demo/app_tidl or demo in other ways. 

There are some relative files as below (custom folder in tidl_j7_01_01_00_10/ and model):

custom.zip  custom_model.zip

  • Hi Ted,

    1. "Required system library libtidl_custom.a" is simply telling that it needs this library for linking. The linker will throw error if its NOT finding the required library. So if you don't get any errors after printing this line your library linking is successful.

    2. You are on the right path, but you might need some debug to root-cause the issue. As you are building for host-emulation please use CCS to debug the executable. This will tell you where and why exactly the crash occurred.

    Also when you say "crash" do you see a "segmentation fault" ?

    Regards,
    Shyam

  • Hi Shyam:

    Sorry about I click the wrong button, may you cancel the resolved?

    I found that the crash is happened at tivx_alg_vision.c:365.

    Then, I can't go into this function.

    Also, can't exactly tell you where the problem is occurred.

    The "crash" is meaning the "segmentation fault"

    Thanks and Regards,

    Ted

  • Hi Ted,

    Can you please also confirm that you have initialized this call-back function pointer and ensured that your function signature matches the one below?

    /* defined in itidl_ti.h*/

    typedef struct {

    ...

      /** Call back Function pointer to Custom Layer Process API */
      int32_t (*TIDL_CustomLayerProcess)(sTIDL_Layer_t *tidlLayer,
                                         void *inPtrs[],
                                         void *outPtrs[],
                                         void *params,
                                         void *udmaDrvObj,
                                         int32_t tensorStats);
    }
    TIDL_CreateParams;

    And make sure you initialize the function pointer in setCreateParams function in app_tidl/main.c file

            if (VX_SUCCESS == status)
            {
                if(createParams_buffer)
                {
                    TIDL_CreateParams *prms = createParams_buffer;

                    TIDL_createParamsInit(prms);

                    prms->isInbufsPaded                 = 1;
                    prms->quantRangeExpansionFactor     = 1.0;
                    prms->quantRangeUpdateFactor        = 0.0;
    #ifdef APP_TIDL_TRACE_DUMP
                    prms->traceLogLevel                 = 1;
                    prms->traceWriteLevel               = 1;
    #else
                    prms->traceLogLevel                 = 0;
                    prms->traceWriteLevel               = 0;
    #endif
                    /* Initialize callback function pointer here */
                    /* prms->TIDL_CustomLayerProcess = <custom layer function> */
                }
                else
                {
                    printf("app_tidl: ERROR: Unable to allocate memory for create time params! %d bytes\n", capacity);
                }

                vxUnmapUserDataObject(createParams, map_id);
            }
        }

    Regards,
    Shyam

  • Hi Shyam:

     

    Thanks for your help.

    It works !

     

    Thanks and Regards,

    Ted

  • Hi Shyam:

    I have some problems about layer-level output in app_tidl.

    1. I found some difference in first layer between this program on x86_64 and TIDL standalone test on PC emulation.
      As I know, the output should be the same with same input model.
      Here is my configs ,models, first layer outputs and build_flags.
      1031.first_layer_output.zip4645.Models.zip4454.build_flags.zip1440.Configs.zip


    2. Is it possible to get the same result  between TIDL standalone test and app_tidl?

    Thanks and Regards,

    Ted

  • Hi Ted,

    results (and layer level traces) shall match between the TIDL standalone test and app_tidl for a given model.

    Please make sure the input tensor provided to both the applications are same.