This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA2HG: 【IPU2 RTOS】Display Link stop working as the RTOS stop schedule

Other Parts Discussed in Thread: TDA2HG

art Number: TDA2HG

Hello:

Description

visionSDK 3.05

Linux+RTOS

IPU2 overview

ssue description

in our case, when apps.out  startup, the display will stop working after several seconds, and report " UTILS: MBX: Utils_mbxSendCmd(): Msg Alloc Failed (0)!!! "after debug, we found the display link stop working that block on 

         Vps_printf("DisplayLink_tsk instance=%d  DisplayLink_tskRun Utils_tskRecvMsg mbx= %p rcvque= %p \n", pObj->displayInstId,&pTsk->mbx,&pTsk->mbx.recvQue);
        status = Utils_tskRecvMsg(pTsk, &pRunMsg, BSP_OSAL_WAIT_FOREVER);
        if (status != SYSTEM_LINK_STATUS_SOK)
        {
              Vps_printf("DisplayLink_tskRun error %d \n",runDone);
            break;
        }

we add kinds of trace into  

Int32 Utils_quePut(Utils_QueHandle * handle, Ptr data, UInt32 timeout)
{
    Int32 status = SYSTEM_LINK_STATUS_EFAIL;/* init status to error */
    UInt32 cookie;
    volatile Bool doBreak = FALSE;
    UInt32 curSafetyMode = BspSafetyOsal_getSafetyMode();
    BspSafetyOsal_setSafetyMode(BSP_SAFETY_OSAL_MODE_ASIL);

    do
    {
        /*
         * disable interrupts
         */
        cookie = Hwi_disable();

        if (handle->count < handle->maxElements)
        {
            /*
             * free space available in que
             */

            /* MISRA.PTR.ARITH
             * MISRAC_2004_Rule_11.1
             * MISRAC_WAIVER:
             * Pointer is accessed as an array.
             * Queue user always allocates queue of size maxElements, queue
             * will not be accessed out of bound.
             */
            /*
             * insert element
             */
            handle->queue[handle->curWr] = data;

            /*
             * increment put pointer
             */
            handle->curWr = (handle->curWr + 1) % handle->maxElements;

            /*
             * increment count of number element in que
             */
            handle->count++;

            /*
             * restore interrupts
             */
            Hwi_restore(cookie);

            /*
             * mark status as success
             */
            status = SYSTEM_LINK_STATUS_SOK;

            if (handle->flags & UTILS_QUE_FLAG_BLOCK_QUE_GET)
            {
                /*
                 * blocking on que get enabled
                 */

                /*
                 * post semaphore to unblock, blocked tasks
                 */
 
                 Vps_printf("sempost to wakeup %p ",handle);
                BspOsal_semPost(handle->semRd);

            }

            /*
             * exit, with success
             */
            doBreak = (Bool)TRUE;

        }
        else
        {
            /*
             * que is full
             */

            /*
             * restore interrupts
             */
            Hwi_restore(cookie);

            if (timeout == BSP_OSAL_NO_WAIT)
            {
                doBreak = (Bool)TRUE;                    /* non-blocking
                                                            * function call,
                                                            * exit with error
                                                            */
            }
            else if (handle->flags & UTILS_QUE_FLAG_BLOCK_QUE_PUT)
            {
                Bool semPendStatus;

                /*
                 * blocking on que put enabled
                 */

                /*
                 * take semaphore and block until timeout occurs or
                 * semaphore is posted
                 */
                handle->blockedOnPut = (Bool)TRUE;
                semPendStatus = BspOsal_semWait(handle->semWr, timeout);
                handle->blockedOnPut = (Bool)FALSE;
                if (!semPendStatus)
                {
                    /* UNREACH.GEN :  MISRAC_2004_Rule_14.1
                     * Unreachable Code
                     * MISRAC_WAIVER:
                     * Value in the if condition is dependent on the
                     * return of a function BspOsal_semWait and
                     * this function is implemented when OS BIOS is
                     * included/defined.
                     */
                    handle->forceUnblockPut = (Bool)FALSE;
                    doBreak = (Bool)TRUE;                /* timeout
                                                            * happend, exit
                                                            * with error */
                }
                else if (handle->forceUnblockPut)
                {
                    handle->forceUnblockPut = (Bool)FALSE;
                    doBreak = (Bool)TRUE;                /* timeout
                                                            * happend, exit
                                                            * with error */
                }
                else
                {
                    doBreak = (Bool)FALSE;
                }
                /*
                 * received semaphore, recheck for available space in the que
                 */
            }
            else
            {
                /*
                 * blocking on que put disabled
                 */

                /*
                 * exit with error
                 */
                doBreak = (Bool)TRUE;
            }
        }

        if ((Bool)TRUE == doBreak)
        {
            break;
        }
    }
    while (1);

    BspSafetyOsal_setSafetyMode(curSafetyMode);
    return status;
}
Int32 Utils_queGet(Utils_QueHandle * handle, Ptr * data,
                   UInt32 minCount, UInt32 timeout)
{
    Int32 status = SYSTEM_LINK_STATUS_EFAIL;/* init status to error */
    UInt32 cookie;
    volatile Bool doBreak = (Bool)FALSE;
    UInt32 curSafetyMode = BspSafetyOsal_getSafetyMode();
    BspSafetyOsal_setSafetyMode(BSP_SAFETY_OSAL_MODE_ASIL);

    /*
     * adjust minCount between 1 and handle->maxElements
     */
    if (0U == minCount)
    {
        minCount = 1U;
    }
    if (minCount > handle->maxElements)
    {
        minCount = handle->maxElements;
    }

    do
    {
        /*
         * disable interrupts
         */
        cookie = Hwi_disable();

        if (handle->count >= minCount)
        {
            /*
             * data elements available in que is >=
             * minimum data elements requested by user
             */

            /*
             * extract the element
             */
            *data = handle->queue[handle->curRd];
            
            Vps_printf("que handle %p que count=%d max %d curRd %d minCount %d",handle,handle->count,handle->maxElements,handle->curRd,minCount);
            /*
             * increment get pointer
             */
            handle->curRd = (handle->curRd + 1) % handle->maxElements;

            /*
             * decrmeent number of elements in que
             */
            handle->count--;

            /*
             * restore interrupts
             */
            Hwi_restore(cookie);

            /*
             * set status as success
             */
            status = SYSTEM_LINK_STATUS_SOK;

            if (handle->flags & UTILS_QUE_FLAG_BLOCK_QUE_PUT)
            {
                /*
                 * blocking on que put enabled
                 */
                if ((handle->count + 1U) == handle->maxElements)
                {
                    /*
                     * post semaphore to unblock, blocked tasks
                     */
                    BspOsal_semPost(handle->semWr);
                }
            }

            /*
             * exit with success
             */
            doBreak = (Bool)TRUE;

        }
        else
        {
          Vps_printf("not enough que handle %p count=%d max %d curRd %d minCount %d",handle, handle->count,handle->maxElements,handle->curRd,minCount);
            /*
             * no elements or not enough element (minCount) in que to extract
             */

            /*
             * restore interrupts
             */
            Hwi_restore(cookie);

            if (timeout == BSP_OSAL_NO_WAIT)
            {
                doBreak = (Bool)TRUE;                      /* non-blocking
                                                            * function call,
                                                            * exit with error
                                                            */
                status = SYSTEM_LINK_STATUS_EFAIL;
            }
            else
            if (handle->flags & UTILS_QUE_FLAG_BLOCK_QUE_GET)
            {
                Bool semPendStatus;

                /*
                 * blocking on que get enabled
                 */

                /*
                 * take semaphore and block until timeout occurs or
                 * semaphore is posted
                 */

                handle->blockedOnGet = (Bool)TRUE;
                 Vps_printf("not enough que handle %p  wait sem",handle);
                semPendStatus = BspOsal_semWait(handle->semRd, timeout);
                 Vps_printf("not enough que handle %p  wait sem ok",handle);
                handle->blockedOnGet = (Bool)FALSE;
                if (!semPendStatus)
                {
                    /* UNREACH.GEN :  MISRAC_2004_Rule_14.1
                     * Unreachable Code
                     * MISRAC_WAIVER:
                     * Value in the if condition is dependent on the
                     * return of a function BspOsal_semWait and
                     * this function is implemented when OS BIOS is
                     * included/defined.
                     */
                    handle->forceUnblockGet = (Bool)FALSE;
                    doBreak = (Bool)TRUE;                  /* timeout
                                                            * happened, exit
                                                            * with error */
                     Vps_printf("not enough que handle %p ETIMEOUT ",handle);                                        
                    status = SYSTEM_LINK_STATUS_ETIMEOUT;
                }
                else if (handle->forceUnblockGet == (Bool)TRUE)
                {
                    handle->forceUnblockGet = (Bool)FALSE;
                    doBreak = (Bool)TRUE;                  /* timeout
                                                            * happened, exit
                                                            * with error */
                    Vps_printf("not enough que handle %p ETIMEOUT break",handle);                                            
                    status = SYSTEM_LINK_STATUS_ETIMEOUT;
                }
                else
                {
                    doBreak = (Bool)FALSE;
                }
                /*
                 * received semaphore, check que again
                 */
            }
            else
            {
                /*
                 * blocking on que get disabled
                 */

                /*
                 * exit with error
                 */
                 Vps_printf("not enough que handle %p error ",handle);    
                doBreak = (Bool)TRUE;
            }
        }

        if ((Bool)TRUE == doBreak)
        {
            break;
        }
    }
    while (1);

    BspSafetyOsal_setSafetyMode(curSafetyMode);
    return status;
}

please check the trace in attchament serial-enable msg-15.log

line 112259 t0 112264

 display Link is blocked to wait for the message

line 112392

previous link has send a new message to this message que and call semaphorepost(), but semaphorewait() is not be waken up.

actually, all link on IPU2 stop working.

Extra Testing 1:

 we create two tasks to test the semaphore as following:

BspOsal_SemHandle TestSemHandle =NULL;
BspOsal_TaskHandle Testtsk;

#pragma DATA_ALIGN(gTest_tskStack, 32)
#pragma DATA_SECTION(gTest_tskStack, ".bss:taskStackSection")
UInt8 gTest_tskStack[DISPLAY_LINK_TSK_STACK_SIZE];
static Int16 FirstFlag = 0;
Void Test_tskMain(UArg arg0, UArg arg1)
{
while(1){
     Vps_printf("Test_tskMain wait post\n");
  BspOsal_semWait(TestSemHandle,BSP_OSAL_WAIT_FOREVER);
   Vps_printf("TestSemHandle wait ok\n");
}
}


BspOsal_TaskHandle Testtsk1;

#pragma DATA_ALIGN(gTest_tskStack1, 32)
#pragma DATA_SECTION(gTest_tskStack1, ".bss:taskStackSection")
UInt8 gTest_tskStack1[DISPLAY_LINK_TSK_STACK_SIZE];
Void Test_tskMain1(UArg arg0, UArg arg1)
{
while(1){

    Vps_printf("Test_tskMain1 TestSemHandle post\n");

  BspOsal_sleep(1000);
   Vps_printf("TestSemHandle post\n");
   BspOsal_semPost(TestSemHandle);

}
}

 we found, when it happened,

  • this test semaphore also did not work that we can not see the trace"
  • the test tasks also not run again(scheduled)

 

Extra Testing 2

  1. Move dup1 to DSP, issue remains.
  2. Comment out process code in EVE, issue remain
  3. Do not active pushpsld data flow, issue disappeared.
  4. Move the EVE code to DSP, issue remain

 

  • Hi,

    Can you tell me which usecase are you trying?

    Please share the stats log after pressing p after running the usecase for 1 min.

    It may happen that some previous link goes to hang state and it is not sending data to display link.

    Please check your previous links also.

    Regards,

    Anuj

  • Anuj Gupta said:
    Can you tell me which usecase are you trying?

    it's our custom link. with tda2xx_evm_infoadas

    Anuj Gupta said:
    Please share the stats log after pressing p after running the usecase for 1 min.

    will update it later.

    Anuj Gupta said:
    Please check your previous links also.

      the previous link is dispdist_weston which run on linux side, always send the message to the display link.

      please check the log line 112444 DispDistSrcLink_processData

  • Anuj Gupta said:
    Please share the stats log after pressing p after running the usecase for 1 min.

     please see the attachment, when it happened stats also hung on.6232.IPU2-semaphore failed.log

  • Hi all:

      simply, in our case, the RTOS on IPU2 stop scheduling...

  • Hi,

    As i can see in your log you are using IPU1_0 and IPU2 both as a part of Vision SDK.

    When you use linux then you can only use 1 IPU core not both as a part of VSDK.

    You can run something else on other IPU core but it should not be VSDK.

    Can you share output of make -s showconfig

    Please share your usecase text file also.

    Regards,

    Anuj

  • Hi:

      

    Anuj Gupta said:
    As i can see in your log you are using IPU1_0 and IPU2 both as a part of Vision SDK.

      yes, we are not run only link on IPU1, as we known, the IPU1 is used for memory allocation of SR1

    Anuj Gupta said:
    Can you share output of make -s showconfig

      

    root@ubuntu:/mnt/workshop/KunPeng/application/tda2_app/build#  make -s showconfig
    #
    # Build Config is [ tda2xx_evm_linux_all ]
    # Build Config file is @ /mnt/workshop/KunPeng/application/tda2_app/configs/tda2xx_evm_linux_all/cfg.mk
    # Build Config .h file is @ /mnt/workshop/KunPeng/application/tda2_app/links_fw/include/config/apps/tda2xx_evm_linux_all/system_cfg.h
    # Build CPUs is @ ipu1_0 ipu2 a15_0 dsp1 eve1
    #
    # CPUs included in application,
    # PROC_IPU1_0_INCLUDE=yes
    # PROC_IPU1_1_INCLUDE=no
    # PROC_IPU2_INCLUDE=yes
    # PROC_DSP1_INCLUDE=yes
    # PROC_DSP2_INCLUDE=no
    # PROC_EVE1_INCLUDE=yes
    # PROC_EVE2_INCLUDE=no
    # PROC_EVE3_INCLUDE=no
    # PROC_EVE4_INCLUDE=no
    # PROC_A15_0_INCLUDE=yes
    #
    # Platform config,
    # VSDK_BOARD_TYPE=TDA2XX_EVM [options: TDA2XX_EVM TDA2EX_EVM TDA3XX_EVM TDA3XX_RVP TDA2XX_RVP]
    # PLATFORM=tda2xx-evm
    # DUAL_A15_SMP_BIOS=no
    # IPU1_SMP_BIOS=no
    # DDR_MEM=DDR_MEM_1024M [options: DDR_MEM_128M DDR_MEM_512M DDR_MEM_1024M]
    # EMIFMODE=SINGLE_EMIF_512MB [options: SINGLE_EMIF_512MB SINGLE_EMIF_1GB ref build_pdk.mk]
    # NDK_PROC_TO_USE=none [options: a15_0 ipu1_0 ipu1_1 ipu2 none]
    # NSP_TFDTP_INCLUDE=no [options: yes no]
    # TDA2EX_ETHSRV_BOARD=no [options: yes no]
    # FATFS_PROC_TO_USE=a15_0 [options: ipu1_0 none]
    # RADAR_BOARD=none [options: TDA3XX_AR12_ALPS TDA3XX_AR12_VIB_DAB_BOOSTER TDA3XX_RADAR_RVP none]
    #
    # Build config,
    # BUILD_OS=Linux [options: Windows_NT Linux]
    # BUILD_DEPENDENCY_ALWAYS=no
    # BUILD_ALGORITHMS=no
    # BUILD_INFOADAS=no
    # PROFILE=release [options: debug release]
    # KW_BUILD=no
    # CPLUSPLUS_BUILD=no
    # IPU_PRIMARY_CORE=ipu2 [options: ipu1_0 ipu2]
    # IPU_SECONDARY_CORE=ipu1_0 [options: ipu1_0 ipu2]
    # A15_TARGET_OS=Linux [options: Bios Linux Qnx]
    # BSP_STW_PACKAGE_SELECT=all [options: all vps-iss-dss-only vps-vip-vpe]
    #
    # Safety Module config,
    # RTI_INCLUDE=no
    # ECC_FFI_INCLUDE=no
    # DCC_ESM_INCLUDE=no
    #
    # Video Module config,
    # IVAHD_INCLUDE=yes
    # VPE_INCLUDE=yes
    # CAL_INCLUDE=no
    # ISS_INCLUDE=no
    # ISS_ENABLE_DEBUG_TAPS=no
    # WDR_LDC_INCLUDE=no
    # DSS_INCLUDE=yes
    #
    # Open Compute config,
    # OPENCL_INCLUDE=no
    # TARGET_ROOTDIR=/mnt/workshop/KunPeng/base/ti_vision_sdk_v3.4/ti_components/../../../dist/apa/targetfs
    # ENABLE_OPENCV=no
    # ENABLE_OPENCV_TESTS=no
    # OPENVX_INCLUDE=no
    #
    # Log config,
    # ENABLE_UART_LOG=yes
    # ENABLE_NETWORK_LOG=no
    # ENABLE_CCS_LOG=no
    # CIO_REDIRECT=yes
    #
    # IPC config,
    # WORKQ_INCLUDE=no
    # IPC_LIB_INCLUDE=no
    #
    # AUTOSAR_APP=no
    #
    # Surround View config,
    # SRV_FAST_BOOT_INCLUDE=no
    #
    # Other Module config,
    # AVB_INCLUDE=no
    # DCAN_INCLUDE=no
    # RADAR_ONLY=no
    # CPU_IDLE_ENABLED=yes
    # FAST_BOOT_INCLUDE=no
    # DATA_VIS_INCLUDE=no
    # HS_DEVICE=no
    # ULTRASONIC_INCLUDE=no
    # PGA450=
    # PGA460=
    # ENABLE_ALL_DEPTH=
    #
    # Linux config,
    # DEFAULT_UBOOT_CONFIG=dra7xx_evm_vision_config
    # DEFAULT_KERNEL_CONFIG=ti_sdk_dra7x_release_defconfig
    # DEFAULT_DTB=dra7-evm-infoadas.dtb
    # CMEM_INCLUDE=yes
    # IPUMM_INCLUDE=no
    # IPU1_EVELOADER_INCLUDE=yes
    # ROBUST_RVC_INCLUDE=no
    # BUILD_ADAM_CAR=no
    #
    # Alg plugins included in build,
    # ALG_dmaSwMs ALG_framecopy ALG_safe_framecopy ALG_mosaic ALG_display ALG_camera ALG_psld_ecarx
    #
    # Use-cases included in build,
    # UC_csi2Cal_multi_cam_view UC_disp_dist_src_display UC_iss_capture_apademo UC_csi2Cal_multi_cam_psld_hw
    #
    #
    # CPUs that are NOT required but included in config [ tda2xx_evm_linux_all ],
    #
    #
    # CPUs that are required but not included in config [ tda2xx_evm_linux_all ],
    #
    #
    # Edit /mnt/workshop/KunPeng/application/tda2_app/build/configs/tda2xx_evm_linux_all/cfg.mk to include or exclude CPUs in an application

    Anuj Gupta said:
    Please share your usecase text file also.

       

    UseCase: chains_csi2CalMultiCam_psld_hw

    Capture -> Alg_Camera(A15)
    DispDistSrc_weston -> Display_m4
    Capture_dsswb ->Display_yuv422

    PushFramePsld(A15) -> Dup1 -> Alg_psld_imageprocess (EVE1) -> Alg_psld_parkingdetection (DSP1) -> AlgoResultPsld(A15)

  • Hi,

    You should not even include IPU1_0 is your config.

    Please disable it from your config and try a clean build and then run your usecase.

    Regards,

    Anuj

  • Hi Anuj:

    Anuj Gupta said:
    You should not even include IPU1_0 is your config.

       we need IP1 to run the APA control algorithm and communicate with external MCU.

  • Hi,

    We do not recommend to use both IPU while using linux.

    So please disable IPU1 from your system and try running the usecase and check.

    Regards,

    Anuj