This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VL-Q1: Linux sometimes hangs without logs

Part Number: TDA4VL-Q1
Other Parts Discussed in Thread: TDA4VL, TDA4VH

Hi TI experts,

HW: our custom board

SDK: j721s2, 8.6

We boot Linux with SBL. The Linux hangs after a couple hours of running without any logs on the serial ports or kernel panic, but at the same time the mcu1_0 is still running.

We are using the recommended power supply from the specsheet(TPS65941421+TPS65941120+LP876411B5). 

Could you give me some suggestions?

Thanks

Regards

quanfeng

  • Could you give me some suggestions?

    Have you tried connecting to A72 using a debugger?
    Are you able to connect?

    - Keerthy

  • Hi Keerthy,

    Apologies for the delayed reply.

    Are you able to connect?

    I can't connect to the A72 with the debugger when Linux hangs.

    I heard from a coworker that turning on the MSAA feature of the GPU on tda4vl will lead to insufficient DDR bandwidth, which in turn will lead to the probabilistic death of the A72, is that correct?

    Thanks

    Regards

    quanfeng

  • Hi,

    Can you confirm that GPU is being actively used when the hang happens?

    - Keerthy

  • Can you confirm that GPU is being actively used when the hang happens?

    Yes, the GPU is always running.

    Here are the logs from our application, which ran until Linux hung. The logs show that the DDR bandwidth is up to 161661 MB/s.

    HWA performance statistics,
    ===========================
    
    HWA:   LDC : LOAD =  58.33 % ( 232 MP/s )
    HWA:   MSC0: LOAD =  20. 8 % ( 87 MP/s )
    HWA:   MSC1: LOAD =  13.90 % ( 65 MP/s )
    HWA:   GPU : LOAD =  85.38 % ( 90 MP/s )
    
    
    DDR performance statistics,
    ===========================
    
    DDR: READ  BW: AVG =   4551 MB/s, PEAK = 100104 MB/s
    DDR: WRITE BW: AVG =   2497 MB/s, PEAK =  61557 MB/s
    DDR: TOTAL BW: AVG =   7048 MB/s, PEAK = 161661 MB/s

    Thanks

    Regards

    quanfeng

  • Hi,

    Thanks for the data. I will loop in GPU expert to give feedback on the above hypothesis.

    - Keerthy

  • Hi TI experts,

    Last night's test turned off both gpu DDR qos priority and MSAA antialiasing and it ran all night (>12 hours).
    Turning off GPU DDR qos priority alone and MSAA antialiasing alone didn't work and only ran for about 3 hours.

    a. Controlling MSAA antialiasing is accomplished with the following code:

    /* Creates texture and eglimage objects
     * flag_msaa: 1-enable 0-disable
     */
    void appEglBindFrameBuffer(void *eglWindow, app_egl_tex_prop_t *prop, int32_t flag_msaa)
    {
        int32_t texFound;
        int32_t texIndex;
        uint32_t i;
        PFNGLEGLIMAGETARGETTEXTURE2DOESPROC glEGLImageTargetTexture2DOES;
        PFNGLCLIPCONTROLEXTPROC glClipControlEXT;
        app_egl_obj_t *obj = (app_egl_obj_t*)eglWindow;
        app_egl_tex_obj_t *tex_obj;
    
        glEGLImageTargetTexture2DOES =
            (PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)eglGetProcAddress("glEGLImageTargetTexture2DOES");
        glClipControlEXT =
                (PFNGLCLIPCONTROLEXTPROC)eglGetProcAddress("glClipControlEXT");
        PFNGLFRAMEBUFFERTEXTURE2DMULTISAMPLEEXTPROC glFramebufferTexture2DMultisampleEXT;
        glFramebufferTexture2DMultisampleEXT =
            (PFNGLFRAMEBUFFERTEXTURE2DMULTISAMPLEEXTPROC)eglGetProcAddress("glFramebufferTexture2DMultisampleEXT");
    
        texIndex = -1;
        texFound = 0;
    
        for(i = 0; i < APP_EGL_MAX_RENDER_TEXTURES; i++)
        {
            tex_obj = &obj->texRender[i];
    
            if(tex_obj->isAlloc
                && tex_obj->dmaBufFd == prop->dmaBufFd[0]
                && tex_obj->dmaBufFdOffset == prop->dmaBufFdOffset[0]
                )
            {
                texIndex = i;
                texFound = 1;
                break;
            }
        }
    
        if(!texFound)
        {
            /* find free slot amd create texture */
            for(i = 0; i < APP_EGL_MAX_RENDER_TEXTURES; i++)
            {
                tex_obj = &obj->texRender[i];
    
                if(!tex_obj->isAlloc)
                {
                    int32_t status;
    
                    status = appEglWindowSetupRenderTex(obj, prop, i, flag_msaa);
                    if(status==0)
                    {
                        texIndex = i;
                        texFound = 1;
                    }
                    break;
                }
            }
        }
    
        if(texFound)
        {
            tex_obj = &obj->texRender[texIndex];
    
            glActiveTexture(GL_TEXTURE2);
            glBindTexture(GL_TEXTURE_EXTERNAL_OES, tex_obj->tex); // GL_TEXTURE_EXTERNAL_OES
            appEglCheckEglError("glBindTexture", EGL_TRUE);
    
            glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, (GLeglImageOES)tex_obj->img); //GL_TEXTURE_EXTERNAL_OES
            appEglCheckEglError("glEGLImageTargetTexture2DOES", EGL_TRUE);
    
            glBindFramebuffer(GL_FRAMEBUFFER, tex_obj->fboId);
            appEglCheckEglError("glBindFramebuffer", EGL_TRUE);
            //need set from GL_TEXTURE_2D to GL_TEXTURE_EXTERNAL_OES
            if (flag_msaa)
            {
                glFramebufferTexture2DMultisampleEXT(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                    GL_TEXTURE_EXTERNAL_OES, tex_obj->tex, 0, obj->samples);
                appEglCheckEglError("glFramebufferTexture2DMultisampleEXT", EGL_TRUE);
            }
            else
            {
                glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                    GL_TEXTURE_EXTERNAL_OES, tex_obj->tex, 0);
                appEglCheckEglError("glFramebufferTexture2D", EGL_TRUE);
            }
    
    #if defined(GL_DEBUG)
            GLenum fbstatus = glCheckFramebufferStatus(GL_FRAMEBUFFER);
            if (fbstatus != GL_FRAMEBUFFER_COMPLETE)
                printf("EGL: ERROR: Frambuffer complete check failed 0x%x\n", fbstatus);
    #endif
        }
        /* Binding FBO: move the origin to upper left */
        glClipControlEXT(GL_UPPER_LEFT_EXT, GL_NEGATIVE_ONE_TO_ONE_EXT);
    }

    b. Turning off GPU DDR qos priority means unapplying the patch in the link below, i.e. unsetting the QOS atype to 3 and leaving it at default:

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1218318/faq-tda4vm-are-there-any-known-bugs-and-patches-that-i-should-use-in-my-gpu-driver

    1. For the gpu patch, I've already used the patches in the two links above and below, do I just need to use one of them?

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1316731/faq-tda4vl-q1-what-are-the-gpu-driver-bug-fixes-for-sdk-8-6-or-earlier

    2. Turning off GPU antialiasing affects our application, can we use FXAA antialiasing if we can't use MSAA antialiasing, are there any examples for FXAA ?

    Thanks

    Regards

    quanfeng

  • We are using the recommended power supply from the specsheet(TPS65941421+TPS65941120+LP876411B5). 

    What power supply are you using with the EVM? For example, what is the voltage and current the external power supply can provide? Some of the standard power supplies we use are only 5A, but we have seen instances where more amperage is needed depending on the application. Can you please share the datasheet/part number of your external power supply? Do you have a way to test with a power supply with a higher amperage?

  • Hello quanfeng,

    Last night's test turned off both gpu DDR qos priority and MSAA antialiasing and it ran all night (>12 hours).
    Turning off GPU DDR qos priority alone and MSAA antialiasing alone didn't work and only ran for about 3 hours.

    Are you seeing any logs during the hang? Or is the system simply freezing and not returning?

    If there are no logs and the system hangs, perhaps this is a power-related issue, and we can test a bigger power supply.

    If there are logs, please share them.

    b. Turning off GPU DDR qos priority means unapplying the patch in the link below, i.e. unsetting the QOS atype to 3 and leaving it at default:

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1218318/faq-tda4vm-are-there-any-known-bugs-and-patches-that-i-should-use-in-my-gpu-driver

    1. For the gpu patch, I've already used the patches in the two links above and below, do I just need to use one of them?

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1316731/faq-tda4vl-q1-what-are-the-gpu-driver-bug-fixes-for-sdk-8-6-or-earlier

    The GPU patches will only help with GPU driver issues. If the application is running OK, and there are no GPU related issues then changing the GPU driver won't help.

    2. Turning off GPU antialiasing affects our application, can we use FXAA antialiasing if we can't use MSAA antialiasing, are there any examples for FXAA ?

    Will need to see if the failure is due to the MSAA antialiasing feature before we conclude that you need to turn it off.

    We currently do not have an example of FXAA but you could use any example you find online.

    Regards,

    Erick

  • Hi Erick,

    What power supply are you using with the EVM?

    We use our custom board with only 1A max current, using a max 3A power supply is sufficient.

    Are you seeing any logs during the hang? Or is the system simply freezing and not returning?

    Still no logs during the hang. When the A72 hangs, we can't connect to A72 with the debugger, but we can connect to other cores (R5F, C7X) in the main and mcu domains. We compared the psc, pll, and bus error registers for normal operation and hang, but found no anomalies.

    Last night's test turned off both gpu DDR qos priority and MSAA antialiasing and it ran all night (>12 hours).

    Performed this test again last night but could only run it for about 3 hours. Turning off GPU DDR QoS and MSAA only reduces the probability of hangs, it does not fix this issue. Can you give us some suggestions?

    Thanks

    Regards

    quanfeng

  • Hi Erick,

    I found the following link, MSAA for TDA4VH consumes a lot of DDR bandwidth, does TDA4VL have this problem too?

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1256239/tda4vh-q1-msaa-costs-a-lot-of-bandwidth

    Thanks

    Regards

    quanfeng

  • quanfeng,

    I found the following link, MSAA for TDA4VH consumes a lot of DDR bandwidth, does TDA4VL have this problem too?

    Yes, it is true that it will consume more bandwidth. But consuming too much bandwidth should not cause your core to halt/hang. It should lower your performance.

    Still no logs during the hang. When the A72 hangs, we can't connect to A72 with the debugger, but we can connect to other cores (R5F, C7X) in the main and mcu domains. We compared the psc, pll, and bus error registers for normal operation and hang, but found no anomalies.

    Interesting, it would be good to understand what the loading looks like before the hang happens, but still should not cause a core to go down in the way you described.

    Still no logs during the hang. When the A72 hangs, we can't connect to A72 with the debugger, but we can connect to other cores (R5F, C7X) in the main and mcu domains. We compared the psc, pll, and bus error registers for normal operation and hang, but found no anomalies.

    What debugger are you using? Is this with code composer studio?

    Regards,

    Erick

  • Hi Erick,

    What debugger are you using? Is this with code composer studio?

    We compared the registers using the USB560 v2 System Trace debugger with CCS.

    Here are our load logs. What are the possible causes of a core hanging like this and can you give us some direction on how to find the cause of the problem?

    Summary of CPU load,
    ====================
    
    CPU: mpu1_0: TOTAL LOAD =  24.89 % ( HWI =   1.86 %, SWI =   0.43 % )
    CPU: mcu2_0: TOTAL LOAD =  38. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
    CPU:  c7x_1: TOTAL LOAD =  61. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
    CPU:  c7x_2: TOTAL LOAD =  29. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
    
    
    HWA performance statistics,
    ===========================
    
    HWA:   LDC : LOAD =  58.33 % ( 232 MP/s )
    HWA:   MSC0: LOAD =  20. 8 % ( 87 MP/s )
    HWA:   MSC1: LOAD =  13.90 % ( 65 MP/s )
    HWA:   GPU : LOAD =  85.38 % ( 90 MP/s )
    
    
    DDR performance statistics,
    ===========================
    
    DDR: READ  BW: AVG =   4551 MB/s, PEAK = 100104 MB/s
    DDR: WRITE BW: AVG =   2497 MB/s, PEAK =  61557 MB/s
    DDR: TOTAL BW: AVG =   7048 MB/s, PEAK = 161661 MB/s
    
    
    Detailed CPU performance/memory statistics,
    ===========================================
    
    DDR_SHARED_MEM: Alloc's: 238 alloc's of 287948696 bytes
    DDR_SHARED_MEM: Free's : 6 free's  of 443712 bytes
    DDR_SHARED_MEM: Open's : 232 allocs  of 287504984 bytes
    DDR_SHARED_MEM: Total size: 469762048 bytes
    
    CPU: mcu2_0: TASK:           IPC_RX:   1.27 %
    CPU: mcu2_0: TASK:       REMOTE_SRV:   0.14 %
    CPU: mcu2_0: TASK:        LOAD_TEST:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CPU_0:   0. 0 %
    CPU: mcu2_0: TASK:        TIVX_V1NF:   0. 0 %
    CPU: mcu2_0: TASK:      TIVX_V1LDC1:  12.16 %
    CPU: mcu2_0: TASK:       TIVX_V1SC1:   4.15 %
    CPU: mcu2_0: TASK:      TIVX_V1MSC2:   6. 3 %
    CPU: mcu2_0: TASK:       TIVXVVISS1:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT1:   3.41 %
    CPU: mcu2_0: TASK:       TIVX_CAPT2:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_DISP1:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_DISP2:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CSITX:   1.93 %
    CPU: mcu2_0: TASK:       TIVX_CAPT3:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT4:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT5:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT6:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT7:   0. 0 %
    CPU: mcu2_0: TASK:       TIVX_CAPT8:   0. 0 %
    CPU: mcu2_0: TASK:      TIVX_DPM2M1:   0. 0 %
    CPU: mcu2_0: TASK:      TIVX_DPM2M2:  10.86 %
    CPU: mcu2_0: TASK:      TIVX_DPM2M3:   0. 0 %
    CPU: mcu2_0: TASK:      TIVX_DPM2M4:   0. 0 %
    
    CPU: mcu2_0: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =   16701952 B ( 99 % unused)
    CPU: mcu2_0: HEAP:           L3_MEM: size =     262144 B, free =     261888 B ( 99 % unused)
    
    CPU:  c7x_1: TASK:           IPC_RX:   0.15 %
    CPU:  c7x_1: TASK:       REMOTE_SRV:   0. 0 %
    CPU:  c7x_1: TASK:        LOAD_TEST:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P1:  60.58 %
    CPU:  c7x_1: TASK:      TIVX_C71_P2:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P3:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P4:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P5:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P6:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P7:   0. 0 %
    CPU:  c7x_1: TASK:      TIVX_C71_P8:   0. 0 %
    CPU:  c7x_1: TASK:      IPC_TEST_RX:   0. 0 %
    CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
    CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
    CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
    
    CPU:  c7x_1: HEAP:    DDR_LOCAL_MEM: size =  335544320 B, free =  275970304 B ( 82 % unused)
    CPU:  c7x_1: HEAP:           L3_MEM: size =    3702784 B, free =     688128 B ( 18 % unused)
    CPU:  c7x_1: HEAP:           L2_MEM: size =     458752 B, free =     458752 B (100 % unused)
    CPU:  c7x_1: HEAP:           L1_MEM: size =      16384 B, free =          0 B (  0 % unused)
    CPU:  c7x_1: HEAP:  DDR_SCRATCH_MEM: size =   67108864 B, free =   66577152 B ( 99 % unused)
    
    CPU:  c7x_2: TASK:           IPC_RX:   0.42 %
    CPU:  c7x_2: TASK:       REMOTE_SRV:   0. 0 %
    CPU:  c7x_2: TASK:        LOAD_TEST:   0. 0 %
    CPU:  c7x_2: TASK:         TIVX_CPU:  28.40 %
    CPU:  c7x_2: TASK:      IPC_TEST_RX:   0. 0 %
    CPU:  c7x_2: TASK:      IPC_TEST_TX:   0. 0 %
    CPU:  c7x_2: TASK:      IPC_TEST_TX:   0. 0 %
    CPU:  c7x_2: TASK:      IPC_TEST_TX:   0. 0 %
    
    CPU:  c7x_2: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =    9114624 B ( 54 % unused)
    CPU:  c7x_2: HEAP:           L2_MEM: size =     458752 B, free =     229376 B ( 50 % unused)
    CPU:  c7x_2: HEAP:           L1_MEM: size =      16384 B, free =      16384 B (100 % unused)
    CPU:  c7x_2: HEAP:  DDR_SCRATCH_MEM: size =   67108864 B, free =   67108864 B (100 % unused)
    
    
    GRAPH:        avm_graph (#nodes =   9, #executions =  57779)
     NODE:       CAPTURE1:             capture_node: avg =  14011 usecs, min/max =    173 /  71541 usecs, #executions =      57779
     NODE:      VPAC_LDC1:                  ldc_avm: avg =  14349 usecs, min/max =   8797 /  33655 usecs, #executions =      57779
     NODE:          A72-2:      OpenGL_APS_SRV_Node: avg =  30639 usecs, min/max =  20550 / 266660 usecs, #executions =      57779
     NODE:          DSP-1:            ImgSelectNode: avg =   3598 usecs, min/max =   2759 /   5370 usecs, #executions =      57779
     NODE:       DSS_M2M2:             CsitxM2mNode: avg =   4609 usecs, min/max =   4068 /   6661 usecs, #executions =      57779
     NODE:          CSITX:                CsitxNode: avg =  17150 usecs, min/max =  16697 /  23966 usecs, #executions =      57779
     NODE:       DSS_M2M2:           DisplayM2mNode: avg =   4917 usecs, min/max =   2464 /  11441 usecs, #executions =      57779
     NODE:          A72-0:     PARKLANE_DETECT_NODE: avg =     27 usecs, min/max =     15 /    926 usecs, #executions =      57779
     NODE:      VPAC_LDC1:                 ldc_node: avg =  11094 usecs, min/max =   8870 /  21440 usecs, #executions =      57779
    
     PERF:         AvmGraph: avg =  41607 usecs, min/max =  26870 / 271940 usecs, #executions =      57780
     PERF:         AvmGraph:   24. 3 FPS
    
    GRAPH:   avm_graph_tidl (#nodes =   8, #executions =  57780)
     NODE:      VPAC_MSC2:               ScalerNode: avg =   3894 usecs, min/max =   3324 /  12594 usecs, #executions =      57780
     NODE:          DSP-1:            SDPreProcNode: avg =   1523 usecs, min/max =   1442 /   1791 usecs, #executions =      57780
     NODE:       DSP_C7-1:          StopperTIDLNode: avg =   8412 usecs, min/max =   5795 /  11667 usecs, #executions =      57780
     NODE:          DSP-1:      PostProcStopperNode: avg =     38 usecs, min/max =     17 /    189 usecs, #executions =      57780
     NODE:       DSP_C7-1:               SDTIDLNode: avg =   6365 usecs, min/max =   5286 /  10893 usecs, #executions =      57780
     NODE:          DSP-1:           PostProcSdNode: avg =     97 usecs, min/max =     57 /    188 usecs, #executions =      57780
     NODE:          A72-0:        LZQSlotFusionNode: avg =     54 usecs, min/max =     18 /   1078 usecs, #executions =      57780
     NODE:          DSP-1:    LZQDrawDetectionsNode: avg =   3078 usecs, min/max =   1717 /   6600 usecs, #executions =      57780
    
     PERF:     AvmTidlGraph: avg =  41607 usecs, min/max =  26849 / 271809 usecs, #executions =      57780
     PERF:     AvmTidlGraph:   24. 3 FPS
    
    GRAPH:       tidl_graph (#nodes =   9, #executions =  28280)
     NODE:      VPAC_MSC1:               ScalerNode: avg =  17428 usecs, min/max =  13635 /  26175 usecs, #executions =      28280
     NODE:          A72-0:   DIRTDETECT_DETECT_NODE: avg =     64 usecs, min/max =     24 /   1263 usecs, #executions =      28280
     NODE:          DSP-1:           img_merge_node: avg =    341 usecs, min/max =    233 /    606 usecs, #executions =      28280
     NODE:          DSP-1:            ODPreProcNode: avg =   2034 usecs, min/max =   1925 /   2521 usecs, #executions =      28280
     NODE:       DSP_C7-1:               ODTIDLNode: avg =  21861 usecs, min/max =  17420 /  26182 usecs, #executions =      28280
     NODE:          DSP-1:    DrawBoxDetectionsNode: avg =   4330 usecs, min/max =   2149 /   6150 usecs, #executions =      28280
     NODE:      VPAC_MSC2:            mosaic_node_1: avg =   5580 usecs, min/max =   3468 /  34944 usecs, #executions =      28280
     NODE:       DSS_M2M2:           ConvertM2mNode: avg =   4416 usecs, min/max =   3502 /   8505 usecs, #executions =      28280
     NODE:          A72-0:           AVM Putod Node: avg =     43 usecs, min/max =     17 /   1081 usecs, #executions =      28280
    
     PERF:        TidlGraph: avg =  84975 usecs, min/max =     44 / 387972 usecs, #executions =      28283
     PERF:        TidlGraph:   11.76 FPS

    Thanks

    Regards

    quanfeng

  • Quanfeng,

    Let me check with our team internally, to see if they've ever seen such a hang and what we can check for A72 heart-beat.

    Regards,

    Erick

  • quanfeng,

    Is there a way to reproduce this on the TI EVM? We want to rule out a hardware related failure.

    Regards,

    Erick

  • Hi Erick,

    Is there a way to reproduce this on the TI EVM?

    The TI EVM uses the TDA4VE in speed grade T, which has a larger DDR bandwidth than the TDA4VL in speed grade H. Additionally, we customized our boards with DDR and deserializer chip changes compared to the TI EVM. Reproducing the problem in TI EVM won't help.

    We want to rule out a hardware related failure.

    What is the hardware related failure that is causing this problem? Can an external monitoring device (e.g., logic analyzer) be used to rule out the hardware failure?

    Regards

    quanfeng

  • Hi,

    The TI EVM uses the TDA4VE in speed grade T, which has a larger DDR bandwidth than the TDA4VL in speed grade H. Additionally, we customized our boards with DDR and deserializer chip changes compared to the TI EVM. Reproducing the problem in TI EVM won't help.

    This would help to understand if it is a hardware issue or a software issue. If the issue reproduces on the EVM we can have more confidence that it is not a hardware issue.

    What is the hardware related failure that is causing this problem? Can an external monitoring device (e.g., logic analyzer) be used to rule out the hardware failure?

    I'm not sure about this. I'll loop in our hardware team to see if they have any suggestions.

    Thanks,

    Erick

  • Also, can you provide on how many units this issue occurs on?

    Thanks,

    Erick

  • Hello Quanfeng,

    Can you check how the PMIC BUCK1_CTRL registers are set?

    Can you try overwriting the setting as follows:

    TI recommends that the PMIC mode should be modified via boot software at the beginning of the SoC boot process, before the SoC is fully operational. 
    The PMIC buck modes can be changed with the following control register writes:
    a)VDD_CPU_AVS rail:

    “PMIC-A” Addr: 0x48

    “BUCK1_CTRL” Reg Addr: 0x04

    “Set bit 2 (BUCK1_FPWM_MP)” = 1 for Forced to Multi-Phase (FMP) operation

    b)VDD_CORE_0V8 rail:

    “PMIC-C” Addr: 0x58

    “BUCK1_CTRL” Reg Addr: 0x04

    “Set bit 2 (BUCK1_FPWM_MP)” = 1 for Forced to Multi-Phase (FMP) operation

    Regards,

    Kyle

  • Hi TI experts,

    Also, can you provide on how many units this issue occurs on?

    We've tested more than a dozen boards, and this issue occurs on every board tested.

    Can you check how the PMIC BUCK1_CTRL registers are set?

    The BUCK1_CTRL register value for both PMIC-A and PMIC-C is 0x33.

    When I set the BUCK1_CTRL register value to 0x37 for PMIC-A and PMIC-C, A72 still only runs for about 3 hours. 

    When I run the following script in the background to regularly clear the IO cache, A72 can run for about 10 hours before it hangs. Why does this happen?

    #!/bin/sh
    #set buff cache limit
    config_max_buffer=10240
    # config_cycle_time s
    config_cycle_time=3
    while [ 1 ]
    do
        Buffer=$(cat /proc/meminfo | awk 'NR==4' | awk '{print $2}')
        #echo Buffer:$Buffer
        if [ ${config_max_buffer} -lt ${Buffer} ]
        then
            #Buffer > config_max_buffer
            sync
            echo  3 > /proc/sys/vm/drop_caches
            #echo "Trigger release cache"
        fi
        sleep ${config_cycle_time}
    done

    Thanks

    Regards

    quanfeng

  • Hello Quanfeng,

      When the system crash, could you please use the below scrip works on MCU2_0 to dump the currently  PC of A72_Core0 and Core1  to check.

    /cfs-file/__key/communityserver-discussions-components-files/791/dump_5F00_a72_5F00_core.gel

      Thanks.

    Linjun

  • Hi Linjun,

    The output of the script is as follows:

    MAIN_Cortex_R5_0_0: GEL Output: core0_PC : 0xFFFF8000100E0674
    MAIN_Cortex_R5_0_0: GEL Output: core1_PC : 0xFFFF8000100186F0

    Here is our memory map, it seems that A72 hangs in timer interrupt and context switching. How to solve it ?

    kallsyms.log

    Thanks

    Regards

    quanfeng

  • Hello Quanfeng,

        I am not sure I can get the clue by these two register.   Could you please share your vmlinux image to us?

        And Please use the below patch to get the __log_buf info do further check.

        /cfs-file/__key/communityserver-discussions-components-files/791/log_5F00_buf_5F00_phy.patch

     Thanks.

    Linjun

  • Hi TI experts,

    We found a conflict between the vpu_sram node of msmc_ram in the Linux device tree and the existing MSMC rtos memory allocation. We suspect that this is the cause of the problem, as it has now been running for 6 days without hanging when we don't load the VPU driver.

    Thanks

    Regards

    quanfeng

  • Hi quanfeng,

    Thanks for digging into this. Please share the code changes on E2E for future reference.

    Thanks,
    Keerthy

  • Hi Keerthy,

    Our changes are as follows, we are still doing more testing and will update the ticket when we have results.

    Thanks
    Regards
    quanfeng

  • Hi Quanfeng,

         Could you please update the log_buf info to us when sytem crashed.  

    Thanks.

    LinJun

  • Hi TI experts,

    After commenting insmod the VPU driver to avoid memory conflicts, it still hangs, and the frequency of hangs is variable, sometimes it can run for more than 6 days, sometimes it can only run for 1 day.

    Here is our log_buf info, there is no valid information in log_buf.

    Booting Linux on physical CPU 0x0000000000 [0x411fd080]
    Linux version 5.10.162-g76b3e88d56 (zhubing@RD17) (aarch64-none-linux-gnu-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025, GNU ld (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209) #1 SMP PREEMPT Wed Dec 27 15:14:14 CST 2023
    Machine model: Texas Instruments J721S2 EVM
    earlycon: ns16550a0 at MMIO32 0x0000000002850000 (options '')
    printk: bootconsole [ns16550a0] enabled
    efi: UEFI not found.
    Reserved memory: created DMA memory pool at 0x00000000a0000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a0000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a0100000, size 15 MiB
    OF: reserved mem: initialized node vision-apps-r5f-memory@a0100000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a1000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a1000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a1100000, size 15 MiB
    OF: reserved mem: initialized node vision-apps-r5f-memory@a1100000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a2000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a2000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a2100000, size 31 MiB
    OF: reserved mem: initialized node vision-apps-r5f-memory@a2100000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a4000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a4000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a4100000, size 31 MiB
    OF: reserved mem: initialized node vision-apps-r5f-memory@a4100000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000a8000000, size 32 MiB
    OF: reserved mem: initialized node vision-apps-rtos-ipc-memory-region@a8000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000aa000000, size 96 MiB
    OF: reserved mem: initialized node vision-apps-dma-memory@aa000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000b0000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-c71-dma-memory@b0000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000b0100000, size 95 MiB
    OF: reserved mem: initialized node vision-apps-c71_0-memory@b0100000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000b6000000, size 1 MiB
    OF: reserved mem: initialized node vision-apps-c71_1-dma-memory@b6000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000b6100000, size 31 MiB
    OF: reserved mem: initialized node vision-apps-c71_1-memory@b6100000, compatible id shared-dma-pool
    OF: reserved mem: initialized node vision_apps_shared-memories, compatible id dma-heap-carveout
    Reserved memory: created DMA memory pool at 0x00000000d4000000, size 48 MiB
    OF: reserved mem: initialized node vision-apps-core-heap-memory-lo@d4000000, compatible id shared-dma-pool
    Reserved memory: created DMA memory pool at 0x00000000d7000000, size 464 MiB
    OF: reserved mem: initialized node vision-apps-core-heap-memory-hi@d7000000, compatible id shared-dma-pool
    Reserved memory: created CMA memory pool at 0x00000000f4000000, size 192 MiB
    OF: reserved mem: initialized node linux-cma-buffers@f4000000, compatible id shared-dma-pool
    Zone ranges:
      DMA      [mem 0x0000000080000000-0x00000000ffffffff]
      DMA32    empty,
      Normal   empty-
    Movable zone start for each node.
    Early memory node ranges/
      node   0: [mem 0x0000000080000000-0x000000009e7fffff]
      node   0: [mem 0x000000009e800000-0x00000000a5ffffff]
      node   0: [mem 0x00000000a6000000-0x00000000a7ffffff]
      node   0: [mem 0x00000000a8000000-0x00000000b7ffffff]
      node   0: [mem 0x00000000b8000000-0x00000000d3ffffff]
      node   0: [mem 0x00000000d4000000-0x00000000f3ffffff]
      node   0: [mem 0x00000000f4000000-0x00000000ffffffff]
    Initmem setup node 0 [mem 0x0000000080000000-0x00000000ffffffff]7
    On node 0 totalpages: 524288
      DMA zone: 8192 pages used for memmap
      DMA zone: 0 pages reserved
      DMA zone: 524288 pages, LIFO batch:63
    psci: probing for conduit method from DT.
    psci: PSCIv1.1 detected in firmware.
    psci: Using standard PSCI v0.2 function IDs
    psci: MIGRATE_INFO_TYPE not supported.
    psci: SMC Calling Convention v1.2
    percpu: Embedded 22 pages/cpu s50392 r8192 d31528 u90112A
    pcpu-alloc: s50392 r8192 d31528 u90112 alloc=22*4096
    pcpu-alloc: [0] 0 [0] 1 C
    Detected PIPT I-cache on CPU0
    CPU features: detected: GIC system register CPU interface
    CPU features: detected: EL2 vector hardening
    CPU features: detected: ARM errata 1165522, 1319367, or 1530923
    CPU features: detected: Spectre-BHB
    CPU features: detected: ARM erratum 1742098
    Built 1 zonelists, mobility grouping on.  Total pages: 516096
    Kernel command line: console=ttyS2,115200n8 earlycon=ns16550a,mmio32,0x02850000 mtdparts=47040000.spi.0:512k(ospi.tiboot3),512k(ospi.tifs),128k(ospi.env),128k(ospi.env.backup),256k(ospi.upflag),3072k(ospi.mcu_a),3072k(ospi.mcu_b),2560k(ospi.linux_a),2560k(ospi.linux_b),19904k(ospi.data),-@32704k(ospi.phypattern) loglevel=0 root=/dev/mmcblk0p2 rw rootfstype=ext4 rootwait longhorn_uboot_version_major=2 longhorn_uboot_version=20240312
    Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
    Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)M
    mem auto-init: stack:off, heap alloc:off, heap free:off
    Memory: 474684K/2097152K available (11136K kernel code, 1156K rwdata, 4268K rodata, 1856K init, 433K bss, 1425860K reserved, 196608K cma-reserved)
    SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
    rcu: Preemptible hierarchical RCU implementation.
    rcu: 	RCU event tracing is enabled.
    rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=2.
    	Trampoline variant of Tasks RCU enabled.
    	Tracing variant of Tasks RCU enabled.
    rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
    rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
    NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
    GICv3: GIC: Using split EOI/Deactivate mode
    GICv3: 960 SPIs implemented
    GICv3: 0 Extended SPIs implemented
    GICv3: Distributor has no Range Selector support\
    GICv3: 16 PPIs implemented
    GICv3: CPU0: found redistributor 0 region 0:0x0000000001900000
    ITS [mem 0x01820000-0x0182ffff]
    GIC: enabling workaround for ITS: Socionext Synquacer pre-ITS
    ITS@0x0000000001820000: Devices Table too large, reduce ids 20->19
    ITS@0x0000000001820000: allocated 524288 Devices @80800000 (flat, esz 8, psz 64K, shr 0)b
    ITS: using cache flushing for cmd queue
    GICv3: using LPI property table @0x0000000080030000
    GIC: using cache flushing for LPI property tablee
    GICv3: CPU0: using allocated LPI pending table @0x0000000080040000
    arch_timer: cp15 timer(s) running at 200.00MHz (phys).
    clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x2e2049d3e8, max_idle_ns: 440795210634 ns
    sched_clock: 56 bits at 200MHz, resolution 5ns, wraps every 4398046511102ns
    Console: colour dummy device 80x25
    Calibrating delay loop (skipped), value calculated using timer frequency.. 400.00 BogoMIPS (lpj=800000)
    pid_max: default: 32768 minimum: 301
    LSM: Security Framework initializing
    Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
    Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
    rcu: Hierarchical SRCU implementation.
    Platform MSI: msi-controller@1820000 domain created
    PCI/MSI: /bus@100000/interrupt-controller@1800000/msi-controller@1820000 domain created
    EFI services will not be available.
    smp: Bringing up secondary CPUs ...
    Detected PIPT I-cache on CPU1
    GICv3: CPU1: found redistributor 1 region 0:0x0000000001920000
    GICv3: CPU1: using allocated LPI pending table @0x0000000080050000
    CPU1: Booted secondary processor 0x0000000001 [0x411fd080]
    smp: Brought up 1 node, 2 CPUs
    SMP: Total of 2 processors activated.
    CPU features: detected: 32-bit EL0 Support
    CPU features: detected: CRC32 instructions
    CPU: All CPU(s) started at EL2
    alternatives: patching kernel code
    devtmpfs: initialized
    KASLR disabled due to lack of seed
    clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
    futex hash table entries: 512 (order: 3, 32768 bytes, linear)
    pinctrl core: initialized pinctrl subsystem
    DMI not present or invalid.
    NET: Registered protocol family 16
    DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
    DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
    DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
    thermal_sys: Registered thermal governor 'step_wise'
    thermal_sys: Registered thermal governor 'power_allocator'
    hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
    ASID allocator initialised with 65536 entries
    HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
    HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
    HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
    HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
    cryptd: max_cpu_qlen set to 1000
    k3-chipinfo 43000014.chipid: Family:J721S2 rev:SR1.0 JTAGID[0x0bb7502f] Detected
    vdd_mmc1: supplied by evm_12v0
    iommu: Default domain type: Translated 
    SCSI subsystem initialized
    mc: Linux media interface: v0.10
    videodev: Linux video capture interface: v2.00
    pps_core: LinuxPPS API ver. 1 registered
    pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    PTP clock support registered
    EDAC MC: Ver: 3.0.0
    FPGA manager framework
    Advanced Linux Sound Architecture Driver Initialized.
    clocksource: Switched to clocksource arch_sys_counter
    VFS: Disk quotas dquot_6.6.0
    VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    Carveout Heap: Exported 448 MiB at 0x00000000b8000000
    NET: Registered protocol family 2
    IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
    tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
    TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
    TCP bind hash table entries: 16384 (order: 6, 262144 bytes, linear)
    TCP: Hash tables configured (established 16384 bind 16384)
    UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
    UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
    NET: Registered protocol family 1
    RPC: Registered named UNIX socket transport module.
    RPC: Registered udp transport module.
    RPC: Registered tcp transport module.
    RPC: Registered tcp NFSv4.1 backchannel transport module.
    NET: Registered protocol family 44
    PCI: CLS 0 bytes, default 64
    hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
    Initialise system trusted keyrings
    workingset: timestamp_bits=46 max_order=18 bucket_order=0
    squashfs: version 4.0 (2009/01/31) Phillip Lougher
    NFS: Registering the id_resolver key type
    Key type id_resolver registered
    Key type id_legacy registered
    nfs4filelayout_init: NFSv4 File Layout Driver Registering...
    nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
    9p: Installing v9fs 9p2000 file system support
    Key type asymmetric registered
    Asymmetric key parser 'x509' registered
    Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
    io scheduler mq-deadline registered
    io scheduler kyber registered
    pinctrl-single 4301c000.pinctrl: 101 pins, size 404
    pinctrl-single 11c000.pinctrl: 72 pins, size 288
    Serial: 8250/16550 driver, 10 ports, IRQ sharing enabled
    brd: module loaded
    loop: module loaded
    megasas: 07.714.04.00-rc1
    tun: Universal TUN/TAP device driver, 1.6
    igbvf: Intel(R) Gigabit Virtual Function Network Driver
    igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
    sky2: driver version 1.30
    VFIO - User Level meta-driver version: 0.3
    i2c /dev entries driver
    sdhci: Secure Digital Host Controller Interface driver
    sdhci: Copyright(c) Pierre Ossman
    sdhci-pltfm: SDHCI platform and OF driver helper
    ledtrig-cpu: registered to indicate activity on CPUs
    SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
    NET: Registered protocol family 17
    9pnet: Installing 9P2000 support
    Key type dns_resolver registered
    Loading compiled-in X.509 certificates
    OF: /gpio-regulator-TLV71033: could not get #gpio-cells for /bus@100000/gpio@600000
    ti-sci 44083000.system-controller: ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
    ti-sci-intr 42200000.interrupt-controller: Interrupt Router 125 domain created
    ti-sci-intr bus@100000:interrupt-controller@a00000: Interrupt Router 148 domain created
    ti-sci-intr 310e0000.interrupt-controller: Interrupt Router 227 domain created
    ti-sci-inta 33d00000.msi-controller: Interrupt Aggregator domain 265 created
    ti-udma 311a0000.dma-controller: Number of rings: 48
    ti-udma 311a0000.dma-controller: Channels: 24 (bchan: 0, tchan: 8, rchan: 16)
    k3-ringacc 2b800000.ringacc: Ring Accelerator probed rings:286, gp-rings[96,20] sci-dev-id:272
    k3-ringacc 2b800000.ringacc: dma-ring-reset-quirk: disabled
    k3-ringacc 2b800000.ringacc: RA Proxy rev. 66349100, num_proxies:64
    k3-ringacc 3c000000.ringacc: Ring Accelerator probed rings:1024, gp-rings[878,128] sci-dev-id:259
    k3-ringacc 3c000000.ringacc: dma-ring-reset-quirk: disabled
    k3-ringacc 3c000000.ringacc: RA Proxy rev. 66349100, num_proxies:64
    printk: console [ttyS2] disabled
    2850000.serial: ttyS2 at MMIO 0x2850000 (irq = 16, base_baud = 3000000) is a 8250
    printk: console [ttyS2] enabled
    printk: bootconsole [ns16550a0] disabled
    cadence-qspi 47040000.spi: error -ENODEV: No Rx DMA available
    spi-nor spi0.0: gd25lx256e (32768 Kbytes)
    11 cmdlinepart partitions found on MTD device 47040000.spi.0
    Creating 11 MTD partitions on "47040000.spi.0":
    0x000000000000-0x000000080000 : "ospi.tiboot3"
    0x000000080000-0x000000100000 : "ospi.tifs"
    0x000000100000-0x000000120000 : "ospi.env"
    0x000000120000-0x000000140000 : "ospi.env.backup"
    0x000000140000-0x000000180000 : "ospi.upflag"
    0x000000180000-0x000000480000 : "ospi.mcu_a"
    0x000000480000-0x000000780000 : "ospi.mcu_b"
    0x000000780000-0x000000a00000 : "ospi.linux_a"
    0x000000a00000-0x000000c80000 : "ospi.linux_b"
    0x000000c80000-0x000001ff0000 : "ospi.data"
    0x000001ff0000-0x000002000000 : "ospi.phypattern"
    mmc0: CQHCI version 5.10
    mmc1: CQHCI version 5.10
    sdhci-am654 4fb0000.mmc: SD Card not insert, Cancel Power on by zhub
    davinci_gpio 600000.gpio: IRQ index 1 not found
    davinci_gpio 600000.gpio: error -ENXIO: IRQ not populated
    omap-mailbox 31f80000.mailbox: omap mailbox rev 0x66fca100
    omap-mailbox 31f81000.mailbox: omap mailbox rev 0x66fca100
    omap-mailbox 31f84000.mailbox: omap mailbox rev 0x66fca100
    ti-udma 285c0000.dma-controller: Channels: 26 (tchan: 13, rchan: 13, gp-rflow: 8)
    ti-udma 31150000.dma-controller: Channels: 60 (tchan: 30, rchan: 30, gp-rflow: 16)
    debugfs: Directory 'pd:39' with parent 'pm_genpd' already present!
    debugfs: Directory 'pd:38' with parent 'pm_genpd' already present!
    debugfs: Directory 'pd:276' with parent 'pm_genpd' already present!
    debugfs: Directory 'pd:154' with parent 'pm_genpd' already present!
    mmc0: SDHCI controller on 4f80000.mmc [4f80000.mmc] using ADMA 64-bit
    mmc1: SDHCI controller on 4fb0000.mmc [4fb0000.mmc] using ADMA 64-bit
    ALSA device list:
      No soundcards found.
    Waiting for root device /dev/mmcblk0p2...
    mmc0: Command Queue Engine enabled
    mmc0: new HS400 MMC card at address 0001
    mmcblk0: mmc0:0001 8GUF4R 7.28 GiB 
    mmcblk0boot0: mmc0:0001 8GUF4R partition 1 31.9 MiB
    mmcblk0boot1: mmc0:0001 8GUF4R partition 2 31.9 MiB
    mmcblk0rpmb: mmc0:0001 8GUF4R partition 3 4.00 MiB, chardev (237:0)
     mmcblk0: p1 p2 p3 p4 < p5 p6 p7 p8 p9 p10 p11 p12 >
    EXT4-fs (mmcblk0p2): recovery complete
    EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
    VFS: Mounted root (ext4 filesystem) on device 179:2.
    devtmpfs: mounted
    Freeing unused kernel memory: 1856K
    Run /sbin/init as init process
      with arguments:
        /sbin/init
      with environment:
        HOME=/
        TERM=linux
        longhorn_uboot_version_major=2
        longhorn_uboot_version=20240312
    EXT4-fs (mmcblk0p5): recovery complete
    EXT4-fs (mmcblk0p5): mounted filesystem with ordered data mode. Opts: (null)
    EXT4-fs (mmcblk0p5): re-mounted. Opts: (null)
    EXT4-fs (mmcblk0p6): recovery complete
    EXT4-fs (mmcblk0p6): mounted filesystem with ordered data mode. Opts: (null)
    EXT4-fs (mmcblk0p6): re-mounted. Opts: (null)
    EXT4-fs (mmcblk0p9): recovery complete
    EXT4-fs (mmcblk0p9): mounted filesystem with ordered data mode. Opts: (null)
    EXT4-fs (mmcblk0p9): re-mounted. Opts: (null)
    EXT4-fs (mmcblk0p10): recovery complete
    EXT4-fs (mmcblk0p10): mounted filesystem with ordered data mode. Opts: (null)
    EXT4-fs (mmcblk0p10): re-mounted. Opts: (null)
    EXT4-fs (mmcblk0p11): recovery complete
    EXT4-fs (mmcblk0p11): mounted filesystem with ordered data mode. Opts: (null)
    EXT4-fs (mmcblk0p11): re-mounted. Opts: (null)
    squashfs: Unknown parameter 'iocharset'
    FAT-fs (mmcblk0p12): utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
    FAT-fs (mmcblk0p12): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
    platform 41000000.r5f: R5F core may have been powered on by a different host, programmed state (0) != actual state (1)
    platform 41000000.r5f: configured R5F for IPC-only mode
    platform 41000000.r5f: assigned reserved memory node vision-apps-r5f-dma-memory@a00000001
    remoteproc remoteproc0: 41000000.r5f is available
    remoteproc remoteproc0: attaching to 41000000.r5f
    platform 41000000.r5f: R5F core initialized in IPC-only mode
     remoteproc0#vdev0buffer: assigned reserved memory node vision-apps-r5f-dma-memory@a0000000
    virtio_rpmsg_bus virtio0: rpmsg host is online
     remoteproc0#vdev0buffer: registered virtio0 (type 7)
    remoteproc remoteproc0: remote processor 41000000.r5f is now attached
    platform 41400000.r5f: R5F core may have been powered on by a different host, programmed state (0) != actual state (1)
    platform 41400000.r5f: configured R5F for IPC-only mode
    platform 41400000.r5f: assigned reserved memory node vision-apps-r5f-dma-memory@a1000000;
    remoteproc remoteproc1: 41400000.r5f is available
    platform 5c00000.r5f: configured R5F for IPC-only mode
    platform 5c00000.r5f: assigned reserved memory node vision-apps-r5f-dma-memory@a2000000
    remoteproc remoteproc2: 5c00000.r5f is available?
    remoteproc remoteproc2: attaching to 5c00000.r5f@
    platform 5c00000.r5f: R5F core initialized in IPC-only mode
     remoteproc2#vdev0buffer: assigned reserved memory node vision-apps-r5f-dma-memory@a2000000
    virtio_rpmsg_bus virtio1: rpmsg host is online
     remoteproc2#vdev0buffer: registered virtio1 (type 7)
    remoteproc remoteproc2: remote processor 5c00000.r5f is now attached
    platform 5d00000.r5f: configured R5F for IPC-only mode
    platform 5d00000.r5f: assigned reserved memory node vision-apps-r5f-dma-memory@a4000000
    remoteproc remoteproc3: 5d00000.r5f is availableH
    remoteproc remoteproc3: attaching to 5d00000.r5fI
    platform 5d00000.r5f: R5F core initialized in IPC-only mode
     remoteproc3#vdev0buffer: assigned reserved memory node vision-apps-r5f-dma-memory@a4000000
    virtio_rpmsg_bus virtio2: rpmsg host is online
     remoteproc3#vdev0buffer: registered virtio2 (type 7)
    remoteproc remoteproc3: remote processor 5d00000.r5f is now attached
    k3-dsp-rproc 64800000.dsp: assigned reserved memory node vision-apps-c71-dma-memory@b0000000
    virtio_rpmsg_bus virtio1: creating channel rpmsg_chrdev addr 0xdP
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existS
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existV
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existY
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already exist\
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already exist_
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existb
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existe
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x38
    virtio_rpmsg_bus virtio2: channel rpmsg_chrdev:ffffffff:38 already existh
    virtio_rpmsg_bus virtio2: rpmsg_create_channel failed
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x37
    virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x15
    virtio_rpmsg_bus virtio2: creating channel ti.ipc4.ping-pong addr 0xe
    k3-dsp-rproc 64800000.dsp: configured DSP for IPC-only mode
    remoteproc remoteproc4: 64800000.dsp is available
    remoteproc remoteproc4: attaching to 64800000.dsp
    remoteproc remoteproc4: unsupported resource 65538
    k3-dsp-rproc 64800000.dsp: DSP initialized in IPC-only mode
     remoteproc4#vdev0buffer: assigned reserved memory node vision-apps-c71-dma-memory@b0000000
    virtio_rpmsg_bus virtio3: rpmsg host is online
     remoteproc4#vdev0buffer: registered virtio3 (type 7)
    remoteproc remoteproc4: remote processor 64800000.dsp is now attached
    virtio_rpmsg_bus virtio3: creating channel rpmsg_chrdev addr 0xdv
    k3-dsp-rproc 65800000.dsp: assigned reserved memory node vision-apps-c71_1-dma-memory@b6000000
    k3-dsp-rproc 65800000.dsp: configured DSP for IPC-only mode
    remoteproc remoteproc5: 65800000.dsp is available
    remoteproc remoteproc5: attaching to 65800000.dsp
    remoteproc remoteproc5: unsupported resource 65538
    k3-dsp-rproc 65800000.dsp: DSP initialized in IPC-only mode
     remoteproc5#vdev0buffer: assigned reserved memory node vision-apps-c71_1-dma-memory@b6000000
    virtio_rpmsg_bus virtio4: rpmsg host is online
     remoteproc5#vdev0buffer: registered virtio4 (type 7)
    remoteproc remoteproc5: remote processor 65800000.dsp is now attached
    virtio_rpmsg_bus virtio4: creating channel rpmsg_chrdev addr 0xd
    virtio_rpmsg_bus virtio4: creating channel rpmsg_chrdev addr 0x15
    virtio_rpmsg_bus virtio3: creating channel rpmsg_chrdev addr 0x15
    virtio_rpmsg_bus virtio1: creating channel rpmsg_chrdev addr 0x15
    virtio_rpmsg_bus virtio3: creating channel ti.ipc4.ping-pong addr 0xe
    virtio_rpmsg_bus virtio4: creating channel ti.ipc4.ping-pong addr 0xe
    pvrsrvkm: loading out-of-tree module taints kernel.
    PVR_K:  187: Read BVNC 36.53.104.796 from HW device registers
    PVR_K:  187: RGX Device registered BVNC 36.53.104.796 with 1 core in the system
    [drm] Initialized pvr 1.15.6133109 20170530 for 4e20000000.gpu on minor 0
    virtio_rpmsg_bus virtio1: creating channel ti.ipc4.ping-pong addr 0xe
    virtio_rpmsg_bus virtio0: creating channel rpmsg_chrdev addr 0x21
    virtio_rpmsg_bus virtio0: creating channel rpmsg_chrdev addr 0x22
    PVR_K:  201: RGX Firmware image 'rgx.fw.36.53.104.796' loaded
    PVR_K:  201: Shader binary image 'rgx.sh.36.53.104.796' loaded
    usbcore: registered new interface driver usbfs
    usbcore: registered new interface driver hub
    usbcore: registered new device driver usb
    usbcore: registered new interface driver usbhid
    usbhid: USB HID core driver
    am65-cpts 310d0000.cpts: CPTS ver 0x4e8a010c, freq:200000000, add_val:4 pps:0
    davinci_mdio c200f00.mdio: Configuring MDIO in manual mode
    davinci_mdio c200f00.mdio: davinci mdio revision 9.7, bus freq 1000000
    davinci_mdio c200f00.mdio: phy[0]: device c200f00.mdio:00, driver unknown
    am65-cpsw-nuss c200000.ethernet: initializing am65 cpsw nuss version 0x6BA02102, cpsw version 0x6BA82102 Ports: 2 quirks:00000000
    am65-cpsw-nuss c200000.ethernet: Use random MAC address
    am65-cpsw-nuss c200000.ethernet: initialized cpsw ale version 1.4
    am65-cpsw-nuss c200000.ethernet: ALE Table size 64
    am65-cpsw-nuss c200000.ethernet: CPTS ver 0x4e8a010b, freq:200000000, add_val:4 pps:0
    am65-cpsw-nuss c200000.ethernet: set new flow-id-base 82
    dp83812_read_straps: Strap is 0x11C0
    dp83812_read_straps: Strap is 0x11C0
    random: avp_master.out: uninitialized urandom read (8 bytes read)
    random: avp_master.out: uninitialized urandom read (8 bytes read)
    am65-cpsw-nuss c200000.ethernet eth0: PHY [c200f00.mdio:00] driver [TI DP83TC813R-Q1] (irq=POLL)
    am65-cpsw-nuss c200000.ethernet eth0: configuring for phy/rgmii-id link mode
    8021q: 802.1Q VLAN Support v1.8
    8021q: adding VLAN 0 to HW filter on device eth0
    am65-cpsw-nuss c200000.ethernet: Adding vlan 69 to vlan filter
    am65-cpsw-nuss c200000.ethernet: get: wrong ale fld id 2
    am65-cpsw-nuss c200000.ethernet: get: wrong ale fld id 1
    device eth0 entered promiscuous mode
    file system registered
    read descriptors
    read strings
    random: crng init done
    "\Ke
    "\Ke
    1GB.max
    1GB.rsvd.max
    1GB.current
    1GB.rsvd.current
    1GB.events
    1GB.events.local
    1GB.limit_in_bytes
    1GB.rsvd.limit_in_bytes
    1GB.usage_in_bytes
    1GB.rsvd.usage_in_bytes
    1GB.max_usage_in_bytes
    1GB.rsvd.max_usage_in_bytes
    1GB.failcnt
    1GB.rsvd.failcnt
    hugepages-1048576kB
    32MB.max
    32MB.rsvd.max
    32MB.current
    32MB.rsvd.current
    32MB.events
    32MB.events.local
    32MB.limit_in_bytes
    32MB.rsvd.limit_in_bytes
    32MB.usage_in_bytes
    32MB.rsvd.usage_in_bytes
    32MB.max_usage_in_bytes
    32MB.rsvd.max_usage_in_bytes
    32MB.failcnt
    32MB.rsvd.failcnt
    hugepages-32768kB
    2MB.max
    2MB.rsvd.max
    2MB.current
    2MB.rsvd.current
    2MB.events
    2MB.events.local
    2MB.limit_in_bytes
    2MB.rsvd.limit_in_bytes
    2MB.usage_in_bytes
    2MB.rsvd.usage_in_bytes
    2MB.max_usage_in_bytes
    2MB.rsvd.max_usage_in_bytes
    2MB.failcnt
    2MB.rsvd.failcnt
    hugepages-2048kB
    64KB.max
    64KB.rsvd.max
    64KB.current
    64KB.rsvd.current
    64KB.events
    64KB.events.local
    64KB.limit_in_bytes
    64KB.rsvd.limit_in_bytes
    64KB.usage_in_bytes
    64KB.rsvd.usage_in_bytes
    64KB.max_usage_in_bytes
    64KB.rsvd.max_usage_in_bytes
    64KB.failcnt
    64KB.rsvd.failcnt
    hugepages-64kB
    linux-cma-buffers@f4000000
     !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
    stdrng
    drbg_pr_hmac_sha1
    stdrng
    drbg_pr_hmac_sha384
    stdrng
    drbg_pr_hmac_sha512
    stdrng
    drbg_pr_hmac_sha256
    stdrng
    drbg_nopr_hmac_sha1
    stdrng
    drbg_nopr_hmac_sha384
    stdrng
    drbg_nopr_hmac_sha512
    stdrng
    drbg_nopr_hmac_sha256
    								
    Texas Instruments J721S2 EVM (DT)
    $oJP
    	1-A=
    performance
    eK\ .
    eK\ 
     \Ke
    /bin/busybox.nosuid
    busybox.nosuid
    "\Ke.
    lOgf
    eK\"
    4s}C
    eK\ 	L_
    eK\ 
    eK\"
    eK\!(J
    eK\"
    eK\"
    eK\!.
    eK\!#
    4BO8
    eK\!
    eK\ 
    eK\ .
    eK\ 5*
    eK\ 
    eK\""
    eK\"*pB
    eK\ 
    eK\"*
    "5T7
    eK\!3
    eK\!8DL
    e	D;
    eK\"
    eK\"
    eK\!'
    eK\!
    eK\  l
    eK\ 
    eK\ 8DL
    UnZ*
    eK\ )
    eK\")
    eK\"#
    eK\ 
    eK\"2
    eK\!9
    eK\!/q
    eK\"
    eK\"
    eK\!&
    eK\!
    eK\ 
    eK\ 
    eK\ 3
    eK\ /
     qN^
    F`z1
    eK\ 
    eK\!
    eK\"6
    eK\ .
    eK\"
    eK\ #
    eK\"
    eK\ 
    eK\"(
    j?J_
    eK\!
    eK\"
    eK\!
    eK\!7
    ]}{Z
    eK\ ,
    eK\!
    eK\ 
    eK\!
    eK\"5*
    eK\ 0
    eK\!
    eK\"!`
    eK\ 
    eK\"	
    

    In one of the hangs we found that we could connect to core0 but not core1 with the debugger, the following is the function call stack for core0. Can you give us some suggestions?

    Thanks
    Regards
    quanfeng

  • Hi TI experts,

    In today's test there is backtrace output which reads as follows, what is the problem causing this? Can you give us some suggestions?

    Thanks
    Regards
    quanfeng

  • Quanfeng,

    Can you run the same test but with vdd_cpu boosted by ~50 mV?

    Thanks,

    Kyle

  • Hi Quanfeng,

       What's the page size?  Could you please enable the CONFIG_ARM64_64K_PAGES  and disable the CONFIG_ARM64_4K_PAGES in tisdk_j721s2-evm_defconfig to try and feedback?

    Linjun

       

      

  • Hi Linjun,

     What's the page size?

    Our linux page size is 4K.

    The following are, in order, the A72 register values for our normal operation and the full backtrace output log.

    521177 83
    R PC 0x0000000B 0x0000FFFF7F2CA978
    R SP 0x0000000B 0x0000FFFF60A183C0
    R LR 0x0000000B 0x0000FFFF7F2CA654
    R A64 Core Registers_PC 0x0000000B 0x0000FFFF7F2CA978
    R A64 Core Registers_SP 0x0000000B 0x0000FFFF60A183C0
    R A64 Core Registers_DSPSR_EL0 0x00000013 0x80000000
    R A64 Core Registers_LR 0x0000000B 0x0000FFFF7F2CA654
    R A64 Core Registers_X0 0x0000000B 0x0000FFFF381D7400
    R A64 Core Registers_X1 0x0000000B 0x0000FFFF381D8240
    R A64 Core Registers_X2 0x0000000B 0x0000000000000E02
    R A64 Core Registers_X3 0x0000000B 0x0000000000000000
    R A64 Core Registers_X4 0x0000000B 0x7FFFFFFF80000000
    R A64 Core Registers_X5 0x0000000B 0x0000FFFF60A18830
    R A64 Core Registers_X6 0x0000000B 0x0000000000000007
    R A64 Core Registers_X7 0x0000000B 0x0000FFFF381E7820
    R A64 Core Registers_X8 0x0000000B 0x0000000000000004
    R A64 Core Registers_X9 0x0000000B 0x0000FFFF60A18710
    R A64 Core Registers_X10 0x0000000B 0x0000000000000000
    R A64 Core Registers_X11 0x0000000B 0x00000000000000FF
    R A64 Core Registers_X12 0x0000000B 0x0000000000000040
    R A64 Core Registers_X13 0x0000000B 0x0000000000000000
    R A64 Core Registers_X14 0x0000000B 0x0000000000000000
    R A64 Core Registers_X15 0x0000000B 0x00000000FFFFFFFF
    R A64 Core Registers_X16 0x0000000B 0x0000FFFF7F6F5968
    R A64 Core Registers_X17 0x0000000B 0x0000FFFF7F350848
    R A64 Core Registers_X18 0x0000000B 0x00000000FFFFFFFF
    R A64 Core Registers_X19 0x0000000B 0x0000FFFF60A18530
    R A64 Core Registers_X20 0x0000000B 0x0000000000000E11
    R A64 Core Registers_X21 0x0000000B 0x0000FFFF381D7A10
    R A64 Core Registers_X22 0x0000000B 0x0000FFFF381D6BD0
    R A64 Core Registers_X23 0x0000000B 0x0000000000000000
    R A64 Core Registers_X24 0x0000000B 0x0000000000000E11
    R A64 Core Registers_X25 0x0000000B 0x0000000000000E11
    R A64 Core Registers_X26 0x0000000B 0x0000000000000001
    R A64 Core Registers_X27 0x0000000B 0x0000FFFF60A184D0
    R A64 Core Registers_X28 0x0000000B 0x0000000000000830
    R A64 Core Registers_X29 0x0000000B 0x0000FFFF60A183C0
    R A64 Core Registers_X30 0x0000000B 0x0000FFFF7F2CA654
    R A32 Core Registers_PC 0x00000013 0x7F2CA978
    R A32 Core Registers_SP 0x00000013 0x60A183C0
    R A32 Core Registers_LR 0x00000013 0x7F2CA654
    R A32 Core Registers_DSPSR 0x00000013 0x80000000
    R SIMD Registers_V0_0 0x0000000B 0x0101010101010101
    R SIMD Registers_V0_1 0x0000000B 0x0101010101010101
    R SIMD Registers_V1_0 0x0000000B 0x7D7D7D7D7D7D7D7D
    R SIMD Registers_V1_1 0x0000000B 0x7D7D7D7D7D7D7D7D
    R SIMD Registers_V2_0 0x0000000B 0x0101010101010101
    R SIMD Registers_V2_1 0x0000000B 0x0101010101010101
    R SIMD Registers_V3_0 0x0000000B 0xFFFFFFFFFFFFFFFF
    R SIMD Registers_V3_1 0x0000000B 0xFFFFFFFFFFFFFFFF
    R SIMD Registers_V4_0 0x0000000B 0x42A80000C0C00000
    R SIMD Registers_V4_1 0x0000000B 0x42A0000042900000
    R SIMD Registers_V5_0 0x0000000B 0x0000FFFF60A18988
    R SIMD Registers_V5_1 0x0000000B 0x0000FFFF60A189D0
    R SIMD Registers_V6_0 0x0000000B 0x00000000EEEF2732
    R SIMD Registers_V6_1 0x0000000B 0x00000000EEEF2732
    R SIMD Registers_V7_0 0x0000000B 0x0000000042FF0000
    R SIMD Registers_V7_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V8_0 0x0000000B 0x0000000000000001
    R SIMD Registers_V8_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V9_0 0x0000000B 0x3FF0000000000000
    R SIMD Registers_V9_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V10_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V10_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V11_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V11_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V12_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V12_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V13_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V13_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V14_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V14_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V15_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V15_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V16_0 0x0000000B 0x0000000000000000
    R SIMD Registers_V16_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V17_0 0x0000000B 0x00000000EEEEEEEE
    R SIMD Registers_V17_1 0x0000000B 0x00000000EEEEEEEE
    R SIMD Registers_V18_0 0x0000000B 0x8020080280200802
    R SIMD Registers_V18_1 0x0000000B 0x8020080280200802
    R SIMD Registers_V19_0 0x0000000B 0xBF78237D80000000
    R SIMD Registers_V19_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V20_0 0x0000000B 0xBF78237DD877C8C0
    R SIMD Registers_V20_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V21_0 0x0000000B 0xBFEFFF8E80000000
    R SIMD Registers_V21_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V22_0 0x0000000B 0xC06BBC0843D1DE00
    R SIMD Registers_V22_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V23_0 0x0000000B 0xBFEFFF8ED337A580
    R SIMD Registers_V23_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V24_0 0x0000000B 0xBF71402C60000000
    R SIMD Registers_V24_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V25_0 0x0000000B 0xC062AF5280000000
    R SIMD Registers_V25_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V26_0 0x0000000B 0x00000000C3157A94
    R SIMD Registers_V26_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V27_0 0x0000000B 0xFFFFFFFFFFFFFFFF
    R SIMD Registers_V27_1 0x0000000B 0xFFFFFFFFFFFFFFFF
    R SIMD Registers_V28_0 0x0000000B 0xEE0040A6A400FCD5
    R SIMD Registers_V28_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V29_0 0x0000000B 0xE600E0F98400FCD5
    R SIMD Registers_V29_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V30_0 0x0000000B 0xDE00804C6300FCD5
    R SIMD Registers_V30_1 0x0000000B 0x0000000000000000
    R SIMD Registers_V31_0 0x0000000B 0xD600209F4400FCD5
    R SIMD Registers_V31_1 0x0000000B 0x0000000000000000
    R Control Registers_DLR_EL0 0x0000000B 0x0000FFFF7F2CA978
    R Control Registers_DSPSR_EL0 0x00000013 0x80000000
    R Control Registers_ELR_EL1 0x0000000B 0x0000FFFF7F024970
    R Control Registers_ELR_EL2 0x0000000B 0x0000000082F40008
    R Control Registers_ELR_EL3 0x0000000B 0xFFFF800010F75DD4
    R Control Registers_FPCR 0x00000013 0x00000000
    R Control Registers_FPSR 0x00000013 0x00000010
    R Control Registers_SP_EL1 0x0000000B 0xFFFF80001205C000
    R Control Registers_SP_EL2 0x0000000B 0x00000000FDEEC4C0
    R Control Registers_SPSR_abt 0x00000013 0x00000000
    R Control Registers_SPSR_EL1 0x00000013 0x80000000
    R Control Registers_SPSR_EL2 0x00000013 0x80000000
    R Control Registers_SPSR_EL3 0x00000013 0x80000000
    R Control Registers_SPSR_fiq 0x00000013 0x00000000
    R Control Registers_SPSR_irq 0x00000013 0x00000000
    R Control Registers_SPSR_und 0x00000013 0x00000000
    R A32 Banked Registers_R8_usr 0x00000013 0x00000004
    R A32 Banked Registers_R9_usr 0x00000013 0x60A18710
    R A32 Banked Registers_R10_usr 0x00000013 0x00000000
    R A32 Banked Registers_R11_usr 0x00000013 0x000000FF
    R A32 Banked Registers_R12_usr 0x00000013 0x00000040
    R A32 Banked Registers_SP_usr 0x00000013 0x00000000
    R A32 Banked Registers_LR_usr 0x00000013 0x00000000
    R A32 Banked Registers_R8_fiq 0x00000013 0x00000E11
    R A32 Banked Registers_R9_fiq 0x00000013 0x00000E11
    R A32 Banked Registers_R10_fiq 0x00000013 0x00000001
    R A32 Banked Registers_R11_fiq 0x00000013 0x60A184D0
    R A32 Banked Registers_R12_fiq 0x00000013 0x00000830
    R A32 Banked Registers_SP_fiq 0x00000013 0x60A183C0
    R A32 Banked Registers_LR_fiq 0x00000013 0x7F2CA654
    R A32 Banked Registers_SPSR_fiq 0x00000013 0x00000000
    R A32 Banked Registers_SP_irq 0x00000013 0x7F350848
    R A32 Banked Registers_LR_irq 0x00000013 0x7F6F5968
    R A32 Banked Registers_SPSR_irq 0x00000013 0x00000000
    R A32 Banked Registers_SP_svc 0x00000013 0x60A18530
    R A32 Banked Registers_LR_svc 0x00000013 0xFFFFFFFF
    R A32 Banked Registers_SPSR_svc 0x00000013 0x80000000
    R A32 Banked Registers_SP_abt 0x00000013 0x381D7A10
    R A32 Banked Registers_LR_abt 0x00000013 0x00000E11
    R A32 Banked Registers_SPSR_abt 0x00000013 0x00000000
    R A32 Banked Registers_SP_und 0x00000013 0x00000000
    R A32 Banked Registers_LR_und 0x00000013 0x381D6BD0
    R A32 Banked Registers_SPSR_und 0x00000013 0x00000000
    R A32 Banked Registers_SP_hyp 0x00000013 0x00000000
    R A32 Banked Registers_ELR_hyp 0x00000013 0x82F40008
    R A32 Banked Registers_SPSR_hyp 0x00000013 0x80000000
    R A32 Banked Registers_SPSR_mon 0x00000013 0x80000000
    R System Registers_ACTLR_EL1 0x00000013 0x00000000
    R System Registers_ACTLR_EL2 0x00000013 0x00000000
    R System Registers_ACTLR_EL3 0x00000013 0x00000000
    R System Registers_AFSR0_EL1 0x00000013 0x00000000
    R System Registers_AFSR0_EL2 0x00000013 0x00000000
    R System Registers_AFSR0_EL3 0x00000013 0x00000000
    R System Registers_AFSR1_EL1 0x00000013 0x00000000
    R System Registers_AFSR1_EL2 0x00000013 0x00000000
    R System Registers_AFSR1_EL3 0x00000013 0x00000000
    R System Registers_AIDR_EL1 0x00000013 0x00000000
    R System Registers_AMAIR_EL1 0x0000000B 0x0000000000000000
    R System Registers_AMAIR_EL2 0x0000000B 0x0000000000000000
    R System Registers_AMAIR_EL3 0x0000000B 0x0000000000000000
    R System Registers_CBAR_EL1 0x0000000B 0x000000006F000000
    R System Registers_CCSIDR_EL1 0x00000013 0x201FE012
    R System Registers_CLIDR_EL1 0x00000013 0x0A200023
    R System Registers_CONTEXTIDR_EL1 0x00000013 0x00000000
    R System Registers_CPACR_EL1 0x00000013 0x00300000
    R System Registers_CPTR_EL2 0x00000013 0x000033FF
    R System Registers_CPTR_EL3 0x00000013 0x00000000
    R System Registers_CPUACTLR_EL1 0x0000000B 0x0000000000000010
    R System Registers_CPUECTLR_EL1 0x0000000B 0x0000001B00000040
    R System Registers_CPUMERRSR_EL1 0x0000000B 0x0000000000000000
    R System Registers_CSSELR_EL1 0x00000013 0x00000001
    R System Registers_CTR_EL0 0x00000013 0x8444C004
    R System Registers_DACR32_EL2 0x00000013 0x00000000
    R System Registers_DCZID_EL0 0x00000013 0x00000004
    R System Registers_ESR_EL1 0x00000013 0x56000000
    R System Registers_ESR_EL2 0x00000013 0x00000000
    R System Registers_ESR_EL3 0x00000013 0x5E000000
    R System Registers_FAR_EL1 0x0000000B 0x0000FFFF61B9F000
    R System Registers_FAR_EL2 0x0000000B 0x0000000000000000
    R System Registers_FAR_EL3 0x0000000B 0x0000000000000000
    R System Registers_FPEXC32_EL2 0x00000013 0x00000700
    R System Registers_HACR_EL2 0x00000013 0x00000000
    R System Registers_HCR_EL2 0x0000000B 0x0000000080000000
    R System Registers_HPFAR_EL2 0x0000000B 0x0000000000000000
    R System Registers_HSTR_EL2 0x00000013 0x00000000
    R System Registers_ID_AA64AFR0_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AA64AFR1_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AA64DFR0_EL1 0x0000000B 0x0000000010305106
    R System Registers_ID_AA64DFR1_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AA64ISAR0_EL1 0x0000000B 0x0000000000011120
    R System Registers_ID_AA64ISAR1_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AA64MMFR0_EL1 0x0000000B 0x0000000000001124
    R System Registers_ID_AA64MMFR1_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AA64PFR0_EL1 0x0000000B 0x1100000001002222
    R System Registers_ID_AA64PFR1_EL1 0x0000000B 0x0000000000000000
    R System Registers_ID_AFR0_EL1 0x00000013 0x00000000
    R System Registers_ID_DFR0_EL1 0x00000013 0x03010066
    R System Registers_ID_ISAR0_EL1 0x00000013 0x02101110
    R System Registers_ID_ISAR1_EL1 0x00000013 0x13112111
    R System Registers_ID_ISAR2_EL1 0x00000013 0x21232042
    R System Registers_ID_ISAR3_EL1 0x00000013 0x01112131
    R System Registers_ID_ISAR4_EL1 0x00000013 0x00011142
    R System Registers_ID_ISAR5_EL1 0x00000013 0x00011121
    R System Registers_ID_MMFR0_EL1 0x00000013 0x10201105
    R System Registers_ID_MMFR1_EL1 0x00000013 0x40000000
    R System Registers_ID_MMFR2_EL1 0x00000013 0x01260000
    R System Registers_ID_MMFR3_EL1 0x00000013 0x02102211
    R System Registers_ID_PFR0_EL1 0x00000013 0x00010131
    R System Registers_ID_PFR1_EL1 0x00000013 0x10011011
    R System Registers_IFSR32_EL2 0x00000013 0x00000000
    R System Registers_ISR_EL1 0x00000013 0x00000080
    R System Registers_L2CTLR_EL1 0x00000013 0x01C03402
    R System Registers_L2ACTLR_EL1 0x00000013 0x00000010
    R System Registers_L2ECTLR_EL1 0x00000013 0x00000000
    R System Registers_L2MERRSR_EL1 0x0000000B 0x0000000000000000
    R System Registers_MAIR_EL1 0x0000000B 0x000C0400BB44FFFF
    R System Registers_MAIR_EL2 0x0000000B 0x000000FF440C0400
    R System Registers_MAIR_EL3 0x0000000B 0x00000000004404FF
    R System Registers_MIDR_EL1 0x00000013 0x411FD080
    R System Registers_MPIDR_EL1 0x0000000B 0x0000000080000000
    R System Registers_MVFR0_EL1 0x00000013 0x10110222
    R System Registers_MVFR1_EL1 0x00000013 0x12111111
    R System Registers_MVFR2_EL1 0x00000013 0x00000043
    R System Registers_PAR_EL1 0x0000000B 0x000000000000081F
    R System Registers_REVIDR_EL1 0x00000013 0x00000000
    R System Registers_RMR_EL3 0x00000013 0x00000001
    R System Registers_RVBAR_EL3 0x0000000B 0x0000000070000000
    R System Registers_SCR_EL3 0x00000013 0x0000073D
    R System Registers_SCTLR_EL1 0x00000013 0x34D4D91D
    R System Registers_SCTLR_EL2 0x00000013 0x30C50830
    R System Registers_SCTLR_EL3 0x00000013 0x30CD183F
    R System Registers_TCR_EL1 0x0000000B 0x00000034B5503510
    R System Registers_TCR_EL2 0x00000013 0x8081351C
    R System Registers_TCR_EL3 0x0000000B 0x0000000080803520
    R System Registers_TPIDR_EL0 0x0000000B 0x0000FFFF60A1D8A0
    R System Registers_TPIDR_EL1 0x0000000B 0xFFFF800016D78000
    R System Registers_TPIDR_EL2 0x0000000B 0x0000000000000000
    R System Registers_TPIDR_EL3 0x0000000B 0x0000000070011D00
    R System Registers_TPIDRR0_EL0 0x0000000B 0x0000000000000000
    R System Registers_TTBR0_EL1 0x0000000B 0x0000000081C31000
    R System Registers_TTBR0_EL2 0x0000000B 0x00000000FFFF0000
    R System Registers_TTBR0_EL3 0x0000000B 0x0000000070014E00
    R System Registers_TTBR1_EL1 0x0000000B 0x00B6000082F37000
    R System Registers_VBAR_EL1 0x0000000B 0xFFFF800010013000
    R System Registers_VBAR_EL2 0x0000000B 0x0000000082AF0000
    R System Registers_VBAR_EL3 0x0000000B 0x000000007000A000
    R System Registers_VMPIDR_EL2 0x0000000B 0x0000000080000000
    R System Registers_VPIDR_EL2 0x00000013 0x411FD080
    R System Registers_VTCR_EL2 0x00000013 0x80000000
    R System Registers_VTTBR_EL2 0x0000000B 0x0000000000000000
    R Debug Registers_DBGAUTHSTATUS_EL1 0x00000013 0x000000FF
    R Debug Registers_DBGBCR0_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBCR1_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBCR2_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBCR3_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBCR4_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBCR5_EL1 0x00000013 0x00000000
    R Debug Registers_DBGBVR0_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGBVR1_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGBVR2_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGBVR3_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGBVR4_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGBVR5_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGCLAIMCLR_EL1 0x00000013 0x00000000
    R Debug Registers_DBGCLAIMSET_EL1 0x00000013 0x000000FF
    R Debug Registers_DBGPRCR_EL1 0x00000013 0x00000000
    R Debug Registers_DBGVCR32_EL2 0x00000013 0x00000000
    R Debug Registers_DBGWCR0_EL1 0x00000013 0x00000000
    R Debug Registers_DBGWCR1_EL1 0x00000013 0x00000000
    R Debug Registers_DBGWCR2_EL1 0x00000013 0x00000000
    R Debug Registers_DBGWCR3_EL1 0x00000013 0x00000000
    R Debug Registers_DBGWVR0_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGWVR1_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGWVR2_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DBGWVR3_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_DLR_EL0 0x00000013 0x7F2CA978
    R Debug Registers_DSPSR_EL0 0x00000013 0x80000000
    R Debug Registers_MDCCINT_EL1 0x00000013 0x00000000
    R Debug Registers_MDCCSR_EL0 0x00000013 0x00000000
    R Debug Registers_MDCR_EL2 0x00000013 0x00000006
    R Debug Registers_MDCR_EL3 0x00000013 0x00018000
    R Debug Registers_MDRAR_EL1 0x0000000B 0x0000000000000000
    R Debug Registers_MDSCR_EL1 0x00000013 0x00005000
    R Debug Registers_OSDLR_EL1 0x00000013 0x00000000
    R Debug Registers_OSECCR_EL1 0x00000013 0x00000000
    R Debug Registers_OSLAR_EL1 0x00000013 0x00000000
    R Debug Registers_OSLSR_EL1 0x00000013 0x00000008
    R Debug Registers_SDER32_EL3 0x00000013 0x00000000
    R Interrupt Controller Registers_ICC_AP1R0_EL1 0x00000013 0x00000000
    R Interrupt Controller Registers_ICC_BPR1_EL1 0x00000013 0x00000003
    R Interrupt Controller Registers_ICC_CTLR_EL1 0x00000013 0x00000402
    R Interrupt Controller Registers_ICC_CTLR_EL3 0x00000013 0x00000410
    R Interrupt Controller Registers_ICC_HPPIR1_EL1 0x00000013 0x0000001E
    R Interrupt Controller Registers_ICC_IAR1_EL1 0x00000013 0x0000001E
    R Interrupt Controller Registers_ICC_IGRPEN0_EL1 0x00000013 0x00000001
    R Interrupt Controller Registers_ICC_IGRPEN1_EL1 0x00000013 0x00000001
    R Interrupt Controller Registers_ICC_IGRPEN1_EL3 0x00000013 0x00000003
    R Interrupt Controller Registers_ICC_PMR_EL1 0x00000013 0x000000F0
    R Interrupt Controller Registers_ICC_RPR_EL1 0x00000013 0x000000A0
    R Interrupt Controller Registers_ICC_SRE_EL1 0x00000013 0x00000007
    R Interrupt Controller Registers_ICC_SRE_EL2 0x00000013 0x0000000F
    R Interrupt Controller Registers_ICC_SRE_EL3 0x00000013 0x0000000F
    R Interrupt Controller Registers_ICH_AP0R0_EL2 0x00000013 0x00000000
    R Interrupt Controller Registers_ICH_AP1R2_EL2 0x00000013 0x00000000
    R Interrupt Controller Registers_ICH_EISR_EL2 0x00000013 0x00000000
    R Interrupt Controller Registers_ICH_ELSR_EL2 0x00000013 0x0000000F
    R Interrupt Controller Registers_ICH_HCR_EL2 0x00000013 0x00000000
    R Interrupt Controller Registers_ICH_LR0_EL2 0x0000000B 0x0000000000000000
    R Interrupt Controller Registers_ICH_LR1_EL2 0x0000000B 0x0000000000000000
    R Interrupt Controller Registers_ICH_LR2_EL2 0x0000000B 0x0000000000000000
    R Interrupt Controller Registers_ICH_LR3_EL2 0x0000000B 0x0000000000000000
    R Interrupt Controller Registers_ICH_MISR_EL2 0x00000013 0x00000000
    R Interrupt Controller Registers_ICH_VMCR_EL2 0x00000013 0x004C0008
    R Interrupt Controller Registers_ICH_VTR_EL2 0x00000013 0x90000003
    R Timer Registers_CNTFRQ_EL0 0x00000013 0x0BEBC200
    R Timer Registers_CNTHCTL_EL2 0x00000013 0x00000003
    R Timer Registers_CNTHP_CTL_EL2 0x00000013 0x00000000
    R Timer Registers_CNTHP_CVAL_EL2 0x0000000B 0x0000000000000000
    R Timer Registers_CNTHP_TVAL_EL2 0x00000013 0xC93E2E80
    R Timer Registers_CNTKCTL_EL1 0x00000013 0x000000D6
    R Timer Registers_CNTP_CTL_EL0 0x00000013 0x00000005
    R Timer Registers_CNTP_CVAL_EL0 0x0000000B 0x000000054A57C3CD
    R Timer Registers_CNTP_TVAL_EL0 0x00000013 0x13629471
    R Timer Registers_CNTPCT_EL0 0x0000000B 0x0000002337013A24
    R Timer Registers_CNTPS_CTL_EL1 0x00000013 0x00000000
    R Timer Registers_CNTPS_CVAL_EL1 0x0000000B 0x0000000000000000
    R Timer Registers_CNTPS_TVAL_EL1 0x00000013 0xC8D930B8
    R Timer Registers_CNTV_CTL_EL0 0x00000013 0x00000000
    R Timer Registers_CNTV_CVAL_EL0 0x0000000B 0x0000000000000000
    R Timer Registers_CNTV_TVAL_EL0 0x00000013 0xC8B60D54
    R Timer Registers_CNTVCT_EL0 0x0000000B 0x00000023375630CF
    R Timer Registers_CNTVOFF_EL2 0x0000000B 0x0000000000000000
    R Performance Monitor Registers_PMCCFILTR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMCCNTR_EL0 0x0000000B 0x0000000000000000
    R Performance Monitor Registers_PMCEID0_EL0 0x00000013 0x7FFF0F3F
    R Performance Monitor Registers_PMCEID1_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMCNTENCLR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMCNTENSET_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMCR_EL0 0x00000013 0x41023040
    R Performance Monitor Registers_PMEVCNTR0_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVCNTR1_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVCNTR2_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVCNTR3_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVCNTR4_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVCNTR5_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER0_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER1_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER2_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER3_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER4_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMEVTYPER5_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMINTENCLR_EL1 0x00000013 0x00000000
    R Performance Monitor Registers_PMINTENSET_EL1 0x00000013 0x00000000
    R Performance Monitor Registers_PMOVSCLR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMOVSSET_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMSELR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMUSERENR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMXEVCNTR_EL0 0x00000013 0x00000000
    R Performance Monitor Registers_PMXEVTYPER_EL0 0x00000013 0x00000000
    R External Debug Registers_DBGAUTHSTATUS_EL1 0x00000013 0x000000FF
    R External Debug Registers_DBGBCR0_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBCR1_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBCR2_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBCR3_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBCR4_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBCR5_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGBVR0_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGBVR1_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGBVR2_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGBVR3_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGBVR4_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGBVR5_EL1 0x0000000B 0x0000000000000000
    R External Debug Registers_DBGCLAIMCLR_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGCLAIMSET_EL1 0x00000013 0x000000FF
    R External Debug Registers_DBGWCR0_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWCR1_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWCR2_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWCR3_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWVR0_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWVR1_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWVR2_EL1 0x00000013 0x00000000
    R External Debug Registers_DBGWVR3_EL1 0x00000013 0x00000000
    R External Debug Registers_EDAA32PFR 0x0000000B 0x0000000000000000
    R External Debug Registers_EDACR 0x00000013 0x00000000
    R External Debug Registers_EDCIDR0 0x00000013 0x0000000D
    R External Debug Registers_EDCIDR1 0x00000013 0x00000090
    R External Debug Registers_EDCIDR2 0x00000013 0x00000005
    R External Debug Registers_EDCIDR3 0x00000013 0x000000B1
    R External Debug Registers_EDCIDSR 0x00000013 0x00000000
    R External Debug Registers_EDDEVAFF0 0x00000013 0x80000000
    R External Debug Registers_EDDEVAFF1 0x00000013 0x00000000
    R External Debug Registers_EDDEVARCH 0x00000013 0x47706A15
    R External Debug Registers_EDDEVID0 0x00000013 0x01000003
    R External Debug Registers_EDDEVID1 0x00000013 0x00000002
    R External Debug Registers_EDDEVID2 0x00000013 0x00000000
    R External Debug Registers_EDDEVTYPE 0x00000013 0x00000015
    R External Debug Registers_EDDFR 0x0000000B 0x0000000010305106
    R External Debug Registers_EDECCR 0x00000013 0x00000000
    R External Debug Registers_EDECR 0x00000013 0x00000000
    R External Debug Registers_EDESR 0x00000013 0x00000000
    R External Debug Registers_EDITCTRL 0x00000013 0x00000000
    R External Debug Registers_EDITR 0x00000013 0x00000000
    R External Debug Registers_EDLAR 0x00000013 0x00000000
    R External Debug Registers_EDLSR 0x00000013 0x00000000
    R External Debug Registers_EDPFR 0x0000000B 0x0000000001002222
    R External Debug Registers_EDPIDR0 0x00000013 0x00000008
    R External Debug Registers_EDPIDR1 0x00000013 0x000000BD
    R External Debug Registers_EDPIDR2 0x00000013 0x0000001B
    R External Debug Registers_EDPIDR3 0x00000013 0x00000000
    R External Debug Registers_EDPIDR4 0x00000013 0x00000004
    R External Debug Registers_EDPRCR 0x00000013 0x00000000
    R External Debug Registers_EDRCR 0x00000013 0x00000000
    R External Debug Registers_EDSCR 0x00000013 0x43007F13
    R External Debug Registers_EDWAR 0x0000000B 0x00000000000000C3
    R External Debug Registers_MIDR_EL1 0x00000013 0x411FD080
    R External Debug Registers_OSLAR_EL1 0x00000013 0x00000000
    R CTI Registers_CTIASICCTL 0x00000013 0x00000000
    R CTI Registers_CTIAPPCLEAR 0x00000013 0x00000000
    R CTI Registers_CTIAPPPULSE 0x00000013 0x00000000
    R CTI Registers_CTIAPPSET 0x00000013 0x00000000
    R CTI Registers_CTIAUTHSTATUS 0x00000013 0x0000000F
    R CTI Registers_CTICHINSTATUS 0x00000013 0x00000000
    R CTI Registers_CTICHOUTSTATUS 0x00000013 0x00000000
    R CTI Registers_CTICIDR0 0x00000013 0x0000000D
    R CTI Registers_CTICIDR1 0x00000013 0x00000090
    R CTI Registers_CTIDIDR2 0x00000013 0x00000005
    R CTI Registers_CTICIDR3 0x00000013 0x000000B1
    R CTI Registers_CTICLAIMCLR 0x00000013 0x00000000
    R CTI Registers_CTICLAIMSET 0x00000013 0x0000000F
    R CTI Registers_CTICONTROL 0x00000013 0x00000001
    R CTI Registers_CTIDEVAFF0 0x00000013 0x00000000
    R CTI Registers_CTIDEVAFF1 0x00000013 0x00000000
    R CTI Registers_CTIDEVARCH 0x00000013 0x00000000
    R CTI Registers_CTIDEVID0 0x00000013 0x00040800
    R CTI Registers_CTIDEVID1 0x00000013 0x00000000
    R CTI Registers_CTIDEVID2 0x00000013 0x00000000
    R CTI Registers_CTIDEVTYPE 0x00000013 0x00000014
    R CTI Registers_CTIGATE 0x00000013 0x0000000F
    R CTI Registers_CTIINEN0 0x00000013 0x00000000
    R CTI Registers_CTIINEN1 0x00000013 0x00000000
    R CTI Registers_CTIINEN2 0x00000013 0x00000000
    R CTI Registers_CTIINEN3 0x00000013 0x00000000
    R CTI Registers_CTIINEN4 0x00000013 0x00000000
    R CTI Registers_CTIINEN5 0x00000013 0x00000000
    R CTI Registers_CTIINEN6 0x00000013 0x00000000
    R CTI Registers_CTIINEN7 0x00000013 0x00000000
    R CTI Registers_CTIINEN8 0x00000013 0x00000000
    R CTI Registers_CTIINEN9 0x00000013 0x00000000
    R CTI Registers_CTIINEN10 0x00000013 0x00000000
    R CTI Registers_CTIINEN11 0x00000013 0x00000000
    R CTI Registers_CTIINEN12 0x00000013 0x00000000
    R CTI Registers_CTIINEN13 0x00000013 0x00000000
    R CTI Registers_CTIINEN14 0x00000013 0x00000000
    R CTI Registers_CTIINEN15 0x00000013 0x00000000
    R CTI Registers_CTIINEN16 0x00000013 0x00000000
    R CTI Registers_CTIINEN17 0x00000013 0x00000000
    R CTI Registers_CTIINEN18 0x00000013 0x00000000
    R CTI Registers_CTIINEN19 0x00000013 0x00000000
    R CTI Registers_CTIINEN20 0x00000013 0x00000000
    R CTI Registers_CTIINEN21 0x00000013 0x00000000
    R CTI Registers_CTIINEN22 0x00000013 0x00000000
    R CTI Registers_CTIINEN23 0x00000013 0x00000000
    R CTI Registers_CTIINEN24 0x00000013 0x00000000
    R CTI Registers_CTIINEN25 0x00000013 0x00000000
    R CTI Registers_CTIINEN26 0x00000013 0x00000000
    R CTI Registers_CTIINEN27 0x00000013 0x00000000
    R CTI Registers_CTIINEN28 0x00000013 0x00000000
    R CTI Registers_CTIINEN29 0x00000013 0x00000000
    R CTI Registers_CTIINEN30 0x00000013 0x00000000
    R CTI Registers_CTIINEN31 0x00000013 0x00000000
    R CTI Registers_CTIINTACK 0x00000013 0x00000000
    R CTI Registers_CTIITCTRL 0x00000013 0x00000000
    R CTI Registers_CTILAR 0x00000013 0x00000000
    R CTI Registers_CTILSR 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN0 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN1 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN2 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN3 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN4 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN5 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN6 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN7 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN8 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN9 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN10 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN11 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN12 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN13 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN14 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN15 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN16 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN17 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN18 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN19 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN20 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN21 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN22 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN23 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN24 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN25 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN26 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN27 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN28 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN29 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN30 0x00000013 0x00000000
    R CTI Registers_CTIOUTEN31 0x00000013 0x00000000
    R CTI Registers_CTIPIDR0 0x00000013 0x00000006
    R CTI Registers_CTIPDR1 0x00000013 0x000000B9
    R CTI Registers_CTIPDR2 0x00000013 0x0000004B
    R CTI Registers_CTIPDR3 0x00000013 0x00000000
    R CTI Registers_CTIPDR4 0x00000013 0x00000004
    R CTI Registers_CTITRIGINSTATUS 0x00000013 0x00000000
    R CTI Registers_CTITRIGOUTSTATUS 0x00000013 0x00000000
    

    [MCU2_0]    637.423018 s:  VX_ZONE_ERROR:[tivxCaptureProcess:1094]  CAPTURE: timestamp 80 $$$$$$$$$$$$$$$$
    [hl info avmsdk.cpp|AvmFactory@3526] After imagestitch use avgtime=2.6445 ms  maxtime = 32.1960 ms
    
    [hl info avmsdk.cpp|AvmFactory@3526] After imagestitch use avgtime=2.3796 ms  maxtime = 32.1960 ms
    
    [hl info avmsdk.cpp|AvmFactory@3526] After imagestitch use avgtime=2.4115 ms  maxtime = 32.1960 ms
    
    [C7x_2 ]    638.808115 s: PreDell cost time: 14861 us; costTime: 14103 us
    [hl info avmsdk.cpp|AvmFactory@3526] After imagestitch use avgtime=2.6503 ms  maxtime = 32.1960 ms
    
    [2024-05-22 10-22-49:599.095] [main_test.c:0444] [INF] tidl_graph fps : 11.824686
    tivxApsSrvProcess: timeuse = 23.1173 ms; maxCostTime: 51250
    [hl info avmsdk.cpp|AvmFactory@3526] After imagestitch use avgtime=2.3568 ms  maxtime = 32.1960 ms
    
     bWheelAngleInvalid: 0; WheelAngle: -9.000000
    bReversGearOn=0; DoorState,LF,RF,LR,RR,Trunk,Hood=0x0,0,0,0,0,0,0; RearViewMirrorStatus,LF,RR=0x0,0,0;
    PulseFL,FR,RL,RR=34825,33958,34553,33560
    [C7x_2 ]    640.180712 s: WarpAffine cost time: 5531 us; costTime: 5896 us
    [ 1595.205157] rcu: INFO: rcu_preempt self-detected stall on CPU
    [ 1595.210900] rcu:     0-....: (5387 ticks this GP) idle=a32/1/0x4000000000000004 softirq=196229/196231 fqs=1933
    [ 1595.220702]  (t=5250 jiffies g=337241 q=13)
    [ 1595.224872] Task dump for CPU 0:
    [ 1595.228087] task:avp_master.out  state:R  running task     stack:    0 pid:  242 ppid:     1 flags:0x00000002
    [ 1595.237982] Call trace:
    [ 1595.240432]  dump_backtrace+0x0/0x1a0
    [ 1595.244084]  show_stack+0x18/0x68
    [ 1595.247388]  sched_show_task+0x13c/0x168
    [ 1595.251299]  dump_cpu_task+0x44/0x54
    [ 1595.254859]  rcu_dump_cpu_stacks+0xb0/0xf0
    [ 1595.258944]  rcu_sched_clock_irq+0x978/0xc98
    [ 1595.263201]  update_process_times+0x60/0xa0
    [ 1595.267370]  tick_sched_handle.isra.0+0x34/0x50
    [ 1595.271884]  tick_sched_timer+0x4c/0xa8
    [ 1595.275706]  __hrtimer_run_queues+0x114/0x1b8
    [ 1595.280047]  hrtimer_interrupt+0xe8/0x248
    [ 1595.284046]  arch_timer_handler_phys+0x34/0x48
    [ 1595.288477]  handle_percpu_devid_irq+0x84/0x148
    [ 1595.292995]  generic_handle_irq+0x30/0x48
    [ 1595.296989]  __handle_domain_irq+0x64/0xc0
    [ 1595.301073]  gic_handle_irq+0x58/0x128
    [ 1595.304809]  el1_irq+0xcc/0x180
    [ 1595.307938]  _raw_spin_unlock_irqrestore+0xc/0x48
    [ 1595.312629]  run_rebalance_domains+0x3c/0x78
    [ 1595.316883]  efi_header_end+0x120/0x268
    [ 1595.320705]  irq_exit+0xc0/0xe0
    [ 1595.323834]  __handle_domain_irq+0x68/0xc0
    [ 1595.327916]  gic_handle_irq+0x58/0x128
    [ 1595.331650]  el1_irq+0xcc/0x180
    [ 1595.334778]  _raw_spin_unlock_irq+0x10/0x50
    [ 1595.338948]  schedule+0x70/0x108
    [ 1595.342164]  do_notify_resume+0x1cc/0x6f8
    [ 1595.346158]  work_pending+0xc/0x618
    [E][2024-05-22 10:39:08][mcu_event_proxy_send_to_io_handle:802] Heart Beat Stopped!!!!
    [I][2024-05-22 10:39:08][mcu_event_proxy_send_to_io_handle:807] eventhub alive:1612.402 s! msgcnt:120888
    "
    > ^C
    > ^C
    > ^C
    >
    >
    > [E][2024-05-22 10:39:38][mcu_event_proxy_send_to_io_handle:802] Heart Beat Stopped!!!!
    [I][2024-05-22 10:39:38][mcu_event_proxy_send_to_io_handle:807] eventhub alive:1642.422 s! msgcnt:120888
    [ 1658.217156] rcu: INFO: rcu_preempt self-detected stall on CPU
    [ 1658.222898] rcu:     0-....: (21105 ticks this GP) idle=a32/1/0x4000000000000004 softirq=196229/196231 fqs=7718
    [ 1658.232786]  (t=21003 jiffies g=337241 q=48)
    [ 1658.237042] Task dump for CPU 0:
    [ 1658.240256] task:avp_master.out  state:R  running task     stack:    0 pid:  242 ppid:     1 flags:0x00000002
    [ 1658.250149] Call trace:
    [ 1658.252599]  dump_backtrace+0x0/0x1a0
    [ 1658.256249]  show_stack+0x18/0x68
    [ 1658.259553]  sched_show_task+0x13c/0x168
    [ 1658.263465]  dump_cpu_task+0x44/0x54
    [ 1658.267025]  rcu_dump_cpu_stacks+0xb0/0xf0
    [ 1658.271110]  rcu_sched_clock_irq+0x978/0xc98
    [ 1658.275366]  update_process_times+0x60/0xa0
    [ 1658.279535]  tick_sched_handle.isra.0+0x34/0x50
    [ 1658.284048]  tick_sched_timer+0x4c/0xa8
    [ 1658.287871]  __hrtimer_run_queues+0x114/0x1b8
    [ 1658.292215]  hrtimer_interrupt+0xe8/0x248
    [ 1658.296214]  arch_timer_handler_phys+0x34/0x48
    [ 1658.300644]  handle_percpu_devid_irq+0x84/0x148
    [ 1658.305159]  generic_handle_irq+0x30/0x48
    [ 1658.309154]  __handle_domain_irq+0x64/0xc0
    [ 1658.313238]  gic_handle_irq+0x58/0x128
    [ 1658.316973]  el1_irq+0xcc/0x180
    [ 1658.320101]  _raw_spin_unlock_irqrestore+0xc/0x48
    [ 1658.324791]  run_rebalance_domains+0x3c/0x78
    [ 1658.329045]  efi_header_end+0x120/0x268
    [ 1658.332868]  irq_exit+0xc0/0xe0
    [ 1658.335994]  __handle_domain_irq+0x68/0xc0
    [ 1658.340076]  gic_handle_irq+0x58/0x128
    [ 1658.343810]  el1_irq+0xcc/0x180
    [ 1658.346937]  _raw_spin_unlock_irq+0x10/0x50
    [ 1658.351108]  schedule+0x70/0x108
    [ 1658.354322]  do_notify_resume+0x1cc/0x6f8
    [ 1658.358315]  work_pending+0xc/0x618
    NOTICE:  BL31: v2.7(debug):fe40ba9023-dirty
    NOTICE:  BL31: Built : 16:17:40, Mar 30 2024
    INFO:    GICv3 without legacy support detected.
    INFO:    ARM GICv3 driver initialized in EL3
    INFO:    Maximum SPI INTID supported: 991
    INFO:    SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
    INFO:    BL31: Initializing runtime services
    INFO:    BL31: cortex_a72: CPU workaround for 1319367 was applied
    INFO:    BL31: cortex_a72: CPU workaround for cve_2018_3639 was applied
    INFO:    BL31: cortex_a72: CPU workaround for cve_2022_23960 was applied
    INFO:    BL31: Initializing BL32
    I/TC:
    I/TC: OP-TEE version: dc3f981862-dev () #1 Sat Mar 30 07:28:53 UTC 2024 aarch64
    I/TC: WARNING: This OP-TEE configuration might be insecure!
    I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
    I/TC: Primary CPU initializing
    I/TC: SYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
    I/TC: HUK Initialized
    I/TC: Activated SA2UL device
    I/TC: Fixing SA2UL firewall owner for GP device
    I/TC: Enabled firewalls for SA2UL TRNG device
    I/TC: SA2UL TRNG initialized
    I/TC: SA2UL Drivers initialized
    I/TC: Primary CPU switching to normal world boot
    INFO:    BL31: Preparing for EL3 exit to normal world
    INFO:    Entry point address = 0x80080000
    INFO:    SPSR = 0x3c9
    
    U-Boot SPL 2021.01 (May 20 2024 - 14:17:01 +0800)
    ti_sci system-controller@44083000: Message not acknowledgedti_sci system-controller@44083000: Message not acknowledgedSYSFW ABI: 3.1 (firmware rev 0x0008 '8.6.3--1-g2249f (Chill Capybara')
    Trying to boot from MMC2
    
    
    U-Boot 2021.01 (May 20 2024 - 14:17:01 +0800)
    
    DRAM:  2 GiB
    Flash: 0 Bytes
    MMC:   mmc@4f80000: 0, mmc@4fb0000: 1
    Loading Environment from MMC... *** Warning - bad CRC, using default environment
    
    In:    serial@2850000
    Out:   serial@2850000
    Err:   serial@2850000
    
    Hello, welcome enter boot__init
    
    **** sd card present ****
    **** rootfs from /dev/mmcblk1p2 ****
    Autoboot in 0 seconds
    k3_r5f_rproc r5f@41000000: Core 1 is already in use. No rproc commands work
    k3_r5f_rproc r5f@41400000: Core 2 is already in use. No rproc commands work
    871560 bytes read in 26 ms (32 MiB/s)
    Load Remote Processor 2 with data@addr=0x82000000 871560 bytes: Success!
    142400 bytes read in 10 ms (13.6 MiB/s)
    Load Remote Processor 3 with data@addr=0x82000000 142400 bytes: Success!
    14815296 bytes read in 147 ms (96.1 MiB/s)
    Load Remote Processor 6 with data@addr=0x82000000 14815296 bytes: Success!
    9600896 bytes read in 44 ms (208.1 MiB/s)
    Load Remote Processor 7 with data@addr=0x82000000 9600896 bytes: Success!
    19079680 bytes read in 396 ms (45.9 MiB/s)
    74830 bytes read in 7 ms (10.2 MiB/s)
    ## Flattened Device Tree blob at 88000000
       Booting using the fdt blob at 0x88000000
       Loading Device Tree to 000000008ffea000, end 000000008ffff44d ... OK
    
    Starting kernel ...
    
    [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd080]
    [    0.000000] Linux version 5.10.162-g76b3e88d56 (zhubing@RD17) (aarch64-none-linux-gnu-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025, GNU ld (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209) #2 SMP PREEMPT Fri May 17 16:10:14 CST 2024
    [    0.000000] Machine model: Texas Instruments J721S2 EVM
    [    0.000000] earlycon: ns16550a0 at MMIO32 0x0000000002850000 (options '')
    [    0.000000] printk: bootconsole [ns16550a0] enabled
    WARNING: GTC: Debug access doesn't stop time. Fix Bootloader
    I/TC: Secondary CPU 1 initializing
    I/TC: Secondary CPU 1 switching to normal world boot
    

    Thanks
    Regards
    quanfeng

  • HI quanfeng,

       Could you please add  "K3_EXCLUSIVE_SNOOP_DELAY := 1"  in the file board-support/trusted-firmware-a-2.8+gitAUTOINC+2fcd408bb3/plat/ti/k3/board/generic/board.mk  and rebuild the ATF file to try and feedback?

       Thanks.

    Linjun

      

  • HI,

      Has this issue been resolved?

  • Hi,

    The problem hasn't been solved yet. Are you experiencing the same problem?

      Has this issue been resolved?

    Thanks
    Regards
    quanfeng