This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4AL-Q1: Vision Apps runtime error with 2GB of memory

Part Number: TDA4AL-Q1

Tool/software:

Hello

I have a custom board with a TDA4AL and 2GB of RAM. It's running a modified PSDK Linux 10.

I've successfully modified linux and u-boot to handle 2GB of RAM. I'm trying to modify the vision-apps package with a new memory map that fits in the 2GB. Memory map generation proceeds without issue, and I was able to apply the device tree overlay and deploy the modified vision-apps library. However, at runtime when I try to run TIOVX powered applications, I get the following errors:

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
APP: Init ... !!!
759.971222 s: MEM: Init ... !!!
759.971322 s: MEM: Initialized DMA HEAP (fd=5) !!!
759.971672 s: MEM: Init ... Done !!!
759.971709 s: IPC: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio0.rpmsg_chrdev.-1.13
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio1.rpmsg_chrdev.-1.13
760.013731 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_1] !!!
760.013753 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_2] !!!
760.013759 s: IPC: Init ... Done !!!
APP: ERROR: IPC init failed !!!
REMOTE_SERVICE: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio3.rpmsg_chrdev.-1.21
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio4.rpmsg_chrdev.-1.21
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio0.rpmsg_chrdev.-1.21
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio1.rpmsg_chrdev.-1.21
REMOTE_SERVICE: Init ... Done !!!
760.021440 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
760.032273 s: VX_ZONE_INIT:Enabled
760.032305 s: VX_ZONE_ERROR:Enabled
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Output of vx_app_arm_remote_log.out:

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[MCU2_0] 4.386927 s: CIO: Init ... Done !!!
[MCU2_0] 4.386982 s: ### CPU Frequency = 1000000000 Hz
[MCU2_0] 4.387007 s: CPU is running FreeRTOS
[MCU2_0] 4.387024 s: APP: Init ... !!!
[MCU2_0] 4.387057 s: SCICLIENT: Init ... !!!
[MCU2_0] 4.387174 s: SCICLIENT: DMSC FW version [10.0.8--v10.00.08 (Fiery Fox)]
[MCU2_0] 4.387203 s: SCICLIENT: DMSC FW revision 0xa
[MCU2_0] 4.387222 s: SCICLIENT: DMSC FW ABI revision 4.0
[MCU2_0] 4.387243 s: SCICLIENT: Init ... Done !!!
[MCU2_0] 4.387262 s: UDMA: Init ... !!!
[MCU2_0] 4.388087 s: UDMA: Init ... Done !!!
[MCU2_0] 4.388122 s: UDMA: Init for CSITX/CSIRX ... !!!
[MCU2_0] 4.388540 s: UDMA: Init for CSITX/CSIRX ... Done !!!
[MCU2_0] 4.388571 s: MEM: Init ... !!!
[MCU2_0] 4.388595 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ b9000000 of size 14680064 bytes !!!
[MCU2_0] 4.388635 s: MEM: Created heap (L3_MEM, id=1, flags=0x00000000) @ 60000000 of size 524288 bytes !!!
[MCU2_0] 4.388670 s: MEM: Created heap (DDR_CACHE_WT_MEM, id=7, flags=0x00000000) @ b9e00000 of size 2097152 bytes !!!
[MCU2_0] 4.388707 s: MEM: Init ... Done !!!
[MCU2_0] 4.388724 s: IPC: Init ... !!!
[MCU2_0] 4.388754 s: IPC: 5 CPUs participating in IPC !!!
[MCU2_0] 4.388789 s: IPC: Waiting for HLOS to be ready ... !!!
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

output of `dmesg | grep remoteproc`

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[ 9.610532] remoteproc remoteproc0: 64800000.dsp is available
[ 9.621601] remoteproc remoteproc0: attaching to 64800000.dsp
[ 9.647452] remoteproc remoteproc0: unsupported resource 65538
[ 9.721599] remoteproc remoteproc0: remote processor 64800000.dsp is now attached
[ 9.765863] remoteproc remoteproc1: 65800000.dsp is available
[ 9.772063] remoteproc remoteproc1: attaching to 65800000.dsp
[ 9.783829] remoteproc remoteproc1: unsupported resource 65538
[ 9.841450] remoteproc remoteproc1: remote processor 65800000.dsp is now attached
[ 10.018024] remoteproc remoteproc2: 41000000.r5f is available
[ 10.024179] remoteproc remoteproc2: attaching to 41000000.r5f
[ 10.124506] remoteproc remoteproc2: remote processor 41000000.r5f is now attached
[ 10.220402] remoteproc remoteproc3: 5c00000.r5f is available
[ 10.246007] remoteproc remoteproc3: attaching to 5c00000.r5f
[ 10.308062] remoteproc remoteproc3: remote processor 5c00000.r5f is now attached
[ 10.345950] remoteproc remoteproc4: 5d00000.r5f is available
[ 10.354722] remoteproc remoteproc4: attaching to 5d00000.r5f
[ 10.429819] remoteproc remoteproc4: remote processor 5d00000.r5f is now attached
[ 10.440568] platform 5e00000.r5f: configured R5F for remoteproc mode
[ 10.466077] remoteproc remoteproc5: 5e00000.r5f is available
[ 10.478323] remoteproc remoteproc5: loading /lib/firmware/j721s2-main-r5f1_0-fw failed with error -22
[ 10.489112] remoteproc remoteproc5: Direct firmware load for j721s2-main-r5f1_0-fw failed with error -22
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

output of `ls -lah /lib/firmware`

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
total 620K
drwxr-xr-x 6 root root 4.0K Mar 9 2018 .
drwxr-xr-x 57 root root 40K Mar 9 2018 ..
drwxr-xr-x 2 root root 4.0K Mar 9 2018 cadence
drwxr-xr-x 2 root root 4.0K Mar 9 2018 cnm
lrwxrwxrwx 1 root root 62 Mar 9 2018 j721s2-c71_0-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_c7x_1.out
lrwxrwxrwx 1 root root 69 Mar 9 2018 j721s2-c71_0-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_c7x_1.out.signed
lrwxrwxrwx 1 root root 62 Mar 9 2018 j721s2-c71_1-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_c7x_2.out
lrwxrwxrwx 1 root root 69 Mar 9 2018 j721s2-c71_1-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_c7x_2.out.signed
lrwxrwxrwx 1 root root 63 Mar 9 2018 j721s2-main-r5f0_0-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu2_0.out
lrwxrwxrwx 1 root root 70 Mar 9 2018 j721s2-main-r5f0_0-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu2_0.out.signed
lrwxrwxrwx 1 root root 63 Mar 9 2018 j721s2-main-r5f0_1-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu2_1.out
lrwxrwxrwx 1 root root 70 Mar 9 2018 j721s2-main-r5f0_1-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu2_1.out.signed
lrwxrwxrwx 1 root root 63 Mar 9 2018 j721s2-main-r5f1_0-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu3_0.out
lrwxrwxrwx 1 root root 70 Mar 9 2018 j721s2-main-r5f1_0-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu3_0.out.signed
lrwxrwxrwx 1 root root 63 Mar 9 2018 j721s2-main-r5f1_1-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu3_1.out
lrwxrwxrwx 1 root root 70 Mar 9 2018 j721s2-main-r5f1_1-fw-sec -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu3_1.out.signed
lrwxrwxrwx 1 root root 63 Mar 9 2018 j721s2-mcu-r5f0_1-fw -> /usr/lib/firmware/vision_apps_eaik/vx_app_rtos_linux_mcu1_1.out
lrwxrwxrwx 1 root root 79 Mar 9 2018 j721s2-mcu-r5f0_1-fw-sec -> /usr/lib/firmware/ti-ipc/j721s2/ipc_echo_test_mcu1_1_release_strip.xer5f.signed
-rw-r--r-- 1 root root 140K Mar 9 2018 rgx.fw.36.53.104.796
-rw-r--r-- 1 root root 375K Mar 9 2018 rgx.sh.36.53.104.796
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

7128.system_memory_map.html
Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<!DOCTYPE html>
<html>
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;border-color:#999;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#999;color:#444;background-color:#F7FDFA;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#999;color:#fff;background-color:#26ADE4;}
.tg .tg-kftd{background-color:#efefef;text-align:left;vertical-align:top}
.tg .tg-6sgx{background-color:#ffffff;text-align:left;vertical-align:top}
.tg .tg-fjir{background-color:#343434;color:#ffffff;text-align:left;vertical-align:top}
</style>
<head>
<title>System Memory Map for Linux+RTOS mode</title>
</head>
<body>
<h1>System Memory Map for Linux+RTOS mode</h1>
<p>Note, this file is auto generated using PyTI_PSDK_RTOS tool</p>
<table class="tg">
<tr>
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Modified `gen_linker_mem_map.py`

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/usr/bin/env python3
#
# Copyright (c) 2018 Texas Instruments Incorporated
#
# All rights reserved not granted herein.
#
# Limited License.
#
# Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive
# license under copyrights and patents it now or hereafter owns or controls to make,
# have made, use, import, offer to sell and sell ("Utilize") this software subject to the
# terms herein. With respect to the foregoing patent license, such license is granted
# solely to the extent that any such patent is necessary to Utilize the software alone.
# The patent license shall not apply to any combinations which include this software,
# other than combinations with devices manufactured by or for TI ("TI Devices").
# No hardware patent is licensed hereunder.
#
# Redistributions must preserve existing copyright notices and reproduce this license
# (including the above copyright notice and the disclaimer and (if applicable) source
# code license limitations below) in the documentation and/or other materials provided
# with the distribution
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • Hello, 

    The engineer assigned is out of the office until 9/26.

    Please expect a delay in response.

    -Josue

  • Hi,

    There seems to be a crash in the C7x_1 and C7x_2 cores from your remote logs in memory initialization

    [C7x_2 ]      5.249438 s: MEM: Init ... !!!
    [C7x_2 ]      5.249453 s: A0 =0xffff00 A1 =0x27000000
    [C7x_2 ]      5.249469 s: A2 =0x1 A3 =0xaa020020
    [C7x_2 ]      5.249479 s: A4 =0xb685ab18 A5 =0x27000000
    [C7x_2 ]      5.249491 s: A6 =0x1000000 A7 =0x0

    Please check why this error occurs. Please check if you have updated the mmu mapping of C7x cores accordingly.

    Regards,

    Nikhil

  • Yeah I was wondering about that. It does sort of look like a stack dump. I'll start looking into what's going on.

    Please check if you have updated the mmu mapping of C7x cores accordingly

    Could you be more specific please? I was under the impression that the python script generates a header used by all of the cores to setup their mmu mapping. Is there an additional step?

  • Alright I've solved my problem. There were a few issues.

    I had missed one step here which was to reduce the number of DDR instances in `app_utils`.

    The bigger issue is that yocto does not rebuild the firmware. So changes to `ti-vision-apps` and the repos pointed to by `psdk_repo_manifests` don't get applied to the edgeai firmware which is prebuilt. I followed instructions here to rebuild the firmware.

    Nikhil thanks for pointing me in the right direction.