Can't run am57xx IPC example

Alexey Arbuzov

Hello,

I'd like to compile and run IPC example for am57xx processor.

I have an am57xx evm devkit.

I downloaded & installed both ti-processor-sdk-rtos-am57xx-evm-02.00.01.07-Linux-x86-Install.bin & ti-processor-sdk-linux-am57xx-evm-02.00.01.07-Linux-x86-Install.bin.

Also downloaded & installed Code Composer Studio.

Then I successfully built IPC libraries within ipc_3_41_00_08 using instruction from http://processors.wiki.ti.com/index.php/IPC_Install_Guide_Linux.

Then I copy IPC libraries to the sd card using make install prefix=<path to card>/usr/local.

Now I want to build example ipc_3_41_00_08/examples/DRA7XX_linux_elf/ex02_messageq and run it on the board.

I edit example's makefile to set proclist to: dsp1 & host:

ALL = dsp1 dsp2 host
PROCLIST = dsp1 host

Then I build example & copy it to the board.

On the board side I:

- run linux with rproc_mem=32M@0x90000000

- set symlink within /lib/firmware to example's dsp1 executable (release version)

- unbind & bind dsp1 processor

- and run linux example, but got an error:

# /root/ipc/ex/app_host -l
--> main:
Processor List
Error: main_host.c, line 197: Ipc_start failed
<-- main:

As I see in dmesg firmware is successfully loaded:

root@am57xx-evm:/sys/bus/platform/drivers/omap-rproc# echo 40800000.dsp > unbind

[ 2840.742781] omap_hwmod: mmu1_dsp1: _wait_target_disable failed
[ 2840.755798] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
[ 2840.761745] remoteproc2: stopped remote processor 40800000.dsp
[ 2840.767901] remoteproc2: releasing 40800000.dsp

root@am57xx-evm:/sys/bus/platform/drivers/omap-rproc# echo 40800000.dsp > bind

[ 2844.242717] omap-rproc 40800000.dsp: assigned reserved memory node dsp1_cma@99000000
[ 2844.250583] remoteproc2: 40800000.dsp is available
[ 2844.255481] remoteproc2: Note: remoteproc is still under development and considered experimental.
[ 2844.264502] remoteproc2: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[ 2844.282831] remoteproc2: powering up 40800000.dsp
[ 2844.287685] remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4558976
[ 2844.301850] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
[ 2844.307784] omap-iommu 40d01000.mmu: 40d01000.mmu: version 3.0
[ 2844.313700] omap-iommu 40d02000.mmu: 40d02000.mmu: version 3.0
[ 2844.328682] remoteproc2: remote processor 40800000.dsp is now up
[ 2844.335192] virtio_rpmsg_bus virtio1: rpmsg host is online
[ 2844.340768] remoteproc2: registered virtio1 (type 7)
[ 2844.346445] virtio_rpmsg_bus virtio1: creating channel rpmsg-proto addr 0x3d

# cat /debug/remoteproc/remoteproc2/trace0
[ 0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
[ 0.000] 17 Resource entries at 0x95000000
[ 0.000] [t=0x0001f476] xdc.runtime.Main: --> main:
[ 0.000] registering rpmsg-proto:rpmsg-proto service on 61 with HOST
[ 0.000] [t=0x0033f40c] xdc.runtime.Main: NameMap_sendMessage: HOST 53, port=61
[ 0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
[ 0.000] [t=0x0035346b] xdc.runtime.Main: --> smain:
[ 0.000] [t=0x003648b8] Server: Server_create: server is ready
[ 0.000] [t=0x00369e06] Server: <-- Server_create: 0
[ 0.000] [t=0x0036dbe3] Server: --> Server_exec:

Can anybody promt me what did wrong?

Thank you.

over 9 years ago

0 Biser Gatchev-XID over 9 years ago

TI__Guru**** 393215 points

Hi,

I will ask the software team to check this.

0 Rex Chang over 9 years ago in reply to Biser Gatchev-XID

TI__Guru 50170 points

Hi, Alexey,

From the remoteproc trace, the DSP seems to be up and running. Could you check if Lad process is running on the ARM side? if it is, could you see what says in the log file, lad.txt?

Rex

0 Alexey Arbuzov over 9 years ago in reply to Rex Chang

Prodigy 230 points

Hi Rex,

Thank you for response.

Lad is running:

root@am57xx-evm:~# ps | grep lad
 1177 root      9828 S    /usr/local/bin/lad_dra7xx -g -l lad.txt
 1180 root      1992 S    grep lad

Log messages from the Lad:

root@am57xx-evm:~# cat /var/volatile/tmp/LAD/lad.txt
[0.736139] 
Initializing LAD... [0.737055] NameServer_setup: entered, refCount=0
[0.737107] NameServer_setup: creating listener thread
[0.737195] NameServer_setup: exiting, refCount=1
[0.737263] NameServer_create(): 'GateMP'
[0.737287] 
    opening FIFO: /tmp/LAD/LADCMDS
[0.737351] listener_cb: Entered Listener thread.
[0.737368] NameServer: waiting for unblockFd: 0, and socks: maxfd: 0

Just in case I've tried again - the same error:

root@am57xx-evm:~# /root/ipc/ex/app_host -l
--> main:
Processor List
Error: main_host.c, line 197: Ipc_start failed
<-- main:

Then I've tried to run Lad that I build when building examples:

root@am57xx-evm:~# ps | grep lad
  974 root      9840 S    /usr/bin/lad_dra7xx -g -l lad.txt
 1172 root      1992 S    grep lad
root@am57xx-evm:~# killall lad_dra7xx
root@am57xx-evm:~# ps | grep lad
 1175 root      1992 S    grep lad
root@am57xx-evm:~# /usr/local/bin/lad_dra7xx -g -l lad.txt

GateMP support enabled on host

Opened log file: lad.txt

numProcessors = 5 id = 0 baseId = 0

Spawned daemon: /usr/local/bin/lad_dra7xx

And again example does not work:

root@am57xx-evm:~# /root/ipc/ex/app_host -l
--> main:
Processor List
Error: main_host.c, line 197: Ipc_start failed
<-- main:

Below log messages of Lad from /usr/local/bin:

root@am57xx-evm:~# cat /var/volatile/tmp/LAD/lad.txt
[0.151211] 
Initializing LAD... [0.151617] NameServer_setup: entered, refCount=0
[0.151677] NameServer_setup: creating listener thread
[0.151776] NameServer_setup: exiting, refCount=1
[0.151807] listener_cb: Entered Listener thread.
[0.151829] NameServer: waiting for unblockFd: 0, and socks: maxfd: 0
[0.151842] NameServer_create(): 'GateMP'
[0.151868] 
    opening FIFO: /tmp/LAD/LADCMDS

BTW: copycodectest from the SD-card that shipped with DevKit worked normally when I switched to appropriate DSP firmware.

0 Rex Chang over 9 years ago in reply to Alexey Arbuzov

TI__Guru 50170 points

Hi, Alexey,

The logs seem to be ok but somehow the ipc_start still fails. Could you try to remove the following files in /lib/firmware and restart the system and retry it?

dra7-dsp2-fw.xe66

dra7-ipu1-fw.xem4

dra7-ipu2-fw.xem4

At the mean time, I'll see if I can reproduce what you saw.

Rex

0 Alexey Arbuzov over 9 years ago in reply to Rex Chang

Prodigy 230 points

Hi Rex,

I've tried to remove files as you ask, but nothing changed. See below:

Initial state:

root@am57xx-evm:/lib/firmware# ls -l
-rw-r--r--    1 root     root         53488 Oct  7 18:49 TIInit_10.6.15.bts
-rwxr-xr-x    1 root     root         70311 Oct  7 18:49 TIInit_11.8.32.bts
-rw-r--r--    1 root     root         17084 Oct  7 18:49 TIInit_12.10.28.bts
-rwxr-xr-x    1 root     root         70316 Oct  7 18:49 TIInit_12.8.32.bts
-rw-r--r--    1 root     root         50695 Oct  7 18:49 TIInit_7.6.15.bts
lrwxrwxrwx    1 root     root            16 Oct  8 18:54 dra7-dsp1-fw.xe66 -> server_dsp1.xe66
-rw-r--r--    1 root     root        870888 Oct  7 18:49 dra7-dsp1-fw.xe66.dce
-rw-r--r--    1 root     root      21306292 Oct  7 19:08 dra7-dsp1-fw.xe66.ocl
-rwxr-xr-x    1 root     root       6937149 Oct  7 17:53 dra7-dsp1-fw.xe66.test
lrwxrwxrwx    1 root     root            21 Oct  7 19:58 dra7-dsp2-fw.xe66 -> dra7-dsp2-fw.xe66.ocl
-rw-r--r--    1 root     root      21306292 Oct  7 19:08 dra7-dsp2-fw.xe66.ocl
-rwxr-xr-x    1 root     root       6937149 Oct  7 17:53 dra7-dsp2-fw.xe66.test
-rwxr-xr-x    1 root     root       6142375 Oct  7 17:53 dra7-ipu1-fw.xem4.test
-rw-r--r--    1 root     root       3480916 Oct  7 17:53 dra7-ipu2-fw.xem4
-rwxr-xr-x    1 root     root       6142375 Oct  7 17:53 dra7-ipu2-fw.xem4.test
-rw-r--r--    1 root     root       4558976 Oct  8 19:28 server_dsp1.xe66
drwxr-xr-x    2 root     root          4096 Oct  7 18:51 ti-connectivity
-rw-r--r--    1 root     root          4002 Oct  7 19:03 vpdma-1b8.bin

Removing files:

root@am57xx-evm:/lib/firmware# mv dra7-dsp2-fw.xe66 ~/backup/lib-firmware/   
root@am57xx-evm:/lib/firmware# mv  dra7-ipu1-fw.xem4 ~/backup/lib-firmware/
mv: can't rename 'dra7-ipu1-fw.xem4': No such file or directory
root@am57xx-evm:/lib/firmware# mv dra7-ipu2-fw.xem4 ~/backup/lib-firmware/

After remove:

root@am57xx-evm:/lib/firmware# ls -l
-rw-r--r--    1 root     root         53488 Oct  7 18:49 TIInit_10.6.15.bts
-rwxr-xr-x    1 root     root         70311 Oct  7 18:49 TIInit_11.8.32.bts
-rw-r--r--    1 root     root         17084 Oct  7 18:49 TIInit_12.10.28.bts
-rwxr-xr-x    1 root     root         70316 Oct  7 18:49 TIInit_12.8.32.bts
-rw-r--r--    1 root     root         50695 Oct  7 18:49 TIInit_7.6.15.bts
lrwxrwxrwx    1 root     root            16 Oct  8 18:54 dra7-dsp1-fw.xe66 -> server_dsp1.xe66
-rw-r--r--    1 root     root        870888 Oct  7 18:49 dra7-dsp1-fw.xe66.dce
-rw-r--r--    1 root     root      21306292 Oct  7 19:08 dra7-dsp1-fw.xe66.ocl
-rwxr-xr-x    1 root     root       6937149 Oct  7 17:53 dra7-dsp1-fw.xe66.test
-rw-r--r--    1 root     root      21306292 Oct  7 19:08 dra7-dsp2-fw.xe66.ocl
-rwxr-xr-x    1 root     root       6937149 Oct  7 17:53 dra7-dsp2-fw.xe66.test
-rwxr-xr-x    1 root     root       6142375 Oct  7 17:53 dra7-ipu1-fw.xem4.test
-rwxr-xr-x    1 root     root       6142375 Oct  7 17:53 dra7-ipu2-fw.xem4.test
-rw-r--r--    1 root     root       4558976 Oct  8 19:28 server_dsp1.xe66
drwxr-xr-x    2 root     root          4096 Oct  7 18:51 ti-connectivity
-rw-r--r--    1 root     root          4002 Oct  7 19:03 vpdma-1b8.bin

server_dsp1.xe66 - it is DSP firmware built by myself from example02.

After reboot the same fail:

root@am57xx-evm:/lib/firmware# sync
root@am57xx-evm:/lib/firmware# reboot
...
root@am57xx-evm:~# /root/ipc/ex/app_host -l
--> main:
Processor List
Error: main_host.c, line 197: Ipc_start failed
<-- main:

Some logs:

root@am57xx-evm:~# dmesg | grep remote
[    3.237737]  remoteproc0: 58820000.ipu is available
[    3.290564]  remoteproc0: Note: remoteproc is still under development and considered experimental.
[    3.424548]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[    3.548467]  remoteproc0: Direct firmware load for dra7-ipu1-fw.xem4 failed with error -2
[    3.556687]  remoteproc0: Falling back to user helper
[    3.633817]  remoteproc1: 55020000.ipu is available
[    3.698167]  remoteproc1: Note: remoteproc is still under development and considered experimental.
[    3.707175]  remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[    3.847438]  remoteproc1: Direct firmware load for dra7-ipu2-fw.xem4 failed with error -2
[    3.882033]  remoteproc1: Falling back to user helper
[    4.423689]  remoteproc2: 40800000.dsp is available
[    4.446669]  remoteproc2: Note: remoteproc is still under development and considered experimental.
[    4.489445]  remoteproc2: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[    4.554295]  remoteproc3: 41000000.dsp is available
[    4.561097]  remoteproc3: Note: remoteproc is still under development and considered experimental.
[    4.572932]  remoteproc3: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[    4.586350]  remoteproc3: Direct firmware load for dra7-dsp2-fw.xe66 failed with error -2
[    4.598461]  remoteproc0: failed to load dra7-ipu1-fw.xem4
[    4.604205]  remoteproc1: failed to load dra7-ipu2-fw.xem4
[    4.612008]  remoteproc3: Falling back to user helper
[    4.623897]  remoteproc3: failed to load dra7-dsp2-fw.xe66
[    5.035290]  remoteproc2: registered virtio0 (type 7)
[    5.059087]  remoteproc2: powering up 40800000.dsp
[    5.068044]  remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4558976
[    5.145709]  remoteproc2: remote processor 40800000.dsp is now up
root@am57xx-evm:~# cat /debug/remoteproc/remoteproc2/trace0
[      0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
[      0.000] 17 Resource entries at 0x95000000
[      0.000] [t=0x0001e811] xdc.runtime.Main: --> main:
[      0.000] registering rpmsg-proto:rpmsg-proto service on 61 with HOST
[      0.000] [t=0x0087ab01] xdc.runtime.Main: NameMap_sendMessage: HOST 53, port=61
[      0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
[      0.000] [t=0x0088e5ed] xdc.runtime.Main: --> smain:
[      0.000] [t=0x0089f53c] Server: Server_create: server is ready
[      0.000] [t=0x008a4942] Server: <-- Server_create: 0
[      0.000] [t=0x008a86a8] Server: --> Server_exec:

0 Rex Chang over 9 years ago in reply to Alexey Arbuzov

TI__Guru 50170 points

Hi, Alexey,

I am not sure why the -l has an issue, but please try to run the example and send message to DSP. That works. However, in this release, there is an known issue. To work around, you need to restart the lad_dra7xx without -g, that is, /usr/bin/lad_dra7xx -l lad.txt. I also rebuilt the example with only dsp1 and host in the prolist. Here is the capture of running the example when sending message to DSP1.

root@am57xx-evm:~# ./app_host DSP1
--> main:
--> Main_main:
--> App_create:
App_create: Host is ready
<-- App_create:
--> App_exec:
App_exec: sending message 1
App_exec: sending message 2
App_exec: sending message 3
App_exec: message received, sending message 4
App_exec: message received, sending message 5
App_exec: message received, sending message 6
App_exec: message received, sending message 7
App_exec: message received, sending message 8
App_exec: message received, sending message 9
App_exec: message received, sending message 10
App_exec: message received, sending message 11
App_exec: message received, sending message 12
App_exec: message received, sending message 13
App_exec: message received, sending message 14
App_exec: message received, sending message 15
App_exec: message received
App_exec: message received
App_exec: message received
<-- App_exec: 0
--> App_delete:
<-- App_delete:
<-- Main_main:
<-- main:
root@am57xx-evm:~#

0 Rex Chang over 9 years ago in reply to Rex Chang

TI__Guru 50170 points

Hi, Alexey,

There is a bug in the main_host.c. I modified the code by moving the Ipc_transportConfig() before Main_parseArgs(), and "app_host -l" works. Please see code change and console logs below. Please note that "-g" still need to be removed when running lad_dra7xx.

Int main(Int argc, Char* argv[])
{
Int status;

printf("--> main:\n");

/* configure the transport factory */
Ipc_transportConfig(&TransportRpmsg_Factory);

/* parse command line */
status = Main_parseArgs(argc, argv);

    if (status < 0) {
        goto leave;
    }
    /* configure the transport factory */
    /* Ipc_transportConfig(&TransportRpmsg_Factory);*/

/* Ipc initialization */
status = Ipc_start();

root@am57xx-evm:~# ./app_host -l
--> main:
Processor List
Ipc_start: GateMP_start failed: -5
Error: main_host.c, line 201: Ipc_start failed
<-- main:

root@am57xx-evm:~# ps | grep lad
1106 root 9844 S /usr/bin/lad_dra7xx -g -l lad.txt
1132 root 1992 S grep lad
root@am57xx-evm:~# kill 1106
root@am57xx-evm:~# /usr/bin/lad_dra7xx -l lad.txt

Opened log file: lad.txt

numProcessors = 5 id = 0 baseId = 0

Spawned daemon: /usr/bin/lad_dra7xx

root@am57xx-evm:~# ./app_host -l
--> main:
Processor List
    procId=0, procName=HOST
    procId=1, procName=IPU2
    procId=2, procName=IPU1
    procId=3, procName=DSP2
    procId=4, procName=DSP1
root@am57xx-evm:~#

0 Alexey Arbuzov over 9 years ago in reply to Rex Chang

Prodigy 230 points

Hi Rex,

It seems your advice regarding -g options did the trick.

But I must run lad_dra7xx built by myself (before examples), lad_dra7xx that came with SD-card don't work.

Some logs:

Arago 2015.09 am57xx-evm

am57xx-evm login: root
root@am57xx-evm:~# 
root@am57xx-evm:~# killall lad_dra7xx
root@am57xx-evm:~# /usr/local/bin/lad_dra7xx -l lad.txt

Opened log file: lad.txt

numProcessors = 5 id = 0 baseId = 0

Spawned daemon: /usr/local/bin/lad_dra7xx

root@am57xx-evm:~# 
root@am57xx-evm:~# /root/ipc/ex/app_host DSP1
--> main:
--> Main_main:
--> App_create:
App_create: Host is ready
<-- App_create:
--> App_exec:
App_exec: sending message 1
App_exec: sending message 2
App_exec: sending message 3
App_exec: message received, sending message 4
App_exec: message received, sending message 5
App_exec: message received, sending message 6
App_exec: message received, sending message 7
App_exec: message received, sending message 8
App_exec: message received, sending message 9
App_exec: message received, sending message 10
App_exec: message received, sending message 11
App_exec: message received, sending message 12
App_exec: message received, sending message 13
App_exec: message received, sending message 14
App_exec: message received, sending message 15
App_exec: message received
App_exec: message received
App_exec: message received
<-- App_exec: 0
--> App_delete:
<-- App_delete:
<-- Main_main:
<-- main:
root@am57xx-evm:~#

Thank you for support.

Processors

Processors forum

Can't run am57xx IPC example