• State Resolved
  • Locked Locked
  • Replies 6 replies
  • Answers 1 answer
  • Subscribers 98 subscribers
  • Views 639 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

Linux/AM5728: IPC examples issues

Linux/AM5728: IPC example error

shi Li
Prodigy 30 points
Part Number: AM5728


Tool/software: Linux

Hi,

I am trying to run the ex02_messageq example on a AM5728 board and using the sdk as follows:

ti-processor-sdk-linux-am57xx-evm-05.03.00.07 

ti-processor-sdk-rtos-am57xx-evm-05.03.00.07

When running the example , I always get the following error. Can anyone 

./app_host DSP1
--> main:
--> Main_main:
--> App_create:
App_create: Host is ready
<-- App_create:
--> App_exec:
App_exec: sending message 1
App_exec: sending message 2
App_exec: sending message 3
[ 450.017945] remoteproc remoteproc2: crash detected in 40800000.dsp: type watchdog
[ 450.025572] remoteproc remoteproc2: handling crash #1 in 40800000.dsp
[ 450.032270] remoteproc remoteproc2: recovering 40800000.dsp
recvfrom failed: Link has been severed (67)
rpmsgThreadFxn: transportGet failed on fd 11, returned -20
[ 450.059996] remoteproc remoteproc2: stopped remote processor 40800000.dsp
<-- App_exec: -20
<-- Main_main:
[ 450.070285] remoteproc remoteproc2: powering up 40800000.dsp
<-- main:
[ 450.086765] remoteproc remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4487396
[ 450.102100] omap-iommu 40d01000.mmu: 40d01000.mmu: version 3.0
[ 450.108039] omap-iommu 40d02000.mmu: 40d02000.mmu: version 3.0
root@am57xx-evm:~# [ 450.114955] alloc_contig_range: [99004, 99007) PFNs busy
[ 450.130659] virtio_rpmsg_bus virtio0: rpmsg host is online
[ 450.136346] remoteproc remoteproc2: registered virtio0 (type 7)
[ 450.142474] remoteproc remoteproc2: remote processor 40800000.dsp is now up
[ 450.150198] virtio_rpmsg_bus virtio0: creating channel rpmsg-proto addr 0x3d

What's the problem ?

Thanks a lot.

  • 0
    TI__Guru 50170 points
    Hi, Shi,

    Let me download the 5.3 release and give it a try. I'll get back as soon as I get the restult either way.

    Rex
  • 0 in reply to Rex Chang
    TI__Guru 50170 points
    Hi, Shi,

    I tried with the prebuilt images in the PSDK 5.3 filesystem and don't see the issue. Please see my logs at the end of the post,
    Did you run the ex02 example binaries which you built or using the prebuilt?
    If you built the example images, did you use the high level build as described in IPC Quick Start Guide?
    software-dl.ti.com/.../Foundational_Components_IPC.html

    If not, could you try follow the steps in the IPC Quick Start Guide to rebuilt the ex02_messageq example and give it a try?

    Rex

    am57xx-evm login: root
    root@am57xx-evm:~# uname -a
    Linux am57xx-evm 4.14.79-ge669d52447 #1 SMP PREEMPT Sat Apr 6 02:23:36 UTC 2019 armv7l GNU/Linux
    root@am57xx-evm:~# cd /lib/firmware
    root@am57xx-evm:/lib/firmware# ls -l dra7-dsp1-fw.xe66
    lrwxrwxrwx 1 root root 46 Apr 6 2019 dra7-dsp1-fw.xe66 -> /lib/firmware/dra7-dsp1-fw.xe66.opencl-monitor
    root@am57xx-evm:/lib/firmware# ln -sf /usr/bin/ipc/examples/ex02_messageq/debug/server_dsp1.xe66 dra7-dsp1-fw.xe66
    root@am57xx-evm:/lib/firmware# ln -sf /usr/bin/ipc/examples/ex02_messageq/debug/server_dsp2.xe66 dra7-dsp2-fw.xe66
    root@am57xx-evm:/lib/firmware# reboot -f now
    :::::::::::::::::
    am57xx-evm login: root
    root@am57xx-evm:~# dmesg | grep dsp
    [ 0.000000] OF: reserved mem: initialized node dsp1-memory@99000000, compatible id shared-dma-pool
    [ 0.000000] OF: reserved mem: initialized node dsp2-memory@9f000000, compatible id shared-dma-pool
    [ 0.471262] iommu: Adding device 40800000.dsp to group 0
    [ 0.471536] iommu: Adding device 41000000.dsp to group 3
    [ 10.102137] omap-rproc 40800000.dsp: assigned reserved memory node dsp1-memory@99000000
    [ 10.112241] remoteproc remoteproc2: 40800000.dsp is available
    [ 10.181017] omap-rproc 41000000.dsp: assigned reserved memory node dsp2-memory@9f000000
    [ 10.189387] remoteproc remoteproc3: 41000000.dsp is available
    [ 10.584916] remoteproc remoteproc2: powering up 40800000.dsp
    [ 10.590860] remoteproc remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4760892
    [ 10.605519] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 10.672411] remoteproc remoteproc2: remote processor 40800000.dsp is now up
    [ 12.286309] remoteproc remoteproc3: powering up 41000000.dsp
    [ 12.286318] remoteproc remoteproc3: Booting fw image dra7-dsp2-fw.xe66, size 4598740
    [ 12.293033] omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    [ 12.409979] remoteproc remoteproc3: remote processor 41000000.dsp is now up
    root@am57xx-evm:~# cat /sys/kernel/debug/remoteproc/remoteproc2/name
    40800000.dsp
    root@am57xx-evm:~# cat /sys/kernel/debug/remoteproc/remoteproc2/trace0
    [ 0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
    [ 0.000] 17 Resource entries at 0x95000000
    [ 0.000] [t=0x000525d8] xdc.runtime.Main: --> main:
    [ 0.000] registering rpmsg-proto:rpmsg-proto service on 61 with HOST
    [ 0.000] [t=0x003de90f] xdc.runtime.Main: NameMap_sendMessage: HOST 53, port=61
    [ 0.000] Watchdog disabled: TimerBase = 0x48086000 ClkCtrl = 0x4a009728
    [ 0.000] [t=0x00407c43] xdc.runtime.Main: --> smain:
    [ 0.000] [t=0x00427105] Server: Server_create: server is ready
    [ 0.000] [t=0x004308c4] Server: <-- Server_create: 0
    [ 0.000] [t=0x0043836d] Server: --> Server_exec:
    root@am57xx-evm:~# [ 140.866347] NET: Registered protocol family 15
    [ 140.924373] Initializing XFRM netlink socket

    root@am57xx-evm:~# cd /usr/bin/ipc/examples/ex02_messageq/debug/
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ls
    app_host server_dsp2.xe66 server_ipu2.xem4
    server_dsp1.xe66 server_ipu1.xem4
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ./app_host DSP1
    --> main:
    [ 210.450591] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
    --> Main_main:
    --> App_create:
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    App_exec: sending message 1
    App_exec: sending message 2
    App_exec: sending message 3
    App_exec: message received, sending message 4
    App_exec: message received, sending message 5
    App_exec: message received, sending message 6
    App_exec: message received, sending message 7
    App_exec: message received, sending message 8
    App_exec: message received, sending message 9
    App_exec: message received, sending message 10
    App_exec: message received, sending message 11
    App_exec: message received, sending message 12
    App_exec: message received, sending message 13
    App_exec: message received, sending message 14
    App_exec: message received, sending message 15
    App_exec: message received
    App_exec: message received
    App_exec: message received
    <-- App_exec: 0
    --> App_delete:
    <-- App_delete:
    <-- Main_main:
    <-- main:
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug#
  • 0 in reply to Rex Chang
    Prodigy 30 points

    Hi Chang,

    Thank you for your quickly investigation.

    Yes, at the beginning,  I followed the quick start guide http://processors.wiki.ti.com/index.php?oldid=223643&title=IPC_Quick_Start_Guide and got the problem. Then I found the link as you mentioned. It seems that the quick start guide miss out  IPC_INSTALL_PATH environment variable setting.  I set it to corresponding value, rebuild the core and run the example again, "./app_host DSP1" dose work, but "./app_host DSP2" dose not.

    I tried many times and got the same result. I didn't get into the details of the example source code, so not sure if the result is what it is.

    Another two interesting things:

    1, I always got the following error 1~2 minutes after "./app_host DSP1" executting correctly:

    root@am57xx-evm:~# [ 5106.379761] remoteproc remoteproc2: crash detected in 40800000.dsp: type watchdog
    [ 5106.388580] remoteproc remoteproc2: handling crash #1 in 40800000.dsp
    [ 5106.395054] remoteproc remoteproc2: recovering 40800000.dsp
    [ 5106.423715] remoteproc remoteproc2: stopped remote processor 40800000.dsp
    [ 5106.430564] remoteproc remoteproc2: powering up 40800000.dsp
    [ 5106.443624] remoteproc remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4684680
    [ 5106.458577] omap-iommu 40d01000.mmu: 40d01000.mmu: version 3.0
    [ 5106.464552] omap-iommu 40d02000.mmu: 40d02000.mmu: version 3.0
    [ 5106.480707] virtio_rpmsg_bus virtio1: rpmsg host is online
    [ 5106.486274] remoteproc remoteproc2: registered virtio1 (type 7)
    [ 5106.492283] remoteproc remoteproc2: remote processor 40800000.dsp is now up
    [ 5106.499988] virtio_rpmsg_bus virtio1: creating channel rpmsg-proto addr 0x3d
    [ 5106.507284] omap-rproc 40800000.dsp: received echo reply from 40800000.dsp

    To make it run, I must use the following commands:

    echo stop > /sys/class/remoteproc/remoteproc2/state

    echo start > /sys/class/remoteproc/remoteproc2/state

    2, Yesterday, "./app_host DSP1" runs OK, whilest "./app_host DSP2" not.  I powered off the board after work.  But today when I rerun the test, both commands get the same watchdog error. Nothing couldn't make it work again.

    I am totally confused.

  • 0 in reply to shi Li
    TI__Guru 50170 points
    Hi, Shi,

    I don't see the issue on my side. I waited for 12 minutes and don't see any messages in console. I re-ran the test to both DSP1 and DSP2, they ran fine. Could you try prebuilt image to see if the issue happens? My console capture is shown below:


    am57xx-evm login: root
    root@am57xx-evm:~# uname -a
    Linux am57xx-evm 4.14.79-ge669d52447 #1 SMP PREEMPT Sat Apr 6 02:23:36 UTC 2019 armv7l GNU/Linux
    root@am57xx-evm:~# ls -l /lib/firmware/dra7-dsp?-fw.xe66
    lrwxrwxrwx 1 root root 58 Apr 6 01:43 /lib/firmware/dra7-dsp1-fw.xe66 -> /usr/bin/ipc/examples/ex02_messageq/debug/server_dsp1.xe66
    lrwxrwxrwx 1 root root 58 Apr 6 01:44 /lib/firmware/dra7-dsp2-fw.xe66 -> /usr/bin/ipc/examples/ex02_messageq/debug/server_dsp2.xe66
    root@am57xx-evm:~# cd /usr/bin/ipc/examples/ex02_messageq/debug/

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ./app_host DSP2
    --> main:
    [ 65.769590] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
    --> Main_main:
    --> App_create:
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    App_exec: sending message 1
    App_exec: sending message 2
    App_exec: sending message 3
    App_exec: message received, sending message 4
    App_exec: message received, sending message 5
    App_exec: message received, sending message 6
    App_exec: message received, sending message 7
    App_exec: message received, sending message 8
    App_exec: message received, sending message 9
    App_exec: message received, sending message 10
    App_exec: message received, sending message 11
    App_exec: message received, sending message 12
    App_exec: message received, sending message 13
    App_exec: message received, sending message 14
    App_exec: message received, sending message 15
    App_exec: message received
    App_exec: message received
    App_exec: message received
    <-- App_exec: 0
    --> App_delete:
    <-- App_delete:
    <-- Main_main:
    <-- main:

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ./app_host DSP1
    --> main:
    --> Main_main:
    --> App_create:
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    App_exec: sending message 1
    App_exec: sending message 2
    App_exec: sending message 3
    App_exec: message received, sending message 4
    App_exec: message received, sending message 5
    App_exec: message received, sending message 6
    App_exec: message received, sending message 7
    App_exec: message received, sending message 8
    App_exec: message received, sending message 9
    App_exec: message received, sending message 10
    App_exec: message received, sending message 11
    App_exec: message received, sending message 12
    App_exec: message received, sending message 13
    App_exec: message received, sending message 14
    App_exec: message received, sending message 15
    App_exec: message received
    App_exec: message received
    App_exec: message received
    <-- App_exec: 0
    --> App_delete:
    <-- App_delete:
    <-- Main_main:
    <-- main:

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# date
    Sat Apr 6 01:38:40 UTC 2019

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# [ 135.069428] NET: Registered protocol family 15
    [ 135.123851] Initializing XFRM netlink socket

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# date
    Sat Apr 6 01:39:59 UTC 2019
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug#
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug#

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# date
    Sat Apr 6 01:51:07 UTC 2019

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ./app_host DSP1
    --> main:
    [ 884.547527] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
    --> Main_main:
    --> App_create:
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    App_exec: sending message 1
    App_exec: sending message 2
    App_exec: sending message 3
    App_exec: message received, sending message 4
    App_exec: message received, sending message 5
    App_exec: message received, sending message 6
    App_exec: message received, sending message 7
    App_exec: message received, sending message 8
    App_exec: message received, sending message 9
    App_exec: message received, sending message 10
    App_exec: message received, sending message 11
    App_exec: message received, sending message 12
    App_exec: message received, sending message 13
    App_exec: message received, sending message 14
    App_exec: message received, sending message 15
    App_exec: message received
    App_exec: message received
    App_exec: message received
    <-- App_exec: 0
    --> App_delete:
    <-- App_delete:
    <-- Main_main:
    <-- main:

    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug# ./app_host DSP2
    --> main:
    --> Main_main:
    --> App_create:
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    App_exec: sending message 1
    App_exec: sending message 2
    App_exec: sending message 3
    App_exec: message received, sending message 4
    App_exec: message received, sending message 5
    App_exec: message received, sending message 6
    App_exec: message received, sending message 7
    App_exec: message received, sending message 8
    App_exec: message received, sending message 9
    App_exec: message received, sending message 10
    App_exec: message received, sending message 11
    App_exec: message received, sending message 12
    App_exec: message received, sending message 13
    App_exec: message received, sending message 14
    App_exec: message received, sending message 15
    App_exec: message received
    App_exec: message received
    App_exec: message received
    <-- App_exec: 0
    --> App_delete:
    <-- App_delete:
    <-- Main_main:
    <-- main:
    root@am57xx-evm:/usr/bin/ipc/examples/ex02_messageq/debug#
  • 0 in reply to Rex Chang
    Prodigy 30 points
    Hi Rex,
    Thanks so much.
    Today, I tried with v4.1 prebuilt bin and they worked pretty well. Then I came back to v5.03 that I built last time. It works and no crash again for both DSP1 and DSP2. Amazing!!
    Actually I am totally confused now. Have no idea with the issue. But at least I can start off with my own tests.

    More Questions:
    1, what's the difference between sdk-linux and sdk-linux-rt? I saw they have the same linux core version.
    2, where can I get the older version of the these packages??
  • 0 in reply to shi Li
    TI__Guru 50170 points
    HI, Shi,

    Linux -RT is the Real Time Linux. Please see more info on RT Linux in internet, such as en.wikipedia.org/.../RTLinux

    At the end of each release download page, there is a link to the previous release version. Please keep in mind that TI only supports one version back from current release. The PSDK 6.0 release is around the corner. After 6.0 is released, only 5.x will be supported.

    If this post answers your questions, please click "Resolved".

    Rex