• State Resolved
  • Locked Locked
  • Replies 31 replies
  • Answers 1 answer
  • Subscribers 98 subscribers
  • Views 1376 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

CCS/AM5728: Debugging IPC Example

Linux/TMDXIDK5728: IPC - ARM15 to DSP - MessageQ: Custom Application

Alec Phillips
Expert 2280 points

Part Number: TMDXIDK5728

Tool/software: Linux

Hello Linux Support,

I am able to run the makefile MessageQ example. (e2e.ti.com/.../2839504)

I want to build a custom application using the MessageQ, with IPC between the Linux ARM core and the RTOS DSP core. Can I build it on Linux for this use-case? The following link indicates the CCS project should be starting based of the DSP/BIOS examples. 


Any advice you can give to aid me in developing a small custom test application would be really helpful. My current goal is to communicate/pass data from a Linux PC Redis server to the Sitara Client (I was thinking the ARM core first), and then pass that data from the Linux core to the DSP core. I am not very familiar with building custom makefile applications. The Redis part doesn't really matter, I can implement my own functions and method calls once I figure out how to get the custom project built. 

Thank you in advance.

Regards,
Alec

  • 0
    TI__Guru 50170 points
    Hi, Alec,

    Please take a look at the IPC Quick Start Guide,
    software-dl.ti.com/.../Foundational_Components_IPC.html
    in which it shows how to rebuild IPC examples using Linux SDk high level build.

    You can also take a look at the out-of-box demo, MessageQBench, and see how it works. The source code for Linux/ARM and RTOS/DSP are indicated in the Quick Start Guide.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex Chang said:

    You can also take a look at the out-of-box demo, MessageQBench, and see how it works. The source code for Linux/ARM and RTOS/DSP are indicated in the Quick Start Guide.

    Hi Rex,

    Can you help me locate the mentioned source code? I cannot seem to find this demo program. 

    Thank you,

    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    The IPC Quick Start Guide at
    software-dl.ti.com/.../Foundational_Components_IPC.html

    It describes how to build from high level and also shows where the source code is located.

    Rex
  • 0 in reply to Rex Chang
    TI__Guru 50170 points
    Sorry, correct url for QUick Start Guide should be
    software-dl.ti.com/.../Foundational_Components_IPC.html

    You can always click it from IPC Page any way.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex,

    I've looked at it and now see where it is located. Would have been nice if you provided a screenshot, since the small text box is below the images. I will see how I can use this. 

    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points

    Hi, Alec,

    The document says:

    The source code are located in:

Linux side application: <RTOS_SDK_INSTALL_DIR>/ipc_x_xx_xx_xx/linux/src/tests/MessageQBench.c
DSP side application:   <RTOS_SDK_INSTALL_DIR>/ipc_x_xx_xx_xx/packages/ti/ipc/tests/messageq_single.c

    The x_xx_xx_xx is the IPC version in the SDK. Are you able to find them?

    Rex

  • 0 in reply to Rex Chang
    Expert 2280 points
    Rex,

    Yes I located the files. They are up to date (within the /tests/ folder i ran "make MessageQBench"). I tried executing the files, "./MessageQBench" and I got the following output:
    "Line 117: Cannot execute binary file: exec format error" and "Line 117: Success".

    How do I execute the files?

    Alec
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    I have never tried to rebuild MessageQBench by the way you do, but always rebuild from high level as document described. There is an error on the location of ARM binary in the document which I just noticed last week. The executable should be in ipc/linux/src/tests/.lib.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points
    Rex,

    I'm getting "Cannot execute binary file: exec format error" when I try to run any of the executable files. I just checked, and they're all 32-bit executable files, which I am then attempting to be run on a 64-bit Ubuntu OS. This doesn't work... What can I do?

    Alec
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points

    Hi, Alec,

    The code is supposed to run on the EVM. You should copy it to the EVM Linux.

    Rex

  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex Chang said:

    The code is supposed to run on the EVM. You should copy it to the EVM Linux.

    This is what makes no sense to me about the TI Linux file system. Half your references are to EVM Linux, the other half are to the Linux OS; but there's no indication anywhere in the documentation that says one way or the other. How is the user supposed to determine the correct destination for the instructions if they are unclear to begin with? 

    How do you suggest getting it copied to my EVM? I'm using a pre-built SD card kernel for U-boot. Do I have to keep re-building the SD card? What methods are available? 

  • 0 in reply to Alec Phillips
    Expert 2280 points

    I'm able to find the MessageQBench files in /usr/share/ti/ti-ipc-tree/qnx/src/tests/MessageQBench/ on my SD card through minicom terminal connection.

    Having issues with the makefile. How do I build the example on the EVM?

  • 0 in reply to Alec Phillips
    Expert 2280 points
    Ok, I'm slowly piecing this together. Now I'm trying to run the "make -f ipc-qnx.mak all" command but do not have the environment variables setup yet. I will attempt to follow the procedure outlined in the TI documentation and will post again when I run into trouble.
  • 0 in reply to Alec Phillips
    Expert 2280 points

    Hi Rex,

    I'm on this page:

    What is the default Linux SDK Install DIR for the SD card?

    $ cd <TI_LINUX_PROC_SDK_INSTALL_DIR>
$ make ti-ipc-linux

    Thank you,

    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    We don't support native build of the IPC on EVM. The development is done on Ubuntu machine. The <TI_LINUX_PROC_SDK_INSTALL_DIR> is where you install the Linux SDK at on the Ubuntu machine, and RTOS_INSTALL_PATH will be where the RTOS SDK is installed. You rebuilt the ti-ipc-linux on Ubuntu machine and copy the executable to the EVM. I don't believe you can build within RTOS/ipc/linux/src/tests directory. It requires some libraries in Linux, so I suggest you do the high level build from Linux SDK as described in IPC Quick Start Guide.

    I am not sure if I mentioned. The MessageQBench prebuilt binary is in EVM filesystem in /usr/bin, and MessageQ_single.xe66 is in /lib/firmware/ipc/ti_platforms_evmDRA7XX_dspX directory. You can try running them first.

    QNX is not our support scope. It belongs to the other Business Unit. If you have question on IPC for QNX, please submit a separate thread.

    You can either scp from ubuntu to the EVM, or mount the SD card on your Ubuntu machine and copy the executable to the file system on the SD card.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex,

    When I run the MessageQBench application, I'm getting omap errors:

    "omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    omap_iommu: 41501000.mmu: 41501000.mmu: version 3.0
    omap_iommu: 41502000.mmu: 41502000.mmu: version 3.0"
    Then the application begins to execute; but does not appear to complete. I left it running when I left last night and it made no progress by my return today. 

    After looking into the error on the TI E2E forums, it seems this occurs when the core goes into a "suspended" state. So it should have no effect on the MessageQBench application. I tried running the app with only 100 loops instead of the default 1000; and still there is no output from the application. Please help me understand what I should expect to see and also how to debug why I'm not seeing anything. 


    Thank you,
    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    Those messages are irrelevant. Which ProcID did you send to? I believe on AM57x, the ProcID for DSP1 is 4. Try this command:
    MessageQBench 100 8 4

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points
    Rex,

    I know that now. Still no output from that command. The ProcID I used initially was the default, which is 1.

    Now I've tried ProcID 1, 2, 3, and 4. No output other than:
    "Using numLoops: 10; payloadSize: 8, procID: 4"
    "Entered MessageQApp_execute"
    "Local MessageQID: 0x87"
    The Local ID increments each time I run the application.

    I wait a few minutes and nothing happens.

    Alec
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    Do you use the prebuilt binaries or the ones you build? If not prebuilt, try to use prebuilt binaries first. That should rule out build issue.
    If you are using prebuilt, then that's something else. The prebuilt should work and it passed System Test before each SDK release.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex,

    I'm using pre-built, since I still have no idea what I'm doing. I found the pre-built binary in the /usr/bin/ folder as you pointed out to me yesterday. So it's something else.

    I'm trying to think of any changes I made. When I ran the MessageQ application following the instructions, there was a change made to the DSP launch target.

    Could this be the issue?

    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Ok. I know what the problem is now.

    In IPC, we have tests and examples. ex02_messageq is an example which DSP as server receives messages from Linux IPC (app_host), and reverse the source and destination address (ProcID) before sending them back to host. There isn't anything more to it, but ping-pong'ing the messages.

    MessageQBench is a test (under tests directory and system test uses it for its test). The difference from ex02 is the host send a message first to tell DSP how many packets it's going to send and waits for the acknowledge back to verify the connection. Then it sends the negotiated number of packets which DSP will be expecting and checking the msgID. Both also measure the time to calculate average round trip time.

    The MessageQBehcn is considered as out-of-box demo and you should only look at the section of "Rebuilding the demo". The texts you quoted are for "building IPC Linux examples". You used ex02 DSP binary with MessageQBench which breaks the handshaking.

    MessageQBench host binary is located in /usr/bin and its corresponding DSP binaries are located in /lib/firmware/ipc/ti_platform_[evm]_dspX
    Both of the host and dsp binaries for ex02_messageq example are in /usr/bin/ipc/examples/ex02_messageq/debug

    Hope this resolves your issue.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points
    Rex,

    Thanks for the explanation. That helps.

    The only change I made to the SD card file system is what was shown in the screenshot. Is there a way to fix this without rebuilding the entire SDK? Is the original version of the dra7-dsp1-fw.xe66 located anywhere else on the disk so I can replace the one I modified? I did a search and found /usr/lib/opkg/alternatives/ has the file, but I don't know if there's a difference between it and the one in /lib/firmware/. I do not see the file in /lib/firmware/ipc/. The 4 files in that directory are "ti_platforms_evmDRA7XX_" dsp1, dsp2,ipu1,ipu2.

    Alec
  • 0 in reply to Alec Phillips
    Expert 2280 points
    I back-tracked my steps in the example. I saw what the other cores were linked to, reassigned the Linked file in /lib/firmware/, unbound and bound the DSP core. Now I'm getting the _wait_target_disable warning for DSP1 core. I tried running the MessageQBench pre-built binary, but it still is not doing anything. And I targeted ProcID 4, as you suggested earlier.

    Please let me know if there's anything else I can try before going back to the Ubuntu system.

    Thanks,
    Alec
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    dra7-dsp1-fw.xe66 is a link to dra7-dsp1-fw.xe66.opencl-monitor in the default setup for other demo. You only removed the link to it. You can always restore it by "ln -sf" command to link it back. the dsp1 opencl-monitor binary is in the same folder. In the worst case, you can always copy it from the filesystem in the PLSDK release. Just untar the tisdk-rootfs-image-am57xx-evm.tar.xz in Linux_SDK/filesystem on your Ubuntu machine and find the file under the same directory, then scp to your EVM filesystem.

    My EVM is tied up trying to reproduce an issue which may run overnight. So, I can't execute MessageQBench to show you, but it is loading messageq_single.xe66, the DSP binary for MessageQBench. I am not sure if I mentioned it, messageq_single.xe66 is the binray you should load on DSP for MessageQBench test. you want to link dra7-dsp1-fw.xe66 to /lib/firmware/ipc/ti-platform_DRA7xx_dsp1/messageq_single.xe66.

    That message is irrelevant. The logs for DSP loading from my EVM show both DSP binaries are loaded and DSP's are up.

    am57xx-evm login: [ 19.678557] omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    [ 19.692533] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 27.404519] NET: Registered protocol family 15
    [ 27.459421] Initializing XFRM netlink socket
    root
    root@am57xx-evm:~# dmesg | grep dsp
    [ 0.000000] OF: reserved mem: initialized node dsp1-memory@99000000, compatible id shared-dma-pool
    [ 0.000000] OF: reserved mem: initialized node dsp2-memory@9f000000, compatible id shared-dma-pool
    [ 0.471592] iommu: Adding device 40800000.dsp to group 0
    [ 0.471866] iommu: Adding device 41000000.dsp to group 3
    [ 7.890166] omap-rproc 40800000.dsp: assigned reserved memory node dsp1-memory@99000000
    [ 7.922399] remoteproc remoteproc2: 40800000.dsp is available
    [ 7.954863] omap-rproc 41000000.dsp: assigned reserved memory node dsp2-memory@9f000000
    [ 7.969485] remoteproc remoteproc3: 41000000.dsp is available
    [ 8.694728] remoteproc remoteproc2: powering up 40800000.dsp
    [ 8.704636] remoteproc remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4721340
    [ 8.711307] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 8.768308] remoteproc remoteproc2: remote processor 40800000.dsp is now up
    [ 8.998536] remoteproc remoteproc3: powering up 41000000.dsp
    [ 9.005601] remoteproc remoteproc3: Booting fw image dra7-dsp2-fw.xe66, size 4721136
    [ 9.012245] omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    [ 9.043713] remoteproc remoteproc3: remote processor 41000000.dsp is now up
    [ 19.678557] omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    [ 19.692533] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    root@am57xx-evm:~# reboot -f now
    Rebooting with argument 'now'.


    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points
    Ok Rex,
    I have decided to rebuild the SD card since I had already replaced the linked file with "dra7-dsp1-fw.xe66.opencl-monitor" and it didn't work. I'd rather start fresh at this point. I will post again if I have trouble running the MessageQBench pre-built binary.
    Thanks.
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    You don't really need to care about the original dra7-dsp1-fw.xe66 link and the opencl-monitor binary.
    You need to overwrite the link to use the messageq_single.xe66 any way.

    At the current state, you should be able to do the following:
    $ cd /lib/firmware
    $ ln -sf /lib/firmware/ipc/tisdk_platform_DRA7xx_dsp1/messageq_single.xe66 dra7-dsp1-fw.xe66

    Usually, unbind/bind should work, but the worse case is to power cycle the EVM to get a fresh start in case DSP is in a bad state. .
    Recreating the SD card will be the last resort. It did happen to me before that I needed to recreate the SD card to clean up the mess.

    I'll close the thread for now, but reply to this thread when you still have issue after recreating the SD card.
    Replying to this thread will re-open it.

    Rex
  • 0 in reply to Rex Chang
    Expert 2280 points

    Rex,

    Rex Chang said:

    You don't really need to care about the original dra7-dsp1-fw.xe66 link and the opencl-monitor binary.
    You need to overwrite the link to use the messageq_single.xe66 any way.

    When viewing the list of files in /lib/firmware/ the link to the DSP2 is to the opencl-monitor binary. How can you say I don't need to care about that? 

    I set the link as you suggested and it did not repair the MessageQBench application's functionality. It still gets held up after execution. Previously, before I was changing around the binary pointer for DSP1, the MessageQ Example 02 app worked. I tried it out now, after changing the pointer to the messageq_single.xe66, and that application also doesn't work. 

    I tried to "fix" the IPC by copying over the files from the Ubuntu system, but everything is read-only. I checked my physical card and it is unlocked. This is very frustrating to be going in circles with no improvement. 

    A fresh SD card appears to be my only option at this point. 

    Alec

  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    It should be a very simple task, but I am not sure where it went wrong and causes all these issues.
    The reason I said dsp2 using opencl-monitor binary won't matter because it's not the core we run MessageQBench against. Only if you want to run the bench binary to dsp2, then the dsp2 needs to use the messageq_single.xe66. By linking the binary is not enough. I assume you had either unbind/bind the dsp after changing the link, or reboot the system. My EVM is freed, and I had both links pointed to the messageq_single.xe66 earlier, so I modified the dsp2 to point to opencl-monitor. My console logs are shown below. Did you do anything differently from mine?

    root@am57xx-evm:/lib/firmware#
    root@am57xx-evm:/lib/firmware# ls -l dra7-dsp?-fw.xe66
    lrwxrwxrwx 1 root root 66 Dec 16 03:38 dra7-dsp1-fw.xe66 -> /lib/firmware/ipc/ti_platforms_evmDRA7XX_dsp1/messageq_single.xe66
    lrwxrwxrwx 1 root root 66 Dec 16 03:38 dra7-dsp2-fw.xe66 -> /lib/firmware/ipc/ti_platforms_evmDRA7XX_dsp2/messageq_single.xe66
    root@am57xx-evm:/lib/firmware# ls *.opencl-monitor
    dra7-dsp1-fw.xe66.opencl-monitor dra7-ipu1-fw.xem4.opencl-monitor
    dra7-dsp2-fw.xe66.opencl-monitor
    root@am57xx-evm:/lib/firmware# ln -sf dra7-dsp2-fw.xe66.opencl-monitor dra7-dsp2-fw.xe66
    root@am57xx-evm:/lib/firmware# ls -l dra7-dsp?-fw.xe66
    lrwxrwxrwx 1 root root 66 Dec 16 03:38 dra7-dsp1-fw.xe66 -> /lib/firmware/ipc/ti_platforms_evmDRA7XX_dsp1/messageq_single.xe66
    lrwxrwxrwx 1 root root 32 Dec 16 06:21 dra7-dsp2-fw.xe66 -> dra7-dsp2-fw.xe66.opencl-monitor
    root@am57xx-evm:/lib/firmware# reboot -f now

    [EVM reboot]

    am57xx-evm login: root
    root@am57xx-evm:~# uname -a
    Linux am57xx-evm 4.14.79-gbde58ab01e #1 SMP PREEMPT Thu Dec 20 04:51:24 UTC 2018 armv7l GNU/Linux
    root@am57xx-evm:~# l[ 28.565773] NET: Registered protocol family 15
    s [ 28.621955] Initializing XFRM netlink socket

    root@am57xx-evm:~#
    root@am57xx-evm:~#
    root@am57xx-evm:~# ls -l /lib/firmware/dra7-dsp?-fw.xe66
    lrwxrwxrwx 1 root root 66 Dec 16 03:38 /lib/firmware/dra7-dsp1-fw.xe66 -> /lib/firmware/ipc/ti_platforms_evmDRA7XX_dsp1/messageq_single.xe66
    lrwxrwxrwx 1 root root 32 Dec 16 2018 /lib/firmware/dra7-dsp2-fw.xe66 -> dra7-dsp2-fw.xe66.opencl-monitor
    root@am57xx-evm:~# MessageQBench 100 8 4
    [ 59.056679] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
    [ 59.100664] omap_hwmod: mmu0_dsp2: _wait_target_disable failed
    [ 59.106551] omap-iommu 41501000.mmu: 41501000.mmu: version 3.0
    [ 59.112576] omap-iommu 41502000.mmu: 41502000.mmu: version 3.0
    [ 59.125900] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 59.131782] omap-iommu 40d01000.mmu: 40d01000.mmu: version 3.0
    [ 59.137835] omap-iommu 40d02000.mmu: 40d02000.mmu: version 3.0
    Using numLoops: 100; payloadSize: 8, procId : 4
    Entered MessageQApp_execute
    Local MessageQId: 0x80
    Remote queueId [0x40080]
    Exchanging 100 messages with remote processor DSP1...
    DSP1: Avg round trip time: 141 usecs
    Leaving MessageQApp_execute

    root@am57xx-evm:~#
  • 0 in reply to Rex Chang
    Expert 2280 points
    Hi Rex,

    Thanks for going through it for me. I had forgotten to select ProcID 4 when re-running the fixed file link. Now when I execute the MessageQBench program, it works.

    Now if I want to modify the application, and rebuild it; what do I do? Once I can accomplish my original goal, I can close the thread.

    Thank you,
    Alec
  • 0 in reply to Alec Phillips
    TI__Guru 50170 points
    Hi, Alec,

    I know it is a bit confusion especially that the same code runs on different platforms with different number of DSP cores. Originally, it was for KS2 family which only has DSP cores, but with Sitara family, it has IPUs. So, the default target of dsp1/CORE0 becomes CORE3 (procid 4).

    Any way, I am glad that you got the demo working. You can modify the "RTOS_SDK"/ipc/linux/src/tests/MessageQBench.c code on Ubuntu machine. Once you are done modifying, you just do a high level build (make ti-ipc-linux) in "Linux" SDK on Ubuntu machine. The DSP binary will be in "RTOS_SDK"/ipc/linux/src/tests/.lib. I put "" around to bing your attention on the SDK changes
    Don't forget to set the environment variables as described in Quick Start Guide.

    If you modify messageq_single.c, then follow the instruction in "Rebuilding the demo" under "DSP RTOS" section to build ipc_bios (make ipc_bios) in RTOS_SDK/processor_sdk_rtos_<platform>_<version> directory. Again, set the environment variables and "source ./setupenv.sh" as described in the Quick Start guide.

    I suggest that you exercise the build before making any changes to be sure you can build and run the MessageQBench from what you build. Then, start modifying the code.

    Rex
  • 0 in reply to Rex Chang
    TI__Guru 50170 points
    Hi, Alec,

    I believe you can at least build the MessageQBench in the release RTOS SDK by now. If that is the case, I'll close this thread. If you have other issue down the road, Please create a new thread.

    Rex