IPC multi threading question (Keystone II)

Anthony Diak

Intellectual 430 points

Other Parts Discussed in Thread: SYSBIOS

I've got an EVMK2H and i'm trying to setup a simple IPC project that does the following:

On each DSP core, create a message q, listen, and respond back to a message when received.
On the ARM, create a thread per DSP core and send messages from that thread to a DSP core.

For creating the threads on the ARM, I used the smp_test as an example, which uses the clone() function.

The questions/problems I have are more on the ARM (Linux) side. I've successfully got one thread talking to a DSP core.

Do I need to call Ipc_transportConfig() (and/or) Ipc_Start() on each thread, or just once before I start creating the threads?
My first MessageQ_Create() succeeds, but subsequent threads all return NULL
- Each thread has a unique name that is being passed to the MessageQ_Create
- Is there any way to figure out why this failed? some sort of error code?

over 8 years ago

0 Titusrathinaraj Stalin over 8 years ago

TI__Guru** 116100 points

Hi,
Did you visit to our MCSDK processing demo TI wiki page at any time ?
We have IPC image processing demo code for K1 and K2 devices.

C:\ti\mcsdk_bios_3_01_01_04\demos\image_processing\ipc\evmtci6638k2k

processors.wiki.ti.com/.../MCSDK_Image_Processing_Demonstration_Guide

0 Shankari G over 8 years ago

TI__Mastermind 43955 points

Hi Anthony,

Already an example ( Image processing ) exists which demonstrates the IPC communication between ARM cores and DSP cores .

The image processing's master side code runs on ARM Core and the slave side code runs on DSP core.

The master side code runs on top of linux and the salve side code runs on top of Sys/Bios. The MPM running on the ARM core will load the salve code on DSP cores and the DSP cores in turn process the image.

It is always good to start with something which already works. I would recommend you to take the Image processing demo and run it once and have a look at the source code. With reference to that code, you can develop your own.

Source code info - Enclosed in package: mcsdk_bios_3_00_03_15

Slave side : ti\mcsdk_bios_3_00_03_15\demos\image_processing\ipc\evmtci6638k2k\slave

Master side: \ti\mcsdk_bios_3_00_03_15\demos\image_processing\ipc\master\src

Regards,

Shankari

-------------------------------------------------------------------------------------------------------

Please click the Verify Answer button on this post if it answers your question.

--------------------------------------------------------------------------------------------------------

0 Anthony Diak over 8 years ago in reply to Shankari G

Intellectual 430 points

That is what I based my code on so far, but that demo looks like it only has one ARM thread talking to DSP cores (which I have successfully done). I want to get multiple ARM threads talking to DSP cores. So there would be 8 ARM threads, each with their own messageQ, and 8 tasks (one on each DSP core), each with their own messageQ.

Each ARM thread would talk to a DSP core task (1:1). The DSP tasks all seem to start up fine and a single ARM task can create a messageQ and talk to any DSP task. My problem is when I try to create more than 1 ARM thread. The first one will work fine, but the second (and subsequent) threads won't create a messageQ.

0 Shankari G over 8 years ago in reply to Anthony Diak

TI__Mastermind 43955 points

Hi Anthony,

I aprreciate your effort in creating multiple ARM threads. But what all we have, is just the Image processing demo for IPC.

For creating multiple message Q, please refer to the project, " \ti\mcsdk_2_01_02_06\demos\image_processing\ipc\evmc6678l\master" in which multiple meassage Qs are created for each core in the master code. But, that code is for C6678, not for ARM core. You can just have a look at it and try creating MessageQ in a similar way in your ARM Code.

Update us how it goes. If you succeed, post your insights too so that it would be beneficial for other community members.

0 Anthony Diak over 8 years ago in reply to Shankari G

Intellectual 430 points

I got my project working. I was able to get the MessageQ_create to work with pthread_create() instead of using clone() to create a new thread. I was using the clone() from the example smp_test project.

Another important configuration item that I was missing (once i got the MessageQ creation issue fixed) was that I needed to turn off the cache using the following config:

Cache.setMarMeta(0xA0000000,0x1FFFFFF,0);

Cache.setMarMeta(0x80000000,0x20000000,0);

I'm not sure why these need set -- I thought it was leftover from the DDR memory mapping of the image processing demo, but obviously it is important. I'm not explicitly loading code/data into any of these memory locations via the linker.

Everything now works as I expected, but the memory pointers do not act as I expected them to (I expect the round trip message to have the same address, and the DSP pointer keeps incrementing).

The flow of my program is this:

ARM allocates message using MessageQ_alloc (0xb64004a0)
ARM sends message to DSP
DSP receives message (0x800258) <-- this value changes every receive (it increments)
DSP reuses message and sends back to ARM
ARM receives message (0xb6200468) <--- why is this not the same pointer as before (0xb64004a0)?
ARM frees message using MessageQ_free

To answer my original questions:

Do I need to call Ipc_transportConfig() (and/or) Ipc_Start() on each thread, or just once before I start creating the threads?
- You only need to call it once before starting the threads.
My first MessageQ_Create() succeeds, but subsequent threads all return NULL
Each thread has a unique name that is being passed to the MessageQ_Create
Is there any way to figure out why this failed? some sort of error code?
- No idea why this failed, but the clone() function for thread creation did not work (at least not with CLONE_VM | CLONE_SIGHAND | CLONE_FILES)

0 Robert Tivy over 8 years ago in reply to Anthony Diak

TI__Mastermind 18260 points

Anthony Diak said:

Another important configuration item that I was missing (once i got the MessageQ creation issue fixed) was that I needed to turn off the cache using the following config:

Cache.setMarMeta(0xA0000000,0x1FFFFFF,0);

Cache.setMarMeta(0x80000000,0x20000000,0);

I'm not sure why these need set -- I thought it was leftover from the DDR memory mapping of the image processing demo, but obviously it is important. I'm not explicitly loading code/data into any of these memory locations via the linker.

I'm not sure about the 0x80000000, 0x20000000 range, but IPC needs the vring buffers to be non-cached, and those buffers start at 0xA0000000. Linux likely owns the whole area from 0x80000000 -> 0xA0000000, and as such it probably doesn't need to be marked non-cached from the DSP's perspective (in fact, the DSP should *never* be accessing this range).

Anthony Diak said:

ARM allocates message using MessageQ_alloc (0xb64004a0)

ARM sends message to DSP

DSP receives message (0x800258) <-- this value changes every receive (it increments)

DSP reuses message and sends back to ARM

ARM receives message (0xb6200468) <--- why is this not the same pointer as before (0xb64004a0)?

ARM frees message using MessageQ_free

MessageQ_put() frees the message that is passed to it, and even though you MessageQ_alloc()ed it you are giving up ownership when passing it to MessageQ_put(). Likewise, MessageQ_get() allocates a new message, causing ownership of that message to pass to you when MessageQ_get() returns (which means that you're responsible for either freeing it, or otherwise giving up ownership as happens with MessageQ_put()).

Regards,

- Rob

0 Anthony Diak over 8 years ago in reply to Robert Tivy

Intellectual 430 points

Thanks for the reply -- So it looks like my plan on trying to avoid dynamic memory allocation has been thwarted if I want to use MessageQ.

I was hoping that If I only called MessageQ_alloc() once, that only one item would be actually used from the heap. My plan was to allocate a bunch at startup and then be able to reuse them (I was going to enforce a response back to return the message), but now I see that MessageQ_get actually allocates memory off the heap.

Is there any other IPC that can go Linux <-> SYSBIOS without doing the dynamic memory allocation? Is there one for SYSBIOS <-> SYSBIOS?

Processors

Processors forum

IPC multi threading question (Keystone II)