Hi,
Sorry for the long post but it summarizes our concerns.
Problem Statement
Our board has a single keystone2(KS2) device connected to two keystone1(KS1) devices via hyperlink.
On KS2 ARM we run several linux processes which need to communicate with local and remote DSPs, it is critical for our application that ARMßàDSP communication will be high speed and low latency.
We looked at TI’s software ecosystem but couldn’t find a suitable package for our key requirements:
- ARM should be able to send different messages + data to any local/remote DSP
- DSP should be able to send different messages + data to ARM
- DSP/ARM should not poll (busy wait) in order to observe new messages
- Several ARM processes should be able to send messages simultaneously (even to same DSP)
- Several DSPs should be able to send messages to ARM simultaneously
- No ARMßàDSP message should be lost (i.e. reliable mechanism)
Unless there is some ready package we can use, we’ll have to implement a software package which fulfills above requirements.
Suggested Solution Concept
We thought of having some shared memory mapped by ARM & remote device in addition to bidirectional triggering mechanism (interrupt based). This will enable us implementing application specific protocols over it.
Ideally we would like to achieve the following:
Triggering
ARM side (linux – user mode):
void raiseInterrupt(DeviceID, DSPIndex, int data) – raises an interrupt on the given device dsp (DeviceID - local/remote1/remote2, DSPIndex – 1-8), preferably passing an additional 32bit data.
void registerInterrupt(DeviceID, DSPIndex, void (*ISR)(DeviceID, DSPIndex, int data)) – registers a callback with an interrupt of a specific device dsp.
DSP side (SYS/BIOS):
void raiseInterrupt(int data) – raises an interrupt on ARM. This will cause all registered callbacks on the raising device dsp to be called on ARM (possibly on several processes). Preferably passing an additional 32bit data.
void registerInterrupt(void (*ISR)(int data)) – registers a callback with an interrupt from ARM. Preferably passing an additional 32bit data.
For ARMàDSP triggering, we consider using hyperlink interrupts and “wiring” them to remote core interrupts (i.e. hyperlink interrupt0 to dsp0, interrupt1 to dsp1 and so on).
For DSPàARM triggering, we consider using hyperlink interrupts too, “wiring” them to linux and associate it with the relevant registered processes callbacks ISRs (recall ISR in user mode).
Shared Memory
ARM side (linux – user mode):
Using mmap, we consider mapping a continuous segment of ARM’s physical memory to be shared with remote devices (i.e. 2 segments, each of 128MB).
We’ll configure the TX (KS1 side) & RX (KS2 side) hyperlink settings so that the remote device hyperlink space will map to above ARM’s physical memory. In order to allow concurrency among remote device DSPs, we’ll have a dedicated shared memory area per DSP in the shared segment (i.e. 128MB/8DSPs = 16MB per DSP)
Messages Passing
Combining the triggering and shared memory as suggested, we can achieve message passing as follows:
ARMàDSP
[ARM]: Write message data to target DSP shared memory area
Raise interrupt on target DSP
[DSP]: On ISR, read message data from shared memory area and process it.
DSPàARM
[DSP]: Write message data to shared memory area
Raise interrupt on ARM
[ARM]: On ISR and according to raising DSP, read message data from shared memory area of the DSP
and process it
Questions and Concerns
- Is there any package we can use? If not what is suggested by TI for inter-devices/ARM-DSP communication over hyperlink?
- Following our suggested concept, does TI can provide an example for the triggering & shared memory mechanisms? While shared memory over hyperlink seems to be feasible from TI’s docs, the triggering part (and hyperlink interrupts especially involving linux) seems to lack docs and examples.
- Observing the IPC package, we found no implementation for hyperlink transport. Does TI has such one or intend to provide this?
- In keystone2 mcsdk, I could find libraries and code for C66 LLD (low level drivers) but couldn’t find the one for ARM-linux (although mentioned as existing in docs). Where can I find the libs and sources for ARM-linux?
- Is it possible to perform concurrent hyperlink transactions (i.e. several DSPs to ARM or several ARM processes to DSPs), and what about interrupts (can we raise different interrupts in parallel)?
Thanks,
Assaf - Applied Materials
