Hi,
I have this intermittent problem which more so seems like timing related.
My setup:
Custom Board with OMAP L138, ipc_1_25_03_15, bios_6_35_04_50, xdctools_3_25_03_72, syslink_2_21_02_10
At startup on ARM, the dsp "slaveloader_release startup DSP dsp_image.dsp"
Then start ARM side module that creates the Heaps, MessageQ.
these two operations are done in the /etc/init.d/rcS which is run as part of the Linux load process.
All this works and there is communication between the ARM and the DSP.
However, when I manually start the DSP from linux prompt and start the ARM module I get a stack dump.
I have modified the "syslink_2_21_02_10/packages/ti/syslink/family/hlos/knl/omapl1xx/Platform.c" and also the target.cfg to have HeapBufMP config.trackAllocs=TRUE and MessageQConfig.numHeaps=20.
The Heap creation, MessageQ creation and their open, the IPC events are all based on the example "syslink_2_21_02_10\examples\archive\OMAPL138_linux_elf\ex02_messageq"
The SR0 in target.cfg is:
var SharedRegion = xdc.useModule('ti.sdo.ipc.SharedRegion');
/* configure SharedRegion #0 (IPC) */
var SR0Mem = Program.cpu.memoryMap["DDR_SHARED"];
SharedRegion.setEntryMeta(0,
new SharedRegion.Entry({
name: "SR0",
base: 0xC7000000,
len: 0x00200000, /* 2MB shared space */
ownerProcId: MultiProc.getIdMeta("HOST"),
cacheEnable: true,
isValid: true
})
);
I also use SR1 directly:
var SR1Mem = Program.cpu.memoryMap["DDR_SR1"];
SharedRegion.setEntryMeta(1,
new SharedRegion.Entry({
name: "DDR_CDM",
base: 0xC7200000,
len: 0x00600000,
ownerProcId: MultiProc.getIdMeta("HOST"),
cacheEnable: true,
isValid: true
})
);
the target.bld defines these memories as:
["DDR_SHARED",
{
name: "DDR_SHARED",
base: 0xC7000000,
len: 0x00200000,
space: "data",
access: "RW",
}
],
["DDR_SR1",
{
name: "DDR_SR1",
base: 0xC7200000,
len: 0x00600000,
space: "data",
access: "RW",
}
],
The stack dump is below. It comes from the ISR that handles the events from the DSP.
Assertion at Line no: 1605 in syslink/syslink_2_21_02_10/packages/ti/syslink/utils/hlos/knl/Linux/../../../../../../ti/syslink/ipc/hlos/knl/MessageQ.c: (UInt16)queueId < MessageQ_module->numQueues : failed
Unable to handle kernel paging request at virtual address c7ab9a5c
pgd = c0004000
[c7ab9a5c] *pgd=c5a49011, *pte=00000000, *ppte=00000000
Internal error: Oops: 7 [#1] PREEMPT
last sysfs file: [[removed]]
Modules linked in: [[removed]] syslink
CPU: 0 Not tainted (2.6.37 #1)
PC is at MessageQ_put+0x338/0x500 [syslink]
LR is at pick_next_task_fair+0xa4/0xc4
pc : [<bf0315b0>] lr : [<c0037874>] psr: 60000013
sp : c38cdee8 ip : c03a9ab8 fp : c38cdf1c
r10: ce401900 r9 : 00000000 r8 : 00000100
r7 : 00000000 r6 : bf0b5664 r5 : 00007297 r4 : ce418808
r3 : c7a9d000 r2 : a0000093 r1 : 00000000 r0 : 0000013d
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 0005317f Table: c4b84000 DAC: 00000017
Process kernelThread_0 (pid: 538, stack limit = 0xc38cc270)
Stack: (0xc38cdee8 to 0xc38ce000)
dee0: ce418808 00000294 c38cdf1c c38cdf00 bf0203d0 ce35b000
df00: 0000ffff ce418808 ce401800 00000100 c38cdf4c c38cdf20 bf04e084 bf031288
df20: 00000000 00000002 ce35b000 00000000 00000000 ce346000 00000002 00000000
df40: c38cdf74 c38cdf50 bf02ef74 bf04dfdc 00000000 00000000 ce401900 ce33d000
df60: 00000000 00000002 c38cdfa4 c38cdf78 bf04a18c bf02eec0 bf04a0a8 ce343000
df80: ce343020 bf024ac0 00000013 00000000 00000000 00000000 c38cdfbc c38cdfa8
dfa0: bf024b50 bf04a0b8 c341fba8 ce343000 c38cdff4 c38cdfc0 c0057884 bf024ad0
dfc0: c341fba8 00000000 ce343000 00000000 c38cdfd0 c38cdfd0 00000000 c341fba8
dfe0: c00577f4 c0040db0 00000000 c38cdff8 c0040db0 c0057804 6ae25d13 c7c756b1
Backtrace:
[<bf031278>] (MessageQ_put+0x0/0x500 [syslink]) from [<bf04e084>] (_TransportShm_notifyFxn+0xb8/0x114 [syslink])
r8:00000100 r7:ce401800 r6:ce418808 r5:0000ffff r4:ce35b000
[<bf04dfcc>] (_TransportShm_notifyFxn+0x0/0x114 [syslink]) from [<bf02ef74>] (Notify_exec+0xc4/0x110 [syslink])
r6:00000000 r5:00000002 r4:ce346000
[<bf02eeb0>] (Notify_exec+0x0/0x110 [syslink]) from [<bf04a18c>] (_NotifyDriverShm_ISR+0xe4/0x144 [syslink])
r6:00000002 r5:00000000 r4:ce33d000
[<bf04a0a8>] (_NotifyDriverShm_ISR+0x0/0x144 [syslink]) from [<bf024b50>] (Thread_callback+0x90/0xec [syslink])
[<bf024ac0>] (Thread_callback+0x0/0xec [syslink]) from [<c0057884>] (kthread+0x90/0x98)
r5:ce343000 r4:c341fba8
[<c00577f4>] (kthread+0x0/0x98) from [<c0040db0>] (do_exit+0x0/0x6c8)
r6:c0040db0 r5:c00577f4 r4:c341fba8
Code: e59f31c4 ebffca8d e5963000 e5933078 (e7936105)
---[ end trace 1395434b69e498f3 ]---
The default value of MessageQ_module->numQueues is 32 and I have max 24 messageQs between ARM and DSP. So this condition should not assert. This is also validated by the fact that when DSP is started and ARM module started in /etc/init.d/rcS it all works.
Tracing back the failure it seems that when _TransportShm_notifyFxn extracts the queueId from the Message it is garbled.
I added a sleep of 1 second between starting DSP and starting my ARM module and the problem persists.
Why is this Assertion and that too not when the DSP and app started from /etc/init.d/rcS file but when started manually from the linux prompt? What undocumented timing considerations here with cache enabled?
-----------------------------------
Suggestion: In a future release of syslink, can you also please print the values used in the Asserts? In this case either of the two values hard coded 32 or the dynamic queueId are messed up. A print of these values in the Assert would help.
I can definitely add printks, but many a times I cannot and adding printk changes timing and the problem disappears.
