AM5728: Static allocated message causes program crash

Hello Pavol,

Most exception causes fall into the following three categories.

• Stack overflow or corruption leads to arbitrary code execution.
• A NULL pointer has been dereferenced and written to.
• A peripheral module (like UART, Timer, and so forth) is accessed without being powered.

You can refer to the guide Deciphering CPU Exceptions for more details and debugging techniques. 

Can you please check ROV for errors? (ROV Classic > BIOS > Scan for Errors). I suspect you may be running out of stack. 

You can refer to the video Debugging Common Application Issues with TI-RTOS for guidance on using ROV to debug. 

I hope this helps.

Hi Sahin,

thank you for the answer.
Yes, I see some errors in ROV, but the project I'm working on is the IPC example (ex02_messageq) provided by TI. Thus I assume these errors are not important. Anyway I don't know how to get rid of them. These errors are:

,ti.sdo.ipc.GateMP,Gate Resources,N/A,N/A,
Caught exception in view init code: Problem fetching remoteSystemInUseJavaException: java.lang.Exception: Target memory read failed at address: 0x8e000300, length: 256This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt.

,ti.sdo.ipc.ListMP,Lists,0x8b06a980,index,
Problem retrieving the instance's Attrs struct: JavaException: java.lang.Exception: Target memory read failed at address: 0x8e002c80, length: 16This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt.

,ti.sdo.ipc.ListMP,Lists,0x8b06a9a8,index,
Problem retrieving the instance's Attrs struct: JavaException: java.lang.Exception: Target memory read failed at address: 0x8e002c00, length: 16This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt.

,ti.sdo.ipc.nsremote.NameServerRemoteNotify,Basic,(0x8b06a870),localRequestStatus,
Problem retrieving local request status

,ti.sysbios.heaps.HeapBuf,Detailed,(0x8b06a9d0),N/A,
Caught exception in view init code: "C:/ti/xdctools_3_51_01_18_core/packages/xdc/rov/StructureDecoder.xs", line 518: java.lang.Exception: Target memory read failed at address: 0x8e002d00, length: 8This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt.

,ti.sysbios.heaps.HeapBuf,Detailed,(0x8b06a9d0),freeList,
Error fetching Queue Elem struct: JavaException: java.lang.Exception: Target memory read failed at address: 0x8e002d00, length: 8This read is at an INVALID address according to the application's section map. The application is likely either uninitialized or corrupt.

The example project works well if the messages between ARM and DSP are dynamically allocated no matters about the errors in ROV. When I try to use statically allocated message, the program ends in exception. After the exception I can see in ROV new one error:

,ti.sysbios.family.arm.exc.Exception,Module,N/A,exception,
An exception has occurred!

To see details, I added to Host.cfg file these lines:

var excpt = xdc.useModule('ti.sysbios.family.arm.exc.Exception');
excpt.enableDecode = true;

And here is the content of ROV->Exception:

Decoded exception, core 1,
Decoded,Prefetch Abort
Exception context 1,
$addr,0x8b068028
$type,ti.sysbios.family.arm.exc.Exception.ExcContext
threadType,ti.sysbios.BIOS.ThreadType_Hwi
threadHandle,0x0
threadStack,0x8b073110
threadStackSize,57344
type,ti.sysbios.family.arm.exc.Exception.Type_PreAbort
r0,0x0
r1,0x1
r2,0x8b03dbe0
r3,0x2
r4,0x0
r5,0x8b03bc1c
r6,0x8b03649a
r7,0x8b03b004
r8,0x2
r9,0x0
r10,0x0
r11,0x8
r12,0x600001df
sp,0x8b05a0a0
lr,0x8b030a48
pc,0x8b034158
psr,0x600001df
dfsr,0x0
ifsr,0x222
dfar,0x0
ifar,0x0
Exception call stack,
0 _exit(int) at syscalls.c:30,PC = 0x8B034158 FP = 0x8B05A0A0
1 ti_sysbios_family_arm_a15_smp_Core_getRevisionNumber__E() at Core.c:488,PC = 0x8B030A48 FP = 0x8B05A0A0

I don't see anything that could help me understand where the issue is. The code is from IPC example from TI, the only one thing I changed is the type of a message. So I don't know what I should repair to achieve the good working project. I assume the execption originates somewhere inside of IPC, so probably I didn't use the static message by the correct way? Is somewhere any static message example available?
Of course I tried to increase stack size from 0x1000 to 0xE000, but that didn't help.

Any ideas? what else I could try?

Hello,

Can you please share what is shown in ROV under the IPC and LoggerBuf modules after the crash occurs?

Also, please make sure your application is not calling messageQ_free() since messages are not being allocated using messageQ_alloc(), otherwise an assert will occur. This is explained in the IPC User's Guide:

 http://software-dl.ti.com/processor-sdk-rtos/esd/docs/latest/rtos/index_Foundational_Components.html#messageq-allocation-without-a-heap

Hello Sahin,
my program doesn't contain messageQ_free(), but might be called by IPC libraries during transport process, I don't know.

Meanwhile I migrated to the most recent software, so currently I have:
AM5729 on BeagleBoard-X15
bios_6_75_02_00 running on both cores ARM Cortex-A15 core 0 and DSP C66x core 0
processor_sdk_rtos_am57xx_6_00_00_07
pdk_am57xx_1_0_15
ipc_3_50_03_05
CCS 9.1.00010 on Win10

ROV->Ipc contains:
,0,true,true,true
,1,false,true,true
,2,false,true,true
,3,false,true,true

ROV->LoggerBuf contains:
ti.sysbios.smp.LoggerBuf
AppLog
,1,19,0,xdc.runtime.Main,main: -->,0,xdc.runtime.Log_print,-1962716788,0,0,0,0,0,0,0
,2,916,0,xdc.runtime.Main,App_taskFxn: -->,0,xdc.runtime.Log_print,-1962716912,0,0,0,0,0,0,0
,3,6958,0,xdc.runtime.Main,App_taskFxn: ipc ready,0,xdc.runtime.Log_print,-1962716840,0,0,0,0,0,0,0
,4,917762,0,xdc.runtime.Main,App_create: -->,0,xdc.runtime.Log_print,-1962717320,0,0,0,0,0,0,0
,5,192996332,0,xdc.runtime.Main,App_create: Host is ready,0,xdc.runtime.Log_print,-1962717160,0,0,0,0,0,0,0
,6,192996356,0,xdc.runtime.Main,App_create: <-- status=0,0,xdc.runtime.Log_print,-1962717132,0,0,0,0,0,0,0
,7,193823730,0,xdc.runtime.Main,App_exec: -->,0,xdc.runtime.Log_print,-1962717060,0,0,0,0,0,0,0
,8,194051823,0,xdc.runtime.Main,App_exec: sending message 1,0,xdc.runtime.Log_print,-1962717044,1,0,0,0,0,0,0
,9,198680972,0,ti.sdo.ipc.transports.TransportShm,ERROR: "(unknown file)", line 391: assertion failure: A_regionInvalid: Region is invalid,10473,xdc.runtime.Log.L_error,0,391,-1962703156,-1962696024,-1962706992,0,0,0

The numbers from the LoggerBuf row #9 looks like from DDR3
-1962703156 = 0x8B0386CC
-1962696024 = 0x8B03A2A8
-1962706992 = 0x8B0377D0

Here is my MAP file:
.rodata.xdc_runtime_Text_charTab__A
0x8b036984 0x3531 C:\Users\censored\workspace_v9\ex02_messageq_mpu\Debug\configPkg\package\cfg\Host_pa15fg.oa15fg
0x8b036984 xdc_runtime_Text_charTab__A
*fill* 0x8b039eb5 0x3

.rodata.xdc_runtime_Assert_raise__I.str1.4
0x8b03a2a8 0x3 C:\ti\bios_6_75_02_00\packages\gnu\targets\arm\rtsv7A\lib\gnu.targets.arm.rtsv7A.aa15fg(Assert.oa15fg)
0x5 (size before relaxing)
*fill* 0x8b03a2ab 0x1

I have searched through e2e for "A_regionInvalid" and found this thread:
e2e.ti.com/.../890502
If I have understood the thread correctly, then static messages are unsupported on my Sitara device, and are supported by copy transports (thus Concerto devices) only? Could you please confirm this finding?

Pavol Beliancin said:

,9,198680972,0,ti.sdo.ipc.transports.TransportShm,ERROR: "(unknown file)", line 391: assertion failure: A_regionInvalid: Region is invalid,10473,xdc.runtime.Log.L_error,0,391,-1962703156,-1962696024,-1962706992,0,0,0

Line 391 of TransportShm.c refers to this line of code:

    /* This transport only deals with messages allocated from SR's */
    Assert_isTrue(id != SharedRegion_INVALIDREGIONID,
            ti_sdo_ipc_SharedRegion_A_regionInvalid);

Given this, there are a couple of things to try:

1. Try placing your buffer in SR0.

2. Try using the TransportShmNotify transport:

Add this to your .cfg file:

var MessageQ = xdc.module('ti.sdo.ipc.MessageQ');
MessageQ.SetupTransportProxy =
      xdc.module('ti.sdo.ipc.transports.TransportShmNotifySetup');

Please try these and let me know how it goes.

Regards,
Sahin

Hi Sahin,

2. using of TransportShmNotify didn't help, the program still ends in error/exception

1. As I read here:
e2e.ti.com/.../840300
Placing of user variables into SR is not allowed.

But at least, I decided to simulate it. So I created a pointer and set it to the address of the middle of the SR_0 memory. The example project has SR_0 defined as follows:

var SR_0 = {
name: "SR_0", space: "data", access: "RW",
base: 0x8E000000, len: 0x1000000,
comment: "SR#0 Memory (16 MB)"
};

So the pointer looks like:
App_Msg *buffer = (App_Msg *)0x8E800000;

When I put this pointer to MessageQ_put() function, the transfer of the message is working fine and the other side succesfully received the message.

But as I has mentioned above, placing into SharedRegion is not allowed for user variables. Or do you know way how to place a static allocated buffer into SharedRegion? If yes please let me know.
For the messages allocated outside of SharedRegion, there isn't support inside of TransportShm implementation, so probably static messages are really not supported, are they?

Hi Pavol,

I'm checking with the IPC developer. I will get back to you soon.

Regards,
Sahin

Hi Pavol,

Just wanted to check in and let you know we're still looking at this.

Generally speaking, we believe you should be able to allocate static messages using a second SharedRegion and setting createHeap = false in the SR config, but we're setting up a test case to confirm. I'm sorry for the delay.

Regards,
Sahin

Hi Pavol,

Quick update on this:

The method above proved unsuccessful, but the IPC developer has a couple ideas to try first before concluding this. We should have an update for you early next week.

I appreciate your patience. 

Hi Sahin,

thank you for the working example of static messages, I tested it on my EVMAM5728 platform and it works.

I did some minor changes to making it compileable under CCS with EVMAM5728, maybe this will help somebody in the future:
- commented out the definition of 'RB' variable and replaced it by 'false', so it looks like:
//var RB = (cfgArgs.profile == "release" ? true : false);
var RB = false;
- removed unused cores from the 'procNameAry' array in 'ipc.cfg.xs', so it looks like:
var procNameAry = ["DSP1", "HOST"];
- removed memory configuration of unused cores from the file 'config.bld'
- replaced the 'evmDRA7XX' platform with 'evmAM572X' in 'config.bld'