Task Stack Overflow (C6678)

Brandy Jabkiewicz

Other Parts Discussed in Thread: SYSBIOS

Something is wrong with the design intent of my program. I can’t seem to make the stacks and heaps play together correctly.

Here is my program layout:

I am running the same executable on 5 cores of a C6678 DSP EVM. This executable initializes the network stack based on the helloWorldUDP example. It initializes the SRIO based on the DIOIsrExample. This is done by Core 0 only. Core 0 initializes both comms then starts the network task. From the network task (NetworkOpen())it starts its own worker task.

Cores 1 to 4 do not do any initialization but just start the worker task.

Right now, in the worker task, Core 0 only will wait for an SRIO doorbell to determine that there is new data available. Depending on what doorbell it is it sets a flag in the shared mem for the other cores. The other cores wait for the flag to be set and then act on the data by manipulating it to be in a certain form with memcpy.

Before I integrated the network example, the program worked fine. However it was running from MCMSRAM. After integration of the network example, changing interrupts and event managers, ensure that the qmss and cppi initializations weren’t duplicated or destroying each, combining the memory maps etc. the program now runs from DDR3.

Now when I run the application, Cores 1 to 4 almost always have a stack overflow error.

[C66xx_3] ti.sysbios.knl.Task: line 330: E_stackOverflow: Task 0x80000408 stack overflow.

[C66xx_3] xdc.runtime.Error.raise: terminating execution

[C66xx_1] ti.sysbios.knl.Task: line 330: E_stackOverflow: Task 0x80000408 stack overflow.

[C66xx_2] ti.sysbios.knl.Task: line 330: E_stackOverflow: Task 0x80000408 stack overflow.

[C66xx_4] ti.sysbios.knl.Task: line 330: E_stackOverflow: Task 0x80000408 stack overflow.

[C66xx_1] xdc.runtime.Error.raise: terminating execution

[C66xx_2] xdc.runtime.Error.raise: terminating execution

[C66xx_4] xdc.runtime.Error.raise: terminating execution

So I increased this parameter in my cfg to be very large.

Task.defaultStackSize = 64*1024;//4096;

I still get an overflow.

Then I use the ROV tool, it tells me that my tasks (which were created dynamically and not statically in the cfg) are using the system heap as their stack and they indeed are using all of this very large default stack size.

The tool also says that the network task is using all of its stack too.

Then I am trying to think what I must be missing. The default system heap is also already quite large. But I made it larger to test.

var HeapMem = xdc.useModule('ti.sysbios.heaps.HeapMem');

var heapMemParams = new HeapMem.Params();

heapMemParams.size = 0x300000+64*1024;

heapMemParams.sectionName = "systemHeap";

Program.global.heap0 = HeapMem.create(heapMemParams);

It does not work either, still there a stack overflow.

I try another idea and create a separate heap for these tasks and attempt to assign it in the taskparams like so:

cfg

Program.sectMap[".myHeapSection"] = "L2SRAM";

var heapMemParams = new HeapMem.Params;

heapMemParams.size = 5*1024;//0x100;

heapMemParams.sectionName = ".myHeapSection";

Program.global.myHeap0 = HeapMem.create(heapMemParams);

application

taskParams.stackHeap = HeapMem_Handle_to_xdc_runtime_IHeap(myHeap0);

Task_Params_init(&taskParams);

Task_create(dioExampleTask, &taskParams, NULL);

Still I fail. In this case, it never gets assigned to this heap and still goes to the system heap.

I feel like I must have a fundamental design flaw here and I can’t figure out what it is.

I would appreciate some advice!

Thanks,

Brandy

Here is my .cfg file:

* Restricted rights to use, duplicate or disclose this code are

* granted through contract.

/* THIS FILE WAS GENERATED BY ti.sysbios.genx */

* ======== loopbackDioIsr.cfg ========

environment['xdc.cfg.check.fatal'] = 'false';

/* Load and use the various BIOS modules. */

var Memory = xdc.useModule('xdc.runtime.Memory');

var BIOS = xdc.useModule('ti.sysbios.BIOS');

var HWI = xdc.useModule('ti.sysbios.family.c64p.Hwi');

var CPINTC = xdc.useModule('ti.sysbios.family.c66.tci66xx.CpIntc');

var ECM = xdc.useModule('ti.sysbios.family.c64p.EventCombiner');

var HeapMem = xdc.useModule('ti.sysbios.heaps.HeapMem');

var HeapBuf = xdc.useModule('ti.sysbios.heaps.HeapBuf');

var SEM = xdc.useModule('ti.sysbios.knl.Semaphore');

var Task = xdc.useModule('ti.sysbios.knl.Task');

var Idle = xdc.useModule('ti.sysbios.knl.Idle');

var Log = xdc.useModule('xdc.runtime.Log');

var Diags = xdc.useModule('xdc.runtime.Diags');

/* Load and use the CSL, CPPI, QMSS and SRIO packages */

var cslSettings = xdc.useModule ('ti.csl.Settings');

var Timestamp = xdc.useModule('xdc.runtime.Timestamp');

var Cppi = xdc.loadPackage('ti.drv.cppi');

var Qmss = xdc.loadPackage('ti.drv.qmss');

var Srio = xdc.loadPackage('ti.drv.srio');

/* Load the PA package */

var Pa = xdc.useModule('ti.drv.pa.Settings');

/* Load the Platform/NDK Transport packages */

var PlatformLib = xdc.loadPackage('ti.platform.evmc6678l');

var NdkTransport = xdc.loadPackage('ti.transport.ndk');

** Use this load to configure NDK 2.2 and above using RTSC. In previous versions of

** the NDK RTSC configuration was not supported and you should comment this out.

var Global = xdc.useModule('ti.ndk.config.Global');

** This allows the heart beat (poll function) to be created but does not generate the stack threads

** Look in the cdoc (help files) to see what CfgAddEntry items can be configured. We tell it NOT

** to create any stack threads (services) as we configure those ourselves in our Main Task

** thread hpdspuaStart.

Global.enableCodeGeneration = false;

/* Load and use the System Package */

var System = xdc.useModule('xdc.runtime.System');

SysStd = xdc.useModule('xdc.runtime.SysStd');

System.SupportProxy = SysStd;

* Enable Event Groups here and registering of ISR for specific GEM INTC is done

* using EventCombiner_dispatchPlug() and Hwi_eventMap() APIs

ECM.eventGroupHwiNum[0] = 7;

ECM.eventGroupHwiNum[1] = 8;

ECM.eventGroupHwiNum[2] = 9;

ECM.eventGroupHwiNum[3] = 10;

/* Load and use the IPC packages */

var Ipc = xdc.useModule('ti.sdo.ipc.Ipc');

var Settings = xdc.module('ti.sdo.ipc.family.Settings');

var ListMP = xdc.useModule('ti.sdo.ipc.ListMP');

var GateMP = xdc.useModule('ti.sdo.ipc.GateMP');

var SharedRegion = xdc.useModule('ti.sdo.ipc.SharedRegion');

var HeapMemMP = xdc.useModule('ti.sdo.ipc.heaps.HeapMemMP');

var MultiProc = xdc.useModule('ti.sdo.utils.MultiProc');

/* Configure the shared memory heap for shared memory allocations required by the

* CPPI and QMSS Libraries */

SharedRegion.translate = false;

var memmap = Program.cpu.memoryMap;

Startup = xdc.useModule('xdc.runtime.Startup');

Startup.firstFxns.$add('&myStartupFxn');

MultiProc.setConfig(null, ["CORE0", "CORE1", "CORE2", "CORE3", "CORE4"]);

/* Synchronize all processors (this will be done in Ipc_start) */

Ipc.procSync = Ipc.ProcSync_ALL;

/* To avoid wasting shared memory for Notify and MessageQ transports */

for (var i = 0; i < MultiProc.numProcessors; i++) {

Ipc.setEntryMeta({

remoteProcId: i,

setupNotify: false,

setupMessageQ: false,

});

}

/* Create a shared memory */

SharedRegion.setEntryMeta(0,

{ base: 0x0C010000,

len: 0x00100000,

ownerProcId: 0,

isValid: true,

name: "sharemem",

});

/* Create a default system heap using ti.bios.HeapMem. */

//var heapMemParams1 = new HeapMem.Params;

//heapMemParams1.size = 32768;

//heapMemParams1.sectionName = "systemHeap";

//Program.global.heap0 = HeapMem.create(heapMemParams1);

/* This is the default memory heap. */

//Memory.defaultHeapInstance = Program.global.heap0;

//Program.sectMap["systemHeap"] = Program.platform.stackMemory;

** Create a Heap.

var HeapMem = xdc.useModule('ti.sysbios.heaps.HeapMem');

var heapMemParams = new HeapMem.Params();

heapMemParams.size = 0x300000+64*1024;

heapMemParams.sectionName = "systemHeap";

Program.global.heap0 = HeapMem.create(heapMemParams);

/* This is the default memory heap. */

Memory.defaultHeapInstance = Program.global.heap0;

Program.sectMap["sharedL2"] = "DDR3";

Program.sectMap["systemHeap"] = "DDR3";

Program.sectMap[".sysmem"] = "DDR3";

Program.sectMap[".args"] = "DDR3";

Program.sectMap[".cio"] = "L2SRAM";

Program.sectMap[".far"] = "DDR3";

Program.sectMap[".rodata"] = "DDR3";

Program.sectMap[".neardata"] = "DDR3";

Program.sectMap[".bss"] = "DDR3";

Program.sectMap[".cppi"] = "DDR3";

Program.sectMap[".init_array"] = "DDR3";

Program.sectMap[".qmss"] = "DDR3";

Program.sectMap[".cinit"] = "DDR3";

Program.sectMap[".const"] = "DDR3";

Program.sectMap[".text"] = "DDR3";

Program.sectMap[".code"] = "DDR3";

Program.sectMap[".switch"] = "DDR3";

Program.sectMap[".data"] = "DDR3";

Program.sectMap[".fardata"] = "DDR3";

Program.sectMap[".args"] = "DDR3";

Program.sectMap[".vecs"] = "DDR3";

Program.sectMap["platform_lib"] = "DDR3";

Program.sectMap[".far:taskStackSection"] = "L2SRAM";

Program.sectMap[".stack"] = "L2SRAM";

Program.sectMap[".nimu_eth_ll2"] = "L2SRAM";

Program.sectMap[".myLocalMemory"] = "L2SRAM";

Program.sectMap[".myHeapSection"] = "L2SRAM";

Program.sectMap[".resmgr_memregion"] = {loadSegment: "L2SRAM", loadAlign:128}; /* QMSS descriptors region */

Program.sectMap[".resmgr_handles"] = {loadSegment: "L2SRAM", loadAlign:16}; /* CPPI/QMSS/PA Handles */

Program.sectMap[".resmgr_pa"] = {loadSegment: "L2SRAM", loadAlign:8}; /* PA Memory */

Program.sectMap[".far:IMAGEDATA"] = {loadSegment: "L2SRAM", loadAlign: 8};

Program.sectMap[".far:NDK_OBJMEM"] = {loadSegment: "L2SRAM", loadAlign: 8};

Program.sectMap[".far:NDK_PACKETMEM"] = {loadSegment: "L2SRAM", loadAlign: 128};

//Program.sectMap[".srioSharedMem"] = new Program.SectionSpec();

Program.sectMap[".srioSharedMem"] = "MSMCSRAM";

//Program.sectMap[".qmss"] = new Program.SectionSpec();

//Program.sectMap[".qmss"] = "MSMCSRAM";

//Program.sectMap[".cppi"] = new Program.SectionSpec();

//Program.sectMap[".cppi"] = "MSMCSRAM";

Program.sectMap[".windowData"] = new Program.SectionSpec();

Program.sectMap[".windowData"].loadAddress = 0x80400000;

//Program.sectMap[".windowData"] = "DDR3";

//var heapMemParams = new HeapMem.Params;

//heapMemParams.size = 5*1024;//0x100;

//heapMemParams.sectionName = ".myHeapSection";

//Program.global.myHeap0 = HeapMem.create(heapMemParams);

* @(#) ti.sysbios.genx; 2, 0, 0, 0,275; 4-29-2009 15:45:06; /db/vtree/library/trees/avala/avala-k25x/src/

Task.defaultStackSize = 64*1024;//4096;

BIOS.heapSection = null;

Task.common$.namedInstance = true;

var task0Params = new Task.Params();

task0Params.instance.name = "task0";

Program.global.task0 = Task.create("&myTaskTest", task0Params);

over 13 years ago

0 Brandy Jabkiewicz over 13 years ago

Mastermind 6325 points

now I have tested making the stack size and the heap size very large. Also, I moved the stack to DDR3 to make these large tests.

var HeapMem =xdc.useModule('ti.sysbios.heaps.HeapMem');

var heapMemParams = new HeapMem.Params();

heapMemParams.size =0x800000+100*1024;

heapMemParams.sectionName = "systemHeap";

Program.global.heap0= HeapMem.create(heapMemParams);

Task.defaultStackSize =100*1024;

I get different behavoirs.

When the worker task is set to loop 5 times, Cores 1 to 4 will overflow when they call Task_exit().

When the worker task is set to loop 15 times, Core 0 will overflow after 7 loops, then Cores 1 to 4 will over flow 20 seconds later (after waiting and not receiving data for that amount of time).

When the heap is too, too big, the program cannot run at all on any core, they all just overflow immediately.

heapMemParams.size =0x1000000+100*1024;

0 Brandy Jabkiewicz over 13 years ago

Mastermind 6325 points

Ok. New update. If I make the stack size small (4096) and the Heap Size big (0x8000000) then the program will run ok until it is time for the task to exit. Then the stack overflow and sometimes even a register dump occurs.

Please advise.

0 judahvang over 13 years ago in reply to Brandy Jabkiewicz

TI__Mastermind 32475 points

Brandy,

Sounds like you are creating your task dynamically right? If you do, then the Task and its stack are allocated from the heap. So having a big stack could cause the Heap allocation to fail. Looks like you are already using ROV to check the stack peak of your tasks right?

Not sure why a task exit would cause a stack overflow, but you should check the stack before the task exit in ROV and then after. Could it be that something is getting corrupted because it doesn't make sense a task exit would cause a stack overflow.

What version of BIOS/IPC are you using?

Judah

0 Brandy Jabkiewicz over 13 years ago in reply to judahvang

Mastermind 6325 points

Hi Judah,

Yes, I am creating the tasks dynamically. I also tried to assign them to a different heap and that did not work either, they still went to the systemHeap.

I am using Bios_6_32_05_54, ipc_1_24_00_16, and ndk_2_20_04_26.

I will try the test to look at ROV right be for the Task_exit and after right now and get back to you. Often if the stack overflow occurs, the ROV data is corrupt and I can't see anything.

Thanks!

Brandy

0 Brandy Jabkiewicz over 13 years ago in reply to Brandy Jabkiewicz

Mastermind 6325 points

It is harder than I thought to get the info you wanted. The breakpoint right on Task_exit shows the information for the ROV is invalid. I re-ran it and then the overflow occured on the first loop through the worker task. I'll try again.

The ROV tool shows 2000 out of 2048 bytes are being used.

0 Brandy Jabkiewicz over 13 years ago in reply to Brandy Jabkiewicz

Mastermind 6325 points

Wow - okay so this is wierd.

I re-ran my program about 15 times and the worker task was always aborting from stack overflow on the first loop. I power cycled many times etc.

I was wondering, what did I change between when I posted and when you asked this question. What I changed is I commented out (1) System_printf in the Core 0 initialization code.

So I put that System_printf back in and then the other cores would run all the way to my breakpoint at the exit. In which case, of course, the ROV tool says the data is invalid. Specifically it says, "Received exception from ROV server: Target memory read failed at address 0x0, length: 8. This read is at an INVALID address according to the applications section map. The application is likely either uninitilaized or corrupt."

Please advise.

Brandy

0 DanRinkes over 13 years ago in reply to Brandy Jabkiewicz

TI__Expert 8055 points

Brandy,

The re-addition of a System_printf might signify some sort of race condition. The output that you show in your outputLog.txt file is standard CIO Printf output. Executing code with these types of printfs is not real-time (i.e each time you do it, the target is halted)

I am coming in to this thread a bit late, so I am still trying to understand your application.

Looking at your .map file, I see .myHeapSection allocated at 0085a7a0, which is L2SRAM, but I also see the following entries in the map file.

.myHeapSection: load addr=8088667c, load size=00000008 bytes, run addr=0085a7a0, run size=00000800 bytes, compression=zero_init

Additionally, looking at your code and the APIs, Memory_alloc requires specification of the heap. If the heap is NULL, the default HeapInstance is used, which I believe is why you are seeing memory allocated from DDR3 rather than L2SRAM. In your code, you are calling it with NULL.

ARGUMENTS

heap— heap from which the memory is allocated

The heap is created by a module that implements the xdc.runtime.IHeap interface. If heap is NULL, the defaultHeapInstance is used.

size— requested memory block size (in MADUs)

align— alignment (in MADUs) of the block of memory

A value of 0 denotes maximum default type alignment.

eb— pointer to error block

Regards,
Dan

0 Brandy Jabkiewicz over 13 years ago in reply to DanRinkes

Mastermind 6325 points

Hi Dan,

What kind of race condition am I experiencing? the main thing I don't understand is how I should design my program to work and share memory appropiately. It seems to make sense that each task would need its own stack but when I try to do that, it still gets allocated on the default heap. Then it looks like each core's stack get corrupted by another core somehow.

Why would my heap be null? it exists in both the config file and the memory map. Please help me understand this. Also, if I don't use the heap but use the .stack parameter with a global variable, it also defaults. Also, I added a line to check if it was NULL and it wasn't.

if(taskParams.stackHeap == NULL)

{

System_printf("no neap pointer\n");

}

Thanks!

Brandy

0 DanRinkes over 13 years ago in reply to Brandy Jabkiewicz

TI__Expert 8055 points

Brandy,

I didn't say your stack point was null. I said that the parameter that specifies which heap to use is null. It's null because that's what's in your code.

See line 1332 of evmExploitation_rx.c.

myPtr = Memory_alloc(NULL, 0x08, 0, &errorBlock);

I don't know the details of the race condition, or even if there definitely is one. I was just making the point that when you use CIO printf functions, the timing of your application will be significantly altered. It will also affect the usage of your system stack.

Regards,

Dan

0 Brandy Jabkiewicz over 13 years ago in reply to DanRinkes

Mastermind 6325 points

Hi Dan,

You said if the task parameter "taskParams.stackHeap" was NULL then it would default to the system heap as the task stack location.

heap— heap from which the memory is allocated

The heap is created by a module that implements the xdc.runtime.IHeap interface. If heap is NULL, the defaultHeapInstance is used.

I was trying to say that this is not NULL but the task stack is still on the system heap according to the ROV tool.

I see your point on the memory alloc but I think it is unrelated to my issue. And I thought placing NULL there would make this come from the default heap assigned to the task (which in theory should have been myHeap0 anyhow).

I am trying to understand where each of my task stacks should be since each core is running the same task and it looks like from the same memory. Is this ok? I suspect not becuase I am getting the stack overflows.

Thanks,

Brandy

0 Brandy Jabkiewicz over 13 years ago in reply to Brandy Jabkiewicz

Mastermind 6325 points

Hi Dan,

So, if I set a break point right at the beginning of the task in each core, I can use the ROV tool to see the stack details.

It looks like each core is running the same .text code - which makes sense, but the stack base is the same for each core - I think this is the problem but I can't seem to fix it.

Core 1's stack should have been in .myHeapSection aka myHeap0. But it is not. It shows the same base as Core 2. Also, why does it show that the stack is already completely used up? I haven't done anything yet in the function.

Here is the screen shot of Core 1:

Here is the screen shot of Core 2:

Hopefully this clears my question up a bit :)

Thanks!

Brandy

0 Brandy Jabkiewicz over 13 years ago in reply to Brandy Jabkiewicz

Mastermind 6325 points

Hello Dan and Judah,

I solved one problem this morning. I can now assign a different heap to each task. However, when the tasks exit there is still a stack overflow error.

The problem was order of execution. I was re-initializing the task parameters after I was setting them. A rookie mistake - I should be ashamed...

Task_Params_init(&taskParams);

if(CORE_NUM ==1)

{

taskParams.stackHeap = HeapMem_Handle_to_xdc_runtime_IHeap(myHeap0);

}

if(CORE_NUM ==2)

{

taskParams.stackHeap = HeapMem_Handle_to_xdc_runtime_IHeap(myHeap1);

}

if(CORE_NUM ==3)

{

taskParams.stackHeap = HeapMem_Handle_to_xdc_runtime_IHeap(myHeap2);

}

if(CORE_NUM ==4)

{

taskParams.stackHeap = HeapMem_Handle_to_xdc_runtime_IHeap(myHeap3);

}

if(taskParams.stackHeap == NULL)

{

System_printf("What the heck?\n");

}

Task_create(dioExampleTask, &taskParams, NULL);

0 judahvang over 13 years ago in reply to Brandy Jabkiewicz

TI__Mastermind 32475 points

Brandy,

I failed to realise this from the beginning of the thread that you were placing both code/data into DDR3.
If you are using a single executable to do this, that isn't going to work.
The reason being, the data that is placed there by CORE0 is going to be overwritten by CORE1, and so on.

For a single image to be loaded on all cores, you need to make sure that any data is placed in Local L2 memory. This assures that each core has its own copy.
Code can be placed into any shared memory. We recommend placing the Interrupt vectors in local memory.

If its not possible to do this then you will need to build separate executables and have DDR3 split up between the different cores.

Judah

0 Brandy Jabkiewicz over 13 years ago in reply to judahvang

Mastermind 6325 points

Hello Judah,

Thanks. Another struggle I had and resolved was the size of the ndk system heap and stack. It was too big to fit into L2. I resolved this by moving the ndk buffers to a section of MSMCSRAM that I created just for the ndk. Then, I had enough space in the L2 for the necessary system heap and other local variables.

Program.sectMap[".far:NDK_OBJMEM"] = {loadSegment: "MSMCSRAM_NDK", loadAlign: 8};

Program.sectMap[".far:NDK_PACKETMEM"] = {loadSegment: "MSMCSRAM_NDK", loadAlign: 128};

In this way I was able to build one exectuable and have all the data in local ram.

Brandy

Processors

Processors forum

Task Stack Overflow (C6678)