C6678 x IdleTask

weber de souza calixto said:

ti_sysbios_knl_Task_Object__table__V:

Your program has jumped into data.  The label that  you disassembled contains the table of Task_Objects.

Note that your highlighted .word at the NRP memory location is the first data in this table that doesn't correspond to a valid assembly instruction.  If your code jumped to any address between ti_sysbios_knl_Task_Object__table__V and 0x80382414 then the CPU would happily execute all those "nonsense" assembly instructions until it hits the .word value, at which point it generates an exception.

In order to determine how you got to this bogus location, you can combine your knowledge of the code that has executed just prior to the exception and the value of the registers in the exception dump.  There are quite a few registers that contain a value between 0x80382340 and 0x80382414 in your exception dump, and if any one of those have been used as the target of a branch instruction then you will end up in the same situation.

Normally, B3 is a good breadcrumb, holding the address to which a bogus function call is expected to return.  However, in your case, it has 0 so it is of no value.  You need to somehow determine what code was executing just before the bogus branch.  Typically, a bogus branch is taken when a function pointer (perhaps a callback) has been corrupted in memory, so in finding out how you branched to this data area, you might need to then find out how the function pointer got overwritten.

You ask if there might be a bug in BIOS idle processing (or just a general bug in BIOS).  While this is certainly possible, it's not likely, as BIOS is used extensively enough that a problem in idle task processing would have been identified by now.  Since you mention DDR3, the cache needs to be considered, since cacheing issues can easily result in corrupted data.

Regards,

- Rob

Hi,

Thanks for the informations. The weird about this error is that by turning off the idle task seems to solve it, and it happens every time I place stack and heap of a thread into DDR3. That is why I've asked if this could be a sysbios error.

You mention that corrupted data can lead to nasty errors. The problem is, I'm running a code that far exceeds the cache memory capabilities (both for .stack and heap) for only one core (~2MB Heap and 1Mb stack for each slave core + 6Mb heap for master core). Therefore, I must place this symbols into DDR3.

In my case, I'm running a proprietary library for OCR. When called, the library tries to allocate lots of data space, which requires lots of stack and heap. What is the 'etiquette' for these cases? How can I check for corrupted data? Is there any document that deeply explains how should I proceed regarding big stack and heap allocation and DDR3 probllems?

Weber

It shouldn't be a problem to have a large heap and stack and large data allocations from those.  I believe the problem is more related to the thing that you have to do to support the large heap/stack - moving to DDR3.  When your heap and stack are in L2 memory then there is no cacheing in L2 cache, and the L1 cacheing is probably OK since it's handled automatically.  When you move to DDR3, the L2 cache comes into play, and the cacheability of DDR3 accesses depends on the MAR register settings of the C6x+.

As an experiment, you could try turning off all MAR bits, so no cacheing occurs for DDR3 accesses.  This will ensure that no stale cache data is used and all memory accesses go straight to DDR3.  This might also drastically slow down performance, which might in itself mask the problem, or move it to a different place, but it's worth a shot IMO.  If the problem goes away when disabling the MAR bits, then that tells you there might be a cacheing issue (where some flushing or invalidating might need to be done, especially if you're sharing DDR3 across cores).

Regards,

- Rob

Hi,

How do I disable the MAR bits on C66 devices?  This wiki refers to C64x http://processors.wiki.ti.com/index.php/Enabling_64x%2B_Cache, but I cant find similar for C66.

I think that someone at texas should take a careful look at this problem. The user Jean-Baptiste Theou refers similar errors when using big heap/stack. As for myself, the error always happens with big heap/stack placed outside the processor's internal memory.

At this post http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/186916/674511.aspx he complains about the same problem.

Robert,

Could you (or someone else from IT) take a careful look at this problem?

Jean reported the same problem as I did, after increasing heap size weird errors happen.

He reported a disassembly from a crash. There we can see that the NPR points to a NOP instruction, but the next instruction is a .word just as my disassembly here.

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/186916/674729.aspx

I'm waiting your response.

Weber

Robert,

I've been working on the other thread mentioned.  It would appear that a buffer (not sure if it's the heap or what) is being overflowed and the code space is being overwritten, thus when the jump back to that code space they get an exception.

Can you suggest a way for them to track if they're overflowing the buffer and which buffer it is? 

Best Regards,

Chad

Weber,

Are you running a single image on all cores or do you have different executables and memory maps for each core?

If you have a single image that is to run on all cores then any data cannot be placed in shared memory.  This includes any heaps.

Data must be placed in local memory or you need to be able to separate the shared memory between each core.

Judah

My project is based on Image processing demo project for EVM6678.

It has two .out images: One goes on master core, the other runs on each one of the 7 slave cores. So there is a single image that runs on 7 cores (slaves), but master core runs a different image.

There are my .map files for master and slave projects:

8420.evmc6678l_memory_maps.zip

Weber

Weber,

I notice that DDR3 is the same for both programs.  I think this is  a problem.  Could you please split of DDR3 so that the two programs don't overlap?
Then see if your program still crashes.

Judah

Hi,

Is there any errata for EVMC6678 regarding DDR?

I'm doing a few DDR tests, and seems that the default rang is incorrect:

MEMORY
{
L2SRAM (RWX) : org = 0x800000, len = 0x40000
MSMCSRAM_MASTER (RWX) : org = 0xc000000, len = 0x100000
MSMCSRAM_SLAVE (RWX) : org = 0xc100000, len = 0x100000
MSMCSRAM_IPC (RWX) : org = 0xc200000, len = 0x200000
DDR3 (RWX) : org = 0x80000000, len = 0x1000000
}

This gives 256Mbyte for DDR3, however the EVM should have 512Mbyte. When try to use 

DDR3 (RWX) : org = 0x80000000, len = 0x2000000 the EVM goes really slow.

Weber

Chad,

I'm working with the demo project for EVM6678: "Multicore Image Processing Demo".  Following Judah suggestion, I splitted DDR3 in two parts, onde for each project:

DDR3 (RWX) : org = 0x80000000, len = 0x10000000 for the slave core project and 

DDR3 (RWX) : org = 0x90000000, len = 0x10000000 for the master core project. This way I can guarantee that each project runs on a isolated memory range.

By "the EVM runs really slow" I meant that the multicore border detection that the demo implements runs  10x slower than wen I run with 

DDR3 (RWX) : org = 0x80000000, len = 0x10000000 for the two projects.

Length 0x10000000 means ~256Mbytes, right? However, the EVM has 512 MByte DDR3. That's why I've asked about erratas regarding DDR3.

I'm also using the default GEL file. In fact, right now most of the project is the default provided with the EVM, so I can test the memory allocation before putting that big library to run.