Trouble loading DSP with DVRRDK 3.50.00.05 with 512MB DM8148

Can you zip and attached the contents of /dvr_rdk/build/dvr_rdk folder. You can delete the firmware files else the size of attachment becomes large.All configuration looks correct and it is not expected for DSP to fail. Have you tried out the exact same configuration with same binaries on TI 814x evm ? Do you have the ability to connect CCS+JTAG to DSP to check the status of the DSP ? Looks like c674 has crashed.

Hello,

Thanks for your response.

I am attaching a tarball of the folder with the firmware files deleted.

8270.dvr_rdk.tar.gz

I can try the binaries with the EVM, but I will need to boot with a different kernel / u-Boot combination.  Our board is booted via PCIe as a client, so the kernel image will crash on the EVM.  I will try to do this Monday and report the results.

I did connect, once, with the emulator on our board.  It looked like the DSP was in the IDLE loop, but the base of the call stack was "00000000" instead of c_int00, so I am not entirely sure what I was looking at.  it may have crashed.  I will repeat and try to get a snapshot of the state of the system.  Is there a recommended way to add something like:

volatile go = 0;

while(!go);

in the application prior to the Ipc_attach() in order to catch it with CCS during startup and walk through the sequence?  I'm not sure if I can do this as the A8 may expect certain timing/responses during the load sequence with syslink / IPC and bail early before the Ipc_attach is attempted.  I don't really know how syslink / IPC works under the covers.

It was also recommended that we try relaxing our DDR timing to see if it was some SDRAM issue.  I did back off the timing a bit using the relaxed timing values in the TI DDR3 spreadsheets, but that only relaxed things by 1 or 2 clocks.  The behavior was the same.  I may back off the timings some more, but this hardware has been running EZSDK compressing 720P60 video for some time now without issue.  (we moved to DVRRDK because loading the DSP on EZSDK was failing in a similar manner).  Unless the booting the DSP somehow pushes the DDR a little harder?

Thanks.

-Mike

I updated the previous post.  The tarball link was somehow lost when posted.  Should be there now.

-Mike

Hi,

Memory configuration has to be changed in :
ipnc_rdk\ipnc_mcfw\mcfw\src_bios6\cfg\ti814x\config_512M.bld.

for example:
from default:
var DDR3_ADDR_256_REG1_START = 0xB0000000;
var DDR3_ADDR_256_REG1_END   =   0xC0000000;

to :
var DDR3_ADDR_256_REG1_START = 0x90000000;
var DDR3_ADDR_256_REG1_END   = 0xA00000000;

Look at your : 8270.dvr_rdk\dvr_rdk\bin\ti814x-evm\dvr_rdk_c6xdsp_debug_512M_128M.xe674.map
for your current  MEMORY CONFIGURATION.

Regards.

Hello Mr. Shink,

Thank you for the response.

I can make that change, but I'm not sure I understand why.  This config_512M.bld is the one that came out-of-the-box with the DVRRDK for 512MB operation.

If I make this change, won't the memory map violate the nice printout of the intended memory map in the config_512M.bld?

E.G., the print statements indicate that the tiler memory should start at 0xB0000000.  If I change it, the tiler memory would get mapped to 0x90000000.

Does this mean I don't need to use the second LISA mapping in u-Boot?  There are two mappings of the 512MB region.  One at 0x80000000 (to 0xA0000000) and one at 0xA0000000 to 0xC0000000.  

The current LISA mappings put the bottom 256 MB at 0x80000000 to 0x90000000 and the upper 256 MB at 0xB0000000 to 0xC0000000.  This is the same as for the EVM in 512 MB mode.  I do not see any overlap in the memory map file with these mappings.  What is overlapping or out of region?

-Mike

Hi Mike,

If you have changed DDR modules number and sizes,  maybe you have to update other values in uboot:
u-boot\include\configs\ti8148_ipnc.h (CONFIG_NR_DRAM_BANKS, PHYS_DRAM_1_SIZE, ... )
u-boot\board\ti\ti8148_ipnc\evm.c (dram_init)

Regards,
Marko.

I verified the memory map, LISA settings and DSP MAR bits settings are fine. Not sure what the issue is.Need to debug further .Have you written incrementing values to entire 512MB read it back and confirmed values are fine ?  It is a good idea to try your suggestion of adding a emulator wait volatile global in main and connecting CCS and single step thru IPC_attach.I don't think there is any timeout value for IPC_attach. I will check the code and confirm but you could try it out anyway.

Hello Mr. Narayanan,

Thanks for reviewing the memory configuration.

When we first brought up these boards months ago, we did a fair amount of memory testing and performed the leveling procedures for the part.  We tested using the TI gel file tests (JTAG peek/poke, EDMA transfers, etc.) as well as using the uBoot mtest program.  I will also compile and run memtester for linux as well.  But I should clarify that we have been running these boards for months using EZSDK stack with the M3 hardware to perform video capture at 720P, scaling + compression and streaming to a PCIe host.  We have not observed any issues with data corruption or crashes commonly associated with DDR issues.  However, when we tried to run the DSP in order to add an audio encode to our stream, we ran into pretty much the same problem we are seeing here with IPC_attach().  We transitioned to the DVRRDK hoping that it would clear up the problem.  I won't dismiss the DDR (we will test it again on Monday), but I think this may be specific to the DSP somehow.

I have three paths to work on:

1) Test the build on the TI EVM.  If it works on the EVM, it would suggest a hardware problem on our custom board, DDR or otherwise.

2) Run memtester on our custom boards and triple check DDR integrity.

3) try to walk through the DSP boot up attach sequence using the emulator.

Thank you again for validating our memory configuration.

-Mike

This does not look like a DDR timing issue. It looks like some issue with DDR chip select or such.I dont think memtest is useful. I want to confirm if the entire 512MB (0x8000_0000 - 0x9000_0000 & 0xB000_0000 to 0xC000_0000) is accessible by writing unique values.You can confirm if it is a DSP specific or address range specific issue by swapping the DSS_M3 code/data and bss with the DSP sections. You can also try swapping  SR0 and SR2. If you change SR0 you will have to modify the DSP MAR bits correctly.

You could use the below function for setting of MAR bits based on section name rather that hardcoded values so that even if you change SR0 address the MAR bits are updated correctly:

function IntType(Value)
{
    return (Value>>>0);
}

function createBitMask(numBits,startBitOffset)
{
    let mask = 0;
    let i;
   
    for (i = 0; i < numBits; i++)
    {
        mask <<= 1;
        mask  |= 1;
    }
    mask <<= startBitOffset;
    return mask;

}

function pad(number, length) {
    var str = '' + number;
    while (str.length < length) str = '0' + str;
    return str.toUpperCase();
}

function dec2hexStr(number) {
    return "0x" + pad(Number(number >>> 0).toString(16),8);
}

function DisableMARBits(CacheDisableRegions,CacheModule)
{
    let MB = 1024*1024;
    let MAR_REG_SIZE = 512 * MB;
    let MAR_REG_BIT_SIZE = 16 * MB;
    let MAR_REG_NAME_MULTIPLE = 32;

    for (var i in CacheDisableRegions)
    {
        let cacheOffMemSection   = Program.cpu.memoryMap[CacheDisableRegions[i]];
        if (typeof cacheOffMemSection == 'undefined')
        {
            throw new Error("Section for cache disable not defined in memory map: " + CacheDisableRegions[i]);
        }
        let marRegNum = IntType(cacheOffMemSection.base / MAR_REG_SIZE);
        let marRegStartBitOffset = IntType((cacheOffMemSection.base - (marRegNum * MAR_REG_SIZE)) / MAR_REG_BIT_SIZE);
        let marRegNumBits        = IntType((cacheOffMemSection.len + (MAR_REG_BIT_SIZE - 1))/ MAR_REG_BIT_SIZE);
        let marRegNumString      = "MAR" + (MAR_REG_NAME_MULTIPLE * marRegNum) + "_" + ((MAR_REG_NAME_MULTIPLE * marRegNum) + (MAR_REG_NAME_MULTIPLE - 1));
        let marRegBitMask        = createBitMask(marRegNumBits, marRegStartBitOffset);
        marRegBitMask            = ~marRegBitMask;
        print ("MAR BITS DISABLE:" + " SECTION_NAME: " + CacheDisableRegions[i] + " REG NAME: " + marRegNumString + " START_BIT_OFFSET: " + marRegStartBitOffset + " NUMBITS " + marRegNumBits + " MAR REG BITMASK " + dec2hexStr(marRegBitMask));
        print ("Current MAR Reg Value: " + dec2hexStr(CacheModule[marRegNumString]));
        CacheModule[marRegNumString] &= marRegBitMask;
        print ("Modified MAR Reg Value: " + dec2hexStr(CacheModule[marRegNumString]));
    }
}

var Cache = xdc.useModule('ti.sysbios.family.c64p.Cache');

/* Disable caching for HWspinlock addresses */
Cache.MAR0_31    = 0x00000000;
Cache.MAR32_63   = 0x00000000;
/* Config/EDMA registers cache disabled */
Cache.MAR64_95   = 0x00000000;
Cache.MAR96_127  = 0x00000000;
/* CPU access code and data  - 0x80000000 cache enable */
Cache.MAR128_159 = 0xFFFFFFFF;
/* TILER memory cache disabled  - 0xA0000000*/
Cache.MAR160_191 = 0xFFFFFFFF;
/* memory cache disabled  - 0xC0000000*/
Cache.MAR192_223 = 0xFFFFFFFF;
/* memory cache disabled  - 0xE0000000*/
Cache.MAR224_255 = 0xFFFFFFFF;

var CacheDisableRegions = ["SR0","REMOTE_DEBUG_MEM", "LINUX_MEM"];
DisableMARBits(CacheDisableRegions, Cache);

Pls refer /dvr_rdk/mcfw/src_bios6/cfg/ti816x/FC_RMAN_IRES_c6xdsp.cfg for example of how to use this function.

... and you can try my mem-configuration with large DSP space (3+125M).

Regards.

An update:

I was able to swap the DSS_M3_CODE/DATA with the DSP_CODE/DATA and run it on our custom hardware.  Same results, the DSP is hung up in IPC_attach().  

The updated memory map is here:

MEMORY CONFIGURATION

name origin length used unused attr fill
---------------------- -------- --------- -------- -------- ---- --------
DSP_L2_RAM 10800000 00020000 00020000 00000000 RWIX
OCMC0_RAM 40300000 00020000 00000000 00020000 RWIX
OCMC1_RAM 40400000 00020000 00000000 00020000 RWIX
LINUX_MEM 80000000 08000000 00000000 08000000 RWIX
SR1 88000000 04f00000 04f00000 00000000 RWIX
VIDEO_M3_CODE_MEM 8cf00000 00300000 00000000 00300000 RWIX
VIDEO_M3_DATA_MEM 8d200000 00100000 00000000 00100000 RWIX
DSP_CODE_MEM 8df00000 00200000 000ac480 00153b80 RWIX
DDR3_DSP 8e100000 00d00000 009d134a 0032ecb6 RWIX
DSS_M3_CODE_MEM 8ee00000 00200000 00000000 00200000 RWIX
DSS_M3_DATA_MEM 8f000000 00200000 00000000 00200000 RWIX
TILER_MEM b0000000 08000000 00000000 08000000 RWIX
SR2_FRAME_BUFFER_MEM b8000000 07000000 00000000 07000000 RWIX
SR0 bf000000 008c0000 008c0000 00000000 RWIX
VIDEO_M3_EXCEPTION_CT bf8c0000 00020000 00000000 00020000 RWIX
VPSS_M3_EXCEPTION_CTX bf8e0000 00020000 00000000 00020000 RWIX
HDVPSS_SHARED_MEM bf900000 00200000 00000000 00200000 RWIX
HDVPSS_DESC_MEM bfb00000 00200000 00000000 00200000 RWIX
HOST_VPSS_NOTIFYMEM bfd00000 00200000 00000000 00200000 RWIX
REMOTE_DEBUG_MEM bff00000 00100000 0000f060 000f0fa0 RWIX

When I tried to build the configuration provided by Mr. Shink I got the following errors:

# Invoking configuro...
making package.mak (because of package.bld) ...
###
### Total DDR usage in MB = 1152 MB ###
###
Generation of Shell script in progress...
js: "/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6/cfg/ti814x/genaddrinfo.xs", line 53: ReferenceError: "TOTAL_MEM_SIZE" is not defined.
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6/cfg/ti814x/config_512M.bld", line 622
"./config.bld", line 3
making package.mak (because of package.bld) ...
###
### Total DDR usage in MB = 1152 MB ###
###
Generation of Shell script in progress...
js: "/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6/cfg/ti814x/genaddrinfo.xs", line 53: ReferenceError: "TOTAL_MEM_SIZE" is not defined.
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6/cfg/ti814x/config_512M.bld", line 622
"./config.bld", line 3
gmake: *** No rule to make target `linker.cmd'. Stop.
js: "/export/space/dvrrdk/DVRRDK_03.50.00.05/ti_tools/xdc/xdctools_3_23_03_53/packages/xdc/tools/Cmdr.xs", line 51: Error: xdc.tools.configuro: configuration failed due to earlier errors (status = 2); 'linker.cmd' deleted.
make[2]: *** [xdc_configuro] Error 1
make[1]: *** [apps] Error 2
make: *** [dvr_rdk_bios6] Error 2

--------------------------------------------

I was able to load up the EVM with the original build and the DSP loads up without issue, there are some errors about SharedRegion_setEntry() for region 2, but they seem to clear once the m3 cores complete initialization.  In any case, the DSP does appear to be loaded and ready to go.

I went ahead an modified the EVM u-Boot to use a single asynchronous bank of memory in the LISA configurations (matching our custom board), and the DSP is still able to load (same messages as before).

So, there seems to be something else going on with our board configuration, either in hardware somewhere (power supplies?  sequencing?) or in our u-Boot or Kernel port.  That's all that is different.

I am going to try to attach the emulator and walk through the DSP initialization and try to assess where the issue may be.

-Mike

Hi Mike,

I can build amd run configuration that i sent you, but  i may have slightly different  rdk version (DM812xIPNC_MT10_3.5.0).
It seems that you have other problem then memory configuration, but anyway,  if you send me or post your "ipnc_rdk\ipnc_mcfw\mcfw\src_bios6\cfg\ti814x" folder,
i could look at. Or you can look in your original genaddrinfo.xs and config_512M.bld  for TOTAL_MEM_SIZE variable, where your build stops. I don't have it in my files.


Regards,
Marko.

Can anyone tell me how to compile the DSP firmware for debugging?  I get symbols when I load the "debug" image, but it looks like the code is still compiled for release as the PC is jumping around performing out-of-order calls while walking through the software.

Also, is there a procedure to be able to rebuild IPC?  If I want to add statements into Ipc.c, it doesn't seem to be getting rebuilt when I do a make -s sys....  am I missing something?

Thanks.

-Mike

Look at ipnc_rdk\ipnc_mcfw\makerules\rules_c674.mk for compiler options.
Maybe you can remove/change some of the optimisation switches, like -O3 ... under (ifeq ($(PROFILE_$(CORE)), debug)
Also look in  : \ti_tools\cgt6x_7_4_2\doc\SPRU187U.pdf  (3.14.3 ...)

Regards.

As Marko said to disable optimization for dsp side change

/dvr_rdk/makerules/rules_c674.mk

#CFLAGS_INTERNAL += --symdebug:dwarf
CFLAGS_INTERNAL += --symdebug:dwarf -O3 -ms0 --opt_for_speed=5 --optimize_with_debug

to

 CFLAGS_INTERNAL += --symdebug:dwarf
#CFLAGS_INTERNAL += --symdebug:dwarf -O3 -ms0 --opt_for_speed=5 --optimize_with_debug

To compile IPC with debug change

/dvr_rdk/mcfw/src_bios6/cfg/ti814x/BIOS_common.cfg

BIOS.libType = BIOS.LibType_Custom;

to

BIOS.libType = BIOS.LibType_Debug;

Can you try reducing the Linux mem size and move SR0 to that address. IPC_attach uses SR0 addresses and if there is issue with memory mapping then behavior may change by changing SR0 address. Make sure you set

/dvr_rdk/mcfw/src_bios6/cfg/ti814x/SYSLINK_common.cfg

SharedRegion.setEntryMeta( 0,
    {
      base:        sr0MemSection.base,
      len:         sr0MemSection.len,
      name:        sr0MemSection.name,
      isValid:     true,
      ownerProcId: srOwnerProcId,
      cacheEnable: true,
      cacheLineSize: 128,
      createHeap:  true
    } 

Without above change VPSS M3/Video M3 load will fail.

Do I also need to relocate the VIDEO_M3_EXCEPTION_CTX and VPSS_M3_EXCEPTION_CTX sections along with the SR0 as well?  Seems like they are bolted together in the memory mapping...

-Mike

No change only SR0.The EXCEPTION_CTX are independent segments and are unrelated to SR0. I think your comment is based on their size calculation.

Having some problems.  The scripts are using the LINUX_MEM (which I reduced from 128 down to 118 M to fit in the SR0) are trying to make logical links, etc., and I get the error:

# /export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bin/ti814x-evm/dvr_rdk_c6xdsp_debug_512M_128M created.
#
# Linking into /export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bin/ti814x-evm/dvr_rdk_c6xdsp_debug_512M_128M src/ 116M.xe674...
#
fatal error #10296: cannot open output file "src/" for writing
make[2]: *** [src/] Error 1
make[1]: *** [apps] Error 2
make: *** [dvr_rdk_bios6] Error 2

I will try to sort it out.

-Mike

Pls copy paste the changes done to /dvr_rdk/mcfw/src_bios6/cfg/ti814x/config_512M.bld. Don't have

var LINUX_SIZE                 = 128*MB;

commented out when you modify the file.

The build scripts grep for this string.

That was the problem.

Here is the diff between the current and original (it also has the shuffle of the DSP CODE/DATA with the DSS CODE/DATA from before):

diff config_512M.bld config_512M.bld.orig
104,105c104
< var LINUX_SIZE = 118*MB;
< var SR0_SIZE = 10*MB;
---
> var LINUX_SIZE = 128*MB;
119c118
< /* var SR0_SIZE = 9*MB - 256*KB; */
---
> var SR0_SIZE = 9*MB - 256*KB;
173,174c172
< var SR0_ADDR = LINUX_ADDR + LINUX_SIZE;
< var SR1_ADDR = SR0_ADDR + SR0_SIZE;
---
> var SR1_ADDR = LINUX_ADDR + LINUX_SIZE;
185c183
< /* var DSS_M3_CODE_ADDR = VIDEO_M3_BSS_ADDR + VIDEO_M3_BSS_SIZE;
---
> var DSS_M3_CODE_ADDR = VIDEO_M3_BSS_ADDR + VIDEO_M3_BSS_SIZE;
196,209d193
< } */
<
< /*** remove this after test */
< var DSP_CODE_ADDR = VIDEO_M3_BSS_ADDR + VIDEO_M3_BSS_SIZE;
< var DSP_DATA_ADDR = DSP_CODE_ADDR + DSP_CODE_SIZE;
< var DSS_M3_CODE_ADDR = DSP_DATA_ADDR + DSP_DATA_SIZE;
< var DSS_M3_DATA_ADDR = DSS_M3_CODE_ADDR + DSS_M3_CODE_SIZE;
< var DSS_M3_BSS_ADDR = DSS_M3_DATA_ADDR + DSS_M3_DATA_SIZE;
< var DSS_M3_BSS_RUN = DSS_M3_BSS_ADDR - DDR3_ADDR + 0x20000000;
< if ((DSS_M3_BSS_RUN + DSS_M3_BSS_SIZE) > DDR3_ADDR_REG0_END)
< {
< throw xdc.$$XDCException("MEMORY_MAP OVERFLOW ERROR ",
< "\nRegion End: " + "0x" + java.lang.Long.toHexString(DDR3_ADDR_REG0_END) +
< "\nActual End: " + "0x" + java.lang.Long.toHexString(DSP_DATA_ADDR + DSP_DATA_SIZE));
211d194
< /*** */
215,217c198,199
< /* var SR0_ADDR = SR2_FRAME_BUFFER_ADDR + SR2_FRAME_BUFFER_SIZE;
< var VIDEO_M3_EXCEPTION_CTX_ADDR = SR0_ADDR + SR0_SIZE; */
< var VIDEO_M3_EXCEPTION_CTX_ADDR = SR2_FRAME_BUFFER_ADDR + SR2_FRAME_BUFFER_SIZE;
---
> var SR0_ADDR = SR2_FRAME_BUFFER_ADDR + SR2_FRAME_BUFFER_SIZE;
> var VIDEO_M3_EXCEPTION_CTX_ADDR = SR0_ADDR + SR0_SIZE;

I got an IPC build issue (or it didn't build with the change you requested, or something):

# Linking into /export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bin/ti814x-evm/dvr_rdk_m3vpss_release_512M_118M.xem3...
#
<Linking>
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/mcfw/src_bios6/cfg/ti814x/link_hdvpss.cmd", line 71: error:
cannot find file "ipc.lib"
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/mcfw/src_bios6/cfg/ti814x/link_hdvpss.cmd", line 71: warning:
no matching section
error: errors encountered during linking;
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bi
n/ti814x-evm/dvr_rdk_m3vpss_release_512M_118M.xem3" not built

>> Compilation failure
make[2]: *** [/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bin/ti814x-evm/dvr_rdk_m3vpss_release_512M_118M.xem3] Error 1
make[1]: *** [apps] Error 2
make: *** [dvr_rdk_bios6] Error 2

I clearly don't know what I am doing.  I will back out the IPC "debug" comments and start over.

-Mike

Comment out below lines:

/dvr_rdk/makerules/rules_m3.mk

ifeq ($(SOC),ti814x)
#  MISC_LNKCMD_INCLUDE = $(dvr_rdk_PATH)/mcfw/src_bios6/cfg/ti814x/link_hdvpss.cmd
 endif 

and

ifeq ($(SOC),ti814x)
 # MISC_LNKCMD_INCLUDE = $(dvr_rdk_PATH)/mcfw/src_bios6/cfg/ti814x/link_codec.cmd
 endif 

If issue is still not resolved pls attach your full build logs (build without make -s options)

The debug build for the M3 looks too big.

Error: (I can send the whole build text if you want, but I think I can try to bump up the space).

<Linking>
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/obj/ti814x-evm/m3vpss/release/dvr_rdk_configuro/linker_mod.cmd", line 236: error:
run placement fails for object "GROUP_1", size 0xd0c308 (page 0). Available
ranges:
DDR3_M3 size: 0x200000 unused: 0x1274b0 max hole: 0x127434
error: errors encountered during linking;
"/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bi
n/ti814x-evm/dvr_rdk_m3vpss_release_512M_118M.xem3" not built
make[2]: *** [/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/../dvr_rdk/build/dvr_rdk/bin/ti814x-evm/dvr_rdk_m3vpss_release_512M_118M.xem3] Error 1
make[1]: *** [apps] Error 2
make: *** [dvr_rdk_bios6] Error 2

>> Compilation failure
make[2]: Leaving directory `/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6/main_app'
make[1]: Leaving directory `/export/space/dvrrdk/DVRRDK_03.50.00.05/dvr_rdk/mcfw/src_bios6'

^C[1]+ Exit 2 make sys_all > /tmp/build.txt

Were you able to resolve the linker error.If not pls share the map file.

Hi,

I had to turn off all the debugging switches and make clean.   I spent about 4 hours trying to make the code areas for the DSP and the video controllers larger to support debugging builds but I kept running into linker files that appeared to be "hard-coded" vs. the auto-generated map files from the config file area.  So... I punted.

Running the release code in the other memory area you suggested lead to the same resutls.   I also played around in u-Boot writing to difference sections of memory and checking potential "dual-mapped" areas and I could not see any overlap or funny behavior.

Then I spent some time adding logging statements in the IPC initialization code in the DSP, and have noted that at least for the EZSDK (need to repeat on the DVRRDK, but since they are using the same version of syslink I expect this may be the same issue) the DSP is hanging in the Task_sleep() calls while doing the handshaking for the SR0 regions with the A8 host.

This reminded me that one of the differences in our kernel vs. the EVM kernel is that we have a driver for a 1588 clock source fed by a 10 Mhz clock and a PPS signal.  The driver uses a DMTIMER in the kernel via the omap_dm_timer_request(), which is supposed to provide a timer for the next available/free resource.  I can provide a link to the driver if interested.  I am wondering if the kernel is giving our driver the same DMTIMER that (I assume) SYS/BIOS is using on the DSP?  Is there any resource negotiation / allocation management for DMTIMERS in the DVRRDK or EZSDK stack?

I intend to disable the driver when I get back in the office and see if there is a change in performance.

-Mike

This is a great catch and 99% certainly the rootcause as it is specific to c674 DSP and to your board which matches the symptoms.The c674 sysbios does use DMTIMER 3 for the kernel 1ms scheduler and using for any other purpose will cause sleep to hang. There is a assert check in sysbios which should have asserted if it finds wrong frequency values during system initialization phase and I am surprised that it did not hit this assert .

Getting debug build functional is DVR RDK is fairly straightforward.

Pls modify the compiler flags for c674 DSP as I mentioned.

TO compile IPC for debug change the config file as I mentioned.

Comment out the internal memory placement in rules_m3.mk as the code size will not fit in OCMC/L2 as I mentioned.

After above three changes if your build fails pls attach the map file.Adjustment between code and BSS sections should be sufficient to be able to resolve any linker section overflow issues.

So what is the proper way to reserve a Timer Resource on the A8 linux kernel?  We need our driver in the final system to maintain time.  Any suggestions?

Well,

I completely disabled loading of our timer module and verified that the linux kernel was only using DMTIMER1 and DMTIMER2 during initialization.  But, this didn't fix the issue.  It still behaves the same way on the DVRRDK load.

I expect that your code chacking DMTIMER3 may not have caught the problem because I didn't change the internal clock frequency but changed the timer clock source to use the external timer pin.  In any case I removed it -- it should be the same as the EVM configures it now.

Still digging.  Is it possible that the IPC attach code could be getting run prior to SYSBIOS initializing or out of the context of a SYSBIOS task?  

When we sort this out, I think this patch should reserve DMTIMER3 going forward, agree?

diff --git a/arch/arm/plat-omap/dmtimer.c b/arch/arm/plat-omap/dmtimer.c
index ddded0e..c11a769 100644
--- a/arch/arm/plat-omap/dmtimer.c
+++ b/arch/arm/plat-omap/dmtimer.c
@@ -274,7 +274,7 @@ static const int omap4_dm_timer_count = ARRAY_SIZE(omap4_dm_timers);
static struct omap_dm_timer ti81xx_dm_timers[] = {
{ .phys_base = 0x4802E000, .irq = TI81XX_IRQ_GPT1 },
{ .phys_base = 0x48040000, .irq = TI81XX_IRQ_GPT2 },
- { .phys_base = 0x48042000, .irq = TI81XX_IRQ_GPT3 },
+ { .phys_base = 0x48042000, .irq = TI81XX_IRQ_GPT3, .reserved = 1 },
{ .phys_base = 0x48044000, .irq = TI81XX_IRQ_GPT4 },
{ .phys_base = 0x48046000, .irq = TI81XX_IRQ_GPT5 },
{ .phys_base = 0x48048000, .irq = TI81XX_IRQ_GPT6 },

-Mike

 Is it possible that the IPC attach code could be getting run prior to SYSBIOS initializing or out of the context of a SYSBIOS task?

  -- This is not possible. IPC_attach runs after all sysbios initialization is complete.

The Linux kernel timer usage is documented at:  http://processors.wiki.ti.com/index.php/TI81XX_PSP_04.04.00.01_Feature_Performance_Guide#Module.2FSubsystem_Usage

There is a wiki page in processor wiki which mentions the changes to allocate additional dmtimer .Pls check if your driver follows similar steps  :http://processors.wiki.ti.com/index.php/TI81xx_PSP_Porting_Guide#Using_Hardware_Timers_from_Kernel_Module.2FDriver

If the control never came out of Task_sleep then we should concentrate on that.Cn you put a breakpoint in the function ti_sysbios_knl_Clock_doTick__I and check if it hits the breakpoint.If breakpoint is hit repeatedly it means atlest timer tick interrupts are being serviced.

OK,

I will go back and try to rebuild debug.  Will I also need to rebuild sysbios now as well?  Will that get generated with the instructions provided that also rebuild IPC?

Thanks for your help.  I thought we had it.

BTW: my driver does follow those practices referenced.  However, it's not clear to me from your links exactly what timers the DSP will be using.  It looks as though SYSBIOS keeps list of free ones (4-8?) to hand out, and they seem to overlap the A8 kernel's list.

-Mike

lliamsonchael said:

Will I also need to rebuild sysbios now as well?

No sysbios will also get rebuilt with previous BIOS cfg file change

lliamsonchael said:

BTW: my driver does follow those practices referenced.  However, it's not clear to me from your links exactly what timers the DSP will be using.  It looks as though SYSBIOS keeps list of free ones (4-8?) to hand out, and they seem to overlap the A8 kernel's list

The timer resources available on 814x is listed in this <ti_tools>\bios_6_33_05_46\packages\ti\sysbios\timers\dmtimer\doc-files\TimerTables.html.You will find an entry for 814x in that.. I will share any additional link that document this if I find it. You are right about timers 4 - 8 being available to sysbios. But apart from timer used for 1ms tick no other timer resources are used on the c674 so you are free to use any other timer like timer 5.

 There is a wiki with info on dmtimer.: http://processors.wiki.ti.com/index.php/SYS/BIOS_dmtimer_configuration_for_TI81xx .Not sure how useful it is though for the present debug.

 Is it possible to have a remote debug session via webex with CCS+JTAG connected to the target and a debug build loaded ? I would need to connect to the machine which has CCS setup and connected to the target and source code of DVR RDK,BIOS and IPC accessible on that machine. If this is possible pls let me know the time and I will setup webex + conf bridge and share the details.