Enabling A8 cache on SR2 for IPNC 3.0 dm8148 EVM

SharedRegion2 is not mapped on  A8 by default.To enable a shared region on A8 , first make sure the Syslink_common.cfg has isEnabled set to true for shared region 2.

Also set cache enabled to true. (Basically match setting of SharedRegion 1). This will map the SharedRegion2 on A8. We had issues with vmalloc running out of space when SR2 is mapped on A8 so pls check if you are able to load and run the app after this change.

To enable caching of buffers on A8, set Syslink_params in firmware_loader to enable caching on A8 for SHaredRegion2. THis is already done for SharedRegion1. (Done before SYslink_setup in firmware_loader main) You can use it as reference.

SharedRegion property does not actually control the caching. It is more of info to IPC whether cache coherency operations need to be performed.The actual caching of buffers happens by different means on different cores:

1. Video & VPSS M3: AMMU page table setting.

2. DSP: MAR bit setting

3. A8: Memory attribute properties of pages when physical memory is mapped by the OS.

That you for the help. Unfortunately, I am still unable to get it working properly. 

In fw_load_main.c I set     SysLink_params  = "SharedRegion.entry[2].cacheEnable=TRUE;" ;

First question, I would like to keep shared region 1 cached, should I use     SysLink_params  = "SharedRegion.entry[1].cacheEnable=TRUE;SharedRegion.entry[2].cacheEnable=TRUE;" ;   ?

Secondly, when I enable isValid in SYSLINK_common.cfg the application halts after loading m3 core. What is the significance of the isValid flag to the M3 core? 

Thirdly, I am wondering if this is not working because I am accessing the shared region improperly. If I use mmap on the physical address will that disable the caching of that physical memory? Looking at the BitsIn link (using shared region 1) I see them using SharedRegion_getPtr to get a virtual address for the buffer, but for frames In there is a comment stating that sharedRegion_getPtr does not work and they use an mmap. Will this cause caching not to work? Do I need to get sharedRegion_getPtr to work?

Thanks,

Ben

First question, I would like to keep shared region 1 cached, should I use     SysLink_params  = "SharedRegion.entry[1].cacheEnable=TRUE;SharedRegion.entry[2].cacheEnable=TRUE;" ;   ?

  -- Yes this is correct.Below is documentation about Syslink_params in <syslink>/packages/ti/syslink/Syslink.h>:

               This string is a list of semi-colon-delimited "assignments" that can be set by users prior to the initial call to SysLink_setup() to affect system  behavior.

Secondly, when I enable isValid in SYSLINK_common.cfg the application halts after loading m3 core. What is the significance of the isValid flag to the M3 core?

  - Are you getting vmalloc failed error ? If so try increasing vmalloc= param in the kernel bootargs. Setting isValid=TRUE causes the physical memory associated with the sharedRegion to be mapped by syslink to kernel space.This happens when the firmware is loaded during which time the info about the sharedregion is extracted from the fimrware executable.

Thirdly, I am wondering if this is not working because I am accessing the shared region improperly. If I use mmap on the physical address will that disable the caching of that physical memory? Looking at the BitsIn link (using shared region 1) I see them using SharedRegion_getPtr to get a virtual address for the buffer, but for frames In there is a comment stating that sharedRegion_getPtr does not work and they use an mmap. Will this cause caching not to work? Do I need to get sharedRegion_getPtr to work?

 - If you are using ipcFramesIn and ipcFramesOut you dont need buffers to be allocated from a sharedRegion. Once you get the physical address of the buffer you can individually mmap it.You can set appropriate mmap properties so that buffers are cached.,

Again, thank you for the response.

Unfortunately I am not getting a vmalloc error. I was already running with vmalloc=1024M because of previous issues with that. After letting it sit for a while it appears that IPC attach finally times out:

[host] Setting L3 bandwidth regulator for [HDVICP0 ] to [press=[0,0] BW=900, WM Cycles=2500]

[host] Setting DMM priority for [ISS ] to [0] ( 0x4e000634 = 0x00800000 )

[host] Setting DMM priority for [HDVICP0 ] to [1] ( 0x4e000634 = 0x00000009 )
[host] Attached to slave procId 2.

[host] Loaded file ./firmware/ipnc_rdk_fw_m3vpss.xem3 on slave procId 2.

[host] Started slave procId 2.

[host] After Ipc_loadcallback status [0x00000000]
[m3vpss ] ***** SYSTEM : Frequency <ORG> - 200000000, <NEW> - 200000000
[m3vpss ] notify_attach rtnVal 0
[m3vpss ] initProxyServer rtnVal 0

*** Platform_startCallback: Ipc_attach timeout

Error [0xffffffff] at Line no: 2847 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslic
*** Ipc_control: Platform_startCallback failed!
Error [0xffffffff] at Line no: 858 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslinc
Assertion at Line no: 308 in /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslink_2_10_06_28/packages/d

[host] Error in Ipc_control Ipc_CONTROLCMD_STARTCALLBACK [0xffffffff]
[c6xdsp ] Remote Debug Shared Memory @ 0xbff00000
[m3video] Remote Debug Shared Memory @ 0xbff05020
[m3vpss ] Remote Debug Shared Memory @ 0xbff0a040
[host] Attached to slave procId 1.

[host] Attached to slave procId 0.

[host] Loaded file ./firmware/ipnc_rdk_fw_m3video.xem3 on slave procId 1.

[host] Started slave procId 1.

[host] After Ipc_loadcallback status [0x00000000]
[m3vpss ] ***** SYSTEM : Frequency <ORG> - 200000000, <NEW> - 200000000

[host] Loaded file ./firmware/ipnc_rdk_fw_c6xdsp.xe674 on slave procId 0.

[host] Started slave procId 0.

[host] After Ipc_loadcallback status [0x00000000]
[c6xdsp ] ***** SYSTEM : Frequency <ORG> - 500000000, <NEW> - 500000000
[c6xdsp ]
[c6xdsp ] *** UTILS: CPU KHz = 500000 Khz ***
[c6xdsp ]
[c6xdsp ] 2: SYSTEM : System Common Init in progress !!!
[c6xdsp ] 2: SYSTEM: IPC init in progress !!!
[c6xdsp ] 2: SYSTEM: Attaching to [HOST] ...
*** Platform_startCallback: Ipc_attach timeout
Error [0xffffffff] at Line no: 2847 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslic
*** Platform_startCallback: Ipc_attach timeout
Error [0xffffffff] at Line no: 2847 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslic
*** Ipc_control: Platform_startCallback failed!
Error [0xffffffff] at Line no: 858 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslinc
*** Ipc_control: Platform_startCallback failed!
Error [0xffffffff] at Line no: 858 in file /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslinc
Assertion at Line no: 308 in /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslink_2_10_06_28/packages/d
Assertion at Line no: 308 in /home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ti_tools/syslink_2_10_06_28/packages/d

[host] Error in Ipc_control Ipc_CONTROLCMD_STARTCALLBACK [0xffffffff]

[host] Error in Ipc_control Ipc_CONTROLCMD_STARTCALLBACK [0xffffffff]

It may also be worth mentioning that I am using a modified 1GB memory map. I made this change a while ago and have been able to use everything fine, so I believe that the memory map change itself was not done wrong. I wouldn't think it would have an impact on something like this unless there was a hardcoded parameter somewhere.

I will continue to investigate the issue, let me know if you have any ideas. 

Thanks,

Ben

Can you confirm when insmod of syslink is done you are setting TRACEFAILURE=1.Also can you set TRACEENTER=1 and TRACECLASS=3. THis will print lots of info but could give some clue to debug further.Also pls share the memory map printed by kernel when it boots up and your modified memory map.

Memory Map:

Boot map:

Originally I was inserting syslink with the following:

insmod ./kermod/syslink.ko TRACE=1 TRACEFAILURE=1 2> /dev/null

I have modified as you have suggester and it is indeed printing a LOT of data. I am trying to redirect it to a file but have been unsuccessful so far. Do you have a recommendation  on how to ouput to file?

Thanks,

-Ben

--

Edit: My boot args are 

console=ttyO0,115200n8 noinitrd mem=139M vmalloc=1024M mem=128M@0xAF000000 rootwait vram=4M notifyk.vpssm3_sva=0xBFD00000 root=/dev/nfs rw nfsroot=10.1.1.155:/home/benmcgee/RDK_IPNC_3_0/PrivateTI_IPNC_RDK_DM812x_DM385_v3.0.0/Source/ipnc_rdk/target/filesys ip=dhcp earlyprintk

Couple of points:

1. Linux memory start address and length should be 4MB aligned.Pls ensure you change linux size to 136MB.

2. From the kernel memory map, lowmem is very less of only 32M.Pls try reducing your vmalloc to 384M or 512 M so that lowmem size is not reduced.

3.Remove redirection of syslink trace to /dev/null and try with just insmod ./kermod/syslink.ko TRACE=1 TRACEFAILURE=1.This may give some indication of where the load fails.

4. Is vram=4M sufficient for your application ? Pls confirm that your frame buffer sizes fits within 4M

Thanks for the corrections. I have changed my boot args appropriately.

Here is something interesting. If I rebuild everything and run I get the above error. If I then change the isValid flag to false and rebuild VPSS, HDVICP, and DSP (not syslink or any other arm  components) The application runs fine. To me this sounds like the ARM is not rebuilding correctly and does not see the shared region properly. Which ARM side components use the memory map, Syslink? Is calling 'make syslink' sufficient to encorporate the new cache configuration into ARM settings? How does the ARM read the shared region sections?

Thanks,

Ben

After doing more digging I noticed that the emulator prints out:

[Cortex_M3_ISS] ti.sdo.ipc.SharedRegion: line 331: assertion failure: A_idTooLarge: id cannot be larger than numEntries
[Cortex_M3_ISS] xdc.runtime.Error.raise: terminating execution

It appears this is referencing line 331 of IPC/packages/ti/sdo/ipc/SharedRegion.c in the function ShareRegion_getSRPtr. What is happening is it tries to get  a gate and the gate is a null pointer. 

I am a little confused about the behaviour of the shared region set up. Look at this code from sharedRegion.c:

/*

* Loop through shared regions. If an owner of a region is specified
* and createHeap has been specified for the SharedRegion, then
* the owner creates a HeapMemMP and the other processors open it.
*/
for (i = 0; i < ti_sdo_ipc_SharedRegion_numEntries; i++) {
  printf("Looking at shared region %u\n",i);
  region = &(SharedRegion_module->regions[i]);
  if ((region->entry.isValid) &&
        (region->entry.ownerProcId == MultiProc_self()) &&
        (region->entry.createHeap) &&
        (region->heap == NULL)) {    <-- We only enter this if statement if the isValid flag is set
  /* get the next free address in each region */
  sharedAddr = (Ptr)((UInt32)region->entry.base +
  region->reservedSize);

  /* Create the HeapMemMP in the region. */
  HeapMemMP_Params_init(&params);
  params.sharedAddr = sharedAddr;
  params.sharedBufSize = region->entry.len - region->reservedSize;

  /* Adjust to account for the size of HeapMemMP_Attrs */
  params.sharedBufSize -= (HeapMemMP_sharedMemReq(&params) - params.sharedBufSize);
  printf("Making heap in region %u\n",i);
  heapHandle = HeapMemMP_create(&params); <-- The assertion eventually happens inside this function when i = 2 

  if (heapHandle == NULL) {
    status = SharedRegion_E_FAIL;
  }
  else {
  /* put heap handle into SharedRegion Module state */
  region->heap = (IHeap_Handle)heapHandle;
  }
 }
}

It would appear that this code section looks at the flags to see if this processor should create a heap in this shared region. Whether it enters this loop depends on the isValid flag.

Now, I do not understand why I would want the M3VPSS core to create an extra heap in SR2 as a result of me wanting to cache the region on the A8. It would seem to me that I would not want the M3Video core to do ANYTHING different. Could you clarify this? Do I need to make the ARM the owner of the shared region in order to cache it?

Thanks,

Ben

So I made the ARM the owner of the shared region and that seemed to have worked fine. Are you aware of any negative side effects this could have?

Thank you for the detailed explanation. You have been most helpful.