• State Resolved
  • Locked Locked
  • Replies 9 replies
  • Subscribers 102 subscribers
  • Views 1676 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DM385: Changing memory map to support 1G RAM

Mikael Nelson
Intellectual 645 points
Part Number: DM385

Hi Everyone,

I'm currently working on a custom design based around a DM385. Our software originated from a development board developed by nSketch, our PSP package is "psp_04_04_00_02" if that helps. 

The DevKit contained 512MB of ram, I believe it's address mapping is like this:

256MB at address 0x80000000 - 0x90000000

256MB at address 0xB0000000 - 0xC0000000

Our custom board is using 1GB total, two 512MB chips. I believe it's address mapping is like this:

512MB at address 0x80000000 - 0xA0000000

512MB at address 0xA0000000 - 0xC0000000

As you can see, it's basically filling in the 512MB gap in the original mapping.

Our kernel boot argument is: 

"console=ttyO0,115200n8 rootwait=1 rw ubi.mtd=7,2048 rootfstype=ubifs root=ubi0:rootfs init=/sbin/init mem=92M vram=8M notifyk.vpssm3_sva=0xBFD00000 ip=off eth=$(ethaddr) cmemk.phys_start=0x85C00000 cmemk.phys_end=0x8AE00000 cmemk.allowOverlap=1 earlyprintk"

mem=92M - I take this to mean 92M@0x80000000

cmemk.phys_start=0x85C00000 cmemk.phys_end=0x8AE00000 cmemk.allowOverlap=1 - CMEM region from 0x85C00000 to 0x8AE00000

The .bld file our project uses (mcfw/src_bios6/cfg/ti810x/config_512M.bld) gives the following memory map that seems to match up:

/* first 256MB */
var LINUX_SIZE = 92*MB;
var CMEM_SIZE = 82*MB;
var SR1_SIZE = 50*MB;
var VIDEO_M3_CODE_SIZE = 512*KB;
var VIDEO_M3_DATA_SIZE = 512*KB;
var DSS_M3_CODE_SIZE = 4*MB;
var DSS_M3_DATA_SIZE = 26*MB;
var DSP_CODE_SIZE = 512*KB;
var DSP_DATA_SIZE = 512*KB;

/* second 256MB */
var TILER_SIZE = 16*MB; /* Reducing this to fix Vid Frame Alloc failures. Need to fix */ /* MUST be aligned on 128MB boundary */
var SR2_FRAME_BUFFER_SIZE = 217*MB; // set over 128M if tiler is enabled
var SR0_SIZE = 16*MB;
var VIDEO_M3_EXCEPTION_CTX_SIZE = 128*KB;
var VPSS_M3_EXCEPTION_CTX_SIZE = 128*KB;
var HDVPSS_DESC_SIZE = 2*MB;
var HDVPSS_SHARED_SIZE = 2*MB - 256*KB;
var NOTIFY_SHARED_SIZE = 2*MB;
var REMOTE_DEBUG_SIZE = 1*MB;

What I would like to do is give the 512MB from 0x90000000 - 0xB0000000 to Linux so my user applications can use it. If I can get away with it, I'd like to leave all other allocations in the memory map untouched to minimize changes to the system.

Here are the experiments I've done so far to try to get an understanding of the situation. It should be noted that I have not yet modified the .bld file to account for the entire 1GB of RAM, so this might be my issue.

1. In uboot, I've ran mtest over the range of 0xA0000000 - 0xC0000000 (skipping first 256MB, didn't want to blow up uboot and I wasn't sure how much memory uboot was using). mtest did not detect any issues after a 5 or 6 iterations with different test data. Not a thorough test, but I figured it was a good start.

2. Just add the following to the bootargs "mem=512M@0x90000000", so for example: 

"console=ttyO0,115200n8 rootwait=1 rw ubi.mtd=7,2048 rootfstype=ubifs root=ubi0:rootfs init=/sbin/init mem=92M mem=512M@0x90000000 vram=8M notifyk.vpssm3_sva=0xBFD00000 ip=off eth=$(ethaddr) cmemk.phys_start=0x85C00000 cmemk.phys_end=0x8AE00000 cmemk.allowOverlap=1 earlyprintk"

The system boots normally to command prompt. The kernel initialization prints indicate it has the additional memory accounted for:

Memory: 92MB 502MB 1MB = 595MB total
Memory: 597996k/597996k available, 20500k reserved, 0K highmem

And /proc/meminfo also indicates the memory is there:

MemTotal: 598200 kB

However, when I start loading the MCFW firmware, I have issues:

root@DM385_CBB:~/video# ./remote_debug_client.out 0xbff00000 &
root@DM385_CBB:~/video#  [c6xdsp ] Remote Debug Shared Memory @ 0xbff00000
 [m3video] Remote Debug Shared Memory @ 0xbff05020
 [m3vpss ] Remote Debug Shared Memory @ 0xbff0a040

root@DM385_CBB:~/video# insmod syslink.ko
SysLink version : 2.21.02.10
SysLink module created on Date:Jul 25 2017 Time:12:58:38
root@DM385_CBB:~/video# ./fw_load.out startup VPSS-M3 carbb_rdk_fw_m3vpss.xem3

 [host] Attached to slave procId 1.
vmap allocation for size 21929984 failed: use vmalloc=<size> to increase size.
Unable to handle kernel NULL pointer dereference at virtual address 00000000
pgd = ee718000
[00000000] *pgd=aea20031, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#1]
last sysfs file:
Modules linked in: syslink bat_irq g_ether ti81xx nop_usb_xceiv cppi41dma musb_hdrc
CPU: 0    Not tainted  (2.6.37_DM385_CBB_0.80.00 #2)
PC is at __memzero+0x24/0x80
LR is at 0x0
pc : [<c0194604>]    lr : [<00000000>]    psr: 20000013
sp : eea57d24  ip : 00000000  fp : 0000000a
r10: 00000005  r9 : f1de8000  r8 : 06a85000
r7 : f1d79000  r6 : 00000000  r5 : 014e8b48  r4 : eea57db8
r3 : 00000000  r2 : 00000000  r1 : 014e8b08  r0 : 00000000
Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: ae718019  DAC: 00000015
Process fw_load.out (pid: 173, stack limit = 0xeea562e8)
Stack: (0xeea57d24 to 0xeea58000)
7d20:          bf04a850 8e500000 003a429a 003a429a 00000001 bf0e4948 00000001
7d40: 00000001 00000000 00000000 8e500000 8e500000 bf058094 014e8b48 00000000
7d60: 00000001 bf0e4948 003a429a 8e500000 00000070 f1dd8000 00000001 f1de8000
7d80: f1e4e0c0 f1deb000 f1de8000 00000005 0000000a bf04e400 00000002 bf04cdcc
7da0: ef3bfb80 eecdf280 00000000 00000000 00000002 f1dd801c f1de8000 f1e5d000
7dc0: f7400000 00000001 003a4480 00000000 00000002 00000020 f1deb000 f1deb000
7de0: 00000000 03046000 f1d79000 f1d73000 00000000 00000000 00000000 bf0e4948
7e00: beeefaf4 bf04a274 f1de5000 00000000 00000000 00000000 eea57e34 f1de8000
7e20: ee402080 f1d73000 00000000 f1de5000 00000000 bf0e4948 00000000 bf045c1c
7e40: 00000000 eea57f00 00000000 00000000 bf0e4948 f1d85000 00000000 00000000
7e60: f1de5000 eea57f00 00000000 bf046b48 00000000 eea57f00 00000000 00000000
7e80: beeefa90 00000000 c020e0a2 f1de5000 00000000 eea56000 00000000 bf048ed8
7ea0: 00000000 eea57f00 000006e0 c01c9a50 ef13c400 c01cb7e4 00000002 0000000a
7ec0: ef19e000 000006ba ef19e000 c01d0da0 a0000013 ee9be380 00000000 c01c9a3c
7ee0: c01c9c08 40361000 f1d85000 beeefda8 00000019 00000000 00000000 00000000
7f00: 00000000 ef19a9f0 eea56000 00000000 eecddc80 00000008 beeefa90 00000008
7f20: 00000000 c00c407c ee918000 ee918000 00000025 c01b2440 ee99fbc0 c01b4bd8
7f40: 0000009d 2d4cae03 00000001 ee9be380 40dd7000 eea57f80 00000000 00000025
7f60: eea56000 00000000 ee9be380 eeb0a380 beeefa90 c020e0a2 00000008 00000000
7f80: eea56000 c00c413c 00000008 00000001 0001f1c4 00000000 beeefa90 00000036
7fa0: c0040604 c0040480 0001f1c4 00000000 00000008 c020e0a2 beeefa90 00000008
7fc0: 0001f1c4 00000000 beeefa90 00000036 00000000 00000000 4011b000 beeefaf4
7fe0: c020e0a2 beeefa78 00013840 402f81cc 20000010 00000008 afefe021 afefe421
Code: e52de004 e1a0c002 e1a0e002 e2511040 (a8a0500c)
---[ end trace d633e997e115e877 ]---

3. I get the same issue if I reduce this new linux memory section to 256MB (0x90000000 - 0xA0000000)

4. The closest I've gotten to getting something to work is to move the CMEM location to 0x90000000 - 0xA0000000 (256MB) and allocate from it. For some reason, the mcfw firmware didn't blow up with this configuration. Unfortunately, due to the structure of our application, we cannot use CMEM allocations.

So my question is, what is the right way of going about increasing the memory available to Linux? 

Thanks!

  • 0
    Expert 1610 points
    First place to start is "IPNC_RDK_MemoryMapAnalysis.pdf" part of collaterals in RDK
  • 0 in reply to Vishwanath Patil
    Intellectual 645 points

    Vishwanath,

    Thanks for the help. I'm back working on the memory map now and I believe I've successfully moved CMEM and made the LINUX memory larger by doing the following modifications:

    Move CMEM to location after DSP data (unused portion of memory) and expanded LINUX section to include old CMEM

    Modified CMEM section of kernel boot arguments to indicate the new location of CMEM

    Modified mem section of kernel boot args to indicate the new size of the LINUX section

    Everything seems to work fine with this.

    Here's our current memory map:

    /*

Memory Map - 1024MB DDR

+--------------+ 
| 174MB        | Linux 
|              |
|              |
+--------------+
| 50MB         | (SR1) Bitstream buffer (Cached) 
|              |
+--------------+
| 512KB        | Video M3 code
+--------------+
| 512KB        | Video M3 data
+--------------+
| 4MB          | VPSS / DSS M3 code
+--------------+
| 26MB         | VPSS / DSS M3 data
+--------------+
| 512KB        | DSP code
+--------------+
| 512KB        | DSP data
+--------------+
| 128MB        | CMEM
+--------------+
| 128MB        | Unused
|              | 
+--------------+
| 16MB         | Tiler buffer 
|              |
+--------------+
| 217MB        | SR(2) Frame buffer 
|              |
|              |
|              | VPSS-VID M3 Frame buffer
|              |
|              |
|              |
|              |
|              |
+--------------+
| 16MB         | (SR0) Syslink MsgQ / IPC List MP (Non-cached)
+--------------+
| 128KB        | VPSS M3 - VPDMA descriptor
+--------------+
| 128KB        | VPSS M3 - FBDev
+--------------+
| 2MB          | HDVPSS Desc
+--------------+
| 2MB - 256KB  | HDVPSS Shared
+--------------+
| 2MB          | Host - VPSS M3 Notify (for FBDev)
+--------------+
| 1MB          | Remote Printf
+--------------+

*/


var LINUX_SIZE				= 174*MB;
var SR1_SIZE				= 50*MB;
var VIDEO_M3_CODE_SIZE		= 512*KB;
var VIDEO_M3_DATA_SIZE		= 512*KB;
var DSS_M3_CODE_SIZE		= 4*MB;
var DSS_M3_DATA_SIZE		= 26*MB;
var DSP_CODE_SIZE			= 512*KB;
var DSP_DATA_SIZE			= 512*KB;
var CMEM_SIZE				= 128*MB;
var TILER_SIZE				= 16*MB;		/* Reducing this to fix Vid Frame Alloc failures. Need to fix */ /* MUST be aligned on 128MB boundary */
var SR2_FRAME_BUFFER_SIZE	= 217*MB;	// set over 128M if tiler is enabled
var SR0_SIZE				= 16*MB;
var VIDEO_M3_EXCEPTION_CTX_SIZE	= 128*KB;
var VPSS_M3_EXCEPTION_CTX_SIZE	= 128*KB;
var HDVPSS_DESC_SIZE		= 2*MB;
var HDVPSS_SHARED_SIZE		= 2*MB - 256*KB;
var NOTIFY_SHARED_SIZE		= 2*MB;
var REMOTE_DEBUG_SIZE		= 1*MB;

    var LINUX_ADDR = DDR3_ADDR_512_REG0_START;
    var SR1_ADDR = LINUX_ADDR + LINUX_SIZE;
    var VIDEO_M3_CODE_ADDR = SR1_ADDR + SR1_SIZE;
    var VIDEO_M3_DATA_ADDR = VIDEO_M3_CODE_ADDR + VIDEO_M3_CODE_SIZE;
    var DSS_M3_CODE_ADDR = VIDEO_M3_DATA_ADDR + VIDEO_M3_DATA_SIZE;
    var DSS_M3_DATA_ADDR = DSS_M3_CODE_ADDR + DSS_M3_CODE_SIZE;
    var DSP_CODE_ADDR = DSS_M3_DATA_ADDR + DSS_M3_DATA_SIZE;
    var DSP_DATA_ADDR = DSP_CODE_ADDR + DSP_CODE_SIZE;

    var CMEM_ADDR = DSP_DATA_ADDR + DSP_DATA_SIZE;

    var TILER_ADDR = DDR3_ADDR_512_REG1_START; 
    var SR2_FRAME_BUFFER_ADDR = TILER_ADDR + TILER_SIZE; 
    var SR0_ADDR = SR2_FRAME_BUFFER_ADDR + SR2_FRAME_BUFFER_SIZE; 
    var VIDEO_M3_EXCEPTION_CTX_ADDR = SR0_ADDR + SR0_SIZE;
    var VPSS_M3_EXCEPTION_CTX_ADDR = VIDEO_M3_EXCEPTION_CTX_ADDR + VIDEO_M3_EXCEPTION_CTX_SIZE;
    var HDVPSS_DESC_ADDR = VPSS_M3_EXCEPTION_CTX_ADDR + VPSS_M3_EXCEPTION_CTX_SIZE;
    var HDVPSS_SHARED_ADDR = HDVPSS_DESC_ADDR + HDVPSS_DESC_SIZE;
    var NOTIFY_SHARED_ADDR = HDVPSS_SHARED_ADDR + HDVPSS_SHARED_SIZE;
    var REMOTE_DEBUG_ADDR = NOTIFY_SHARED_ADDR + NOTIFY_SHARED_SIZE;

    Next, I want to move SR1 Bitstream Buffer (Cached) to be after CMEM (also in an unused area) so that I can give the LINUX partition an extra 60MB.

    This would look like the following change:

    /*

Memory Map - 1024MB DDR

+--------------+ 
| 224MB        | Linux 
|              |
|              |
+--------------+
| 512KB        | Video M3 code
+--------------+
| 512KB        | Video M3 data
+--------------+
| 4MB          | VPSS / DSS M3 code
+--------------+
| 26MB         | VPSS / DSS M3 data
+--------------+
| 512KB        | DSP code
+--------------+
| 512KB        | DSP data
+--------------+
| 128MB        | CMEM
+--------------+
| 50MB         | (SR1) Bitstream buffer (Cached) 
|              |
+--------------+
| 78MB         | Unused
|              | 
+--------------+
| 16MB         | Tiler buffer 
|              |
+--------------+
| 217MB        | SR(2) Frame buffer 
|              |
|              |
|              | VPSS-VID M3 Frame buffer
|              |
|              |
|              |
|              |
|              |
+--------------+
| 16MB         | (SR0) Syslink MsgQ / IPC List MP (Non-cached)
+--------------+
| 128KB        | VPSS M3 - VPDMA descriptor
+--------------+
| 128KB        | VPSS M3 - FBDev
+--------------+
| 2MB          | HDVPSS Desc
+--------------+
| 2MB - 256KB  | HDVPSS Shared
+--------------+
| 2MB          | Host - VPSS M3 Notify (for FBDev)
+--------------+
| 1MB          | Remote Printf
+--------------+

*/

var LINUX_SIZE				= 224*MB;
var VIDEO_M3_CODE_SIZE		= 512*KB;
var VIDEO_M3_DATA_SIZE		= 512*KB;
var DSS_M3_CODE_SIZE		= 4*MB;
var DSS_M3_DATA_SIZE		= 26*MB;
var DSP_CODE_SIZE			= 512*KB;
var DSP_DATA_SIZE			= 512*KB;

var CMEM_SIZE				= 128*MB;
var SR1_SIZE				= 50*MB;

var TILER_SIZE				= 16*MB;		/* Reducing this to fix Vid Frame Alloc failures. Need to fix */ /* MUST be aligned on 128MB boundary */
var SR2_FRAME_BUFFER_SIZE	= 217*MB;	// set over 128M if tiler is enabled
var SR0_SIZE				= 16*MB;
var VIDEO_M3_EXCEPTION_CTX_SIZE	= 128*KB;
var VPSS_M3_EXCEPTION_CTX_SIZE	= 128*KB;
var HDVPSS_DESC_SIZE		= 2*MB;
var HDVPSS_SHARED_SIZE		= 2*MB - 256*KB;
var NOTIFY_SHARED_SIZE		= 2*MB;
var REMOTE_DEBUG_SIZE		= 1*MB;

var LINUX_ADDR				= DDR3_ADDR_512_REG0_START;
var VIDEO_M3_CODE_ADDR		= LINUX_ADDR + LINUX_SIZE;
var VIDEO_M3_DATA_ADDR		= VIDEO_M3_CODE_ADDR + VIDEO_M3_CODE_SIZE;
var DSS_M3_CODE_ADDR		= VIDEO_M3_DATA_ADDR + VIDEO_M3_DATA_SIZE;
var DSS_M3_DATA_ADDR		= DSS_M3_CODE_ADDR + DSS_M3_CODE_SIZE;
var DSP_CODE_ADDR			= DSS_M3_DATA_ADDR + DSS_M3_DATA_SIZE;
var DSP_DATA_ADDR			= DSP_CODE_ADDR + DSP_CODE_SIZE;

var CMEM_ADDR				= DSP_DATA_ADDR + DSP_DATA_SIZE;
var SR1_ADDR				= CMEM_ADDR + CMEM_SIZE;

var TILER_ADDR				= DDR3_ADDR_512_REG1_START;				
var SR2_FRAME_BUFFER_ADDR	= TILER_ADDR  + TILER_SIZE;				
var SR0_ADDR				= SR2_FRAME_BUFFER_ADDR + SR2_FRAME_BUFFER_SIZE;	
var VIDEO_M3_EXCEPTION_CTX_ADDR	= SR0_ADDR + SR0_SIZE;													
var VPSS_M3_EXCEPTION_CTX_ADDR	= VIDEO_M3_EXCEPTION_CTX_ADDR + VIDEO_M3_EXCEPTION_CTX_SIZE;
var HDVPSS_DESC_ADDR		= VPSS_M3_EXCEPTION_CTX_ADDR + VPSS_M3_EXCEPTION_CTX_SIZE;
var HDVPSS_SHARED_ADDR		= HDVPSS_DESC_ADDR + HDVPSS_DESC_SIZE;
var NOTIFY_SHARED_ADDR		= HDVPSS_SHARED_ADDR + HDVPSS_SHARED_SIZE;
var REMOTE_DEBUG_ADDR		= NOTIFY_SHARED_ADDR + NOTIFY_SHARED_SIZE;

    I then performed a full rebuild (make sysall) and re-flashed my kernel and filesystem. When I load the xem3 firmware after the reboot, it doesn't seem to work:

    root@DM385_CBB:~/video# ./fw_load.out startup VPSS-M3 carbb_rdk_fw_m3vpss.xem3

 [host] Attached to slave procId 1.

 [host] Loaded file carbb_rdk_fw_m3vpss.xem3 on slave procId 1.

 [host] Started slave procId 1.

 [host] After Ipc_loadcallback status [0x00000000]

 [host] After Ipc_startcallback status [0x00000000]

    Typically, I see the m3vpss prints like this:

     [host] Attached to slave procId 1.

 [host] Loaded file carbb_rdk_fw_m3vpss.xem3 on slave procId 1.

 [host] Started slave procId 1.

 [host] After Ipc_loadcallback status [0x00000000]
 [m3vpss ] ***** SYSTEM  : Frequency <ORG> - 200000000, <NEW> - 150000000
 [m3vpss ] notify_attach  rtnVal  0
 [m3vpss ] m3vpss process -  initProxyServer  rtnVal  0
 [m3vpss ]
 [m3vpss ]  *** UTILS: CPU KHz = 300000 Khz ***
 [m3vpss ]
 [m3vpss ]  88: SYSTEM  : System Common Init in progress !!!
 [m3vpss ]  88: SYSTEM: IPC init in progress !!!
 [m3vpss ]  88: SYSTEM: Attaching to [HOST] ...
 [m3vpss ]  138: SYSTEM: Attaching to [HOST] ...
 [m3vpss ]  140: SYSTEM: Attaching to [HOST] ... SUCCESS !!!
 [m3vpss ]  140: SYSTEM: Creating MsgQ Heap [IPC_MSGQ_MSG_HEAP] ...
 [m3vpss ]  141: SYSTEM: Creating MsgQ [VPSS-M3_MSGQ] ...
 [m3vpss ]  141: SYSTEM: Creating MsgQ [VPSS-M3_ACK_MSGQ] ...
 [m3vpss ]  144: SYSTEM: Notify register to [HOST] line 0, event 10 ...
 [m3vpss ]  144: SYSTEM: IPC init DONE !!!
...

    And so on, many more prints when the firmware is loaded.

    I assume there must be another define that I need to change somewhere to tell the m3vpss where the new start address is of the S1 buffer, but I'm unable to find this in the code. Do you know where it is at?

    Thanks!

  • 0 in reply to Mikael Nelson
    Intellectual 645 points
    Digging into it a little deeper, it looks like it's coming from the SharedRegion code in the ipc package, but I haven't quite traced it back to where the base address and size are stored. Any ideas?
  • 0 in reply to Mikael Nelson
    Guru 13620 points
    Hi Mikael,

    I believe, it comes from Source/ipnc_rdk/ipnc_mcfw/mcfw/src_bios6/cfg/ti810x/BIOS_m3vpss.cfg

    e.g. in Source/ipnc_rdk/ipnc_mcfw/mcfw/src_bios6/cfg/ti810x/BIOS_m3vpss.cfg

    Program.sectMap[".video:extMemNonCache:notify"] = new Program.SectionSpec();
    Program.sectMap[".video:extMemNonCache:notify"].loadSegment = "HOST_VIDEO_NOTIFYMEM";
    Program.sectMap[".video:extMemNonCache:notify"].type = "NOLOAD";

    In the code,

    static UInt8 hostVidNotifyMem[100*1024];
    #pragma DATA_SECTION(hostVidNotifyMem,".video:extMemNonCache:notify")

    Similarly you can trace for the SR1 memory.
  • 0 in reply to Dwarakesh Radhakrishnan
    Intellectual 645 points
    I think I see in SYSLINK_common.cfg where it's taking the memory map values and setting up a SharedRegion struct. I believe this is Java code, does this get converted to C somewhere? I wonder if that process is not getting run? Any ideas?
  • 0 in reply to Mikael Nelson
    Intellectual 645 points

    I think I discovered my issue with the fw_load. I messed up a size or address somewhere so the REMOTE_DEBUG section changed! Once I corrected that when launching remote_debug_client, I now get the prints. 

  • 0 in reply to Mikael Nelson
    Genius 4030 points

    Hi,

    As you said the issue was with the address of REMOTE_DEBUG section.

    So i hope now you are able to use the unused space for increasing tour linux memory.

  • 0 in reply to Anuj Kumar Gupta
    Intellectual 645 points
    Yep, seems like everything is working.

    For anyone doing this, I really only changed the bootargs that uboot gives the kernel, the remote debug address given to the remote debug application, and the BLD file that specifies the memory map. The key trick was to do a complete system clean and build every time I made a change. It took awhile, but I think files were not getting cleaned properly before and causing me a lot of issues.
  • 0 in reply to Mikael Nelson
    Genius 4030 points

    Hi,

    Glad to know the issue is resolved and thanks for sharing these information.