This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS/PROCESSOR-SDK-AM57X: IPU IPC/GateMP configuration

Part Number: PROCESSOR-SDK-AM57X

Tool/software: TI-RTOS

Hi all, 

In our application, we want to use a shared memory between the 3 followings cores : A15 (Linux), IPU (SYS/BIOS) and DSP (SYS/BIOS). I created a CMEM shared memory according to the "AM57x Processor SDK Linux®: Customizing Multicore" (sprac60). Linux allocs this CMEM memory and sends the physical address to the others cores (IPU and DSP), through a messageQ. Each core can use this shared memory. That works well.

Now, we want to use the GateMP mechanism to protect read/write to this shared memory. I found the "gatempapp" example in the IPC/packages directory. I implemented it. I created a shared memory (SR_0), that is owned by the DSP. The GateMP mechanism works between Linux (create/open) and the DSP (create/open), but not in the IPU core.

DSP :
*.cfg :

var BIOS = xdc.useModule('ti.sysbios.BIOS');
BIOS.addUserStartupFunction('&IpcMgr_callIpcStart');

var GateMP = xdc.useModule('ti.sdo.ipc.GateMP');
GateMP.hostSupport = true;

var SharedRegion = xdc.useModule('ti.sdo.ipc.SharedRegion');

var SR0Mem = Program.cpu.memoryMap["SR_0"];

SharedRegion.setEntryMeta(0,
    new SharedRegion.Entry({
        name:           "SR0",
        base:           SR0Mem.base,
        len:            SR0Mem.len,
      	createHeap: 	true,
        ownerProcId:    MultiProc.getIdMeta("DSP1"),
        cacheEnable:    true,
        isValid:        true
    })
);

var Resource = xdc.useModule('ti.ipc.remoteproc.Resource');
Resource.customTable = true;

*.c :

GateMP_Params_init(&gateParams);
gateParams.name             = App_GATE_DSP_NAME;
gateParams.localProtect     = GateMP_LocalProtect_PROCESS;
gateParams.remoteProtect    = GateMP_RemoteProtect_SYSTEM;
gateParams.regionId         = 0;
Module.slaveGateMPHandle = GateMP_create(&gateParams);

IPU :
*.cfg :

var BIOS = xdc.useModule('ti.sysbios.BIOS');
BIOS.addUserStartupFunction('&IpcMgr_callIpcStart');

var GateMP = xdc.useModule('ti.sdo.ipc.GateMP');
GateMP.hostSupport = true;

var SharedRegion = xdc.useModule('ti.sdo.ipc.SharedRegion');

var SR0Mem = Program.cpu.memoryMap["SR_0"];

SharedRegion.setEntryMeta(0,
    new SharedRegion.Entry({
        name:           "SR0",
        base:           SR0Mem.base,
        len:            SR0Mem.len,
      	createHeap: 	true,
        ownerProcId:    MultiProc.getIdMeta("DSP1"),
        cacheEnable:    true,
        isValid:        true
    })
);

var Resource = xdc.useModule('ti.ipc.remoteproc.Resource');
Resource.customTable = true;

*.c

GateMP_Params_init(&gateParams);
gateParams.name             = App_GATE_IPU_NAME;
gateParams.localProtect     = GateMP_LocalProtect_PROCESS;
gateParams.remoteProtect    = GateMP_RemoteProtect_SYSTEM;
gateParams.regionId         = 0;
Module.slaveGateMPHandle = GateMP_create(&gateParams);

IpuAmmu.cfg : same as the ex02_messageQ example

config.bld :

var evmDRA7XX_ExtMemMapDsp = {
    ...
    SR_0: {
        name: evmDRA7XX_SR_0.name,
        base: evmDRA7XX_SR_0.base,
        len:  evmDRA7XX_SR_0.len,
        space: "data",
        access: "RW"
    }
};
Build.platformTable["ti.platforms.evmDRA7XX:dsp1"] = {
    externalMemoryMap: [
        [ "EXT_CODE", evmDRA7XX_ExtMemMapDsp.EXT_CODE ],
        [ "EXT_DATA", evmDRA7XX_ExtMemMapDsp.EXT_DATA ],
        [ "EXT_HEAP", evmDRA7XX_ExtMemMapDsp.EXT_HEAP ],
        [ "TRACE_BUF", evmDRA7XX_ExtMemMapDsp.TRACE_BUF ],
        [ "EXC_DATA", evmDRA7XX_ExtMemMapDsp.EXC_DATA ],
        [ "PM_DATA", evmDRA7XX_ExtMemMapDsp.PM_DATA ],
        [ evmDRA7XX_SR_0.name, evmDRA7XX_ExtMemMapDsp.SR_0 ]
    ],
    codeMemory: "EXT_CODE",
    dataMemory: "EXT_DATA",
    stackMemory: "EXT_DATA",
};

var evmDRA7XX_ExtMemMapIpu1 = {
    ...
    SR_0: {
        name: evmDRA7XX_SR_0.name,
        base: evmDRA7XX_SR_0.base,
        len:  evmDRA7XX_SR_0.len,
        space: "data",
        access: "RW"
    }
};

Build.platformTable["ti.platforms.evmDRA7XX:ipu1"] = {
    externalMemoryMap: [
        [ "EXT_CODE", evmDRA7XX_ExtMemMapIpu1.EXT_CODE ],
        [ "EXT_DATA", evmDRA7XX_ExtMemMapIpu1.EXT_DATA ],
        [ "EXT_HEAP", evmDRA7XX_ExtMemMapIpu1.EXT_HEAP ],
        [ "TRACE_BUF", evmDRA7XX_ExtMemMapIpu1.TRACE_BUF ],
        [ "EXC_DATA", evmDRA7XX_ExtMemMapIpu1.EXC_DATA ],
        [ "PM_DATA", evmDRA7XX_ExtMemMapIpu1.PM_DATA ],        
        [ evmDRA7XX_SR_0.name, evmDRA7XX_ExtMemMapIpu1.SR_0 ]
    ],
    codeMemory: "EXT_CODE",
    dataMemory: "EXT_DATA",
    stackMemory: "EXT_DATA",
};

rsc_table_ipu.h :

#define IPU_SR0_VIRT            0xBFB00000
#define IPU_SR0                 0xBFB00000
#define IPU_SR0_SIZE            (SZ_1M * 1)
struct my_resource_table ti_ipc_remoteproc_ResourceTable = {
    ...
    {
        TYPE_DEVMEM,
        IPU_SR0_VIRT, IPU_SR0,
        IPU_SR0_SIZE, 0, 0, "IPU_SR0",
    },
};

rsc_table_dsp.h :

#define DSP_SR0_VIRT            0xBFB00000
#define DSP_SR0                 0xBFB00000
#define DSP_SR0_SIZE            (SZ_1M * 1)
struct my_resource_table ti_ipc_remoteproc_ResourceTable = {
    ...
    {
        TYPE_DEVMEM,
        DSP_SR0_VIRT, DSP_SR0,
        DSP_SR0_SIZE, 0, 0, "DSP_SR0",
    },
};

Configuration :
BIOS: 6.46.01.38
IPC: 3.44.0.0
XDC tools: 3.32.01.22

In the IPU, the GateMP_create() function returns the following error : "ti.sdo.ipc.GateMP: line 1160: assertion failure: A_noHeap: Region has no heap" (What does "createHeap: true" mean ?)

If a create an HeapBufMP (DSP side), and i try to open it (IPU side), the the HeapBufMP_open() function returns : HeapBufMP_E_NOTFOUND.

Do you have any idea why i can not create a GateMP on the IPU core ? And why i try to open an heap, it doesn't find it ?


Thank you for your help,

Olivier

  • The RTOS team have been notified. They will respond here.
  • Hi Biser,

    We are also facing similar issue on AM57X, and interested in answer.

    I have a related question:

    The physical address specified (DSP_SR0) in rsc_table_dsp.h falls in DDR-SDRAM address space. Does it have to be specified in Linux DTS, similar to CMA carveout and CMEM block; so that Linux do not touch this memory region ?

    Thanks,
    Paritosh

  • Olivier,

    createHeap is defined in packages/ti/ipc/SharedRegion.h:

       Bool createHeap;

       /*!< @brief Whether a heap is created for the region

        *

        *  If 'TRUE', a HeapMemMP instance is created with the size

        *  spanning the length of the shared region minus any memory

        *  that is reserved in the region.  If 'FALSE', no heap

        *  is created in the shared region.

        */

    As you define DSP1 as the shared region owner, the error you observed should be caused by both the DSP and IPU are trying to create the HeapMP in the shared region. If the intention is the owner of the shared region (DSP1) creates the Heap in shared region and IPU uses the heapMP, probably setting the following in the IPU configuration can help.

    IPU:cfg

            createHeap:    false,

    Regards,
    Garrett

  • Hi Garret,

    Quick queries:
    Is SharedRegion configuration mandatory for GateMP ?

    MessageQ works fine without SharedRegion; We use HeapID as 0 when allocating MessageQ messages; Is that heap different ?
  • Hi Garrett,

    Thank you for your reply.

    As you recommend, in the IPU:cfg, I set 'createHeap' parameter to false, but I have the same problem : "A_noHeap: Region has no heap" error for the IPU. GateMP_create() function works for the DSP.

    I try to set the IPU1 as the shared region owner :  

    IPU1:*.cfg :

    SharedRegion.setEntryMeta(0,
        new SharedRegion.Entry({
            name:           "SR0",
            base:           SR0Mem.base,
            len:            SR0Mem.len,
          	createHeap: 	true,
            ownerProcId:    ipc_cfg.MultiProc.getIdMeta("IPU1"),
            cacheEnable:    true,
            isValid:        true
        })
    );

    DSP1:*.cfg:

    SharedRegion.setEntryMeta(0,
        new SharedRegion.Entry({
            name:           "SR0",
            base:           SR0Mem.base,
            len:            SR0Mem.len,
          	createHeap: 	false,
            ownerProcId:    ipc_cfg.MultiProc.getIdMeta("IPU1"),
            cacheEnable:    true,
            isValid:        true
        })
    );
    

    But the IPU1 firmware loading crash ...
    Can it be a problem of MMU ?

    As Paritosh asks : "Does [the SR0 region] have to be specified in Linux DTS, similar to CMA carveout and CMEM block; so that Linux do not touch this memory region ?"

    Regards,
    Olivier

  • Paritosh,

    Yes, the SR0 region has to be specified in Linux DTS, similar to CMA carveout and CMEM block; so that Linux do not touch this memory region.

    &reserved_mem {

                   dsp1_sr0: dsp1_sr0@bfb00000 {

                    reg = <0x0 0xbfb00000 0x0 0x100000>;

                    no-map;

                    status = "okay";

            };

    };

    SharedRegion is supported in BIOS only, and mandatory for GateMP.

    Olivier,

    What kind of crash you observed? Typically there is iommu error in kernel boot log if it's caused MMU configuration, that you need to take care by using customized resource table.

    Regards,
    Garrett

     

  • Hi Garret,

    Does this imply that GateMP can not be used for synchronization of shared memory (like CMEM) across C66 (SYS/BIOS) and A15 (Linux) ?

    Thanks,
    Paritosh

  • Paritosh,

    GateMP is supported across C66 (SYS/BIOS) and A15 (Linux), but just not through the specific SharedRegion module.

    Regards,
    Garrett
  • Thanks Garret.

    What is alternate to SharedRegion for GateMP in this case ?

    Can you please point us to the user guide to configure GateMP for synchronization across C66 (SYS/BIOS) and A15 (Linux) ?

    Thanks,
    paritosh
  • Hi,

    For now and to not modifie the dst file, i use the CMEM space as the SR0 region, so that Linux do not touch this memory. The Linux application doesn't use this CMEM/SR0 region. So, now, SR0 starts at the address : 0xA0000000. Here, my new configuration :

    IpuAmmu.cfg : same as the ex02_messageQ example :

    /*********************** Large Pages *************************/
    /* Instruction Code: Large page  (512M); cacheable */
    /* config large page[0] to map 512MB VA 0x0 to L3 0x0 */
    AMMU.largePages[0].pageEnabled = AMMU.Enable_YES;
    AMMU.largePages[0].logicalAddress = 0x0;
    AMMU.largePages[0].translationEnabled = AMMU.Enable_NO;
    AMMU.largePages[0].size = AMMU.Large_512M;
    AMMU.largePages[0].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
    AMMU.largePages[0].L1_posted = AMMU.PostedPolicy_POSTED;
    
    /* Peripheral regions: Large Page (512M); non-cacheable */
    /* config large page[1] to map 512MB VA 0x60000000 to L3 0x60000000 */
    AMMU.largePages[1].pageEnabled = AMMU.Enable_YES;
    AMMU.largePages[1].logicalAddress = 0x60000000;
    AMMU.largePages[1].translationEnabled = AMMU.Enable_NO;
    AMMU.largePages[1].size = AMMU.Large_512M;
    AMMU.largePages[1].L1_cacheable = AMMU.CachePolicy_NON_CACHEABLE;
    AMMU.largePages[1].L1_posted = AMMU.PostedPolicy_POSTED;
    
    /* Private, Shared and IPC Data regions: Large page (512M); cacheable */
    /* config large page[2] to map 512MB VA 0x80000000 to L3 0x80000000 */
    AMMU.largePages[2].pageEnabled = AMMU.Enable_YES;
    AMMU.largePages[2].logicalAddress = 0x80000000;
    AMMU.largePages[2].translationEnabled = AMMU.Enable_NO;
    AMMU.largePages[2].size = AMMU.Large_512M;
    AMMU.largePages[2].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
    AMMU.largePages[2].L1_posted = AMMU.PostedPolicy_POSTED;
    
    /* TILER & DMM regions: Large page (512M); cacheable */
    /* config large page[3] to map 512MB VA 0xA0000000 to L3 0xA0000000 */
    AMMU.largePages[3].pageEnabled = AMMU.Enable_YES;
    AMMU.largePages[3].logicalAddress = 0xA0000000;
    AMMU.largePages[3].translationEnabled = AMMU.Enable_NO;
    AMMU.largePages[3].size = AMMU.Large_512M;
    AMMU.largePages[3].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
    AMMU.largePages[3].L1_posted = AMMU.PostedPolicy_POSTED;

    config.bld : 

    /* Shared region definition used in GateMP app */
    var evmDRA7XX_SR_0 = {
            name: "SR_0", space: "data", access: "RW",
            base: 0x88000000, len: 0x01000000,
            comment: "SR#0 Memory"
    };
    
    var evmDRA7XX_ExtMemMapIpu1 = {
        EXT_CODE: {
            name: "EXT_CODE",
            base: 0x00004000,
            len:  0x000FC000,
            space: "code",
            access: "RWX"
        },
        EXT_DATA: {
            name: "EXT_DATA",
            base: 0x80000000,
            len:  0x00200000,
            space: "data",
            access: "RW"
        },
        EXT_HEAP: {
            name: "EXT_HEAP",
            base: 0x80200000,
            len:  0x00300000,
            space: "data",
            access: "RW"
        },
        TRACE_BUF: {
            name: "TRACE_BUF",
            base: 0x9F000000,
            len:  0x00060000,
            space: "data",
            access: "RW"
        },
        EXC_DATA: {
            name: "EXC_DATA",
            base: 0x9F060000,
            len:  0x00010000,
            space: "data",
            access: "RW"
        },
        PM_DATA: {
            name: "PM_DATA",
            base: 0x9F070000,
            len:  0x00020000,
            space: "data",
            access: "RWX"  /* should this have execute perm? */
        },
        SR_0: {
            name: evmDRA7XX_SR_0.name,
            base: evmDRA7XX_SR_0.base,
            len:  evmDRA7XX_SR_0.len,
            space: "data",
            access: "RW"
        }
    };
    
    Build.platformTable["ti.platforms.evmDRA7XX:ipu1"] = {
        externalMemoryMap: [
            [ "EXT_CODE", evmDRA7XX_ExtMemMapIpu1.EXT_CODE ],
            [ "EXT_DATA", evmDRA7XX_ExtMemMapIpu1.EXT_DATA ],
            [ "EXT_HEAP", evmDRA7XX_ExtMemMapIpu1.EXT_HEAP ],
            [ "TRACE_BUF", evmDRA7XX_ExtMemMapIpu1.TRACE_BUF ],
            [ "EXC_DATA", evmDRA7XX_ExtMemMapIpu1.EXC_DATA ],
            [ "PM_DATA", evmDRA7XX_ExtMemMapIpu1.PM_DATA ],        
            [ evmDRA7XX_SR_0.name, evmDRA7XX_ExtMemMapIpu1.SR_0 ]
        ],
        codeMemory: "EXT_CODE",
        dataMemory: "EXT_DATA",
        stackMemory: "EXT_DATA",
    };
    

    rsc_table_ipu.h : same as the "rsc_table_vayu.h" file, with the new "IPU_SR0" entry :

    #define IPU_CMEM_IOBUFS         0x88000000
    #define PHYS_CMEM_IOBUFS        0xA0000000
    #define IPU_CMEM_IOBUFS_SIZE    (SZ_1M * 16)
    
    struct my_resource_table {
    [...]
        UInt32 offset[19];  /* Should match 'num' in actual definition */
    [...]
        /* devmem entry */
        struct fw_rsc_devmem devmem13;
    };
    
    struct my_resource_table ti_ipc_remoteproc_ResourceTable = {
    [...]
        19,     /* number of entries in the table */
    [...]
            offsetof(struct my_resource_table, devmem13),
    [...]
        {
            TYPE_DEVMEM,
            IPU_CMEM_IOBUFS, PHYS_CMEM_IOBUFS,
            IPU_CMEM_IOBUFS_SIZE, 0, 0, "IPU_SR0",
        },
    };

    I have the same behavior : 
    - DSP is the owner : "A_noHeap: Region has no heap" error for the IPU. GateMP_create() function works for the DSP.
    - IPU is the owner : IPU firmware loading crash :

    [0][      0.000] Exception occurred at (PC) = 000196e2
    [0][      0.000] CPU context: thread
    [0][      0.000] BIOS Main name: (null task) handle: 0x0.
    [0][      0.000] BIOS Main stack base: 0x8006d428.
    [0][      0.000] BIOS Main stack size: 0x1000.
    [0][      0.000] [t=0x000a1235] ti.sysbios.family.arm.m3.Hwi: ERROR: line 1104: E_hardFault: FORCED
    [0][      0.000] ti.sysbios.family.arm.m3.Hwi: line 1104: E_hardFault: FORCED
    [0][      0.000] [t=0x000ca7ed] ti.sysbios.family.arm.m3.Hwi: ERROR: line 1181: E_busFault: IMPRECISERR: Delayed Bus Fault, exact addr unknown, address: e000ed38
    [0][      0.000] ti.sysbios.family.arm.m3.Hwi: line 1181: E_busFault: IMPRECISERR: Delayed Bus Fault, exact addr unknown, address: e000ed38
    [0][      0.000] R0 = 0x00000001  R8  = 0x00000000
    [0][      0.000] R1 = 0x00000000  R9  = 0x00025978
    [0][      0.000] R2 = 0x4a0f6800  R10 = 0x8006d3c4
    [0][      0.000] R3 = 0x00000000  R11 = 0x8006e3ac
    [0][      0.000] R4 = 0x00000001  R12 = 0x00025f39
    [0][      0.000] R5 = 0x00000014  SP(R13) = 0x8006e350
    [0][      0.000] R6 = 0x00000050  LR(R14) = 0x00019693
    [0][      0.000] R7 = 0x00024a0e  PC(R15) = 0x000196e2
    [0][      0.000] PSR = 0x01000200
    [0][      0.000] ICSR = 0x00400803
    [0][      0.000] MMFSR = 0x00
    [0][      0.000] BFSR = 0x04
    [0][      0.000] UFSR = 0x0000
    [0][      0.000] HFSR = 0x40000000
    [0][      0.000] DFSR = 0x00000000
    [0][      0.000] MMAR = 0xe000ed34
    [0][      0.000] BFAR = 0xe000ed38
    [0][      0.000] AFSR = 0x00000000

    What's wrong in my configuration ?

    Thank,

    Regards,

  • paritosh,

    As Olivier noted, there is "gatempapp" example in the IPC/packages directory, see

    ./linux/src/tests/GateMPApp.c

    ./packages/ti/ipc/tests/gatempapp_rsc_table_vayu_dsp.h

    ./packages/ti/ipc/tests/gatempapp.c

    ./packages/ti/ipc/tests/gatempapp.cfg

    GateMP in Linux implementation is in the files:

    ./linux/src/daemon/GateMP_daemon.c

    ./linux/src/daemon/GateHWSpinlock.c

    ./linux/include/GateMP_config.h

    Olivia,

    "IPU is the owner : IPU firmware loading crash" - have you tried to narrow down the issue by leaving only the GateMP related function calls in the IPU code?

    "DSP is the owner : "A_noHeap: Region has no heap" error for the IPU." - there are a few discussions in the e2e thread regarding the error (but mostly from DSP), not sure if you have went through it?


    Regards, Garrett

  • In the IPU code, if i leave only the GateMP related function (ie remove the shared region configuration in the *.cfg file, and only call the GateMP_create() function in the *.c file), i have the "A_noHeap: Region has no heap". So, it's the shared region configuration/creation that "crash" the IPU. Certainly a memory mapping problem, AMMU or IOMMU side. There is no problem with the DSP. You haven't seen any error in my configuration ?

    I have go through the e2e forum with this error, i saw the thread : "A_noHeap: Region has no heap". I start first the DSP core (owner), so that it initialize the heap in the SR0, and after start the IPU core : i have the same error.