AM5728 - IPC RPMessage_send failed vring description address not consistent when annouce "MessageQ" service to the HOST

Quentin Lohri

Hi TI expert,

I have an issue with the IPC communication between Linux and IPU1 (TI-RTOS on M4).
The issue is on RTOS side when I try to send the message to linux side (RPMessage_send) to annouce the MessageQ service to host.

After several day of debug it's seems that I have good alignment between the AMMU define in .cfg and the resource table entry based on messageQ example 2.
I'm able to start correctly the application form linux side, read the resource table and memories carveout and connect with CC on IPU1 to debug.

all seems align excepted the vring descriptor address that I get during vring_init function. As you can see below the vr->description address is correct regarding AMMU and resource table (0xA0000000)
but the address of the descriptor is inconsistent.

So this have as consequence that when I try to copy the message at this address in RPMessage_send function an exception is raised.

As you can see the vring address are consistent.

You find below my AMMU configuration as my resource table

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/*********************** Small Pages *************************/
/* smallPages[0] & smallPages[1] are auto-programmed by h/w */
/* Overwrite smallPage[1] so that 16K is covered. H/w reset value configures
 * only 4K */
AMMU.smallPages[1].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[1].logicalAddress = 0x40000000;
AMMU.smallPages[1].translatedAddress = 0x55080000;
AMMU.smallPages[1].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[1].size = AMMU.Small_16K;
/* L2RAM: 64K mapped using 4 smallPages(16K); cacheable; translated */
/* config small page[2] to map 16K VA 0x20000000 to PA 0x55020000  */
AMMU.smallPages[2].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[2].logicalAddress = 0x20000000;
AMMU.smallPages[2].translatedAddress = 0x55020000;
AMMU.smallPages[2].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[2].L1_writePolicy = AMMU.WritePolicy_WRITE_BACK;
AMMU.smallPages[2].L1_allocate = AMMU.AllocatePolicy_ALLOCATE;
AMMU.smallPages[2].L1_posted = AMMU.PostedPolicy_POSTED;
AMMU.smallPages[2].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

/*********************** Small Pages *************************/
/* smallPages[0] & smallPages[1] are auto-programmed by h/w */

/* Overwrite smallPage[1] so that 16K is covered. H/w reset value configures
 * only 4K */
AMMU.smallPages[1].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[1].logicalAddress = 0x40000000;
AMMU.smallPages[1].translatedAddress = 0x55080000;
AMMU.smallPages[1].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[1].size = AMMU.Small_16K;

/* L2RAM: 64K mapped using 4 smallPages(16K); cacheable; translated */
/* config small page[2] to map 16K VA 0x20000000 to PA 0x55020000  */
AMMU.smallPages[2].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[2].logicalAddress = 0x20000000;
AMMU.smallPages[2].translatedAddress = 0x55020000;
AMMU.smallPages[2].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[2].L1_writePolicy = AMMU.WritePolicy_WRITE_BACK;
AMMU.smallPages[2].L1_allocate = AMMU.AllocatePolicy_ALLOCATE;
AMMU.smallPages[2].L1_posted = AMMU.PostedPolicy_POSTED;
AMMU.smallPages[2].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.smallPages[2].size = AMMU.Small_16K;

/* config small page[3] to map 16K VA 0x20004000 to PA 0x55024000 */
AMMU.smallPages[3].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[3].logicalAddress = 0x20004000;
AMMU.smallPages[3].translatedAddress = 0x55024000;
AMMU.smallPages[3].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[3].L1_writePolicy = AMMU.WritePolicy_WRITE_BACK;
AMMU.smallPages[3].L1_allocate = AMMU.AllocatePolicy_ALLOCATE;
AMMU.smallPages[3].L1_posted = AMMU.PostedPolicy_POSTED;
AMMU.smallPages[3].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.smallPages[3].size = AMMU.Small_16K;

/* config small page[4] to map 16K VA 0x20008000 to PA 0x55028000 */
AMMU.smallPages[4].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[4].logicalAddress = 0x20008000;
AMMU.smallPages[4].translatedAddress = 0x55028000;
AMMU.smallPages[4].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[4].L1_writePolicy = AMMU.WritePolicy_WRITE_BACK;
AMMU.smallPages[4].L1_allocate = AMMU.AllocatePolicy_ALLOCATE;
AMMU.smallPages[4].L1_posted = AMMU.PostedPolicy_POSTED;
AMMU.smallPages[4].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.smallPages[4].size = AMMU.Small_16K;

/* config small page[5] to map 16K VA 0x2000C000 to PA 0x5502C000 */
AMMU.smallPages[5].pageEnabled = AMMU.Enable_YES;
AMMU.smallPages[5].logicalAddress = 0x2000C000;
AMMU.smallPages[5].translatedAddress = 0x5502C000;
AMMU.smallPages[5].translationEnabled = AMMU.Enable_YES;
AMMU.smallPages[5].L1_writePolicy = AMMU.WritePolicy_WRITE_BACK;
AMMU.smallPages[5].L1_allocate = AMMU.AllocatePolicy_ALLOCATE;
AMMU.smallPages[5].L1_posted = AMMU.PostedPolicy_POSTED;
AMMU.smallPages[5].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.smallPages[5].size = AMMU.Small_16K;

/*********************** Large Pages *************************/
/* Instruction Code: Large page  (512M); cacheable */
/* config large page[0] to map 512MB VA 0x0 to L3 0x0 */ 
AMMU.largePages[0].pageEnabled = AMMU.Enable_YES;
AMMU.largePages[0].logicalAddress = 0x0;
AMMU.largePages[0].translationEnabled = AMMU.Enable_NO;
AMMU.largePages[0].size = AMMU.Large_512M;
AMMU.largePages[0].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.largePages[0].L1_posted = AMMU.PostedPolicy_POSTED;

/* Peripheral regions: Large Page (512M); non-cacheable */
/* config large page[1] to map 512MB VA 0x60000000 to L3 0x60000000 */
AMMU.largePages[1].pageEnabled = AMMU.Enable_YES;
AMMU.largePages[1].logicalAddress = 0x60000000;
AMMU.largePages[1].translationEnabled = AMMU.Enable_NO;
AMMU.largePages[1].size = AMMU.Large_512M;
AMMU.largePages[1].L1_cacheable = AMMU.CachePolicy_NON_CACHEABLE;
AMMU.largePages[1].L1_posted = AMMU.PostedPolicy_POSTED;

/* Private, Shared and IPC Data regions: Large page (512M); cacheable */
/* config large page[2] to map 512MB VA 0x80000000 to L3 0x80000000 */
AMMU.largePages[2].pageEnabled = AMMU.Enable_YES;
AMMU.largePages[2].logicalAddress = 0x80000000;
AMMU.largePages[2].translationEnabled = AMMU.Enable_NO;
AMMU.largePages[2].size = AMMU.Large_512M;
AMMU.largePages[2].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.largePages[2].L1_posted = AMMU.PostedPolicy_POSTED; 

/* TILER & DMM regions: Large page (512M); cacheable */
/* config large page[3] to map 512MB VA 0xA0000000 to L3 0xA0000000 */
AMMU.largePages[3].pageEnabled = AMMU.Enable_YES;
AMMU.largePages[3].logicalAddress = 0xA0000000;
AMMU.largePages[3].translationEnabled = AMMU.Enable_NO;
AMMU.largePages[3].size = AMMU.Large_512M;
AMMU.largePages[3].L1_cacheable = AMMU.CachePolicy_CACHEABLE;
AMMU.largePages[3].L1_posted = AMMU.PostedPolicy_POSTED;

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#define VAYU_IPU_1
/* IPU Memory Map */
#define L4_DRA7XX_BASE          0x4A000000
/* L4_CFG & L4_WKUP */
#define L4_PERIPHERAL_L4CFG     (L4_DRA7XX_BASE)
#define IPU_PERIPHERAL_L4CFG    0x6A000000
#define L4_PERIPHERAL_L4PER1    0x48000000
#define IPU_PERIPHERAL_L4PER1   0x68000000
#define L4_PERIPHERAL_L4PER2    0x48400000
#define IPU_PERIPHERAL_L4PER2   0x68400000
#define L4_PERIPHERAL_L4PER3    0x48800000
#define IPU_PERIPHERAL_L4PER3   0x68800000
#define L4_PERIPHERAL_L4EMU     0x54000000
#define IPU_PERIPHERAL_L4EMU    0x74000000
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

#define VAYU_IPU_1

/* IPU Memory Map */
#define L4_DRA7XX_BASE          0x4A000000

/* L4_CFG & L4_WKUP */
#define L4_PERIPHERAL_L4CFG     (L4_DRA7XX_BASE)
#define IPU_PERIPHERAL_L4CFG    0x6A000000

#define L4_PERIPHERAL_L4PER1    0x48000000
#define IPU_PERIPHERAL_L4PER1   0x68000000

#define L4_PERIPHERAL_L4PER2    0x48400000
#define IPU_PERIPHERAL_L4PER2   0x68400000

#define L4_PERIPHERAL_L4PER3    0x48800000
#define IPU_PERIPHERAL_L4PER3   0x68800000

#define L4_PERIPHERAL_L4EMU     0x54000000
#define IPU_PERIPHERAL_L4EMU    0x74000000

#define L3_PERIPHERAL_DMM       0x4E000000
#define IPU_PERIPHERAL_DMM      0x6E000000

#define L3_IVAHD_CONFIG         0x5A000000
#define IPU_IVAHD_CONFIG        0x7A000000

#define L3_IVAHD_SL2            0x5B000000
#define IPU_IVAHD_SL2           0x7B000000

#define L3_TILER_MODE_0_1       0x60000000
#define IPU_TILER_MODE_0_1      0xA0000000

#define L3_TILER_MODE_2         0x70000000
#define IPU_TILER_MODE_2        0xB0000000

#define L3_TILER_MODE_3         0x78000000
#define IPU_TILER_MODE_3        0xB8000000
#if defined(VAYU_IPU_1)
#define IPU_MEM_TEXT            0x0
#elif defined(VAYU_IPU_2)
#define IPU_MEM_TEXT0            0x0
#define IPU_MEM_TEXT1            0x0200000
#endif

#if defined(VAYU_IPU_1)
#define IPU_MEM_DATA0            0x80000000
#define IPU_MEM_DATA1            0x80400000
#elif defined(VAYU_IPU_2)
#define IPU_MEM_DATA0            0x80000000
#define IPU_MEM_DATA1            0x80800000
#define IPU_MEM_DATA2            0x81800000
#define IPU_MEM_DATA3            0x82800000
#endif

#define IPU_MEM_IPC_DATA        0x9F000000
#define IPU_MEM_IPC_VRING       0x60000000
#define IPU_MEM_RPMSG_VRING0    0x60000000
#define IPU_MEM_RPMSG_VRING1    0x60004000
#define IPU_MEM_VRING_BUFS0     0x60040000
#define IPU_MEM_VRING_BUFS1     0x60080000

#define IPU_MEM_IPC_VRING_SIZE  SZ_1M
#define IPU_MEM_IPC_DATA_SIZE   SZ_1M

#if defined(VAYU_IPU_1)
#define IPU_MEM_TEXT_SIZE       (SZ_1M)
#elif defined(VAYU_IPU_2)
#define IPU_MEM_TEXT0_SIZE       (SZ_1M * 2)
#define IPU_MEM_TEXT1_SIZE       (SZ_1M * 4)
#endif

#if defined(VAYU_IPU_1)
#define IPU_MEM_DATA0_SIZE       (SZ_1M * 4)
#define IPU_MEM_DATA1_SIZE       (SZ_1M * 4)
#elif defined(VAYU_IPU_2)
#define IPU_MEM_DATA0_SIZE       (SZ_1M * 8)
#define IPU_MEM_DATA1_SIZE       (SZ_1M * 16)
#define IPU_MEM_DATA2_SIZE       (SZ_1M * 16)
#define IPU_MEM_DATA3_SIZE       (SZ_1M * 8)
#endif

/*
 * NOTE:
 * To avoid issues with allocation failures with Linux carveout regions, need
 * to use the RSC_CARVEOUT entries with power of 2 page order sizes and aligned
 * on the same page order.
 * The size and the alignment order of entries in the resource table plays a
 * part in avoiding gaps in allocation
 */
#if defined(VAYU_IPU_1)
#define PHYS_MEM_IPC_VRING      0x9D000000
#elif defined (VAYU_IPU_2)
#define PHYS_MEM_IPC_VRING      0x95800000
#endif

/*
 * Sizes of the virtqueues (expressed in number of buffers supported,
 * and must be power of 2)
 */
#define IPU_RPMSG_VQ0_SIZE      256
#define IPU_RPMSG_VQ1_SIZE      256

/* flip up bits whose indices represent features we support */
#define RPMSG_IPU_C0_FEATURES   1

#if defined(VAYU_IPU_1)
#define NUM_RSC_ENTRIES 18
#elif defined (VAYU_IPU_2)
#define NUM_RSC_ENTRIES 21
#endif

struct my_resource_table {
    struct resource_table base;

    UInt32 offset[NUM_RSC_ENTRIES];  /* Should match 'num' in actual definition */

    /* rpmsg vdev entry */
    struct fw_rsc_vdev rpmsg_vdev;
    struct fw_rsc_vdev_vring rpmsg_vring0;
    struct fw_rsc_vdev_vring rpmsg_vring1;

    /* ipcdata carveout entry */
    struct fw_rsc_carveout ipcdata_cout;

    /* text carveout entry */
#if defined(VAYU_IPU_1)
    struct fw_rsc_carveout text_cout;
#elif defined (VAYU_IPU_2)
    struct fw_rsc_carveout text0_cout;
    struct fw_rsc_carveout text1_cout;
#endif
    /* data carveout entries */
    struct fw_rsc_carveout data0_cout;
    struct fw_rsc_carveout data1_cout;
#if defined(VAYU_IPU_2)
    struct fw_rsc_carveout data2_cout;
    struct fw_rsc_carveout data3_cout;
#endif

    /* trace entry */
    struct fw_rsc_trace trace;

    /* devmem entry */
    struct fw_rsc_devmem devmem0;

    /* devmem entry */
    struct fw_rsc_devmem devmem1;

    /* devmem entry */
    struct fw_rsc_devmem devmem2;

    /* devmem entry */
    struct fw_rsc_devmem devmem3;

    /* devmem entry */
    struct fw_rsc_devmem devmem4;

    /* devmem entry */
    struct fw_rsc_devmem devmem5;

    /* devmem entry */
    struct fw_rsc_devmem devmem6;

    /* devmem entry */
    struct fw_rsc_devmem devmem7;

    /* devmem entry */
    struct fw_rsc_devmem devmem8;

    /* devmem entry */
    struct fw_rsc_devmem devmem9;

    /* devmem entry */
    struct fw_rsc_devmem devmem10;

    /* devmem entry */
    struct fw_rsc_devmem devmem11;
};

extern char ti_trace_SysMin_Module_State_0_outbuf__A;
#define TRACEBUFADDR (UInt32)&ti_trace_SysMin_Module_State_0_outbuf__A

#pragma DATA_SECTION(ti_ipc_remoteproc_ResourceTable, ".resource_table")
#pragma DATA_ALIGN(ti_ipc_remoteproc_ResourceTable, 4096)

struct my_resource_table ti_ipc_remoteproc_ResourceTable = {
    1,      /* we're the first version that implements this */
    NUM_RSC_ENTRIES,     /* number of entries in the table */
    0, 0,   /* reserved, must be zero */
    /* offsets to entries */
    {
        offsetof(struct my_resource_table, rpmsg_vdev),
        offsetof(struct my_resource_table, ipcdata_cout),
#if defined(VAYU_IPU_1)
        offsetof(struct my_resource_table, text_cout),
#elif defined (VAYU_IPU_2)
        offsetof(struct my_resource_table, text0_cout),
        offsetof(struct my_resource_table, text1_cout),
#endif
        offsetof(struct my_resource_table, data0_cout),
        offsetof(struct my_resource_table, data1_cout),
#if defined(VAYU_IPU_2)
        offsetof(struct my_resource_table, data2_cout),
        offsetof(struct my_resource_table, data3_cout),
#endif
        offsetof(struct my_resource_table, trace),
        offsetof(struct my_resource_table, devmem0),
        offsetof(struct my_resource_table, devmem1),
        offsetof(struct my_resource_table, devmem2),
        offsetof(struct my_resource_table, devmem3),
        offsetof(struct my_resource_table, devmem4),
        offsetof(struct my_resource_table, devmem5),
        offsetof(struct my_resource_table, devmem6),
        offsetof(struct my_resource_table, devmem7),
        offsetof(struct my_resource_table, devmem8),
        offsetof(struct my_resource_table, devmem9),
        offsetof(struct my_resource_table, devmem10),
        offsetof(struct my_resource_table, devmem11),
    },

    /* rpmsg vdev entry */
    {
        TYPE_VDEV, VIRTIO_ID_RPMSG, 0,
        RPMSG_IPU_C0_FEATURES, 0, 0, 0, 2, { 0, 0 },
        /* no config data */
    },
    /* the two vrings */
    { IPU_MEM_RPMSG_VRING0, 4096, IPU_RPMSG_VQ0_SIZE, 1, 0 },
    { IPU_MEM_RPMSG_VRING1, 4096, IPU_RPMSG_VQ1_SIZE, 2, 0 },

    {
        TYPE_CARVEOUT,
        IPU_MEM_IPC_DATA, 0,
        IPU_MEM_IPC_DATA_SIZE, 0, 0, "IPU_MEM_IPC_DATA",
    },

#if defined(VAYU_IPU_1)
    {
        TYPE_CARVEOUT,
        IPU_MEM_TEXT, 0,
        IPU_MEM_TEXT_SIZE, 0, 0, "IPU_MEM_TEXT",
    },
#elif defined (VAYU_IPU_2)
    {
        TYPE_CARVEOUT,
        IPU_MEM_TEXT0, 0,
        IPU_MEM_TEXT0_SIZE, 0, 0, "IPU_MEM_TEXT0",
    },

    {
        TYPE_CARVEOUT,
        IPU_MEM_TEXT1, 0,
        IPU_MEM_TEXT1_SIZE, 0, 0, "IPU_MEM_TEXT1",
    },
#endif

    {
        TYPE_CARVEOUT,
        IPU_MEM_DATA0, 0,
        IPU_MEM_DATA0_SIZE, 0, 0, "IPU_MEM_DATA0",
    },

    {
        TYPE_CARVEOUT,
        IPU_MEM_DATA1, 0,
        IPU_MEM_DATA1_SIZE, 0, 0, "IPU_MEM_DATA1",
    },

#if defined(VAYU_IPU_2)
    {
        TYPE_CARVEOUT,
        IPU_MEM_DATA2, 0,
        IPU_MEM_DATA2_SIZE, 0, 0, "IPU_MEM_DATA2",
    },

    {
        TYPE_CARVEOUT,
        IPU_MEM_DATA3, 0,
        IPU_MEM_DATA3_SIZE, 0, 0, "IPU_MEM_DATA3",
    },
#endif

    {
        TYPE_TRACE, TRACEBUFADDR, 0x8000, 0, "trace:sysm3",
    },

    {
        TYPE_DEVMEM,
        IPU_MEM_IPC_VRING, PHYS_MEM_IPC_VRING,
        IPU_MEM_IPC_VRING_SIZE, 0, 0, "IPU_MEM_IPC_VRING",
    },

    {
        TYPE_DEVMEM,
        IPU_TILER_MODE_0_1, L3_TILER_MODE_0_1,
        SZ_256M, 0, 0, "IPU_TILER_MODE_0_1",
    },

    {
        TYPE_DEVMEM,
        IPU_TILER_MODE_2, L3_TILER_MODE_2,
        SZ_128M, 0, 0, "IPU_TILER_MODE_2",
    },

    {
        TYPE_DEVMEM,
        IPU_TILER_MODE_3, L3_TILER_MODE_3,
        SZ_128M, 0, 0, "IPU_TILER_MODE_3",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_L4CFG, L4_PERIPHERAL_L4CFG,
        SZ_16M, 0, 0, "IPU_PERIPHERAL_L4CFG",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_L4PER1, L4_PERIPHERAL_L4PER1,
        SZ_2M, 0, 0, "IPU_PERIPHERAL_L4PER1",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_L4PER2, L4_PERIPHERAL_L4PER2,
        SZ_4M, 0, 0, "IPU_PERIPHERAL_L4PER2",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_L4PER3, L4_PERIPHERAL_L4PER3,
        SZ_8M, 0, 0, "IPU_PERIPHERAL_L4PER3",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_L4EMU, L4_PERIPHERAL_L4EMU,
        SZ_16M, 0, 0, "IPU_PERIPHERAL_L4EMU",
    },

    {
        TYPE_DEVMEM,
        IPU_IVAHD_CONFIG, L3_IVAHD_CONFIG,
        SZ_16M, 0, 0, "IPU_IVAHD_CONFIG",
    },

    {
        TYPE_DEVMEM,
        IPU_IVAHD_SL2, L3_IVAHD_SL2,
        SZ_16M, 0, 0, "IPU_IVAHD_SL2",
    },

    {
        TYPE_DEVMEM,
        IPU_PERIPHERAL_DMM, L3_PERIPHERAL_DMM,
        SZ_1M, 0, 0, "IPU_PERIPHERAL_DMM",
    },
};

On linux side we can see that the resource table and carveout memories are correctly loaded

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
root@am5728-gen357a:~# cat /sys/kernel/debug/remoteproc/remoteproc0/resource_table
Entry 0 is of type vdev
  ID 7
  Notify ID 0
  Device features 0x1
  Guest features 0x1
  Config length 0x0
  Status 0x7
  Number of vrings 2
  Reserved (should be zero) [0][0]
  Vring 0
    Device Address 0x60000000
    Alignment 4096
    Number of buffers 256
    Notify ID 0
    Physical Address 0x0
  Vring 1
    Device Address 0x60004000
    Alignment 4096
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

root@am5728-gen357a:~# cat /sys/kernel/debug/remoteproc/remoteproc0/resource_table
Entry 0 is of type vdev
  ID 7
  Notify ID 0
  Device features 0x1
  Guest features 0x1
  Config length 0x0
  Status 0x7
  Number of vrings 2
  Reserved (should be zero) [0][0]

  Vring 0
    Device Address 0x60000000
    Alignment 4096
    Number of buffers 256
    Notify ID 0
    Physical Address 0x0

  Vring 1
    Device Address 0x60004000
    Alignment 4096
    Number of buffers 256
    Notify ID 1
    Physical Address 0x0

Entry 1 is of type carveout
  Device Address 0x9f000000
  Physical Address 0x9d100000
  Length 0x100000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_MEM_IPC_DATA

Entry 2 is of type carveout
  Device Address 0x0
  Physical Address 0x9d200000
  Length 0x100000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_MEM_TEXT

Entry 3 is of type carveout
  Device Address 0x80000000
  Physical Address 0x9d300000
  Length 0x400000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_MEM_DATA0

Entry 4 is of type carveout
  Device Address 0x80400000
  Physical Address 0x9d700000
  Length 0x400000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_MEM_DATA1

Entry 5 is of type trace
  Device Address 0x9f000000
  Length 0x8000 Bytes
  Reserved (should be zero) [0]
  Name trace:sysm3

Entry 6 is of type devmem
  Device Address 0x60000000
  Physical Address 0x9d000000
  Length 0x100000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_MEM_IPC_VRING

Entry 7 is of type devmem
  Device Address 0xa0000000
  Physical Address 0x60000000
  Length 0x10000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_TILER_MODE_0_1

Entry 8 is of type devmem
  Device Address 0xb0000000
  Physical Address 0x70000000
  Length 0x8000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_TILER_MODE_2

Entry 9 is of type devmem
  Device Address 0xb8000000
  Physical Address 0x78000000
  Length 0x8000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_TILER_MODE_3

Entry 10 is of type devmem
  Device Address 0x6a000000
  Physical Address 0x4a000000
  Length 0x1000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_L4CFG

Entry 11 is of type devmem
  Device Address 0x68000000
  Physical Address 0x48000000
  Length 0x200000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_L4PER1

Entry 12 is of type devmem
  Device Address 0x68400000
  Physical Address 0x48400000
  Length 0x400000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_L4PER2

Entry 13 is of type devmem
  Device Address 0x68800000
  Physical Address 0x48800000
  Length 0x800000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_L4PER3

Entry 14 is of type devmem
  Device Address 0x74000000
  Physical Address 0x54000000
  Length 0x1000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_L4EMU

Entry 15 is of type devmem
  Device Address 0x7a000000
  Physical Address 0x5a000000
  Length 0x1000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_IVAHD_CONFIG

Entry 16 is of type devmem
  Device Address 0x7b000000
  Physical Address 0x5b000000
  Length 0x1000000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_IVAHD_SL2

Entry 17 is of type devmem
  Device Address 0x6e000000
  Physical Address 0x4e000000
  Length 0x100000 Bytes
  Flags 0x0
  Reserved (should be zero) [0]
  Name IPU_PERIPHERAL_DMM

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
root@am5728-gen357a:~# cat /sys/kernel/debug/remoteproc/remoteproc0/carveout_memories
Carveout memory entry:
        Name: vdev0vring0
        Virtual address: 9dd8d14f
        DMA address: 0x000000009d000000
        Device address: 0x60000000
        Length: 0x3000 Bytes
Carveout memory entry:
        Name: vdev0vring1
        Virtual address: 1b3fb2ee
        DMA address: 0x000000009d004000
        Device address: 0x60004000
        Length: 0x3000 Bytes
Carveout memory entry:
        Name: IPU_MEM_IPC_DATA
        Virtual address: 21de72d3
        DMA address: 0x000000009d100000
        Device address: 0x9f000000
        Length: 0x100000 Bytes
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

root@am5728-gen357a:~# cat /sys/kernel/debug/remoteproc/remoteproc0/carveout_memories
Carveout memory entry:
        Name: vdev0vring0
        Virtual address: 9dd8d14f
        DMA address: 0x000000009d000000
        Device address: 0x60000000
        Length: 0x3000 Bytes

Carveout memory entry:
        Name: vdev0vring1
        Virtual address: 1b3fb2ee
        DMA address: 0x000000009d004000
        Device address: 0x60004000
        Length: 0x3000 Bytes

Carveout memory entry:
        Name: IPU_MEM_IPC_DATA
        Virtual address: 21de72d3
        DMA address: 0x000000009d100000
        Device address: 0x9f000000
        Length: 0x100000 Bytes

Carveout memory entry:
        Name: IPU_MEM_TEXT
        Virtual address: d0ba1855
        DMA address: 0x000000009d200000
        Device address: 0x0
        Length: 0x100000 Bytes

Carveout memory entry:
        Name: IPU_MEM_DATA0
        Virtual address: dd316bc7
        DMA address: 0x000000009d300000
        Device address: 0x80000000
        Length: 0x400000 Bytes

Carveout memory entry:
        Name: IPU_MEM_DATA1
        Virtual address: 2c289b62
        DMA address: 0x000000009d700000
        Device address: 0x80400000
        Length: 0x400000 Bytes

So my question are:
Why the vring->desrciptor->address seems to be not consistent ?
Who is in charge to fill the IIPU_MEM_RPMSG_VRING0 address with allocated buffer ?

Many thanks in advance

Quentin

over 1 year ago

0 Josue Zamitiz-Ayala over 1 year ago

TI__Mastermind 25149 points

Hello Quentin,

Could you please let us know what SDK versions you are using for both RTOS and LInux?

-Josue

0 Quentin Lohri over 1 year ago in reply to Josue Zamitiz-Ayala

Prodigy 50 points

Hi Josue,

Thanks for the response.
For the rtos side we are on processor_sdk_rtos_am57xx_06_03_02_08.
On linux side we are working with linux-ti-staging-rt-5.10.168. .config - Linux/arm 5.10.168 Kernel Configuration. This dues to the fact that we have a specific board and application requirement.

You find join the kernel module configuration that we have configured - perhaps you have an expert eye and highlight a module that is missed in VIRTIO or other - for info rpmsg-proto is enabled but perhaps something is loss....

68520.config.txt

On the linux side we have build exmaple, LAD and libraries thanks to make -f ipc-linux.mak config and the pdf document IPC_Install_Guide_Linux.pdf present into sdk IPC folder. We are able to run DRA7XX_linux_elf\ex02_messageq on the host by starting previously LAD deamon.

Here is the log.txt for the LAD deamon when start app on host - application not stared on the IPU1

Fullscreen

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
WARN: Client /tmp/LAD/312 has not yet opened, will retry
[65.439971]     FIFO /tmp/LAD/312 opened for writing
[65.440063]     sent response
[65.440093] DONE
[65.440124] Retrieving command...
[65.441101] Sending response...
[65.441131] Retrieving command...
[65.441406] LAD_MULTIPROC_GETCONFIG: calling MultiProc_getConfig()...
[65.441436] MultiProc_getConfig() - 5 procs
[65.441467] # processors in cluster: 5
[65.441497] cluster baseId: 0
[65.441497]     ProcId 0 - "HOST"
[65.441528]     ProcId 1 - "IPU2"
[65.441558]     ProcId 2 - "IPU1"
[65.441558]     ProcId 3 - "DSP2"
[65.441589]     ProcId 4 - "DSP1"
[65.441619]     status = 0
[65.441650] DONE
[65.441650] Sending response...
[65.441711] Retrieving command...
[65.442932] LAD_NAMESERVER_SETUP: calling NameServer_setup()...
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

WARN: Client /tmp/LAD/312 has not yet opened, will retry
[65.439971]     FIFO /tmp/LAD/312 opened for writing
[65.440063]     sent response
[65.440093] DONE
[65.440124] Retrieving command...
[65.441101] Sending response...
[65.441131] Retrieving command...
[65.441406] LAD_MULTIPROC_GETCONFIG: calling MultiProc_getConfig()...
[65.441436] MultiProc_getConfig() - 5 procs
[65.441467] # processors in cluster: 5
[65.441497] cluster baseId: 0
[65.441497]     ProcId 0 - "HOST"
[65.441528]     ProcId 1 - "IPU2"
[65.441558]     ProcId 2 - "IPU1"
[65.441558]     ProcId 3 - "DSP2"
[65.441589]     ProcId 4 - "DSP1"
[65.441619]     status = 0
[65.441650] DONE
[65.441650] Sending response...
[65.441711] Retrieving command...
[65.442932] LAD_NAMESERVER_SETUP: calling NameServer_setup()...
[65.442962] NameServer_setup: entered, refCount=0
[65.442993] NameServer_setup: creating listener thread
[65.444671] NameServer_setup: exiting, refCount=1
[65.444702]     status = 0
[65.444732] DONE
[65.444763] Sending response...
[65.444793] Retrieving command...
[65.446136] LAD_MESSAGEQ_GETCONFIG: calling MessageQ_getConfig()...
[65.446502]     status = 0
[65.446746] DONE
[65.446777] Sending response...
[65.446807] Retrieving command...
[65.447265] LAD_MESSAGEQ_SETUP: calling MessageQ_setup()...
[65.447296] MessageQ_setup: entered, refCount=0
[65.447326] NameServer_create(): 'MessageQ'
[65.447387] MessageQ_setup: exiting, refCount=1
[65.447387]     status = 0
[65.447418] DONE
[65.447448] Sending response...
[65.447479] Retrieving command...
[65.448852] NameServer_attach: --> procId=1, refCount=0
[65.448913] NameServer_attach: created send socket: 5, procId 1
[65.449005] NameServer_attach: connect failed: procId=1, errno=22 (Invalid argument)
[65.449035]     closing send socket: 5
[65.449096] NameServer_attach: <-- refCount=0, status=-1
[65.449127] Sending response...
[65.449157] Retrieving command...
[65.450744] NameServer_attach: --> procId=2, refCount=0
[65.450775] NameServer_attach: created send socket: 5, procId 2
[65.450836] NameServer_attach: connect failed: procId=2, errno=22 (Invalid argument)
[65.450836]     closing send socket: 5
[65.450897] NameServer_attach: <-- refCount=0, status=-1
[65.450927] Sending response...
[65.450958] Retrieving command...
[65.452270] NameServer_attach: --> procId=3, refCount=0
[65.452331] NameServer_attach: created send socket: 5, procId 3
[65.452362] NameServer_attach: connect failed: procId=3, errno=22 (Invalid argument)
[65.452392]     closing send socket: 5
[65.452423] NameServer_attach: <-- refCount=0, status=-1
[65.452453] Sending response...
[65.452514] Retrieving command...
[65.453704] NameServer_attach: --> procId=4, refCount=0
[65.453765] NameServer_attach: created send socket: 5, procId 4
[65.453796] NameServer_attach: connect failed: procId=4, errno=22 (Invalid argument)
[65.453826]     closing send socket: 5
[65.453857] NameServer_attach: <-- refCount=0, status=-1
[65.453887] Sending response...
[65.455352] Retrieving command...
[65.455413] LAD_GATEMP_ISSETUP: calling GateMP_isSetup()...
[65.455444]     status = 0
[65.455444] DONE
[65.455474] Sending response...
[65.456817] Retrieving command...
[65.456848] LAD_MESSAGEQ_CREATE: calling MessageQ_create(0x337c8, 0x337e8)...
[65.456878] MessageQ_create: creating 'HOST:MsgQ:01'
[65.456909] MessageQ_create: returning obj=0x379a8, qid=0x80
[65.456939]     status = 0
[65.456970] DONE
[65.456970] Sending response...
[65.458374] Retrieving command...
[65.458404] LAD_MESSAGEQ_ANNOUNCE: calling MessageQ_announce(0x337c8, 0x379a8)...
[65.458435] MessageQ_announce: announcing 0x379a8
[65.458465] NameServer_add: Entered key: 'HOST:MsgQ:01', data: 0x80
[65.458496]     status = 0
[65.458496] DONE
[65.458526] Sending response...
[65.459564] Retrieving command...
[65.460021] LAD_NAMESERVER_GETUINT32: calling NameServer_getUInt32(0x378c0, 'IPU1:MsgQ:01')...
[65.460052] NameServer_getLocal: entry key: 'IPU1:MsgQ:01' not found!
[65.460083] NameServer_getRemote: no socket connection to processor 1
[65.460144] NameServer_getRemote: no socket connection to processor 2
[65.460144] NameServer_getRemote: no socket connection to processor 3
[65.460235] NameServer_getRemote: no socket connection to processor 4
[65.460235]     value = 0x80
[65.460266]     status = -5

As mentionned in my previous post, when I start the .xem4 on IPU1, resource table seems to be coherent, memories carveout also and I have that log that inform about application is started as you can see here

Fullscreen

1
2
3
4
5
6
7
8
9
10
[  773.818817] remoteproc remoteproc0: Booting fw image dra7-ipu1-fw.xem4, size 7673056
[  773.818969] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
[  773.851257]  remoteproc0#vdev0buffer: assigned reserved memory node ipu1-memory@9d000000
[  773.855072] virtio_rpmsg_bus virtio0: rpmsg host is online
[  773.855163]  remoteproc0#vdev0buffer: registered virtio0 (type 7)
[  773.855163] remoteproc remoteproc0: remote processor 58820000.ipu is now up
[  774.126647] omap-rproc 58820000.ipu: received echo reply from 58820000.ipu
[  774.126861] omap-iommu 58882000.mmu: iommu fault: da 0x80389970 flags 0x0
[  774.126861] remoteproc remoteproc0: crash detected in 58820000.ipu: type mmufault
[  774.126892] omap-iommu 58882000.mmu: 58882000.mmu: errs:0x00000002 da:0x80389970 pgd:0x022db3db *pgd:px9d600002
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[  773.818817] remoteproc remoteproc0: Booting fw image dra7-ipu1-fw.xem4, size 7673056
[  773.818969] omap-iommu 58882000.mmu: 58882000.mmu: version 2.1
[  773.851257]  remoteproc0#vdev0buffer: assigned reserved memory node ipu1-memory@9d000000
[  773.855072] virtio_rpmsg_bus virtio0: rpmsg host is online
[  773.855163]  remoteproc0#vdev0buffer: registered virtio0 (type 7)
[  773.855163] remoteproc remoteproc0: remote processor 58820000.ipu is now up
[  774.126647] omap-rproc 58820000.ipu: received echo reply from 58820000.ipu
[  774.126861] omap-iommu 58882000.mmu: iommu fault: da 0x80389970 flags 0x0
[  774.126861] remoteproc remoteproc0: crash detected in 58820000.ipu: type mmufault
[  774.126892] omap-iommu 58882000.mmu: 58882000.mmu: errs:0x00000002 da:0x80389970 pgd:0x022db3db *pgd:px9d600002

Thanks to CCS and as mentionned in my previous post, I was able to discovery that the MMU issue occurs on the memcpy in the RPMessage_send with dues to the vring->desc->addr that seems not consitent.

It's why I try to understand from where is allocated this address, because all others like vring->desc are defined in resource table and translation defined with AMMU.

Hope my explanation can help

Thanks

Quentin

0 Suman Anna over 1 year ago in reply to Quentin Lohri

TI__Guru** 101665 points

Hi Quentin,

You cannot randomly mix and match an older RTOS SDK with a newer Linux SDK version. You are using a 6.3 SDK (would have been released with a 4.19 Linux kernel) with a 5.10 Linux kernel (8.2 Linux SDK).

The Linux kernel remoteproc/rpmsg framework can change from LTS kernel to LTS kernel, and this requires the correct IPC 3.x version as well. The 5.10 Linux Kernel requires the IPC 3.51.00.00A release. So, please use the appropriate IPC version.

Please see this response specifically on a different thread.

regards

Suman

0 Quentin Lohri over 1 year ago in reply to Suman Anna

Prodigy 50 points

Hi Suman,

Thanks very much for the information after update to version IPC3.51.00.00A find here https://git.ti.com/cgit/ipc/ipcdev/ all works perfectly.
In fact the only issue was the address of the vring that I have to define as below

Thanks for support.

Regards,

Quentin

Processors

Processors forum

AM5728 - IPC RPMessage_send failed vring description address not consistent when annouce "MessageQ" service to the HOST