This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Problems with using insmod cmemk.ko

Other Parts Discussed in Thread: SYSBIOS

Hello,

I am using the EVM 57xx Board and I want to play around with the CMEM functions, to get the possibility to do IPC between ARM and the DSP1 to share big data like images.

After reading (http://processors.wiki.ti.com/index.php/CMEM_Overview ) even the idea to play around with CMEM is not working as described.

The following is an example of installing the cmem kernel module:

/sbin/insmod cmemk.ko pools=4x30000,2x500000 phys_start=0x0 phys_end=0x3000000

When i try to run this in my Linux shell, it will return this:
root@am57xx-evm:~# /sbin/insmod cmemk.ko pools=4x30000,2x500000 phys_start=0x0 phys_end=0x3000000
insmod: ERROR: could not load module cmemk.ko: No such file or directory
root@am57xx-evm:~#

I found out, that there was already some CMEM stuff running:
root@am57xx-evm:~# cat /proc/cmem

Block 0: Pool 0: 1 bufs size 0xc000000 (0xc000000 requested)

Pool 0 busy bufs:

Pool 0 free bufs:
id 0: phys addr 0xa0000000
root@am57xx-evm:~#


So to continue playing around, I have tried this:
root@am57xx-evm:~# rmmod cmemk.ko
[96428.911582] cmemk unregistered

root@am57xx-evm:~# /sbin/insmod cmemk.ko pools=4x30000,2x500000 phys_start=0x0 phys_end=0x3000000
insmod: ERROR: could not load module cmemk.ko: No such file or directory

root@am57xx-evm:~# /sbin/insmod /lib/modules/4.1.13-g8dc6617/extra/cmemk.ko pools=4x30000,2x500000 phys_start=0x0 phys_end=0x3000000
[96486.032190] CMEMK module: reference Linux version 4.1.13
[96486.038208] allocated heap buffer 0x40500000 of size 0x100000
[96486.044236] cmemk initialized

root@am57xx-evm:~# cat /proc/cmem
Block 0: Pool 0: 1 bufs size 0xc000000 (0xc000000 requested)

Pool 0 busy bufs:

Pool 0 free bufs:
id 0: phys addr 0xa0000000



So as you can see, it did not work like I expected. It looks like, that I cannot use new CMEM buffer. There always will be the buffer on 0xa0000000. Does it depend on the am57xx-evm-cmem.dtsi file?
        reserved-memory {
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;

                cmem_block_mem_0: cmem_block_mem@a0000000 {
                        reg = <0x0 0xa0000000 0x0 0x0c000000>;
                        no-map;
                        status = "okay";
                };

        cmem_block_mem_1_ocmc3: cmem_block_mem@40500000 {
            reg = <0x0 0x40500000 0x0 0x100000>;
            no-map;
            status = "okay";
        };
 };



Do I have to change something there?
  • Sorry for the strange display.
    I am using the Linux-SDK 2.0.1 for ARM, IPC 3.41.0.8 and SYSBIOS 6.45.0.19 for the DSP
  • I will forward this to the software team.
  • Kevin,

    Yes, update the DTS file with your own CMEM pool address, reboot Linux with the new DTB, then you should see the CMEM pool created properly.

    To build the DTB, see processors.wiki.ti.com/.../Linux_Kernel_Users_Guide
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- <dt filename>.dtb

    DTB is located in file system /boot directory.

    Regards,
    Garrett
  • Thanks, I think this could be the solution. But I have a problem with generating the new DTB file.

    I did following:

    - Exported the compiler path with
    export PATH=/home/audis/ti-processor-sdk-linux-am57xx-evm-02.00.01.07/linux-devkit/sysroots/x86_64-arago-linux/usr/bin:$PATH

    - Modified my dtsi file am57xx-evm-cmem.dtsi like this:
    / {
    reserved-memory {
    #address-cells = <2>;
    #size-cells = <2>;
    ranges;

    cmem_block_mem_0: cmem_block_mem@80000000 {
    reg = <0x0 0x80000000 0x0 0x0c000000>;
    no-map;
    status = "okay";
    };

    cmem_block_mem_1_ocmc3: cmem_block_mem@40500000 {
    reg = <0x0 0x40500000 0x0 0x100000>;
    no-map;
    status = "okay";
    };
    };

    cmem {
    compatible = "ti,cmem";
    #address-cells = <1>;
    #size-cells = <0>;

    #pool-size-cells = <2>;

    status = "okay";

    cmem_block_0: cmem_block@0 {
    reg = <0>;
    memory-region = <&cmem_block_mem_0>;
    cmem-buf-pools = <1 0x0 0x0c000000>;
    };

    cmem_block_1: cmem_block@1 {
    reg = <1>;
    memory-region = <&cmem_block_mem_1_ocmc3>;
    };
    };
    };

    - Renamed the am57xx-evm.dts to am57xx-evm.dtb and tried to run the mentioned command, which leeds to the attached output:
    audis@AuDisDev:~/ti-processor-sdk-linux-am57xx-evm-02.00.01.07/board-support/linux-4.1.13+gitAUTOINC+8dc66170d9-g8dc6617/arch/arm/boot/dts$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- am57xx-evm.dtb
    make: Nothing to be done for `am57xx-evm.dtb'.
    audis@AuDisDev:~/ti-processor-sdk-linux-am57xx-evm-02.00.01.07/board-support/linux-4.1.13+gitAUTOINC+8dc66170d9-g8dc6617/arch/arm/boot/dts$

    It doesn't matter what I do in am57xx-evm.dtb, I cannot run the command. What am I doing wrong here?
  • Hi, Kevin,

    For kernel newer than v3.14, it takes the CMEM from DTS. However, from quick browsing through the source code, the command line should still work. I will need some time to look into it as why it is broken. For now, please use the dts modification for your need. I also noticed that the device tree binding document is missing. I'll submit an internal request to have it covered.

    To build the dtb file, don't need to rename the dts to dtb, but run the make command

         make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- am57xx-evm.dtb

    That will build the dtb file.

    Rex

  • I tried what you have said, but the result is the same => "Nothing to be done"

    Is there something missing, what I have to do before I am modifing my DTSI / DTS file?
  • By the way, is my entry correct for my DTSI file? Placing the CMEM memory at 0x80000000? Or should it be at 0xBA300000 like described here ( e2e.ti.com/.../1541256 )?

    #define DSP_MEM_IOBUFS 0x80000000 // This is the virtual address, can be anywhere of your choosing
    #define PHYS_MEM_IOBUFS 0xBA300000 // This must match the base of the reserved physical block
    #define DSP_MEM_IOBUFS_SIZE (SZ_1M * 90) // This must match the size of the reserved physical block
    {
    TYPE_DEVMEM,
    DSP_MEM_IOBUFS, PHYS_MEM_IOBUFS,
    DSP_MEM_IOBUFS_SIZE, 0, 0, "DSP_MEM_IOBUFS",
    },
  • Hi, Kevin,

    Ah, I see it. You were building from dts directory. It should be the top level in linux.

    physical address 0x80000000 may have a problem. You need to chose an area not used by kernel. Please see Rob's comment in the thread you mentioned: "CMEM must be given memory that is completely unused by the kernel or any kernel component. ". Actually, I see in am57xx-beagle-x15.dts, the memory is defined as 0x8000 0000 size 0x8000 0000 if I am not mistaken. That may need to change as well.

    Rex

  • Thanks Rex, this works!
    But can you tell me which memory section I could try, so the ARM and also the DSP will be happy??? Can you give me a suggestion, because I am a bit confused how to decide where the memory has to be, without knowing which memory would be allowed to use.
  • Hi, Kevin,

    I did some tests and added the following cmem entries in the dtsi file to reserve physical address 0xb000 0000 for size 0x40000 which will be used as 4x(0x10000) pool.

            reserved-memory {
                    #address-cells = <2>;
                    #size-cells = <2>;
                    ranges;

                    :::::::::::::::    
                    cmem_block_mem_2_test: cmem_block_mem@b0000000 {
                            reg = <0x0 0xb0000000 0x0 0x040000>;
                            no-map;
                            status = "okay";
                    };
            };

            cmem {
                    compatible = "ti,cmem";
                    #address-cells = <1>;
                    #size-cells = <0>;

                    #pool-size-cells = <2>;

                    status = "okay";

                    ::::::::::::::::::::::::
                    cmem_block_2: cmem_block@2 {
                            reg = <2>;
                            memory-region = <&cmem_block_mem_2_test>;
                            cmem-buf-pools = <4 0x0 0x00010000>;
                    };
            };

    This is what I get after kernel boots up:

    root@am57xx-evm:~# cat /proc/iomem
    40500000-405fffff : CMEM
    40d01000-40d010ff : /ocp/mmu@40d01000
    ::::::::::::::::::::::::::::::::::::::::::::
    80000000-9fffffff : System RAM
      80008000-80874073 : Kernel code
      808c8000-809537cb : Kernel data
    a0000000-abffffff : CMEM
    ac000000-afffffff : System RAM
    b0000000-b003ffff : CMEM
    b0040000-ffcfffff : System RAM
    root@am57xx-evm:~#

    What's confusion about the statement "CMEM must be given memory that is completely unused by the kernel or any kernel component" is that it means the areas used by kernel code/data and other components as shown in the dump above. Beyond 0x809537cb, it seems to be free to use.

    Rex

  • I decided to take a block at 0xBA300000, so my IOMEM looks like this:
    80000000-ba2fffff : System RAM
    80008000-80874073 : Kernel code
    808c8000-809537cb : Kernel data
    ba300000-c62fffff : CMEM
    c6300000-ffcfffff : System RAM


    My CMEM looks like this:
    Block 0: Pool 0: 1 bufs size 0xc000000 (0xc000000 requested)

    Pool 0 busy bufs:

    Pool 0 free bufs:
    id 0: phys addr 0xba300000


    But when I try to use CMEM_alloc, it succeed with an address i don't know where it came from and if I try to do a memcpy with this address, it leads to a segmentation fault.
    CMEM_alloc okay (buf: 0xa0161000)
    CMEM_getPhys okay (payloadPhys: ba300000)

    Shouldn't it be around 0xba300000? Or how does CMEM decide to use its virtual address range?

    I thought to use 0xba300000 because of the entry in RSC table:
    #define PHYS_MEM_IOBUFS 0xBA300000

    Is that making sense or would be the address of this better?
    #define DSP_MEM_IOBUFS 0x80000000

    But that is right in the Kernel Code, as you can see in my IOMEM stat. Shouldn't that address be different to the Kernel Code Mem? I mean, shouldn't be the physical memory of the DSP not also somewhere else like right after 0x809537c? I still don't understand, why my example is not working.
  • Hi, Kevin,

    Do you get segmentation fault when coping to 0xa0161000? It shouldn't because that is the virtual address to 0xBA300000 and Linux cannot access physical address. I see there is another thread about the same question on addressing, so I'll link 2 together: e2e.ti.com/.../503515

    The command line approach is broken after moving the configuration to device tree. For now, it will need to use device tree before this is fixed.

    Rex
  • This is also a thread of mine...;) And still not answered yet...

    It seems like I can crash my DSP with using BA300000 after I sent the physical address to the DSP.
    I modified my ARM application, made it simpler like the host example from ex02messageq and I can show you following output:

    root@am57xx-evm:~# ./app_host
    --> main:
    [ 555.152713] NET: Registered protocol family 41
    --> Main_main:
    --> App_create:
    Local MessageQId: 0x80
    MessageQ_create OK!!
    Trying MessageQ_open...
    MessageQ_open OK!!
    App_create: Host is ready
    <-- App_create:
    --> App_exec:
    CMEM_alloc okay (buf: 0xaa645000)
    CMEM_getPhys okay (payloadPhys: ba300000)

    Copying Data OK!!
    [ 556.188954] omap-iommu 40d01000.mmu: iommu fault: da 0xba300000
    flags 0x0
    [ 556.200805] remoteproc2: crash detected in 40800000.dsp: type mmufault
    [ 556.207457] omap-iommu 40d01000.mmu: 40d01000.mmu: errs:0x00000002 da:0xba300
    000 pgd:0xeb7aee8c *pgd:px00000000
    [ 556.217872] remoteproc2: handling crash #1 in 40800000.dsp

    Trying MessageQ_put...
    MessageQ_put okay!
    Trying MessageQ_get...
    [ 556.228821] remoteproc2: recovering 40800000.dsp
    [ 556.261598] omap_hwmod: mmu1_dsp1: _wait_target_disable failed
    [ 556.274453] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 556.280503] remoteproc2: stopped remote processor 40800000.dsp
    [ 556.294766] remoteproc2: powering up 40800000.dsp
    [ 556.299623] remoteproc2: Booting fw image dra7-dsp1-fw.xe66, size 4584968
    [ 556.313543] omap_hwmod: mmu0_dsp1: _wait_target_disable failed
    [ 556.319486] omap-iommu 40d01000.mmu: 40d01000.mmu: version 3.0
    [ 556.325387] omap-iommu 40d02000.mmu: 40d02000.mmu: version 3.0
    [ 556.340452] remoteproc2: remote processor 40800000.dsp is now up
    [ 556.347575] virtio_rpmsg_bus virtio2: rpmsg host is online
    [ 556.353304] remoteproc2: registered virtio2 (type 7)
    [ 556.359430] virtio_rpmsg_bus virtio2: creating channel rpmsg-proto addr 0x3d
    [ 556.366740] omap-rproc 40800000.dsp: received echo reply from 40800000.dsp


    I cannot see my traces for the DSP because it restarted again. So whats wrong here? I can't figure out, where it the crash happens. I did this in my DSP:

    /* Use a static message for outMsg: no need to call MessageQ_alloc(): */
    MessageQ_staticMsgInit((MessageQ_Msg)&inMsg, sizeof(AppMsg));

    while (running)
    {

    Log_print0(Diags_ENTRY | Diags_INFO, "--> MessageQ_get waiting...");
    /* wait for inbound message */
    status = MessageQ_get(messageQ, (MessageQ_Msg *)&inMsg, MessageQ_FOREVER);
    if (status < 0)
    {
    Log_print0(Diags_ENTRY | Diags_INFO, "--> MessageQ_get not working!!!");
    goto leave;
    }
    Log_print0(Diags_ENTRY | Diags_INFO, "--> MessageQ_get okay!");

    Log_print0(Diags_ENTRY | Diags_INFO, "--> Resource_physToVirt waiting...");
    #if 1
    status = Resource_physToVirt(inMsg->inBuf, &va);
    if (status == Resource_S_SUCCESS)
    {
    Log_print0(Diags_ENTRY | Diags_INFO, "--> Resource_physToVirt okay!");
    data_from_host = va; // you can map any data type here, I've chosen an array of UInt32, but could be a struct *
    for(i = 0; i < 10; i++)
    Log_print1(Diags_ENTRY | Diags_INFO, "--> data_from_host: %x\n",data_from_host[i]);
    }
    else
    Log_print0(Diags_ENTRY | Diags_INFO, "--> Resource_physToVirt not working!!!");

    #endif

    Log_print1(Diags_ENTRY | Diags_INFO, "--> inMsg (%p)\n",(unsigned char*)inMsg);
    //Log_print1(Diags_ENTRY | Diags_INFO, "--> 0xBA300000: %02x\n",*((unsigned char*)(0xBA300000)));
    #if 1
    /* send message back */
    remoteQueueId = MessageQ_getReplyQueue(inMsg);
  • I solved the problem and described it here
    e2e.ti.com/.../503515