Is DSPLink configuration valid?

Konstantin,

On a quick look, the DSPLink configuration seems okay.

Can you provide the CMEM configuration. It might help with debugging.

What DMA library are you using?

I'm not understanding your comment regarding notify. Are you saying it does not work?

Would you provide the link to the TI PSP you are using. Are you starting from an example in the PSP?

~Ramsey

Ramsey, thank you for your responce!

CMEM params: phys_start=0x90000000 phys_end=0x9FFFFFFF

I guess there is no need to add some 'pools' to params if I'm using only notify? Or I'm wrong?

Concerning DMA, I'm using 'omap' functions: omap_request_dma, omap_set_dma_params, omap_start_dma, omap_stop_dma. + dma_map_single for buffer pointing.

Yes, notify does not work. I have attached sources for dsp and gpp side. There is a bunch of commented lines from previous experiments with msgq.

6064.notify_dsp_side.zip

8206.notify_gpp_side.rar

All tools are from dvsdk_dm3730-evm_04_03_00_06_setuplinux

http://downloads.ti.com/dsps/dsps_public_sw/sdo_sb/targetcontent/dvsdk/DVSDK_4_00/latest/index_FDS.html

Linux kernel based on:

http://technexion.com/index.php/support/download-center/arm-cpu-modules/tdm-3730

And yes, my application was initially based on example form ti package.

HI Konstantin,

Regarding DMA, I assume you are having the issue on the GPP side and underlying EDMA3 transfers.

A couple things to consider are that:

1. EDMA3 hardware takes physical addresses as source and destination.
2. EDMA3 expects the source and destination memory to be contiguously allocated (in physical memory)

CMEM is a contiguous memory allocator which may explain why #2 is satisfied. I also assume when passing a CMEM allocated buffer you are passing the addresses translated to physical address, satisfying #1.

If you are trying to pass other memory you have to ensure 1 and 2. Are you able to verify that? 
For example, memory allocated (via dynamic or static allocation) will not be necessarily physically contiguous, it would appear contiguous via 'virtual addresses & MMU.

Murat

Hi raormanrat,

In both cases (when address passed to DMA is below or above 0x9a000000) destination memory is allocated by CMEM, so I hope I should not verify after CMEM's work. Source memory is not allocated at all ( = constant address for GPMC access).

So, after several days of another job, I have revised my application on both sides, cleaned some stuff out and fixed *.tci file for dsp-side. Then my app seems to work with bi-directional notifications. But some bug with cmem appears. 

[   23.463714] CMEMK module: built on Apr  1 2014 at 03:30:30
[   23.469696]   Reference Linux version 2.6.37
[   23.474151]   File /media/Work1/dvsdk/linuxutils_2_26_02_05/packages/ti/sdo/linuxutils/cmem/src/module/cmemk.c
[   23.485565] CMEMK Debug: Allocated major number: 251
[   23.491058] CMEMK Debug: calling request_mem_region(0x90000000, 268435456, "CMEM")
[   23.503356] CMEMK Debug: ioremap_nocache(0x90000000, 268435456)=0xd2000000
[   23.512573] CMEMK Debug: Allocating 1 buffers of size 268435456 (requested 268435456)
[   23.523620] CMEMK Debug: alloc_pool_buffer: Called for size 268435456
[   23.530364] CMEMK Debug: alloc_pool_buffer: Fits req 268435456 < avail: 268435456
[   23.538238] CMEMK Debug: alloc_pool_buffer: new available block size is 0
[   23.545318] CMEMK Debug: alloc_pool_buffer: returning allocated buffer at 0xd2000000
[   23.553436] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd2000000 to 0x90000000
[   23.562652] CMEMK Debug: Allocated buffer 0, virtual 0xd2000000 and physical 0x90000000 and size 268435456
[   23.572753] CMEMK Debug: Busylist for pool 0:
[   23.577331] CMEMK Debug: Freelist for pool 0:
[   23.581878] CMEMK Debug: Free: Buffer with id 0 and physical address 0x90000000
[   23.589538] allocated heap buffer 0xd2000000 of size 0x10000000
[   23.595764] CMEMK Debug: <6>cmem initialized 0 pools between 0x90000000 and 0xa0000000
[   23.604125] cmemk initialized
[   23.636413] DSPLINK Module (1.65.01.05_eng) created on Date: Mar 17 2014 Time: 19:51:03


Buffers allocation:

1) protection buffer

[   51.943817] CMEMK Debug: open: called.
[   51.948486] CMEMK Debug: GETVERSION ioctl received, returning 0x3000100.
[   51.956604] CMEMK Debug: ALLOCHEAP ioctl received on heap pool for block 0
[   51.965515] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd2000000 to 0x90000000
[   51.975891] CMEMK Debug: ALLOCHEAP: allocated 0x200000 size buffer at 0x90000000 (phys address)
[   51.985534] CMEMK Debug: mmap: vma->vm_start     = 0x403de000
[   51.991668] CMEMK Debug: mmap: vma->vm_pgoff     = 0x90000
[   51.997467] CMEMK Debug: mmap: vma->vm_end       = 0x405de000
[   52.003570] CMEMK Debug: mmap: size              = 0x200000
[   52.011230] CMEMK Debug: mmap: calling set_noncached(cb974bd0) ...
[   52.017913] CMEMK Debug: GETPHYS ioctl received.
[   52.022766] CMEMK Debug: get_phys: find_vma translated user 0x403de000 to 0x90000000
[   52.030883] CMEMK Debug: GETPHYS: returning 0x90000000

2) In-out buffers

[   52.036926] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.046142] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd2200000 to 0x90200000
[   52.056549] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x90200000 (phys address)
[   52.067230] CMEMK Debug: mmap: vma->vm_start     = 0x4061c000
[   52.073242] CMEMK Debug: mmap: vma->vm_pgoff     = 0x90200
[   52.079010] CMEMK Debug: mmap: vma->vm_end       = 0x419da000
[   52.085021] CMEMK Debug: mmap: size              = 0x13be000
[   52.091033] CMEMK Debug: mmap: calling set_cached(cb974b78) ...

[   52.098052] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.105957] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd35be000 to 0x915be000
[   52.115264] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x915be000 (phys address)
[   52.125122] CMEMK Debug: mmap: vma->vm_start     = 0x41a62000
[   52.131134] CMEMK Debug: mmap: vma->vm_pgoff     = 0x915be
[   52.136901] CMEMK Debug: mmap: vma->vm_end       = 0x42e20000
[   52.142913] CMEMK Debug: mmap: size              = 0x13be000
[   52.148925] CMEMK Debug: mmap: calling set_cached(cb974b20) ...

[   52.156097] CMEMK Debug: GETPHYS ioctl received.
[   52.160919] CMEMK Debug: get_phys: find_vma translated user 0x4061c000 to 0x90200000
[   52.169036] CMEMK Debug: GETPHYS: returning 0x90200000
[   52.174896] CMEMK Debug: GETPHYS ioctl received.
[   52.179748] CMEMK Debug: get_phys: find_vma translated user 0x41a62000 to 0x915be000
[   52.187866] CMEMK Debug: GETPHYS: returning 0x915be000

[   52.193420] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.202545] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd497c000 to 0x9297c000
[   52.213256] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x9297c000 (phys address)
[   52.224243] CMEMK Debug: mmap: vma->vm_start     = 0x42e60000
[   52.231536] CMEMK Debug: mmap: vma->vm_pgoff     = 0x9297c
[   52.237792] CMEMK Debug: mmap: vma->vm_end       = 0x4421e000
[   52.243835] CMEMK Debug: mmap: size              = 0x13be000
[   52.250122] CMEMK Debug: mmap: calling set_cached(cb974ac8) ...

[   52.257232] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.265075] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd5d3a000 to 0x93d3a000
[   52.274200] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x93d3a000 (phys address)
[   52.284027] CMEMK Debug: mmap: vma->vm_start     = 0x44275000
[   52.290069] CMEMK Debug: mmap: vma->vm_pgoff     = 0x93d3a
[   52.295806] CMEMK Debug: mmap: vma->vm_end       = 0x45633000
[   52.301818] CMEMK Debug: mmap: size              = 0x13be000
[   52.307830] CMEMK Debug: mmap: calling set_cached(cb974a70) ...

[   52.314880] CMEMK Debug: GETPHYS ioctl received.
[   52.319732] CMEMK Debug: get_phys: find_vma translated user 0x42e60000 to 0x9297c000
[   52.327972] CMEMK Debug: GETPHYS: returning 0x9297c000
[   52.333831] CMEMK Debug: GETPHYS ioctl received.
[   52.338684] CMEMK Debug: get_phys: find_vma translated user 0x44275000 to 0x93d3a000
[   52.346954] CMEMK Debug: GETPHYS: returning 0x93d3a000

[   52.352478] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.361694] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd70f8000 to 0x950f8000
[   52.372344] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x950f8000 (phys address)
[   52.383300] CMEMK Debug: mmap: vma->vm_start     = 0x456ff000
[   52.390594] CMEMK Debug: mmap: vma->vm_pgoff     = 0x950f8
[   52.396850] CMEMK Debug: mmap: vma->vm_end       = 0x46abd000
[   52.402862] CMEMK Debug: mmap: size              = 0x13be000
[   52.409027] CMEMK Debug: mmap: calling set_cached(cb974a18) ...

[   52.416107] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.423980] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd84b6000 to 0x964b6000
[   52.433074] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x964b6000 (phys address)
[   52.443084] CMEMK Debug: mmap: vma->vm_start     = 0x46b4d000
[   52.449127] CMEMK Debug: mmap: vma->vm_pgoff     = 0x964b6
[   52.454864] CMEMK Debug: mmap: vma->vm_end       = 0x47f0b000
[   52.460906] CMEMK Debug: mmap: size              = 0x13be000
[   52.466918] CMEMK Debug: mmap: calling set_cached(cb9749c0) ...

[   52.473907] CMEMK Debug: GETPHYS ioctl received.
[   52.478759] CMEMK Debug: get_phys: find_vma translated user 0x456ff000 to 0x950f8000
[   52.486877] CMEMK Debug: GETPHYS: returning 0x950f8000
[   52.492736] CMEMK Debug: GETPHYS ioctl received.
[   52.497558] CMEMK Debug: get_phys: find_vma translated user 0x46b4d000 to 0x964b6000
[   52.505676] CMEMK Debug: GETPHYS: returning 0x964b6000

[   52.511199] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.520568] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xd9874000 to 0x97874000
[   52.530975] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x97874000 (phys address)
[   52.541992] CMEMK Debug: mmap: vma->vm_start     = 0x47f95000
[   52.549316] CMEMK Debug: mmap: vma->vm_pgoff     = 0x97874
[   52.555603] CMEMK Debug: mmap: vma->vm_end       = 0x49353000
[   52.561614] CMEMK Debug: mmap: size              = 0x13be000
[   52.567749] CMEMK Debug: mmap: calling set_cached(cb974968) ...

[   52.574829] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.582702] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xdac32000 to 0x98c32000
[   52.591827] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x13be000 size buffer at 0x98c32000 (phys address)
[   52.601684] CMEMK Debug: mmap: vma->vm_start     = 0x49420000
[   52.607696] CMEMK Debug: mmap: vma->vm_pgoff     = 0x98c32
[   52.613464] CMEMK Debug: mmap: vma->vm_end       = 0x4a7de000
[   52.619476] CMEMK Debug: mmap: size              = 0x13be000
[   52.625671] CMEMK Debug: mmap: calling set_cached(cb974910) ...

[   52.632690] CMEMK Debug: GETPHYS ioctl received.
[   52.637512] CMEMK Debug: get_phys: find_vma translated user 0x47f95000 to 0x97874000
[   52.645629] CMEMK Debug: GETPHYS: returning 0x97874000
[   52.651489] CMEMK Debug: GETPHYS ioctl received.
[   52.656341] CMEMK Debug: get_phys: find_vma translated user 0x49420000 to 0x98c32000
[   52.664459] CMEMK Debug: GETPHYS: returning 0x98c32000

3) small buffer

[   52.669952] CMEMK Debug: ALLOCHEAPCACHED ioctl received on heap pool for block 0
[   52.679077] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xdbff0000 to 0x99ff0000
[   52.689941] CMEMK Debug: ALLOCHEAPCACHED: allocated 0x10000 size buffer at 0x99ff0000 (phys address)
[   52.700592] CMEMK Debug: mmap: vma->vm_start     = 0x4010d000
[   52.707916] CMEMK Debug: mmap: vma->vm_pgoff     = 0x99ff0
[   52.714172] CMEMK Debug: mmap: vma->vm_end       = 0x4011d000
[   52.720367] CMEMK Debug: mmap: size              = 0x10000
[   52.726318] CMEMK Debug: mmap: calling set_cached(cb9748b8) ...
[   52.732513] CMEMK Debug: GETPHYS ioctl received.
[   52.737365] CMEMK Debug: get_phys: find_vma translated user 0x4010d000 to 0x99ff0000
[   52.745483] CMEMK Debug: GETPHYS: returning 0x99ff0000

4) DMA allocation

[   52.751037] CMEMK Debug: ALLOCHEAP ioctl received on heap pool for block 0
[   52.759887] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xdc000000 to 0x9a000000
[   52.770446] CMEMK Debug: ALLOCHEAP: allocated 0x1800000 size buffer at 0x9a000000 (phys address)
[   52.780242] CMEMK Debug: mmap: vma->vm_start     = 0x4a7de000
[   52.786285] CMEMK Debug: mmap: vma->vm_pgoff     = 0x9a000
[   52.792022] CMEMK Debug: mmap: vma->vm_end       = 0x4bfde000
[   52.798065] CMEMK Debug: mmap: size              = 0x1800000
[   52.804107] CMEMK Debug: mmap: calling set_noncached(cb974860) ...
[   52.811553] CMEMK Debug: GETPHYS ioctl received.
[   52.816528] CMEMK Debug: get_phys: find_vma translated user 0x4a7de000 to 0x9a000000
[   52.824645] CMEMK Debug: GETPHYS: returning 0x9a000000

[   52.830566] CMEMK Debug: ALLOCHEAP ioctl received on heap pool for block 0
[   52.839233] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xdd800000 to 0x9b800000
[   52.849670] CMEMK Debug: ALLOCHEAP: allocated 0x1800000 size buffer at 0x9b800000 (phys address)
[   52.860260] CMEMK Debug: mmap: vma->vm_start     = 0x4bfea000
[   52.866271] CMEMK Debug: mmap: vma->vm_pgoff     = 0x9b800
[   52.872009] CMEMK Debug: mmap: vma->vm_end       = 0x4d7ea000
[   52.878051] CMEMK Debug: mmap: size              = 0x1800000
[   52.884063] CMEMK Debug: mmap: calling set_noncached(cb974808) ...
[   52.891510] CMEMK Debug: GETPHYS ioctl received.
[   52.896362] CMEMK Debug: get_phys: find_vma translated user 0x4bfea000 to 0x9b800000
[   52.904479] CMEMK Debug: GETPHYS: returning 0x9b800000

[   52.910522] CMEMK Debug: ALLOCHEAP ioctl received on heap pool for block 0
[   52.919189] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xdf000000 to 0x9d000000
[   52.929626] CMEMK Debug: ALLOCHEAP: allocated 0x1800000 size buffer at 0x9d000000 (phys address)
[   52.940216] CMEMK Debug: mmap: vma->vm_start     = 0x4d7fd000
[   52.946228] CMEMK Debug: mmap: vma->vm_pgoff     = 0x9d000
[   52.951965] CMEMK Debug: mmap: vma->vm_end       = 0x4effd000
[   52.958007] CMEMK Debug: mmap: size              = 0x1800000
[   52.964019] CMEMK Debug: mmap: calling set_noncached(cb9747b0) ...
[   52.971496] CMEMK Debug: GETPHYS ioctl received.
[   52.976348] CMEMK Debug: get_phys: find_vma translated user 0x4d7fd000 to 0x9d000000
[   52.984466] CMEMK Debug: GETPHYS: returning 0x9d000000

[   52.990356] CMEMK Debug: ALLOCHEAP ioctl received on heap pool for block 0
[   52.999023] CMEMK Debug: get_phys: block_virtoff[0] translated kernel 0xe0800000 to 0x9e800000
[   53.009796] CMEMK Debug: ALLOCHEAP: allocated 0x1800000 size buffer at 0x9e800000 (phys address)
[   53.020050] CMEMK Debug: mmap: vma->vm_start     = 0x4f079000
[   53.026092] CMEMK Debug: mmap: vma->vm_pgoff     = 0x9e800
[   53.031829] CMEMK Debug: mmap: vma->vm_end       = 0x50879000
[   53.037841] CMEMK Debug: mmap: size              = 0x1800000
[   53.043853] CMEMK Debug: mmap: calling set_noncached(cb974758) ...
[   53.051330] CMEMK Debug: GETPHYS ioctl received.
[   53.056182] CMEMK Debug: get_phys: find_vma translated user 0x4f079000 to 0x9e800000
[   53.064300] CMEMK Debug: GETPHYS: returning 0x9e800000

There i'm calling CMEM_free for protection buffer

[   54.070739] CMEMK Debug: FREEHEAP ioctl received.
[   54.075744] CMEMK Debug: get_phys: find_vma translated user 0x403de000 to 0x90000000
[   54.083984] CMEMK Debug: FREEHEAP: translated 0x403de000 user virtual to 0x90000000 physical
[   54.093383] CMEMK Debug: FREEHEAP: Removing file cb996ec0 from user list of buffer 0x90000000...
[   54.106109] CMEMK Debug: FREEHEAP: invalidated user virtual 0x403de000->0x405de000

Then I've added some additional debug:

[   54.114135] bi=0, entry->kvirtp = d2000000, size=2097152
[   54.119781] curHeader = bf00ff30, addr = d2000000
[   54.124786] curHeader->next =   (null)

And then NULL pointer dereference appears (in cmemk.c, HeapMem_free function)

[   54.130218] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[   54.139099] pgd = cbafc000
[   54.141998] [00000000] *pgd=8e919031, *pte=00000000, *ppte=00000000
[   54.148681] Internal error: Oops: 17 [#1]
[   54.152893] last sysfs file: /sys/devices/virtual/ti/lpm0/uevent
[   54.159240] Modules linked in: lpm_omap3530 dsplinkk cmemk p347_fpga
[   54.165985] CPU: 0    Not tainted  (2.6.37 #49)
[   54.170776] pc : [<bf00a970>]    lr : [<c00f5bc4>]    psr: 60000013
[   54.170806] sp : cba71e68  ip : cba71da8  fp : cba71e8c
[   54.182922] r10: 00000000  r9 : 00040000  r8 : bf00ff30
[   54.188446] r7 : bf00ff30  r6 : 00200000  r5 : 00000000  r4 : d2000000
[   54.195343] r3 : 60000013  r2 : 00000000  r1 : d2000000  r0 : bf00cb61
[   54.202239] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   54.209747] Control: 10c5387d  Table: 8bafc019  DAC: 00000015
[   54.215820]
[   54.215820] LR: 0xc00f5b44:
[   54.220336] 5b44  e3530000 b597300c b5873054 e5972054 ebfffd1d e1540005 e3e03000 e5873054
[   54.229034] 5b64  1affffd1 ea000006 e5932000 e1a04000 e5933004 e2433001 e1540005 1affffe7
[   54.237731] 5b84  e1a04000 e59f3040 e1a00004 e1a01005 e5932054 ebfffd0c e121f006 eaffffb6
[   54.246429] 5ba4  e3a02000 e59f001c e5832020 eb0064ba e121f006 e3580000 089da9f8 ebffff96
[   54.255126] 5bc4  e89da9f8 c04ed120 c04c4608 c04c45d0 e1a0c00d e92dd830 e24cb004 e1a04000
[   54.263824] 5be4  ebfffdab e59f207c e5923000 e1530004 05943034 05823000 0a000007 e3530000
[   54.272521] 5c04  15932034 1a000008 ea000009 e1520004 1a000003 e5942034 e5832034 e3a05000
[   54.281219] 5c24  ea000004 e1a03002 e5922034 e3520000 1afffff5 e3a05001 e59f3028 e5933000
[   54.289916]
[   54.289916] SP: 0xcba71de8:
[   54.294433] 1de8  5d363837 c00f0020 60000013 ffffffff 00000000 ffffffff cba71e54 00200000
[   54.303100] 1e08  bf00ff30 bf00ff30 cba71e8c cba71e20 c00c9c6c c00c41f4 bf00cb61 d2000000
[   54.311798] 1e28  00000000 60000013 d2000000 00000000 00200000 bf00ff30 bf00ff30 00040000
[   54.320495] 1e48  00000000 cba71e8c cba71da8 cba71e68 c00f5bc4 bf00a970 60000013 ffffffff
[   54.329193] 1e68  bec60a14 cba26a00 cba224c0 90000000 403de000 00000000 cba71ef4 cba71e90
[   54.337890] 1e88  bf00b344 bf00a914 90000000 c010ef6c cba26a08 00000000 cb996ec0 00200000
[   54.346588] 1ea8  c010f044 c010f00c cbaea080 c04b4f08 cba71eec 00000000 00000000 cba26a00
[   54.355285] 1ec8  cba71eec ce562718 00000003 bec60a14 00000003 c00ca284 cba70000 00000000
[   54.363952]
[   54.363983] IP: 0xcba71d28:
[   54.368469] 1d28  cba71d6c cba71d38 c00cf79c c00cf554 cba71d6c cba71d48 c02a278c 00000017
[   54.377166] 1d48  00000007 00000000 c04b6130 cba71e20 20000113 00000000 cba71e1c cba71d70
[   54.385864] 1d68  c00c4220 c00cf5e0 0000b472 0000b49c c04c45d0 60000093 cba71da4 cba71d90
[   54.394561] 1d88  c00f504c c00f4f6c c04ed120 0000b49c cba71dcc cba71da8 c00f5bc4 c00f5a2c
[   54.403259] 1da8  c04c45d0 0000002d c04ed1ec c04c45d0 0000000f 00000036 cba71e3c cba71dd0
[   54.411956] 1dc8  c00f5fd0 c00f5a60 00000036 60000013 00000004 205b1e64 34352020 3432312e
[   54.420623] 1de8  5d363837 c00f0020 60000013 ffffffff 00000000 ffffffff cba71e54 00200000
[   54.429321] 1e08  bf00ff30 bf00ff30 cba71e8c cba71e20 c00c9c6c c00c41f4 bf00cb61 d2000000
[   54.438018]
[   54.438018] FP: 0xcba71e0c:
[   54.442535] 1e0c  bf00ff30 cba71e8c cba71e20 c00c9c6c c00c41f4 bf00cb61 d2000000 00000000
[   54.451232] 1e2c  60000013 d2000000 00000000 00200000 bf00ff30 bf00ff30 00040000 00000000
[   54.459930] 1e4c  cba71e8c cba71da8 cba71e68 c00f5bc4 bf00a970 60000013 ffffffff bec60a14
[   54.468627] 1e6c  cba26a00 cba224c0 90000000 403de000 00000000 cba71ef4 cba71e90 bf00b344
[   54.477325] 1e8c  bf00a914 90000000 c010ef6c cba26a08 00000000 cb996ec0 00200000 c010f044
[   54.486022] 1eac  c010f00c cbaea080 c04b4f08 cba71eec 00000000 00000000 cba26a00 cba71eec
[   54.494720] 1ecc  ce562718 00000003 bec60a14 00000003 c00ca284 cba70000 00000000 cba71f04
[   54.503387] 1eec  cba71ef8 c015ab6c bf00acec cba71f7c cba71f08 c015b5bc c015ab4c cba71f84
[   54.512115]
[   54.512115] R1: 0xd1ffff80:
[   54.516601] ff80  ******** ******** ******** ******** ******** ******** ******** ********
[   54.525329] ffa0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.534027] ffc0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.542724] ffe0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.551422] 0000  ******** ******** ******** ******** ******** ******** ******** ********
[   54.560119] 0020  ******** ******** ******** ******** ******** ******** ******** ********
[   54.568817] 0040  ******** ******** ******** ******** ******** ******** ******** ********
[   54.577514] 0060  ******** ******** ******** ******** ******** ******** ******** ********
[   54.586212]
[   54.586212] R4: 0xd1ffff80:
[   54.590728] ff80  ******** ******** ******** ******** ******** ******** ******** ********
[   54.599426] ffa0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.608123] ffc0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.616821] ffe0  ******** ******** ******** ******** ******** ******** ******** ********
[   54.625518] 0000  ******** ******** ******** ******** ******** ******** ******** ********
[   54.634185] 0020  ******** ******** ******** ******** ******** ******** ******** ********
[   54.642883] 0040  ******** ******** ******** ******** ******** ******** ******** ********
[   54.651580] 0060  ******** ******** ******** ******** ******** ******** ******** ********
[   54.660278] Process dsp_test (pid: 850, stack limit = 0xcba702e8)
[   54.666717] Stack: (0xcba71e68 to 0xcba72000)
[   54.671325] 1e60:                   bec60a14 cba26a00 cba224c0 90000000 403de000 00000000
[   54.679992] 1e80: cba71ef4 cba71e90 bf00b344 bf00a914 90000000 c010ef6c cba26a08 00000000
[   54.688629] 1ea0: cb996ec0 00200000 c010f044 c010f00c cbaea080 c04b4f08 cba71eec 00000000
[   54.697296] 1ec0: 00000000 cba26a00 cba71eec ce562718 00000003 bec60a14 00000003 c00ca284
[   54.705932] 1ee0: cba70000 00000000 cba71f04 cba71ef8 c015ab6c bf00acec cba71f7c cba71f08
[   54.714569] 1f00: c015b5bc c015ab4c cba71f84 cba71f18 c010e71c c0413e58 0000c350 00000000
[   54.723236] 1f20: 0000c350 00000001 ce81baa1 c04c62a0 00000000 00000000 16de974d 00000036
[   54.731872] 1f40: 16ddd3fd 00000036 c010d8a4 c04c5e10 00000000 00000000 cb996ec0 bec60a14
[   54.740509] 1f60: 0004fe03 00000003 c00ca284 cba70000 cba71fa4 cba71f80 c015b644 c015b0ec
[   54.749176] 1f80: c010e834 00000000 403de000 00024554 bec60b08 00000036 00000000 cba71fa8
[   54.757812] 1fa0: c00ca100 c015b610 403de000 00024554 00000003 0004fe03 bec60a14 bec60b08
[   54.766448] 1fc0: 403de000 00024554 bec60b08 00000036 00000000 00000000 4010c000 bec60bb4
[   54.775115] 1fe0: 00000000 bec60a10 00019b8c 4037119c 20000010 00000003 00000000 00000000
[   54.783752] Backtrace:
[   54.786315] Function entered at [<bf00a908>] from [<bf00b344>]
[   54.792480] Function entered at [<bf00ace0>] from [<c015ab6c>]
[   54.798645] Function entered at [<c015ab40>] from [<c015b5bc>]
[   54.804809] Function entered at [<c015b0e0>] from [<c015b644>]
[   54.810974]  r9:cba70000 r8:c00ca284 r7:00000003 r6:0004fe03 r5:bec60a14
[   54.817901] r4:cb996ec0
[   54.820678] Function entered at [<c015b604>] from [<c00ca100>]
[   54.826812]  r7:00000036 r6:bec60b08 r5:00024554 r4:403de000
[   54.832824] Code: e51553c0 e59f00b8 e1a01004 e1a02005 (e5953000)
[   54.839965] ---[ end trace 3ab3f903c27ec969 ]---
[   54.856903] CMEMK Debug: close: called.
[   54.860992] CMEMK Debug: close: all references closed, force freeing all busy buffers.
[   54.869293] CMEMK Debug: Forcing free on pool 0

I'm confused, what I'm doing wrong now.

Hi Konstantin,

If I understand correctly, it looks like you allocated a protection buffer and then attempted to free it using CMEM_free, and that is when the crash occurred. Can you show us your code in HeapMem_free so that we can put your additional debug output into context?

A few things I can think of that you can try:

- Make sure that you are passing the same parameters to CMEM_free as you did for CMEM_alloc for the protection buffer.

- Does the CMEM_free call works without all those other memory allocations that happen after the protection buffer has been allocated? It'd be interesting to see if any of those allocations may have resulted in corruption in the module, though I don't see anything that jumps out from the trace.

- Are you calling CMEM_free in the same thread as CMEM_alloc? Or in the same function? Does your CMEM_free call work correctly when performed immediately after the allocation? In general, try to simplify your test case to see if you can better understand what needs to be fixed.

Best regards,
Vincent

Hi Konstantin,

I'm replying to your initial post since there are some unanswered questions...

Konstantin said:

1) memory is allocated by CMEM_alloc with CMEM_HEAP, not CMEM_POOL. I don't understand why.

and in a subsequent post...

Konstantin said:

CMEM params: phys_start=0x90000000 phys_end=0x9FFFFFFF

I guess there is no need to add some 'pools' to params if I'm using only notify? Or I'm wrong?

First, if you want to allocate with CMEM_POOL then the 'pools' parameter is required, since it sets up pools of buffers to be allocated with type CMEM_POOL.  After cmemk.ko configures the pools, any remaining memory from the phys block is given to the heap, which is allocated with type CMEM_HEAP.  In your case, since you have no pools, the whole of 0x90000000 -> 0x9FFFFFFF is a heap.  FYI, the CMEM 'pools' have no relationship to the DSPLINK POOLS.

Second, and more importantly, your CMEM phys block directly clashes with the DSPLINK configuration.  You're configuring DSPLINK to use the range 0x90000000 -> 0x9A000000.  Perhaps this explains why DMA works with only addresses above 0x9A000000.

Your CMEM phys block needs to be completely separate from the memory area(s) used for DSPLINK.  I assume that doing so will solve the issue of DMA not working with buffers < 0x9A000000.

This might also be the reason for the cmemk.ko crash (NULL pointer dereference).  For the CMEM heap, the freelist starts in a kernel-allocated data object but then continues in the free buffers themselves.  If DSPLINK is using the memory occupied by the free 'blocks' in the heap then it could be corrupting the freelist to the point that it follows a NULL pointer.

Regards,

- Rob

Vincent, here is my code in HeapMem_free (lines commented as "uki"):

void HeapMem_free(int bi, void *addr, size_t size)
{
//    long key;
    HeapMem_Header *curHeader, *newHeader, *nextHeader;
    size_t offset;

    /* obj->head never changes, doesn't need Gate protection. */
    curHeader = &heap_head[bi];
    printk("curHeader = %08p, addr = %08p\n",curHeader,addr); //uki
    /* Restore size to actual allocated size */
    if ((offset = size & (HEAP_ALIGN - 1)) != 0) {
        size += HEAP_ALIGN - offset;
    }
    printk("curHeader->next = %08p\n",curHeader->next); //uki
/*
 * We don't need to enter the "gate" since this function is called
 * with it held already.
 */
//    key = Gate_enterModule();

    newHeader = (HeapMem_Header *)addr;
    nextHeader = curHeader->next;
    printk("newHeader = %08p, nextHeader = %08p, nextHeader->next = %08p\n",newHeader,nextHeader,nextHeader->next); //uki

    /* Go down freelist and find right place for buf */
    while (nextHeader != NULL && nextHeader < newHeader) {
        curHeader = nextHeader;
        nextHeader = nextHeader->next;
        printk("next header is %08p\n",nextHeader->next); //uki
    }

    newHeader->next = nextHeader;
    newHeader->size = size;
    curHeader->next = newHeader;

    printk("newHeader->next = %08p, curHeader->next = %08p\n",newHeader->next,curHeader->next); //uki

    /* Join contiguous free blocks */
    /* Join with upper block */
    if ((nextHeader != NULL) && 
        (((unsigned long)newHeader + size) == (unsigned long)nextHeader)) {
        newHeader->next = nextHeader->next;
        newHeader->size += nextHeader->size;
    }

    /*
     *  Join with lower block. Make sure to check to see if not the
     *  first block.
     */
    if ((curHeader != &heap_head[bi]) &&
        (((unsigned long)curHeader + curHeader->size) == (unsigned long)newHeader)) {
        curHeader->next = newHeader->next;
        curHeader->size += newHeader->size;
    }
    
/*
 * See above comment on Gate_enterModule for an explanation of why we
 * don't use the "gate".
 */
//    Gate_leaveModule(key);
}

And concerning your recommendations:

1) Parameters surely are same (I've even modified code to use individual CMEM_AllocParams for each buffer).

2,3) Thread and functions are the same.

I have tried some changes that you're mentioned, and here is results:

* Alloc protection, then free. Alloc another buffer, use it in app (send addr via notify, check on DSP side, send notify back) runs OK once (at next application launch, it fails with DSP-side configuration mismatch - I guess because "shared" buffer was allocated right at unprotected RESETVECTOR place).

* Alloc protection, then alloc any another, then free protection (hmm, no hang), then free another... oops, same "Unable to handle kernel paging request at virtual address 001bd02a", but after next printk:

newHeader = d2200000, nextHeader = 0xd2000000, nextHeader->next = 001bd02a

Robert, maybe you're right, but I cannot completely separate CMEM and DSPLink addresses, because I need to use memory, shared between my app in Linux and my app in DSP. And looks like "NOTIFY" is enough in my case, because in one thread it would cycle like: 

1) Fill some data somewhere in shared space
2) Send getPhys of data address to DSP via notify, start waiting for responce
3) Catch pointer in DSP, call HAL_cacheInv, read data, send notify back.
4) Catch responce in GPP, loop again or exit.

Maybe I should reconsider memory ranges allocation and arrange it in different order...

Robert, I thought that dsplink should cover all shareable memory with one of LINKCFG_MemEntry sections... 

But you're definitely right considering "system" memory sections like RESETENTRY, CODEENTRY, etc. I have moved cmemk init start to 0x90200000 and I need no mad "protection" buffer anymore :) I was stupid :)

Anyway, seems I have discovered a reason that caused cmem fail in my case. It fails at first "free" when CMEM allocates ALL of heap memory it has. I have changed buffers sizes, it uses now 4 ram pages less, and no crashes in CMEM_free now! 

By the way, "POOLENTRYID" section of DSPLINK still overlaps CMEM range.