• Not Answered

LinuxUtils Cmem and linux 2.6.36 git

Dear all,

 

I am trying to use the linux git kernel 2.6.36 on our DM355 based platform. Everything was Ok until i tried to compile cmemk with this kernel. I installed the latest linux utils release :

- LinuxUtils 2.26.00.01

-LinuxUtils 2.26.01.02

But i am still getting the error when i compile cmemk :

WARNING: "arm926_dma_map_area"

Installing the module causes an error.

The cmemk module has a reference to that assembler function when it calls dmac_map_area. Which is defined in cacheflush.h

#define dmac_map_area            __glue(_CACHE,_dma_map_area)

This problem has been described in the kernel mailing list but never solved :

http://davinci-linux-open-source.1494791.n2.nabble.com/GIT-Kernel-and-CMEM-2-25-04-10-td5330175.html

The linuxutils that i am using are described as linux 2.6.36 and linux 2.6.37 releases so it should be possible to find a solution.

Does anyone faced that problem too ?

 

Thanks

 

--

Pierre

19 Replies

  • Hello Pierre,

    I've the same problem for dm365.

    I'm using linux kernel 2.6.37 and linux utils 2.26.01.02.

    WARNING: "arm926_dma_map_area" [/home/claudio/ti-dvsdk_dm365-evm_4_00_00_22/linuxutils_2_26_01_02/packages/ti/sdo/linuxutils/cmem/src/module/cmemk.ko] undefined!

     

    No solution at the moment.

     

    keep in touch

    regards

    Claudio

  • In reply to Claudio Filipponi:

    Hi,

     

    OK, So I am not alone !

    The asm command is present in the system.map, The warning appears at linking and when I try to load the module it says that the arm926_dma_map_area is an unknown symbol. So I cannot use the cmem module for the moment.

    Could it be a compile option for exporting the symbol ? I didn't see any of that for the moment.

    We need a linuxutils guru ! Someone who managed to compile and load Cmemk without this problem ...

     

    Regards,

    Pierre


  • In reply to Pierre19548:

    Hi,

    I tried a thing that seems to work, didn't fully test yet :

    I replaced every calls to dmac_map_area in cmemk.c:

                    dmac_map_area((void *)virtp, block.size, DMA_FROM_DEVICE);
                    outer_inv_range(__pa((u32)(void *)virtp),
                                    __pa((u32)(void *)virtp + block.size));

    by :

            __dma_single_cpu_to_dev((void *)virtp, block.size, DMA_FROM_DEVICE);

     

    This function is the api function that calls dmac_map_area  and  outer_***_range depending on the direction.

    This function is called also by dma_map_single, a function that performs a direction check and returns virt_to_dma for a given device, maybe this function should be use also ...

    I don't know exactly if what I've done is a good solution, but if it's the case maybe the linux utils should use the api function instead of rewriting the existing Kernel functions.

     

    Does anyone has a opinion on that ? how can we contact the linuxUtils Masters ?

    Regards,

     

    PS : Happy New year 2011 everyone !

  • In reply to Pierre19548:

    Oops I talked to fast...

    As it's not a linux memory there is a problem when we are calling ioctls.

    But  BUG_ON disabled it's working.

    I tested it and it seems to be ok.

     

    By the way, irqk needs to include : <linux/slab.h>    to compile.

    Is there where a place for the bug reports on linux utils ?

  • In reply to Pierre19548:

    Hi,

    I'm the engineer within TI that is responsible for maintaining Linux Utils, so you have contacted the right place.

    What is needed here is expert opinion/feedback from a Linux kernel expert, which I am not.  Cache/DMA support is quite complex in the Linux device driver world, especially with all the chip variants of the ARM processor line, and there is no real guidance that I have found when changes are made to kernel interfaces.  Guidance can be found in other Linux device driver implementations, but that leads to copying API usage that might have not been right in the first place.

    CMEM memory is different than pretty much all other memory handled by Linux, in that it lies outside the kernel's assigned physical memory space.  This makes it tough to find a decent model from which to base cache handling.

    The problem being encountered in this thread is not encountered by all users of later Linux kernels (2.6.36/2.6.37).  Here at TI, when developing and validaing CMEM we typically validate it against some stock installation of recent kernels, and for 2.6.36/2.6.37 we have validated CMEM for the newer TI chips TI816X/TI814X, and we (TI) never encountered an error such as this.  At this point I'm guessing that the error depends on the kernel configuration.  So far in this thread we have 2 different cases, both using basically the same chip (DM355/DM365).  I wonder if there is something specific to the configuration of the kernel for these chips that causes this problem.

    I can't look into this without first encountering it, so I will configure a 2.6.37 kernel for use with DM365 and see if I get the error, and if so what we can do about it.

    I don't think it's the correct solution to use __dma_single_cpu_to_dev(), since there is this comment above its declaration:
     * Private support functions: these are not part of the API and are
     * liable to change.  Drivers must not use these.
     */
    static inline void __dma_single_cpu_to_dev(const void *kaddr, size_t size,
            enum dma_data_direction dir)
    {

    Any and all advice regarding correct cache/DMA API calls inside CMEM are welcome.

    Regards,

    - Rob

     

  • In reply to Pierre19548:

    Pierre,

    I don't understand what you are saying in the first part of your message:

    Pierre

    Oops I talked to fast...

    As it's not a linux memory there is a problem when we are calling ioctls.

    But  BUG_ON disabled it's working.

    I tested it and it seems to be ok.

    Please explain.

    Also, thankyou for the report about irqk needing to #include <linux/slab.h>.  The TI forums are a good place for bug reports on TI software.  A customer FAE would also be a good reporting method, but I think the forums are better since everyone has a chance to see the bug report (and possible solution/workaround).

    Regards,

    - Rob

     

  • In reply to Robert Tivy:

    Robert,

     

    Thanks for your answer.

    Sorry for my explanations, I realised I didn't detailed it so much.

    Our current platform is DM355. I am trying to use the 2.6.36  kernel with the latest linux utils 2,26.01.02 if I remember.
    The arm926_dma_map_area is an unknow symbol at compilation ( this gives the warning of my first message). And installing the cmem module causes an unknown symbol error.

    The origin of the symbol is the dmac_map_area which is used in cmemk.c

    This symbol is never exported so cannot be used by kernel modules ( ?, at least this symbol is not visible for dm355 symbol).

    The dmac_map_area is also described in cacheflush.h as "do not use" (If MULTI_CACHE is enabled, depending on architecture, dm355 doesn't seem so)

    /*
    264 * These are private to the dma-mapping API. Do not use directly.
    265 * Their sole purpose is to ensure that data held in the cache
    266 * is visible to DMA, or data written by DMA to system memory is
    267 * visible to the CPU.
    268 */
    269 #define dmac_map_area

    But this function is also use in __dma_single_cpu_to_dev, ( an exported symbol, Yes it's dirty but i wanted to make it work...) This function calls the two functions :

                    dmac_map_area((void *)virtp, block.size, DMA_FROM_DEVICE);

                    outer_inv_range(__pa((u32)(void *)virtp),

                                    __pa((u32)(void *)virtp + block.size));

    Like cmemk.c does.

    But this function also calls the BUG_ON Macro, which assert that the adress space is visible for linux, as it is not the case for the moment with cmemk, i disabled ithis feature in the kernel ,.... And
    it "worked" (At least our application managed to capture and compress jpeg images correctly).

    The "recommended" function to be called is dma_map_single, which is using a device structure.

    I agree with you, this is not a good solution. As cmemk doesn't operate in linux visible memory, using dmac_map_area seems to be the only solution, the key is to manage to call it...

    As you said this might be a linux problem/cache problem, i will send a mail to the mailing list (linux-davinci) tomorow.


    Thanks for your help !

    ( Sorry my English, I'm french and it's quite late...)


  • In reply to Pierre Chevalier:

    Hello,

    is there any updates about this issue?

     

    Thanks

    Regards

    Claudio

  • In reply to Claudio Filipponi:

    Hi Claudio,

     

    I am using the modification i discribed before, which  is not a good solution but it is not a bad  solution either i think.

    We can see in cacheflush.h ( in arch/arm/include/asm/cacheflush.h) that the dmac_map_area macro which calls the arm926_dma_map_area assembler function is supposed to be a private dma-mapping function :

    /*
     * These are private to the dma-mapping API.  Do not use directly.
     * Their sole purpose is to ensure that data held in the cache
     * is visible to DMA, or data written by DMA to system memory is
     * visible to the CPU.
     */
    #define dmac_map_area            __glue(_CACHE,_dma_map_area)
    #define dmac_unmap_area        __glue(_CACHE,_dma_unmap_area)
    #define dmac_flush_range        __glue(_CACHE,_dma_flush_range)

    Its seems that dm355 uses that define and not  this one

    /*
     * These are private to the dma-mapping API.  Do not use directly.
     * Their sole purpose is to ensure that data held in the cache
     * is visible to DMA, or data written by DMA to system memory is
     * visible to the CPU.
     */
    #define dmac_map_area            cpu_cache.dma_map_area
    #define dmac_unmap_area        cpu_cache.dma_unmap_area
    #define dmac_flush_range        cpu_cache.dma_flush_range

    which is reserved for other cache architecture maybe the one, Robert  tested on.

    What I ve done,  (which works with my prototype ) is :

    replacing the calls to

                                dmac_map_area((void *)virtp, size, DMA_FROM_DEVICE);
                                outer_inv_range(__pa((u32)(void *)virtp), __pa((u32)(void *)virtp_end));


    by

               __cmem_dma_single_cpu_to_dev((void *)virtp, block.size, DMA_FROM_DEVICE);

     

    ( __cmem_dma_single_cpu_to_dev is a function based on __dma_single_cpu_to_dev and is described below)

    even if the arch/arm/include/asm/dma_mapping.h specifies that

    /*
     * The DMA API is built upon the notion of "buffer ownership".  A buffer
     * is either exclusively owned by the CPU (and therefore may be accessed
     * by it) or exclusively owned by the DMA device.  These helper functions
     * represent the transitions between these two ownership states.
     *
     * Note, however, that on later ARMs, this notion does not work due to
     * speculative prefetches.  We model our approach on the assumption that
     * the CPU does do speculative prefetches, which means we clean caches
     * before transfers and delay cache invalidation until transfer completion.
     *
     * Private support functions: these are not part of the API and are
     * liable to change.  Drivers must not use these.
     */

    for  the __dma_single_cpu_to_dev function

    This function is defined in arch/arm/mm/dma-mapping.c (with an underscore more at the beginning) :m

    void ___dma_single_cpu_to_dev(const void *kaddr, size_t size,enum dma_data_direction dir)
    {
        unsigned long paddr;

        BUG_ON(!virt_addr_valid(kaddr) || !virt_addr_valid(kaddr + size - 1));

        dmac_map_area(kaddr, size, dir);

        paddr = __pa(kaddr);
        if (dir == DMA_FROM_DEVICE) {
            outer_inv_range(paddr, paddr + size);
        } else {
            outer_clean_range(paddr, paddr + size);
        }
        /* FIXME: non-speculating: flush on bidirectional mappings? */
    }
    EXPORT_SYMBOL(___dma_single_cpu_to_dev);

    The problem with this function is  that the BUG_On instruction causes a crash

    BUG_ON(!virt_addr_valid(kaddr) || !virt_addr_valid(kaddr + size - 1));

    due to the fact that the cmem memory adress space is not in the kernel adress space. (and the bidirectional FIXME)

    So i added an exported function in this file :

    void ___cmem_dma_single_cpu_to_dev(const void *kaddr, size_t size, enum dma_data_direction dir){
        unsigned long paddr;
        dmac_map_area(kaddr, size, dir);
        paddr = __pa(kaddr);
        if (dir == DMA_FROM_DEVICE) {
            outer_inv_range(paddr, paddr + size);
        } else {
                if (dir == DMA_TO_DEVICE)
                       outer_clean_range(paddr, paddr + size);
                  else
                        outer_flush_range(paddr, paddr + size);
         }
    }
    EXPORT_SYMBOL(___cmem_dma_single_cpu_to_dev);

    and in the arch/arm/include/asm/dma_mapping.h file i added

    static inline void __cmem_dma_single_cpu_to_dev(const void *kaddr, size_t size,
        enum dma_data_direction dir)
    {
        extern void ___cmem_dma_single_cpu_to_dev(const void *, size_t,
            enum dma_data_direction);

        if (!arch_is_coherent())
            ___cmem_dma_single_cpu_to_dev(kaddr, size, dir);
    }

     

    I didn't have any other problems after that.

    As the kernel doesn't share the cmem memory, I didn't find other way to do ...

     

    I didn't have Robert Tivy's feedback about his attempt to compile for dm355 architecture.

     

    Regards

     

    Pierre

     

     

     

     

  • In reply to Pierre19548:

    Pierre

    I am using the modification i discribed before, which  is not a good solution but it is not a bad  solution either i think.

    At this point I'm of the impression that there is no good solution.

    Pierre

    We can see in cacheflush.h ( in arch/arm/include/asm/cacheflush.h) that the dmac_map_area macro which calls the arm926_dma_map_area assembler function is supposed to be a private dma-mapping function :

    /*
     * These are private to the dma-mapping API.  Do not use directly.
     * Their sole purpose is to ensure that data held in the cache
     * is visible to DMA, or data written by DMA to system memory is
     * visible to the CPU.
     */
    #define dmac_map_area            __glue(_CACHE,_dma_map_area)
    #define dmac_unmap_area        __glue(_CACHE,_dma_unmap_area)
    #define dmac_flush_range        __glue(_CACHE,_dma_flush_range)

    Its seems that dm355 uses that define and not  this one

    /*
     * These are private to the dma-mapping API.  Do not use directly.
     * Their sole purpose is to ensure that data held in the cache
     * is visible to DMA, or data written by DMA to system memory is
     * visible to the CPU.
     */
    #define dmac_map_area            cpu_cache.dma_map_area
    #define dmac_unmap_area        cpu_cache.dma_unmap_area
    #define dmac_flush_range        cpu_cache.dma_flush_range

    which is reserved for other cache architecture maybe the one, Robert  tested on.

    cacheflush.h contains a #define MULTI_CACHE.  When MULTI_CACHE is defined, the instance-based cpu_cache.* methodology is used, allowing support for more than one type of cache in a given kernel build.  When MULTI_CACHE is *not* defined, a single cache architecture is assumed, and the _CACHE macro is set to the name of that single architecture.  MULTI_CACHE gets defined when more than one CONFIG_CPU_* is defined, or when just one CONFIG_CPU_* is defined for a particular architecture, for instance, when CONFIG_CPU_V6 or CONFIG_CPU_V6 is defined.

    It seems that the non-MULTI_CACHE case no longer builds cleanly.  I have seen this same problem for both OMAPL138 and DM355/DM365.  For these architectures, functions named arm926_dma_map_area/arm926_dma_unmap_area/arm926_dma_flush_range are called by dmac_map_area/dmac_unmap_area/dmac_flush_range macros.  The arm926_* functions are defined in arch/arm/mm/proc-arm926.S, and I see that this file is compiled into a .o file, but apparently the proc-arm926.o file is not included in the final link, since symbols that it defines and exports are reported as "unresolved".  I need to look into this further.

    I recently supplied a "hack" to an internal customer to address this.  This "hack" is just an extension of a hack that has been present for a while - defining the assembly function in question in cmemk.c.  By stealing the function content from arch/arm/mm/proc-arm926.S, and putting it into an asm("") statement in the C file, I'm able to build cmemk.ko and insmod it successfully.  The internal customer reports that this hack has fixed the issue for them and their demo is now working.

    cmemk.c already has definitions of assembly functions arm926_dma_inv_range/arm926_dma_clean_range/arm926_dma_flush_range, which supports builds of cmemk.ko for older LINUX_VERSION_CODEs, for which cmemk.c called those functions.  Newer Linuxes allowed use of dmac_map_area/outer_*_range combinations, putting us where we are today.  I added this definition as well:
    asm("\n \
            .global arm926_dma_map_area\n \
    arm926_dma_map_area:\n \
            add     r1, r1, r0\n \
    @        cmp     r2, #DMA_TO_DEVICE\n \
            cmp     r2, #1\n \
            beq     arm926_dma_clean_range\n \
            bcs     arm926_dma_inv_range\n \
            b       arm926_dma_flush_range\n \
    ");

    These assembly functions are defined only when LINUX_VERSION_CODE < 2.6.34.  Since this problem is cropping up in LINUX_VERSION_CODE >= 2.6.34, this solution requires removing the line
        #if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,34)
    that is present just before the assembly code at the end of cmemk.c (and the corresonding #endif).  It is also requires changing the follwing #if from the 4 places where dmac_map_area() is called:
        #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,34)
                                dmac_map_area((void *)virtp, size, DMA_FROM_DEVICE);
                                outer_inv_range(__pa((u32)(void *)virtp),
                                                __pa((u32)(void *)virtp_end));
        #else
                                dmac_inv_range((void *)virtp, (void *)virtp_end);
        #endif
    by changing #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,34) to a plain dummy #if 1.

    Neither this nor Pierre's solution is clean, and I'm not sure which way cmemk.c will go in the future, but this Forum thread can help shape cmemk.c.

    Both solutions involve calling functions that are marked with "don't call directly" type of messages, but I don't see any alternative.  The functions that are "meant" for direct calls are not quite right for CMEMK memory.  I would much prefer to see Linux fixed for the non-MULTI_CACHE case, but certainly no fix will be done for 2.6.34 (an "old" kernel at this point, they're developing 2.6.38 now) so we must live with the best solution we can. 

    Regards,

    - Rob