This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

tracebuf without Linux Kernel debug fs enabled

Hi,

Just want to know , if we could get all those tracebuf dumps without enabling Linux Kernel debug fs.

Now there is no problem for us to get tracebuf information with Linux Kernel enabled with debug file system. But in one of production environment we don't have debug fs enabled. So looking for a option if any of our Linux application can collect those tracebuf logs and dump it in absence of debug file system.

Thanks,

Rabi

  • Rabindra Paikaray said:
    So looking for a option if any of our Linux application can collect those tracebuf logs and dump it in absence of debug file system

    Rabi,

    There is no other mechanism besides remoteproc's debugfs support.

    If you somehow knew the physical address of a trace buffer then I suppose you could mmap(/dev/mem) to access it.  Typically, the trace buffer is placed in DDR memory, and CMA controls the resulting physical address.

    Regards,

    - Rob

  • Rob,

    Thanks for your reply. No idea how to find the physical address in this case for tracebuf. 

    But can't we follow the same principle of collecting the tracebuf from a separate kernel module to access the trace from IPU1.  How tracebuf able to read those memory from remoteproc driver.

    Any clue here.

    Thanks,

    Rabi

  • Rabindra Paikaray said:
    Thanks for your reply. No idea how to find the physical address in this case for tracebuf

    Yup, only remoteproc knows the phys addr of the trace buffer, unless you were to place it in internal RAM in which case you could look it up in a .map file since internal RAM doesn't have a corresponding virtual address.

    Rabindra Paikaray said:
    But can't we follow the same principle of collecting the tracebuf from a separate kernel module to access the trace from IPU1.  How tracebuf able to read those memory from remoteproc driver

    remoteproc reads the TYPE_TRACE resource table entry when it's loading the firmware, and stores the kernel virt addr corresponding to the trace buffer's remote virt addr (or device addr) in a trace entry attached to the remoteproc instance.  It then creates a "trace file" in the debugfs filsystem that can be read.

    remoteproc knows the right kernel virt addr since it's based on a previous TYPE_CARVEOUT entry from the resource table.

    It might be possible to write a custom kernel module to retrieve the trace record of IPU1's remoteproc instance, but I wouldn't know where to start looking.  If you had access to IPU1's platform_device then you could get the rproc instance and look at its rproc->traces list, eventually getting to the rproc_mem_entry for the trace and read from trace->va for trace->len bytes.  My kernel knowledge is limited in this area (general device handling), and remoteproc data structures are scoped tightly (i.e., declared static so can't be referenced by identifier from other kernel code).

    If you're so inclined you could look in the remoteproc kernel code:
        - <linux>/drivers/remoteproc/remoteproc_core.c (look for 'trace')
        - <linux>/drivers/remoteproc/omap_remoteproc.c (allocates rproc instance)
        - <linux>/drivers/remoteproc/remoteproc_debugfs.c (self-explanatory)
        - <linux>/arch/arm/mach-omap2/remoteproc.c (static data structs and system registration of them)

    Regards,

    - Rob

     

  • Thanks for your pointer .

    Looked in to the  suggested file . Thinking of this line from remoteproc_core.c file.

    /* what's the kernel address of this resource ? */
    ptr = rproc_da_to_va(rproc, rsc->da, rsc->len);

    Is it possible to use similar kind of approach  from user space to get the virtual address ?

    I could able to read this returned virtual address from an independent .ko to read the contents. But few problems here.

    1. How do I know how much valid data present in the memory.

    2. Once it wrote till the end of memory will it do a truncate of the memory. I mean will it start from the beginning again.

    3. Can we have a notification by any means when IPU1 writing to this TRACEBUF.

    Thanks,

    Rabi 

  • Rabindra Paikaray said:

    /* what's the kernel address of this resource ? */
    ptr = rproc_da_to_va(rproc, rsc->da, rsc->len);

    Is it possible to use similar kind of approach  from user space to get the virtual address ?

    No, because user space knows nothing about the trace buffer's physical address, which is what would be needed to do an mmap(/dev/mem) from user space.

    Rabindra Paikaray said:

    1. How do I know how much valid data present in the memory.

    The trace buffer is zeroed out before running the remote exe, so there is automatically a null-termintor at the end of the trace buffer output (except when wraparound occurs, see below).  So you could just pass the trace buffer starting address to printf() (or whatever else string processing you want to use).

    Rabindra Paikaray said:

    2. Once it wrote till the end of memory will it do a truncate of the memory. I mean will it start from the beginning again.

    It depends on the trace buffer properties.  Trace buffers can typically be configured to be either "fixed" or "continuous" (probably not the exact configuration wording).  In "fixed" the trace buffer will stop once it fills up completely, and in "continuous" it will wrap-around when it reaches the end of the buffer (and start overwriting the oldest output at the beginning of the buffer).  The IPC module <ipc>/packages/ti/trace/SysMin is not configurable in this way, and it uses a circular (wrap-around) buffer.  Older entries can be identified by the timestamp prefix on the entries.

    Rabindra Paikaray said:

    3. Can we have a notification by any means when IPU1 writing to this TRACEBUF.

    You could implement your own tracing provider based on the existing <ipc>/packages/ti/trace/SysMin module.  You could change it so that after a line is entered in the trace buffer, you send a Notify event (or MessageQ message) to the host, and when the user app receives this Notify event (or MessageQ message) it could somehow dump the new content (perhaps by keeping track of where it last left off reading the buffer).

    Regards,

    - Rob