• State TI Thinks Resolved
  • Locked Locked
  • Replies 8 replies
  • Answers 1 answer
  • Subscribers 99 subscribers
  • Views 1882 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux: Memory leaks in AM57x kernel

Vikash Singh Rawat
Prodigy 40 points
Other Parts Discussed in Thread: AM5728

Tool/software: Linux

I am using the TI AM57xx Sitara board. But after running the kmemleak,  I am getting the following memory leaks in the kernel. The Linux kernel version is 4.9.28 and TI SDK is ti-processor-sdk-linux-am57xx-evm-04.00.00.04 .  What is the reason for this leaks and how to remove them?

root@am57xx-evm:~# cat /sys/kernel/debug/kmemleak
unreferenced object 0xedc16e00 (size 64):
  comm "swapper/0", pid 1, jiffies 4294937364 (age 455.870s)
  hex dump (first 32 bytes):
    44 18 05 c1 00 00 00 00 01 00 00 00 01 00 00 00  D...............
    00 00 00 00 44 18 05 c1 cc d3 01 c1 cc d3 01 c1  ....D...........
  backtrace:
    [<c031d590>] __kmalloc+0x194/0x210
    [<c0391438>] __register_sysctl_table+0x58/0x630
    [<c0391a30>] register_sysctl+0x20/0x24
    [<c0e132a4>] user_namespace_sysctl_init+0x1c/0x48
    [<c02017bc>] do_one_initcall+0x4c/0x178
    [<c0e00f68>] kernel_init_freeable+0x1d8/0x268
    [<c08c6550>] kernel_init+0x10/0x110
    [<c0207c88>] ret_from_fork+0x14/0x2c
    [<ffffffff>] 0xffffffff
unreferenced object 0xd5650000 (size 1024):
  comm "kworker/0:0", pid 4, jiffies 4294938272 (age 446.860s)
  hex dump (first 32 bytes):
    01 00 00 52 01 00 00 52 01 00 00 52 01 00 00 52  ...R...R...R...R
    01 00 00 52 01 00 00 52 01 00 00 52 01 00 00 52  ...R...R...R...R
  backtrace:
    [<c031d9c4>] kmem_cache_alloc+0x174/0x1d8
    [<c05b8214>] iopte_alloc+0x5c/0xc8
    [<c05b888c>] iopte_alloc_large+0x34/0xb4
    [<c05b8160>] omap_iommu_map+0x1a0/0x1f8
    [<c05b559c>] iommu_map+0x10c/0x180
    [<bf0354b8>] rproc_handle_devmem+0x7c/0x12c [remoteproc]
    [<bf035890>] rproc_handle_resources+0x64/0xe8 [remoteproc]
    [<bf036cac>] __rproc_boot+0x1c4/0x5bc [remoteproc]
    [<bf0370d0>] rproc_auto_boot_callback+0x18/0x24 [remoteproc]
    [<c0637a90>] request_firmware_work_func+0x44/0x6c
    [<c024483c>] process_one_work+0x1dc/0x3f8
    [<c0245494>] worker_thread+0x58/0x574
    [<c024a7cc>] kthread+0x100/0x118
    [<c0207c88>] ret_from_fork+0x14/0x2c
    [<ffffffff>] 0xffffffff
unreferenced object 0xd5650800 (size 1024):
  comm "kworker/0:0", pid 4, jiffies 4294938272 (age 446.860s)
  hex dump (first 32 bytes):
    01 00 00 52 01 00 00 52 01 00 00 52 01 00 00 52  ...R...R...R...R
    01 00 00 52 01 00 00 52 01 00 00 52 01 00 00 52  ...R...R...R...R
  backtrace:
    [<c031d9c4>] kmem_cache_alloc+0x174/0x1d8
    [<c05b8214>] iopte_alloc+0x5c/0xc8
    [<c05b888c>] iopte_alloc_large+0x34/0xb4
    [<c05b8160>] omap_iommu_map+0x1a0/0x1f8
    [<c05b559c>] iommu_map+0x10c/0x180
    [<bf0354b8>] rproc_handle_devmem+0x7c/0x12c [remoteproc]
    [<bf035890>] rproc_handle_resources+0x64/0xe8 [remoteproc]
    [<bf036cac>] __rproc_boot+0x1c4/0x5bc [remoteproc]
    [<bf0370d0>] rproc_auto_boot_callback+0x18/0x24 [remoteproc]
    [<c0637a90>] request_firmware_work_func+0x44/0x6c
    [<c024483c>] process_one_work+0x1dc/0x3f8
    [<c0245494>] worker_thread+0x58/0x574
    [<c024a7cc>] kthread+0x100/0x118
    [<c0207c88>] ret_from_fork+0x14/0x2c
    [<ffffffff>] 0xffffffff
unreferenced object 0xd5651000 (size 1024):
  comm "kworker/0:0", pid 4, jiffies 4294938272 (age 446.860s)
  hex dump (first 32 bytes):
    01 00 30 40 01 00 30 40 01 00 30 40 01 00 30 40  ..0@..0@..0@..0@
    01 00 30 40 01 00 30 40 01 00 30 40 01 00 30 40  ..0@..0@..0@..0@
  backtrace:
    [<c031d9c4>] kmem_cache_alloc+0x174/0x1d8
    [<c05b8214>] iopte_alloc+0x5c/0xc8
    [<c05b888c>] iopte_alloc_large+0x34/0xb4

  • 0
    TI__Guru**** 393215 points
    The software team have been notified. They will respond here.
  • 0
    TI__Guru 65405 points

    Also could you specify the kernel config change you have made?
    Is the only changes "Kernel memory leak detector" enabled (CONFIG_DEBUG_KMEMLEAK = Y)?

    BR
    Tsvetolin Shulev

  • 0 in reply to Cvetolin Shulev-XID
    Prodigy 40 points

    Hi Tsvetolin Shulev,

    Thank you for the response.

    I am sharing the kernel configuration file.

    7282.config.txt

  • 0 in reply to Vikash Singh Rawat
    TI__Guru 65405 points
    Vikash,

    I reproduced the issue with the last Processor SDK release 04.01.00.06. The issue needs additional investigation.

    BR
    Tsvetolin Shulev
  • 0 in reply to Cvetolin Shulev-XID
    Prodigy 40 points
    Tsvetolin Shulev any updates regarding this issue?
  • 0 in reply to Vikash Singh Rawat
    TI__Mastermind 30736 points
    Vikash,

    I too have replicated the results on an AM5728 IDK. Digging into it a bit more, the leaks seem to be associated with remoteproc and the interprocessor communication it is being used for (between the ARM and DSP, for example). If I clear the kmemleak log and let the system run for a while, I show not further errors. Do you see similar results?

    I'm consulting with our kernel developers to get their input on this and should provide some more detail soon.
  • 0 in reply to RonB
    Prodigy 40 points

    RonB,

    As you told that after clearing the kmemleak log, you are not getting the further memory leaks. I have also cleared the logs by using the command  # echo clear  > /sys/kernel/debug/kmemleak and it shows not further leak.

    But I found in Linux Documentation for kmemleaks that the above command (echo clear >  /sys/kernel/debug/kmemleak) clears the  list of current memory leaks.

    New leaks (if exist) will then come up upon reading /sys/kernel/debug/kmemleak again, not the previous one.

  • 0 in reply to Vikash Singh Rawat
    TI__Mastermind 30736 points

    Vikash,

    I've confirmed with our developer that these are false positives. He'll need to see why kmemleak is complaining and mark these allocations to be ignored in a future update.

    Remoteprocs are loaded and booted during the initial bootup and does allocate some of these slab cache entries for IOMMU page table entries. They get freed when you stop the remote processors. So a real memory leak around these should check the behavior before and after a load/start and shutdown combination.

    In the test that I ran, which sounds similar to yours, the remote processors (DSPs, for example) have been brought up and are running (or, waiting for something to do). It is only when they are properly shut down that this memory would be freed.

    The fact that clearing the logs helps to validate this and make sure that nothing else that is currently running is causing a problem.

    So, we will work in the future to try to make sure these don't show up at all, but they are not negatively impacting the system.