• State
  • +1 person also asked this people also asked this
  • Locked Locked
  • Replies 6 replies
  • Subscribers 101 subscribers
  • Views 1085 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM6442: pressure on the eMMC leads to kernel panic

Pierre Buffo
Prodigy 110 points
Part Number: AM6442
Other Parts Discussed in Thread: TMDS64EVM

Hi,

I'm working on a custom board featuring an AM6442 and an eMMC (SFEM020GB2ED1TB from Swissbit). Currently I use an SD card to boot our customized yocto distribution. Under low load the eMMC seems to works fine. But writing bigger files continuously (eg: cp /dev/random /emmcmnt/test) leads to kernel panic as follow:

root@sepcos-cpu:/mnt# cp /dev/random >> /mnt/test2
[  671.800700] mmc0: running CQE recovery
[  671.810809] Unable to handle kernel paging request at virtual address 81c9e5008d1afa96
[  671.818814] Mem abort info:
[  671.821638]   ESR = 0x0000000096000004
[  671.822143] Unable to handle kernel paging request at virtual address 005335b677a61c7c
[  671.825413]   EC = 0x25: DABT (current EL), IL = 32 bits
[  671.825420]   SET = 0, FnV = 0
[  671.825424]   EA = 0, S1PTW = 0
[  671.825428]   FSC = 0x04: level 0 translation fault
[  671.825433] Data abort info:
[  671.825435]   ISV = 0, ISS = 0x00000004
[  671.825438]   CM = 0, WnR = 0
[  671.833596] Mem abort info:
[  671.838641] [81c9e5008d1afa96] address between user and kernel address ranges
[  671.838649] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP
[  671.838658] Modules linked in:
[  671.841791]   ESR = 0x0000000096000004
[  671.844824]  xhci_plat_hcd pci_endpoint_test rpmsg_ctrl rpmsg_char cdns3
[  671.849768]   EC = 0x25: DABT (current EL), IL = 32 bits
[  671.852566]  cdns_usb_common irq_pruss_intc icssg_prueth crct10dif_ce pru_rproc icss_iep hsr overlay phy_can_transceiver ti_k3_r5_remoteproc pci_j721e_host
[  671.857948] ------------[ cut here ]------------
[  671.859373]  pci_j721e pcie_cadence_host cdns3_ti pcie_cadence rti_wdt
[  671.862172] WARNING: CPU: 1 PID: 102 at mm/slab_common.c:923 free_large_kmalloc+0x8c/0xc4
[  671.869284]  virtio_rpmsg_bus
[  671.875530] Modules linked in:
[  671.878571]  rpmsg_ns
[  671.882304]  xhci_plat_hcd
[  671.888984]  ti_k3_m4_remoteproc
[  671.894277]  pci_endpoint_test
[  671.908151]  ti_k3_common
[  671.912749]  rpmsg_ctrl
[  671.919255]  k3_j72xx_bandgap
[  671.927408]  rpmsg_char
[  671.930362]  cfg80211
[  671.933400]  cdns3
[  671.935660]  rfkill
[  671.938353]  cdns_usb_common
[  671.941567]  sa2ul
[  671.944606]  irq_pruss_intc
[  671.947212]  pruss
[  671.949645]  icssg_prueth
[  671.952598]  ti_am335x_tscadc
[  671.955031]  crct10dif_ce
[  671.957291]  m_can_platform
[  671.959290]  pru_rproc
[  671.961377]  m_can
[  671.964243]  icss_iep
[  671.966243]  can_dev
[  671.969021]  hsr
[  671.971021]  rtc_rv3032
[  671.973627]  overlay
[  671.976580]  lm75
[  671.979186]  phy_can_transceiver
[  671.981967]  spi_omap2_mcspi
[  671.984313]  ti_k3_r5_remoteproc
[  671.986313]  optee_rng
[  671.988571]  pci_j721e_host
[  671.990744]  rng_core
[  671.992569]  pci_j721e
[  671.995004]  cryptodev(O)
[  671.997175]  pcie_cadence_host
[  671.999087]  fuse
[  672.002299]  cdns3_ti
[  672.005166]  drm
[  672.008378]  pcie_cadence
[  672.010725]  drm_panel_orientation_quirks
[  672.013503]  rti_wdt
[  672.015763]  ipv6
[  672.018108]  virtio_rpmsg_bus
[  672.020714] 
[  672.023753]  rpmsg_ns
[  672.025670] CPU: 0 PID: 607 Comm: cat Tainted: G           O       6.1.46-gf8110d9ce8 #1
[  672.027927]  ti_k3_m4_remoteproc
[  672.029754] Hardware name: Secheron test 9 (DT)
[  672.032359]  ti_k3_common
[  672.036354] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  672.038524]  k3_j72xx_bandgap
[  672.040438] pc : kmem_cache_free+0x5c/0x360
[  672.043391]  cfg80211
[  672.044871] lr : bvec_free+0x6c/0xbc
[  672.047130]  rfkill
[  672.055196] sp : ffff800008003d50
[  672.058407]  sa2ul
[  672.062920] x29: ffff800008003d50
[  672.065526]  pruss
[  672.072465]  x28: 0000000000000020
[  672.075418]  ti_am335x_tscadc
[  672.079585]  x27: 0000000000000000
[  672.081844]  m_can_platform
[  672.085403] 
[  672.087489]  m_can
[  672.090789] x26: ffff000006369c80
[  672.092789]  can_dev
[  672.096088]  x25: 0000000000000000
[  672.098088]  rtc_rv3032
[  672.101475]  x24: ffff000002fef100
[  672.104428]  lm75
[  672.107814] 
[  672.110594]  spi_omap2_mcspi
[  672.112074] x23: 0000000000001000
[  672.114074]  optee_rng
[  672.117374]  x22: ff80800008455c40
[  672.119546]  rng_core
[  672.122933]  x21: 81c9e5008d1afa96
[  672.125366]  cryptodev(O)
[  672.128752] 
[  672.130666]  fuse
[  672.132145] x20: ffff000001bd5000
[  672.135012]  drm
[  672.138312]  x19: ffff0000019b6d00
[  672.140658]  drm_panel_orientation_quirks
[  672.144045]  x18: 0000000000040193
[  672.146304]  ipv6
[  672.149690] 
[  672.152296] 
[  672.153776] x17: ffff800076964000
[  672.155690] CPU: 1 PID: 102 Comm: kworker/u4:2 Tainted: G           O       6.1.46-gf8110d9ce8 #1
[  672.158989]  x16: ffff800008000000
[  672.160815] Hardware name: Secheron test 9 (DT)
[  672.164201]  x15: 0000ffffa1b8f000
[  672.168197] Workqueue: writeback wb_workfn
[  672.171580] 
[  672.173494]  (flush-179:0)
[  672.174973] x14: 2cedab2461a805b8
[  672.176453] 
[  672.179753]  x13: 39db9250edf2f25a
[  672.188599] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  672.191985]  x12: 6867c5487405885c
[  672.196498] pc : free_large_kmalloc+0x8c/0xc4
[  672.199884] 
[  672.199886] x11: 0000000000000040
[  672.203963] lr : kfree+0x78/0x7c
[  672.205442]  x10: ffff800076964000
[  672.208134] sp : ffff800009ef3610
[  672.211434]  x9 : ffff000001808168
[  672.212914] x29: ffff800009ef3610
[  672.216300] 
[  672.223240]  x28: ffff00000e00b3b0
[  672.226626] x8 : ffff00007fbbd988
[  672.230966]  x27: 0000000000045800
[  672.232446]  x7 : 0000000000000000
[  672.235745] 
[  672.238958]  x6 : 0000000000000000
[  672.242344] x26: ffff000001955380
[  672.245644] 
[  672.249030]  x25: ffff00000d56f000
[  672.252330] x5 : fffffffffffffd10
[  672.253810]  x24: 0000000000000002
[  672.257197]  x4 : 0000000000000000
[  672.260495] 
[  672.263882]  x3 : ffff000001bd5000
[  672.267268] x23: 0000000000000800
[  672.268748] 
[  672.272134]  x22: ffff00000e00b288
[  672.275434] x2 : 000000000000002f
[  672.276914]  x21: 0000000000000000
[  672.280300]  x1 : 81c9e5008d1afa96
[  672.283600] 
[  672.286987]  x0 : 81c9e5008d1afa97
[  672.290373] x20: 0000000000000001
[  672.291853] 
[  672.295239]  x19: fffffc0000065540
[  672.298541] Call trace:
[  672.300019]  x18: 0000000000040193
[  672.303408]  kmem_cache_free+0x5c/0x360
[  672.306704] 
[  672.310091]  bvec_free+0x6c/0xbc
[  672.313477] x17: 0000000000000000
[  672.314957]  bio_free+0x58/0x80
[  672.318344]  x16: 0000000000000000
[  672.321644]  bio_put+0x114/0x1d0
[  672.323122]  x15: ffffffffffffffff
[  672.326509]  __read_end_io+0x3c/0x1a0
[  672.328942] 
[  672.332328]  mpage_end_io+0x20/0x44
[  672.336148] x14: ffffffffffffffff
[  672.337628]  bio_endio+0x178/0x190
[  672.340840]  x13: ffffffffffffffff
[  672.344140]  blk_update_request+0x1d8/0x394
[  672.347266]  x12: ffff00000deb5ff8
[  672.350652]  mmc_blk_cqe_complete_rq+0xfc/0x16c
[  672.353866] 
[  672.357252]  mmc_blk_mq_complete+0x2c/0x50
[  672.360897] x11: 0000000000000400
[  672.362377]  blk_complete_reqs+0x54/0x70
[  672.365850]  x10: ffff00000d56f000
[  672.369150]  blk_done_softirq+0x20/0x30
[  672.372536]  x9 : 0000000000000003
[  672.375923]  _stext+0x124/0x2a4
[  672.380088] 
[  672.383475]  ____do_softirq+0x10/0x20
[  672.387987] x8 : 000000006ce02b81
[  672.389467]  call_on_irq_stack+0x24/0x4c
[  672.393546]  x7 : 000000000000001c
[  672.396846]  do_softirq_own_stack+0x1c/0x30
[  672.400752]  x6 : 0000000000000000
[  672.404139]  __irq_exit_rcu+0xcc/0xf4
[  672.407958] 
[  672.411345]  irq_exit_rcu+0x10/0x20
[  672.414470] x5 : 0000000000000000
[  672.415950]  el1_interrupt+0x38/0x70
[  672.419596]  x4 : 0000000000316019
[  672.422895]  el1h_64_irq_handler+0x18/0x2c
[  672.426801]  x3 : fffffc0000065540
[  672.430187]  el1h_64_irq+0x64/0x68
[  672.434354] 
[  672.437741]  __arch_copy_from_user+0x1b8/0x230
[  672.441386] x2 : 1b7bb27fd0b9d375
[  672.442866]  generic_perform_write+0xe8/0x1e0
[  672.446339]  x1 : ffff000001955380
[  672.449638]  ext4_buffered_write_iter+0x80/0x130
[  672.453197]  x0 : 0403b74f6b0ad13e
[  672.456583]  ext4_file_write_iter+0x58/0x7c4
[  672.460662] 
[  672.464049]  vfs_write+0x220/0x2b0
[  672.467435] Call trace:
[  672.468915]  ksys_write+0x68/0xf4
[  672.473341]  free_large_kmalloc+0x8c/0xc4
[  672.476640]  __arm64_sys_write+0x1c/0x2c
[  672.480980]  kfree+0x78/0x7c
[  672.484366]  invoke_syscall+0x48/0x114
[  672.488965]  ext4_ext_map_blocks+0x200/0x12f0
[  672.492352]  el0_svc_common.constprop.0+0xd4/0xfc
[  672.496605]  ext4_map_blocks+0x214/0x5b0
[  672.498085]  do_el0_svc+0x30/0xd0
[  672.501471]  ext4_writepages+0x568/0xc70
[  672.503905]  el0_svc+0x2c/0x84
[  672.507204]  do_writepages+0x88/0x1b0
[  672.511197]  el0t_64_sync_handler+0xbc/0x140
[  672.515103]  __writeback_single_inode+0x38/0x1a0
[  672.517970]  el0t_64_sync+0x18c/0x190
[  672.521704]  writeback_sb_inodes+0x210/0x410
[  672.526048] Code: 9249e3d6 d1000401 f240001f 9a951035 (f94002a0) 
[  672.530730]  __writeback_inodes_wb+0x50/0x110
[  672.534641] ---[ end trace 0000000000000000 ]---
[  672.537936]  wb_writeback.isra.0+0x1b4/0x1c4
[  672.541845] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[  672.541851] SMP: stopping secondary CPUs
[  672.544882]  wb_workfn+0x1bc/0x3a0
[  672.548537]  process_one_work+0x1d0/0x320
[  672.552799]  worker_thread+0x14c/0x444
[  672.557407]  kthread+0x10c/0x110
[  672.561064]  ret_from_fork+0x10/0x20
[  672.565326] ---[ end trace 0000000000000000 ]---
[  672.571523] Kernel Offset: disabled
[  672.621432] CPU features: 0x00000,00800084,0000420b
[  672.626299] Memory Limit: none
[  672.629347] Rebooting in 5 seconds..

dmesg indeed confirms that CQE is enabled:

root@sepcos-cpu:~# dmesg | grep mmc0
[    1.446746] mmc0: CQHCI version 5.10
[    1.492136] mmc0: SDHCI controller on fa10000.mmc [fa10000.mmc] using ADMA 64-bit
[    1.638398] mmc0: Command Queue Engine enabled
[    1.642913] mmc0: new HS200 MMC card at address 0001
[    1.648828] mmcblk0: mmc0:0001 00010G 9.60 GiB 
[    1.656122] mmcblk0boot0: mmc0:0001 00010G 31.5 MiB 
[    1.662637] mmcblk0boot1: mmc0:0001 00010G 31.5 MiB 
[    1.669516] mmcblk0rpmb: mmc0:0001 00010G 4.00 MiB, chardev (240:0)

Here's what the ios file looks like:

root@sepcos-cpu:~# cat /sys/kernel/debug/mmc0/ios 
clock:		200000000 Hz
actual clock:	200000000 Hz
vdd:		7 (1.65 - 1.95 V)
bus mode:	2 (push-pull)
chip select:	0 (don't care)
power mode:	2 (on)
bus width:	3 (8 bits)
timing spec:	9 (mmc HS200)
signal voltage:	1 (1.80 V)
driver type:	0 (driver type B)

The eMMC schematics are identical to the TMDS64EVM, so is the DTB section for it.

My kernel is a 6.1 from the ti-linux-kernel project (commit f8110d9ce8019f576bb813cade4f66a04a0cd2a3). I should also mention that my custom kernel combined with TI's DTB for the EVM works well on the EVM itself.

My eMMC is supposed to support eMMC5.1 standard, thus supporting command queuing.

Any guess about where the issue might come from ?

Regards
Pierre

  • 0
    TI__Guru**** 167681 points

    Hi Pierre,

    Pierre Buffo said:
    The eMMC schematics are identical to the TMDS64EVM,

    The eMMC design on the EVM has a bug. Please check errata ID #7 in 

    https://www.ti.com/lit/pdf/spruiy9

    to see if this solves the issue.

  • 0 in reply to Bin Liu
    TI__Guru**** 167681 points

    Sorry, the EVM errata #7 is about  MMC1 (SD card). MMC0 (eMMC) should uses different pull-up/-down.

    I am looping our hardware expert for comments.

  • 0 in reply to Bin Liu
    Prodigy 110 points

    Hi Bin,

    Issue #3 however is about the careful matching of the MMC0 signals, I will double check our PCB design.

    Regards
    Pierre

  • 0 in reply to Pierre Buffo
    Prodigy 110 points

    A few more things I observed, sometimes during the boot this happen:

    [    1.752590] Run /sbin/init as init process
[    1.769873] mmc0: Command Queue Engine enabled
[    1.774539] mmc0: new HS200 MMC card at address 0001
[    1.780579] mmcblk0: mmc0:0001 00010G 9.60 GiB 
[    1.787626] mmcblk0boot0: mmc0:0001 00010G 31.5 MiB 
[    1.794448] mmcblk0boot1: mmc0:0001 00010G 31.5 MiB 
[    1.801030] mmcblk0rpmb: mmc0:0001 00010G 4.00 MiB, chardev (240:0)
[    1.930924] systemd[1]: System time before build time, advancing clock.
[    1.972103] vmap allocation for size 479232 failed: use vmalloc=<size> to increase size
[    1.980220] systemd: vmalloc error: size 475136, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0
[    1.992951] CPU: 0 PID: 1 Comm: systemd Not tainted 6.1.46-gf8110d9ce8 #1
[    1.999729] Hardware name: Secheron test 9 (DT)
[    2.004247] Call trace:
[    2.006686]  dump_backtrace.part.0+0xdc/0xf0
[    2.010967]  show_stack+0x18/0x30
[    2.014278]  dump_stack_lvl+0x68/0x84
[    2.017938]  dump_stack+0x18/0x34
[    2.021246]  warn_alloc+0x10c/0x1a4
[    2.024732]  __vmalloc_node_range+0x590/0x6e0
[    2.029081]  module_alloc+0xec/0x100
[    2.032654]  load_module+0xa10/0x1c70
[    2.036316]  __do_sys_finit_module+0xa8/0x100
[    2.040667]  __arm64_sys_finit_module+0x20/0x30
[    2.045191]  invoke_syscall+0x48/0x114
[    2.048935]  el0_svc_common.constprop.0+0x44/0xfc
[    2.053632]  do_el0_svc+0x30/0xd0
[    2.056941]  el0_svc+0x2c/0x84
[    2.059989]  el0t_64_sync_handler+0xbc/0x140
[    2.064252]  el0t_64_sync+0x18c/0x190
[    2.067943] Mem-Info:
[    2.070221] active_anon:1 inactive_anon:105 isolated_anon:0
[    2.070221]  active_file:349 inactive_file:1927 isolated_file:0
[    2.070221]  unevictable:0 dirty:0 writeback:0
[    2.070221]  slab_reclaimable:824 slab_unreclaimable:2889
[    2.070221]  mapped:1054 shmem:0 pagetables:12
[    2.070221]  sec_pagetables:0 bounce:0
[    2.070221]  kernel_misc_reclaimable:0
[    2.070221]  free:407184 free_pcp:561 free_cma:7825
[    2.108219] Node 0 active_anon:4kB inactive_anon:420kB active_file:1396kB inactive_file:7708kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4216kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:1216kB pagetables:48kB sec_pagetables:0kB all_unreclaimable? no
[    2.138247] DMA free:1628736kB boost:0kB min:22528kB low:28160kB high:33792kB reserved_highatomic:0KB active_anon:4kB inactive_anon:420kB active_file:1396kB inactive_file:7708kB unevictable:0kB writepending:0kB present:1835008kB managed:1666036kB mlocked:0kB bounce:0kB free_pcp:2244kB local_pcp:1184kB free_cma:31300kB
[    2.166364] lowmem_reserve[]: 0 0 0 0
[    2.170035] DMA: 2*4kB (MC) 3*8kB (EC) 4*16kB (UC) 3*32kB (EC) 2*64kB (MC) 2*128kB (EC) 2*256kB (UC) 3*512kB (UME) 2*1024kB (UE) 5*2048kB (MEC) 394*4096kB (MC) = 1628736kB
[    2.185412] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[    2.194103] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=32768kB
[    2.202623] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[    2.211052] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=64kB
[    2.219305] 2280 total pagecache pages
[    2.223049] 0 pages in swap cache
[    2.226360] Free swap  = 0kB
[    2.229236] Total swap = 0kB
[    2.232107] 458752 pages RAM
[    2.234983] 0 pages HighMem/MovableOnly
[    2.238814] 42243 pages reserved
[    2.242037] 8192 pages cma reserved
[    2.245520] 0 pages hwpoisoned

    Also here's another log when I try to read for a long period of time from the eMMC:

    [  206.352940] mmc0: running CQE recovery
[  206.357916] kauditd_printk_skb: 3 callbacks suppressed
[  206.357935] audit: type=1701 audit(1700809152.916:13): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=149 comm="systemd-journal" exe="/lib/systemd/systemd-journald" sig=4 res=1
[  206.380739] mmc0: running CQE recovery
[  206.398851] mmc0: running CQE recovery
[  206.417330] I/O error, dev mmcblk0, sector 1000448 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 2
[  206.575294] audit: type=1701 audit(1700809153.132:14): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=500 comm="systemd" exe="/lib/systemd/systemd" sig=4 res=1
[  206.638080] audit: type=1335 audit(1700809153.196:15): pid=1 uid=0 auid=4294967295 tty=(none) ses=4294967295 comm="systemd" exe="/lib/systemd/systemd" nl-mcgrp=1 op=disconnect res=1
[  206.769124] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000038
[  206.769579] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000038
[  206.778011] Mem abort info:
[  206.786715] Mem abort info:
[  206.789495]   ESR = 0x0000000096000005
[  206.792276]   ESR = 0x0000000096000006
[  206.796010]   EC = 0x25: DABT (current EL), IL = 32 bits
[  206.799741]   EC = 0x25: DABT (current EL), IL = 32 bits
[  206.805034]   SET = 0, FnV = 0
[  206.810327]   SET = 0, FnV = 0
[  206.813366]   EA = 0, S1PTW = 0
[  206.816405]   EA = 0, S1PTW = 0
[  206.819533]   FSC = 0x05: level 1 translation fault
[  206.822658]   FSC = 0x06: level 2 translation fault
[  206.827518] Data abort info:
[  206.832377] Data abort info:
[  206.835244]   ISV = 0, ISS = 0x00000005
[  206.838110]   ISV = 0, ISS = 0x00000006
[  206.838113]   CM = 0, WnR = 0
[  206.838117] user pgtable: 4k pages, 48-bit VAs, pgdp=00000000874ac000
[  206.838124] [0000000000000038] pgd=08000000876de003, p4d=08000000876de003, pud=0800000087452003, pmd=0000000000000000
[  206.838147] Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
[  206.838155] Modules linked in:
[  206.838165] CPU: 1 PID: 498 Comm: cp Tainted: G           O       6.1.46-gf8110d9ce8 #1
[  206.841979]   CM = 0, WnR = 0
[  206.845799] Hardware name: Secheron test 9 (DT)
[  206.848752] user pgtable: 4k pages, 48-bit VAs, pgdp=00000000875d8000
[  206.855173] pstate: 400000c5 (nZcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  206.865750] [0000000000000038] pgd=0800000087552003
[  206.871997] pc : kmem_cache_alloc_lru+0x1bc/0x510
[  206.875036] , p4d=0800000087552003
[  206.883016] lr : kmem_cache_alloc_lru+0xc4/0x510
[  206.885969] , pud=0000000000000000
[  206.890482] sp : ffff80000a34b6b0
[  206.896900] 
[  206.903841] x29: ffff80000a34b6b0 x28: ffff000004bd3918 x27: 0000000000000000
[  206.936664] x26: ffff0000076f0e40 x25: 0000000000000240 x24: 0000000000000001
[  206.943791] x23: ffff000001805100 x22: ffff000007578500 x21: ffff000004bd36d0
[  206.950919] x20: 0000000000402800 x19: 0000000000000000 x18: 0000000000038ac2
[  206.958046] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[  206.965173] x14: 0000000000000001 x13: 0000000000000002 x12: 0000000000000000
[  206.972300] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
[  206.979427] x8 : 0000000000000000 x7 : 0007ffffffffffff x6 : 0000000000000402
[  206.986554] x5 : 0000000000013c01 x4 : ffff00006fc55a10 x3 : 0000000000087001
[  206.993681] x2 : 0000000000000240 x1 : ffffffffffffffff x0 : 0000000000000000
[  207.000809] Call trace:
[  207.003247]  kmem_cache_alloc_lru+0x1bc/0x510
[  207.007597]  xas_alloc+0xb4/0xe4
[  207.010824]  xas_create+0x260/0x410
[  207.014306]  xas_store+0x50/0x550
[  207.017616]  __filemap_add_folio+0x1c4/0x410
[  207.021882]  filemap_add_folio+0x44/0xc4
[  207.025797]  page_cache_ra_unbounded+0xf4/0x1ec
[  207.030324]  page_cache_ra_order+0x88/0x310
[  207.034498]  ondemand_readahead+0x1a4/0x320
[  207.038672]  page_cache_async_ra+0x60/0xac
[  207.042760]  filemap_readahead.isra.0+0x50/0x90
[  207.047281]  filemap_get_pages+0x3f4/0x674
[  207.051371]  filemap_read+0xb4/0x360
[  207.054938]  blkdev_read_iter+0xc4/0x200
[  207.058857]  vfs_read+0x1f8/0x2a0
[  207.062168]  ksys_read+0x68/0xf4
[  207.065391]  __arm64_sys_read+0x1c/0x2c
[  207.069220]  invoke_syscall+0x48/0x114
[  207.072967]  el0_svc_common.constprop.0+0xd4/0xfc
[  207.077664]  do_el0_svc+0x30/0xd0
[  207.080974]  el0_svc+0x2c/0x84
[  207.084024]  el0t_64_sync_handler+0xbc/0x140
[  207.088286]  el0t_64_sync+0x18c/0x190
[  207.091948] Code: 9a810273 f9400260 7217001f 9a9f1273 (f9401e60) 
[  207.098029] ---[ end trace 0000000000000000 ]---
[  207.102633] Internal error: Oops: 0000000096000005 [#2] PREEMPT SMP
[  207.102634] note: cp[498] exited with irqs disabled
[  207.102640] Modules linked in:
[  207.108976] note: cp[498] exited with preempt_count 1
[  207.113745] 
[  207.113749] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D    O       6.1.46-gf8110d9ce8 #1
[  207.113757] Hardware name: Secheron test 9 (DT)
[  207.116853] ------------[ cut here ]------------
[  207.121829] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  207.123309] WARNING: CPU: 1 PID: 498 at mm/slab_common.c:923 free_large_kmalloc+0x8c/0xc4
[  207.131721] pc : kmem_cache_free+0x74/0x360
[  207.136234] Modules linked in:
[  207.140834] lr : bvec_free+0x6c/0xbc
[  207.147773] 
[  207.155926] sp : ffff800008003d70
[  207.160093] CPU: 1 PID: 498 Comm: cp Tainted: G      D    O       6.1.46-gf8110d9ce8 #1
[  207.163132] x29: ffff800008003d70
[  207.166691] Hardware name: Secheron test 9 (DT)
[  207.168171]  x28: 0000000000000020
[  207.171471] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  207.179451]  x27: 0000000000000000
[  207.182751] pc : free_large_kmalloc+0x8c/0xc4
[  207.187263] 
[  207.190649] lr : kfree+0x78/0x7c
[  207.197588] x26: ffff8000093ea9c0
[  207.200974] sp : ffff80000a34b210
[  207.205314]  x25: 0000000000000000
[  207.206793] x29: ffff80000a34b210
[  207.210007]  x24: 000000000000000a
[  207.213307]  x28: ffff80000a34b3b3
[  207.216607] 
[  207.219993]  x27: ffff800008f25e18
[  207.223293] x23: 0000000000000000
[  207.226679] 
[  207.230065]  x22: ffff80000845dc40
[  207.231545] x26: 0000000000000001
[  207.234931]  x21: 0000000000000000
[  207.238231]  x25: ffff8000082360d0
[  207.239711] 
[  207.243097]  x24: 00000000000000c0
[  207.246397] x20: ffff00000301ac00
[  207.249784] 
[  207.253170]  x19: ffff0000019b6d00
[  207.254650] x23: 0000000000000000
[  207.258036]  x18: 0000000000000000
[  207.261336]  x22: 000000000000000b
[  207.262815] 
[  207.266202]  x21: 0000000000000000
[  207.269502] x17: ffff8000669dc000
[  207.272888] 
[  207.276274]  x16: ffff800008000000
[  207.277754] x20: ffff000003af3800
[  207.281140]  x15: 000002d909437c6a
[  207.284440]  x19: fffffc00000ebcc0
[  207.285920] 
[  207.289306]  x18: ffff80000a34b238
[  207.292606] x14: 000000000000030b
[  207.295992] 
[  207.299378]  x13: 0000000000000001
[  207.300858] x17: 3030303030303030
[  207.304244]  x12: 0000000000000000
[  207.307544]  x16: ffff800008d9a000
[  207.309024] 
[  207.312410]  x15: 0000000000000008
[  207.315710] x11: 0000000000000040
[  207.319097] 
[  207.322483]  x10: ffff8000669dc000
[  207.323963] x14: ffff0000075cd80c
[  207.327349]  x9 : ffff000001808168
[  207.330649]  x13: 1fffe000006725e1
[  207.332129] 
[  207.335515]  x12: ffff80000a34b238
[  207.338815] x8 : ffff00006fc3d988
[  207.342201] 
[  207.345587]  x7 : ffff8000093e1150
[  207.347067] x11: 0000000000000204
[  207.350453]  x6 : ffff8000093e1138
[  207.353753]  x10: ffff000003392f08
[  207.355233] 
[  207.358619]  x9 : ffff000003392f0c
[  207.361919] x5 : ffff0000076f0e78
[  207.365306] 
[  207.368692]  x4 : 0000000000000000
[  207.370172] x8 : 0000000000000001
[  207.373558]  x3 : ffff00000301ac00
[  207.376858]  x7 : ffff0000075cd880
[  207.378337] 
[  207.381724]  x6 : ffff0000075cd808
[  207.385023] x2 : 000000000000002f
[  207.388410] 
[  207.391796]  x1 : ffffffffffffffff
[  207.393276] x5 : 0000000001de1d12
[  207.396662]  x0 : 0000000000000000
[  207.399962]  x4 : 000000000000009a
[  207.401442] 
[  207.404828]  x3 : fffffc00000ebcc0
[  207.408128] Call trace:
[  207.411514] 
[  207.414901]  kmem_cache_free+0x74/0x360
[  207.416380] x2 : ffffffffffffffff
[  207.419767]  bvec_free+0x6c/0xbc
[  207.422199]  x1 : ffff000003af3800
[  207.423679]  bio_free+0x58/0x80
[  207.427499]  x0 : 0000000000000000
[  207.430799]  bio_put+0x114/0x1d0
[  207.434011] 
[  207.434014] Call trace:
[  207.437398]  mpage_end_io+0xdc/0xf0
[  207.440524]  free_large_kmalloc+0x8c/0xc4
[  207.443910]  bio_endio+0x178/0x190
[  207.447122]  kfree+0x78/0x7c
[  207.448602]  blk_update_request+0x1d8/0x394
[  207.451035]  __audit_free+0x88/0x140
[  207.454508]  mmc_blk_cqe_complete_rq+0xfc/0x16c
[  207.458500]  do_exit+0x748/0x900
[  207.461887]  mmc_blk_mq_complete+0x2c/0x50
[  207.464753]  make_task_dead+0xa0/0x1a0
[  207.468920]  blk_complete_reqs+0x54/0x70
[  207.472479]  die+0x1f4/0x230
[  207.476991]  blk_done_softirq+0x20/0x30
[  207.480203]  die_kernel_fault+0x268/0x358
[  207.484282]  _stext+0x124/0x2a4
[  207.488014]  __do_kernel_fault+0x124/0x1b0
[  207.491920]  ____do_softirq+0x10/0x20
[  207.494787]  do_page_fault+0xac/0x3d0
[  207.498606]  call_on_irq_stack+0x24/0x4c
[  207.502598]  do_translation_fault+0xac/0xc0
[  207.505724]  do_softirq_own_stack+0x1c/0x30
[  207.509803]  do_mem_abort+0x44/0x94
[  207.513449]  __irq_exit_rcu+0xcc/0xf4
[  207.517094]  el1_abort+0x40/0x6c
[  207.521000]  irq_exit_rcu+0x10/0x20
[  207.525167]  el1h_64_sync_handler+0xd8/0xe4
[  207.529334]  el1_interrupt+0x38/0x70
[  207.532806]  el1h_64_sync+0x64/0x68
[  207.536452]  el1h_64_irq_handler+0x18/0x2c
[  207.539664]  kmem_cache_alloc_lru+0x1bc/0x510
[  207.543137]  el1h_64_irq+0x64/0x68
[  207.547303]  xas_alloc+0xb4/0xe4
[  207.550862]  arch_cpu_idle+0x18/0x2c
[  207.554335]  xas_create+0x260/0x410
[  207.558414]  default_idle_call+0x30/0x6c
[  207.562753]  xas_store+0x50/0x550
[  207.566140]  do_idle+0x244/0x2c0
[  207.569353]  __filemap_add_folio+0x1c4/0x410
[  207.572912]  cpu_startup_entry+0x24/0x30
[  207.576385]  filemap_add_folio+0x44/0xc4
[  207.580291]  kernel_init+0x0/0x130
[  207.583591]  page_cache_ra_unbounded+0xf4/0x1ec
[  207.586803]  arch_post_acpi_subsys_init+0x0/0x18
[  207.591056]  page_cache_ra_order+0x88/0x310
[  207.594962]  start_kernel+0x650/0x690
[  207.598868]  ondemand_readahead+0x1a4/0x320
[  207.602255]  __primary_switched+0xbc/0xc4
[  207.606767]  page_cache_async_ra+0x60/0xac
[  207.611371] Code: 9a9f12b5 b6b8005e b2503fd6 d503201f (f9401ea0) 
[  207.615531]  filemap_readahead.isra.0+0x50/0x90
[  207.619179] ---[ end trace 0000000000000000 ]---
[  207.623342]  filemap_get_pages+0x3f4/0x674
[  207.627337] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[  207.627342] SMP: stopping secondary CPUs
[  207.631415]  filemap_read+0xb4/0x360
[  207.637495]  blkdev_read_iter+0xc4/0x200
[  207.642016]  vfs_read+0x1f8/0x2a0
[  207.646625]  ksys_read+0x68/0xf4
[  207.650712]  __arm64_sys_read+0x1c/0x2c
[  207.657574]  invoke_syscall+0x48/0x114
[  207.661490]  el0_svc_common.constprop.0+0xd4/0xfc
[  207.665059]  do_el0_svc+0x30/0xd0
[  207.668975]  el0_svc+0x2c/0x84
[  207.672282]  el0t_64_sync_handler+0xbc/0x140
[  207.675502]  el0t_64_sync+0x18c/0x190
[  207.679330] ---[ end trace 0000000000000000 ]---
[  207.683089] Kernel Offset: 0x8000 from 0xffff800008000000
[  207.711996] PHYS_OFFSET: 0x80000000
[  207.715473] CPU features: 0x00000,00900084,0000420b
[  207.720339] Memory Limit: none
[  207.723389] Rebooting in 5 seconds..

    Once again, any idea is welcomed.

    Regards
    Pierre

  • 0 in reply to Pierre Buffo
    TI__Guru**** 167681 points

    Hi Pierre,

    The the issue in the first log seems irrelevant to MMC. What is the DDR size on your board? Please attach the full console boot log, including u-boot and kernel logs.

    As I mentioned, the MMC0 and MMC1 interfaces on AM64x GPEVM miss pull-up/-down resisters. I am routing your query to our MMC hardware expert for comments.

  • 0 in reply to Bin Liu
    TI__Guru 62855 points

    No external pull resistors are required for the signals that connect an eMMC device to the MMC0 port on AM64x. I suspect Bin was thinking about another device when making his previous reply. The AM62x and AM62Ax devices require external pulls resistors on the signals connecting an eMMC device to MMC0.

    Please reference the appropriate schematic checklist for these details.

    I'm not able to answer any software questions, so assigning this thread back to Bin to answer your question related to Linux log messages.

    Regards,
    Paul