Other Parts Discussed in Thread: AM6546
HI,
We are developing a custom board with AM6546 using SDK7.1, SR 2.0 and while running kernel getting crashed and the related log is attached below.
Please help to solve this kernel panic.
Error - 1 OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. [62414.557380] Unable to handle kernel paging request at virtual address 0007f8f8b221d500 [62414.565324] Mem abort info: [62414.568125] ESR = 0x86000004 [62414.571186] EC = 0x21: IABT (current EL), IL = 32 bits [62414.576509] SET = 0, FnV = 0 [62414.579567] EA = 0, S1PTW = 0 [62414.582711] [0007f8f8b221d500] address between user and kernel address ranges [62414.589844] Internal error: Oops: 86000004 [#1] PREEMPT SMP [62414.595405] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo sha512_generic sha512_arm64 md5 des_generic libdes cbc rpmsg_char 8250_pruss icssg_prueth ti_am335x_adc kfifo_buf icss_iep dwc3 udc_core usb_common crct10dif_ce pru_rproc irq_pruss_intc ti_k3_r5_remoteproc virtio_rpmsg_bus ti_am335x_tscadc ti_cal pruss phy_omap_usb2 dwc3_keystone ov5640 v4l2_fwnode at24 pvrsrvkm(O) sch_fq_codel jailhouse(O) cryptodev(O) ipv6 [62414.646582] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 5.4.74-g9574bba32a #1 [62414.654913] Hardware name: Texas Instruments AM654 Base Board (DT) [62414.661081] pstate: 40000005 (nZcv daif -PAN -UAO) [62414.665865] pc : 0x7f8f8b221d500 [62414.669086] lr : 0xc207f8f8b221d500 [62414.672564] sp : ffff800010fafef0 [62414.675869] x29: ffff800010fd3868 x28: 0000000082e00018 [62414.681170] x27: 00000000ffef44f8 x26: 0000000000000000 [62414.686470] x25: 00000000000000e0 x24: ffff800010fdb700 [62414.691771] x23: ffff800010fdb700 x22: 0000000000000060 [62414.697072] x21: ffff800010fd3948 x20: 0000000000000000 [62414.702373] x19: ffff800010fd3868 x18: 0000000000000000 [62414.707674] x17: 0000000000000000 x16: 0000000000000000 [62414.712975] x15: 0000000000000000 x14: 0000000000000000 [62414.718275] x13: 00002685fa5ee380 x12: 0000000000000000 [62414.723576] x11: 000000000000001d x10: 00000000000009e0 [62414.728876] x9 : ffff800010fafe20 x8 : ffff800010fdc140 [62414.734177] x7 : ffff80086ebc0000 x6 : 00000b5aa790f5c6 [62414.739477] x5 : 00ffffffffffffff x4 : 0003197500000000 [62414.744777] x3 : 4000000000000000 x2 : 0000000000000000 [62414.750078] x1 : 000000000ab329c0 x0 : 00000000000000e0 [62414.755378] Call trace: [62414.757819] 0x7f8f8b221d500 [62414.760699] Code: bad PC value [62414.763756] ---[ end trace 86232a8b3a47fb2e ]--- [62414.768382] Kernel panic - not syncing: Attempted to kill the idle task! [62414.775086] SMP: stopping secondary CPUs [62414.779013] Kernel Offset: disabled [62414.782495] CPU features: 0x0002,2000200c [62414.786494] Memory Limit: none [62414.789549] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- Error - 2 OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. [ 2482.406494] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 2482.415289] Mem abort info: [ 2482.418077] ESR = 0x86000005 [ 2482.421157] EC = 0x21: IABT (current EL), IL = 32 bits [ 2482.426487] SET = 0, FnV = 0 [ 2482.429534] EA = 0, S1PTW = 0 [ 2482.432686] user pgtable: 64k pages, 48-bit VAs, pgdp=00000008c5cdee00 [ 2482.439214] [0000000000000000] pgd=0000000000000000, pud=0000000000000000 [ 2482.446010] Internal error: Oops: 86000005 [#1] PREEMPT SMP [ 2482.451572] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo sha512_generic sha512_arm64 md5 des_generic libdes cbc rpmsg_char 8250_pruss icssg_prueth ti_am335x_adc kfifo_buf icss_iep dwc3 udc_core usb_common crct10dif_ce pru_rproc irq_pruss_intc ti_k3_r5_remoteproc virtio_rpmsg_bus ti_am335x_tscadc at24 ov5640 ti_cal v4l2_fwnode pruss phy_omap_usb2 dwc3_keystone pvrsrvkm(O) sch_fq_codel jailhouse(O) cryptodev(O) ipv6 [ 2482.502745] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 5.4.74-g9574bba32a #1 [ 2482.511078] Hardware name: Texas Instruments AM654 Base Board (DT) [ 2482.517246] pstate: 20000005 (nzCv daif -PAN -UAO) [ 2482.522031] pc : 0x0 [ 2482.524219] lr : __do_softirq+0x120/0x25c [ 2482.528218] sp : ffff80001002fef0 [ 2482.531522] x29: ffff80001002fef0 x28: 0000000000000000 [ 2482.536824] x27: 0000000000000040 x26: ffff800010fd00c0 [ 2482.542124] x25: 0000000000000020 x24: ffff0008401b9a00 [ 2482.547429] x23: 0000000000000101 x22: 0000000000000038 [ 2482.552731] x21: ffff800010fd00c0 x20: ffff800010e816b0 [ 2482.558032] x19: 0000000000000007 x18: 0000000000000000 [ 2482.563334] x17: 0000000000000000 x16: 0000000000000000 [ 2482.568634] x15: 0000000000000000 x14: 0000000000000000 [ 2482.573935] x13: 0000000000000000 x12: 00000000000000af [ 2482.579236] x11: 000000000000012a x10: 000000000002c618 [ 2482.584536] x9 : 00000000000000af x8 : 00000241f36aea00 [ 2482.589837] x7 : 7fffffffffffffff x6 : 0000000040a6e1db [ 2482.595138] x5 : ffff80086ebe0000 x4 : 0000000000000015 [ 2482.600438] x3 : 0000000000ca739e x2 : 000000000000f679 [ 2482.605740] x1 : 0000000000000000 x0 : ffff800010fd00f8 [ 2482.611043] Call trace: [ 2482.613486] 0x0 [ 2482.615326] irq_exit+0xb8/0xd8 [ 2482.618465] __handle_domain_irq+0x64/0xb8 [ 2482.622553] gic_handle_irq+0x5c/0x148 [ 2482.626294] el1_irq+0xb8/0x180 [ 2482.629433] arch_cpu_idle+0x10/0x18 [ 2482.633003] do_idle+0xc0/0x140 [ 2482.636138] cpu_startup_entry+0x20/0x68 [ 2482.640059] secondary_start_kernel+0x148/0x180 [ 2482.644586] Code: bad PC value [ 2482.647645] ---[ end trace 98eafd4c5a2b4c4a ]--- [ 2482.652253] Kernel panic - not syncing: Fatal exception in interrupt [ 2482.658595] SMP: stopping secondary CPUs [ 2482.662514] Kernel Offset: disabled [ 2482.665993] CPU features: 0x0002,2000200c [ 2482.669991] Memory Limit: none [ 2482.673044] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Error - 3 OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. [ 1952.710550] Unable to handle kernel read from unreadable memory at virtual address ffff8000101365c4 [ 1952.719594] Mem abort info: [ 1952.722379] ESR = 0x8600000f [ 1952.725425] EC = 0x21: IABT (current EL), IL = 32 bits [ 1952.730723] SET = 0, FnV = 0 [ 1952.733767] EA = 0, S1PTW = 0 [ 1952.736902] swapper pgtable: 64k pages, 48-bit VAs, pgdp=0000000082df0000 [ 1952.743675] [ffff8000101365c4] pgd=00000008ffff0003, pud=00000008ffff0003, pmd=00000008fffe0003, pte=0040000082130793 [ 1952.754269] Internal error: Oops: 8600000f [#1] PREEMPT SMP [ 1952.759829] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo sha512_generic sha512_arm64 md5 des_generic libdes cbc rpmsg_char ti_am335x_adc kfifo_buf icssg_prueth dwc3 udc_core usb_common icss_iep 8250_pruss crct10dif_ce pru_rproc irq_pruss_intc ti_k3_r5_remoteproc virtio_rpmsg_bus ti_am335x_tscadc ti_cal pruss at24 ov5640 v4l2_fwnode phy_omap_usb2 dwc3_keystone pvrsrvkm(O) sch_fq_codel jailhouse(O) cryptodev(O) ipv6 [ 1952.811001] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G O 5.4.74-g9574bba32a #1 [ 1952.819333] Hardware name: Texas Instruments AM654 Base Board (DT) [ 1952.825501] pstate: 40000085 (nZcv daIf -PAN -UAO) [ 1952.830294] pc : update_process_times+0x2c/0xb8 [ 1952.834815] lr : update_process_times+0x2c/0xb8 [ 1952.839333] sp : ffff80001004fe20 [ 1952.842638] x29: ffff80001004fe20 x28: ffff800010146938 [ 1952.847939] x27: ffff00087fa87f00 x26: 0000000000000006 [ 1952.853239] x25: 0000000000000001 x24: 0000000000000080 [ 1952.858540] x23: 000001c69f13699e x22: ffff00087fa87f60 [ 1952.863841] x21: ffff8000113efe10 x20: ffff0008401ba800 [ 1952.869141] x19: 0000000000000000 x18: 0000000000000000 [ 1952.874442] x17: 0000000000000000 x16: 0000000000000000 [ 1952.879742] x15: 0000000000000000 x14: 0000000000000000 [ 1952.885043] x13: ffff800010fd1000 x12: 0000000000000001 [ 1952.890343] x11: ffff8000110ca000 x10: 0000000000000040 [ 1952.895648] x9 : ffff800010fea688 x8 : ffff800010fea680 [ 1952.900955] x7 : ffff800010feb000 x6 : 0000000100064dd6 [ 1952.906256] x5 : ffff0008401ba800 x4 : ffff800010e8b000 [ 1952.911557] x3 : ffffffffffffeb81 x2 : 0000000100064dd9 [ 1952.916858] x1 : 0000000000019b51 x0 : 0000000000000000 [ 1952.922159] Call trace: [ 1952.924611] update_process_times+0x2c/0xb8 [ 1952.928790] tick_sched_handle.isra.0+0x30/0x48 [ 1952.933313] tick_sched_timer+0x48/0x98 [ 1952.937140] __hrtimer_run_queues+0x110/0x1b0 [ 1952.941487] hrtimer_interrupt+0xe4/0x240 [ 1952.945493] arch_timer_handler_phys+0x30/0x40 [ 1952.949930] handle_percpu_devid_irq+0x80/0x140 [ 1952.954453] generic_handle_irq+0x24/0x38 [ 1952.958454] __handle_domain_irq+0x60/0xb8 [ 1952.962544] gic_handle_irq+0x5c/0x148 [ 1952.966284] el1_irq+0xb8/0x180 [ 1952.969421] arch_cpu_idle+0x10/0x18 [ 1952.972991] do_idle+0xc0/0x140 [ 1952.976125] cpu_startup_entry+0x24/0x68 [ 1952.980042] secondary_start_kernel+0x148/0x180 [ 1952.984568] Code: 97fecd31 97ffffe2 2a1303e0 97ffdaa6 (b9401280) [ 1952.990659] ---[ end trace bd3b8ce06aa46dc2 ]--- [ 1952.995266] Kernel panic - not syncing: Fatal exception in interrupt [ 1953.001609] SMP: stopping secondary CPUs [ 1953.005527] Kernel Offset: disabled [ 1953.009008] CPU features: 0x0002,2000200c [ 1953.013005] Memory Limit: none [ 1953.016057] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Error - 4 OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. OSPI Test Passed. [ 5258.893382] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 5258.899489] rcu: 1-...0: (1 GPs behind) idle=1a2/0/0x1 softirq=48324/48324 fqs=2462 [ 5258.907306] (detected by 0, t=5255 jiffies, g=167721, q=649) [ 5258.913039] Task dump for CPU 1: [ 5258.916260] swapper/1 R running task 0 0 1 0x0000002a [ 5258.923297] Call trace: [ 5258.925748] __switch_to+0x100/0x1c0 [ 5306.513396] Unable to handle kernel paging request at virtual address 7fff8000100ede28 [ 5306.521309] Mem abort info: [ 5306.524094] ESR = 0x86000004 [ 5306.527140] EC = 0x21: IABT (current EL), IL = 32 bits [ 5306.532438] SET = 0, FnV = 0 [ 5306.535483] EA = 0, S1PTW = 0 [ 5306.538616] [7fff8000100ede28] address between user and kernel address ranges [ 5306.545737] Internal error: Oops: 86000004 [#1] PREEMPT SMP [ 5306.551297] Modules linked in: xt_conntrack xt_MASQUERADE xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_tables x_tables br_netfilter bridge stp llc xfrm_user xfrm_algo sha512_generic sha512_arm64 md5 des_generic libdes cbc rpmsg_char 8250_pruss icssg_prueth ti_am335x_adc kfifo_buf dwc3 udc_core usb_common icss_iep crct10dif_ce pru_rproc irq_pruss_intc ti_k3_r5_remoteproc virtio_rpmsg_bus ti_am335x_tscadc pruss ti_cal phy_omap_usb2 ov5640 v4l2_fwnode at24 dwc3_keystone pvrsrvkm(O) sch_fq_codel jailhouse(O) cryptodev(O) ipv6 [ 5306.602479] CPU: 0 PID: 1244 Comm: containerd Tainted: G O 5.4.74-g9574bba32a #1 [ 5306.611160] Hardware name: Texas Instruments AM654 Base Board (DT) [ 5306.617330] pstate: 20000085 (nzCv daIf -PAN -UAO) [ 5306.622116] pc : 0x7fff8000100ede28 [ 5306.625604] lr : scheduler_tick+0x74/0xf8 [ 5306.629604] sp : ffff80001000fde0 [ 5306.632908] x29: ffff80001000fde0 x28: ffff800010146938 [ 5306.638210] x27: ffff00087fa47f00 x26: 0000000000000002 [ 5306.643511] x25: 0000000000000001 x24: 0000000000000080 [ 5306.648812] x23: ffff00084acb1c00 x22: 0000000000000000 [ 5306.654113] x21: ffff800010fd3b58 x20: ffff800010e8ab40 [ 5306.659414] x19: ffff00087fa4ab40 x18: 0000000000000000 [ 5306.664715] x17: 0000000000000000 x16: 0000000000000000 [ 5306.670015] x15: 0000000000000000 x14: 0000000000000000 [ 5306.675316] x13: ffff800010fd1000 x12: 0000000000000000 [ 5306.680617] x11: ffff8000110ca000 x10: 0000000000000040 [ 5306.685918] x9 : ffff800010fea688 x8 : ffff800010fea680 [ 5306.691218] x7 : ffff800010feb000 x6 : 000000cd0f8d6539 [ 5306.696519] x5 : 00ffffffffffffff x4 : 000004d059c09364 [ 5306.701819] x3 : 7fff8000100ede28 x2 : 0000000000000000 [ 5306.707120] x1 : ffff00084acb1c00 x0 : ffff00087fa4ab40 [ 5306.712421] Call trace: [ 5306.714863] 0x7fff8000100ede28 [ 5306.718000] update_process_times+0x3c/0xb8 [ 5306.722177] tick_sched_handle.isra.0+0x30/0x48 [ 5306.726698] tick_sched_timer+0x48/0x98 [ 5306.730526] __hrtimer_run_queues+0x110/0x1b0 [ 5306.734873] hrtimer_interrupt+0xe4/0x240 [ 5306.738879] arch_timer_handler_phys+0x30/0x40 [ 5306.743315] handle_percpu_devid_irq+0x80/0x140 [ 5306.747839] generic_handle_irq+0x24/0x38 [ 5306.751842] __handle_domain_irq+0x60/0xb8 [ 5306.755931] gic_handle_irq+0x5c/0x148 [ 5306.759671] el1_irq+0xb8/0x180 [ 5306.762808] futex_wait+0xc/0x208 [ 5306.766116] do_futex+0x190/0xc98 [ 5306.769425] __arm64_sys_futex+0x100/0x140 [ 5306.773514] el0_svc_common.constprop.0+0x68/0x160 [ 5306.778295] el0_svc_handler+0x20/0x80 [ 5306.782036] el0_svc+0x8/0x204 [ 5306.785089] Code: bad PC value [ 5306.788147] ---[ end trace 1d97ccee225730c7 ]--- [ 5306.792754] Kernel panic - not syncing: Fatal exception in interrupt [ 5306.799097] SMP: stopping secondary CPUs [ 5307.920058] SMP: failed to stop secondary CPUs 0-1 [ 5307.924838] Kernel Offset: disabled [ 5307.928318] CPU features: 0x0002,2000200c [ 5307.932316] Memory Limit: none [ 5307.935368] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
Thanks,
Logan