This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VH-Q1: Kernel panic during large NVMe read/write on PCI0

Part Number: TDA4VH-Q1
Other Parts Discussed in Thread: TDA4VH

Tool/software:

Hello TI E2E team,

I’m working with a TDA4VH-EVM (J784S4) running Processor SDK Linux ADAS (ti-processor-sdk-linux-adas-j784s4-evm-09_02_00_05) and Linux kernel 6.1.80-ti. An NVMe SSD is attached to PCIe0.

Small file transfers work fine, but when I copy a large file (~8 GB) with:

  • Board: TDA4VH (J784S4) (Made by New Board)

  • SDK: ti-processor-sdk-linux-adas-j784s4-evm-09_02_00_05

  • Kernel: 6.1.80-ti-g2e423244f8c0

  • Filesystem: ext4 on NVMe /dev/nvme0n1p1

  • Repro:  

                            root@j784s4-evm:/run/media/nvme0n1p# cp source_8GB.vhdx target.vhdx

the kernel panics partway through. here is logs :

 root@j784s4-evm:~#

[ 246.004290] Unable to handle kernel paging request at virtual address 00562a7a2e722520
[ 246.012202] Mem abort info:
[ 246.014982] ESR = 0x0000000096000004
[ 246.018717] EC = 0x25: DABT (current EL), IL = 32 bits
[ 246.024012] SET = 0, FnV = 0
[ 246.027052] EA = 0, S1PTW = 0
[ 246.030178] FSC = 0x04: level 0 translation fault
[ 246.035038] Data abort info:
[ 246.037904] ISV = 0, ISS = 0x00000004
[ 246.041725] CM = 0, WnR = 0
[ 246.044678] [00562a7a2e722520] address between user and kernel address ranges
[ 246.051794] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP
[ 246.058043] Modules linked in: overlay ti_cpsw_proxy_client nvme nvme_core pci_endpoint_test bluetooth ecdh_generic ecc cfg80211 rfkill xhci_plat_hcd rpmsg_ctrl rpmsg_char ti_am335x_adc cdns3 kfifo_buf omap_rng cdns_usb_common cdns_pltfrm crct10dif_ce ti_j721e_cpsw_virt_mac wave5 display_connector videobuf2_dma_contig phy_can_transceiver videobuf2_memops v4l2_mem2mem cdns_mhdp8546 ti_k3_r5_remoteproc videobuf2_v4l2 cdns_dsi drm_display_helper ti_k3_dsp_remoteproc videobuf2_common drm_kms_helper virtio_rpmsg_bus syscopyarea videodev rpmsg_ns ti_am335x_tscadc ti_k3_common k3_j72xx_bandgap pvrsrvkm(O) pci_j721e_host sysfillrect mc sysimgblt m_can_platform sa2ul fb_sys_fops pci_j721e ti_j721e_ufs m_can pcie_cadence_host cdns_dphy cdns_dphy_rx pcie_cadence cdns3_ti can_dev optee_rng rng_core rti_wdt cryptodev(O) fuse drm drm_panel_orientation_quirks ipv6
[ 246.085527] Unable to handle kernel paging request at virtual address ffff80036c2de040
[ 246.133212] CPU: 7 PID: 2387 Comm: cp Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
[ 246.141109] Unable to handle kernel paging request at virtual address ffff7ffffb2c4d50
[ 246.149606] Hardware name: Texas Instruments J784S4 EVM (DT)
[ 246.157499] Mem abort info:
[ 246.163139] pstate: 000000c5 (nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 246.165918] ESR = 0x0000000096000004
[ 246.172857] pc : irq_work_run_list+0x44/0x70
[ 246.176590] EC = 0x25: DABT (current EL), IL = 32 bits
[ 246.180842] lr : irq_work_tick+0x28/0x6c
[ 246.184296] Insufficient stack space to handle exception!
[ 246.184298] Insufficient stack space to handle exception!
[ 246.184300] ESR: 0x000000009a000000 -- SP Alignment
[ 246.184301] Unable to handle kernel paging request at virtual address ffff7ffffbcae590
[ 246.184302] FAR: 0x6c6172657469810f
[ 246.184304] Mem abort info:
[ 246.184304] Task stack: [0xffff800009610000..0xffff800009614000]
[ 246.184305] ESR = 0x0000000096000004
[ 246.184306] IRQ stack: [0x6c61726574696c5f..0x6c6172657469ac5f]
[ 246.184306] EC = 0x25: DABT (current EL), IL = 32 bits
[ 246.184308] Overflow stack: [0xffff000f5c3d8300..0xffff000f5c3d9300]
[ 246.184309] SET = 0, FnV = 0
[ 246.184310] EA = 0, S1PTW = 0
[ 246.184311] FSC = 0x04: level 0 translation fault
[ 246.184310] CPU: 1735289188 PID: 0 Comm: swapper/6 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
[ 246.184313] Data abort info:
[ 246.184314] ISV = 0, ISS = 0x00000004
[ 246.184315] CM = 0, WnR = 0
[ 246.184315] Hardware name: Texas Instruments J784S4 EVM (DT)
[ 246.184316] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000082f40000
[ 246.184316] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 246.184319] [ffff7ffffbcae590] pgd=0000000000000000, p4d=0000000000000000
[ 246.184320] pc : el1h_64_sync+0x0/0x68
[ 246.184326] lr : call_on_irq_stack+0x24/0x4c
[ 246.184331] sp : 6c6172657469810f
[ 246.184332] x29: ffff800009613c40 x28: ffff0008bc2f3900 x27: 0000000000000000
[ 246.184337] x26: ffff0008bc2f3900 x25: 0000000000000000 x24: 0000000000000000
[ 246.184342] x23: 0000000060000005 x22: ffff800008b6c5bc x21: ffff800009613df0
[ 246.184347] x20: 6874756120726568 x19: ffff800008ff46c8 x18: 0000000000000000
[ 246.184350] x17: ffff800f533ec000 x16: 6c61726574696c5f x15: 0000000000000000
[ 246.184354] x14: 00000000000001bf x13: 0000000000000001 x12: 0000000000000000
[ 246.184358] x11: 0000000000000001 x10: 00000000000009b0 x9 : ffff800009613d70
[ 246.184363] x8 : ffff0008bc2f4310 x7 : 0000000000000000 x6 : 00000000795a097c
[ 246.184367] x5 : ffff80000853f0c0 x4 : ffff800009613c51 x3 : ffff800009613ca0
[ 246.184371] x2 : ffff800009614000 x1 : ffff80000853f0c0 x0 : ffff800009613ca0
[ 246.186135] SET = 0, FnV = 0
[ 246.186138] EA = 0, S1PTW = 0
[ 246.186139] FSC = 0x04: level 0 translation fault
[ 246.186141] Data abort info:
[ 246.186142] ISV = 0, ISS = 0x00000004
[ 246.186143] CM = 0, WnR = 0
[ 246.186144] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000082f40000
[ 246.186147] [ffff7ffffb2c4d50] pgd=0000000000000000, p4d=0000000000000000
[ 246.190042] sp : ffff800009323e30
[ 246.190044] x29: ffff800009323e30 x28: 0000000000000000 x27: ffff8000080f2c40
[ 246.190049] x26: 0000000000000001 x25: 00000000000000c0 x24: 000000393ad26a4c
[ 246.190053] x23: ffff000f5c3f0680 x22: ffff000f5c3f06e0 x21: ffff80001532b5c0
[ 246.190058] x20: 0000000000000000 x19: 61562a7a2e722520 x18: 000000000046c56d
[ 246.190062] x17: ffff800f53400000 x16: ffff800009320000 x15: 00003d0910c8e000
[ 246.190066] x14: 00000000000c3500 x13: 000000000103ff97 x12: 003d0910be8a1cbe
[ 246.190071] x11: 0000000000000000 x10: 0000000100000008 x9 : 000000000103ff97
[ 246.190075] x8 : ffff8000091d2000 x7 : ffff800008ff5000 x6 : ffff800f53400000
[ 246.190079] x5 : ffff800009323d30 x4 : 0000000000000034 x3 : ffff800009323d30
[ 246.190083] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 61562a7a2e722520
[ 246.190087] Call trace:
[ 246.190089] irq_work_run_list+0x44/0x70
[ 246.190092] irq_work_tick+0x28/0x6c
[ 246.190095] update_process_times+0x94/0xac
[ 246.190101] tick_sched_handle+0x34/0x60
[ 246.190107] tick_sched_timer+0x4c/0xa4
[ 246.190111] __hrtimer_run_queues+0x138/0x1b0
[ 246.190115] hrtimer_interrupt+0xe8/0x244
[ 246.190118] arch_timer_handler_phys+0x34/0x44
[ 246.190123] handle_percpu_devid_irq+0x84/0x130
[ 246.190128] generic_handle_domain_irq+0x2c/0x44
[ 246.190134] gic_handle_irq+0x50/0x124
[ 246.190138] call_on_irq_stack+0x24/0x4c
[ 246.190142] do_interrupt_handler+0x80/0x8c
[ 246.190145] el1_interrupt+0x34/0x70
[ 246.190152] el1h_64_irq_handler+0x18/0x2c
[ 246.190157] el1h_64_irq+0x64/0x68
[ 246.190159] _raw_spin_unlock_irqrestore+0xc/0x50
[ 246.190164] mark_buffer_dirty+0x100/0x11c
[ 246.190172] __block_commit_write.constprop.0.isra.0+0xb8/0x160
[ 246.190177] generic_write_end+0x5c/0x194
[ 246.190179] ext4_da_write_end+0xfc/0x1f4
[ 246.190186] generic_perform_write+0x11c/0x1ec
[ 246.190190] ext4_buffered_write_iter+0x80/0x130
[ 246.190193] ext4_file_write_iter+0x5c/0x674
[ 246.190196] do_iter_readv_writev+0xbc/0x150
[ 246.190201] do_iter_write+0x90/0x200
[ 246.190203] vfs_iter_write+0x1c/0x30
[ 246.190206] iter_file_splice_write+0x248/0x3c0
[ 246.190210] direct_splice_actor+0x30/0x4c
[ 246.190213] splice_direct_to_actor+0xd8/0x280
[ 246.190215] do_splice_direct+0x94/0xe4
[ 246.190218] vfs_copy_file_range+0x13c/0x450
[ 246.190222] __arm64_sys_copy_file_range+0x164/0x33c
[ 246.190225] invoke_syscall+0x48/0x114
[ 246.190230] el0_svc_common.constprop.0+0xd4/0xfc
[ 246.190234] do_el0_svc+0x20/0x30
[ 246.190237] el0_svc+0x28/0xa0
[ 246.190241] el0t_64_sync_handler+0xbc/0x140
[ 246.190245] el0t_64_sync+0x18c/0x190
[ 246.190250] Code: 35ffffc2 d5033bbf b40000b3 aa1303e0 (f9400273)
[ 246.195633] ---[ end trace 0000000000000000 ]---

Has anyone seen a similar “kernel paging request” panic during large file writes over NVMe on this platform?

Is there a known issue or patch for the NVMe/PCIe/IOMMU driver in this SDK?

I tried to improve DTSI by applying SMMU & IOMMU, etc., but it didn't work well.

k3-j784s4-main.dtsi 

......
smmu0: iommu@36600000 {
compatible = "arm,mmu-500", "arm,smmu-v3";
reg = <0x00 0x36600000 0x00 0x100000>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&gic500>;

interrupts = <GIC_SPI 772 IRQ_TYPE_EDGE_RISING>,
<GIC_SPI 768 IRQ_TYPE_EDGE_RISING>,
<GIC_SPI 773 IRQ_TYPE_EDGE_RISING>;
#global-interrupts = <2>;
#iommu-cells = <1>;
status = "disabled";
};

......

pcie0_rc: pcie@2900000 {
compatible = "ti,j784s4-pcie-host";
reg = <0x00 0x02900000 0x00 0x1000>,
<0x00 0x02907000 0x00 0x400>,
<0x00 0x0d000000 0x00 0x00800000>,
<0x00 0x10000000 0x00 0x00001000>;
reg-names = "intd_cfg", "user_cfg", "reg", "cfg";
interrupt-names = "link_state";
interrupts = <GIC_SPI 318 IRQ_TYPE_EDGE_RISING>;
device_type = "pci";
ti,syscon-pcie-ctrl = <&scm_conf 0x4070>;
max-link-speed = <3>;
num-lanes = <4>;
power-domains = <&k3_pds 332 TI_SCI_PD_EXCLUSIVE>;
clocks = <&k3_clks 332 0>;
clock-names = "fck";
#address-cells = <3>;
#size-cells = <2>;
bus-range = <0x0 0xff>;
vendor-id = <0x104c>;
device-id = <0xb00d>;
msi-map = <0x0 &gic_its 0x0 0x10000>;
dma-coherent;
ranges = <0x01000000 0x0 0x10001000 0x0 0x10001000 0x0 0x0010000>,
<0x02000000 0x0 0x10011000 0x0 0x10011000 0x0 0x7fef000>;
dma-ranges = <0x02000000 0x0 0x0 0x0 0x0 0x10000 0x0>;

/* Add to IOMMU */

iommu-map = <0x0000 &smmu0 0x0000 0x10000>; 

#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 7>;
interrupt-map = <0 0 0 1 &pcie0_intc 0>,
<0 0 0 2 &pcie0_intc 0>,
<0 0 0 3 &pcie0_intc 0>,
<0 0 0 4 &pcie0_intc 0>;

status = "okay";

pcie0_intc: interrupt-controller {
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&gic500>;
interrupts = <GIC_SPI 312 IRQ_TYPE_EDGE_RISING>;
};
};

         .....

Any suggestions for a workaround or configuration change would be greatly appreciated.

Thank you!

  • Hi Dongho,

    Was the kernel panic seen before IOMMU node was added? A good experiment to do would be to see if a kernel panic is seen running the same cp command when there is no IOMMU node in devicetree.

    To give some extra details, that kernel panic might happen if the driver is trying to access memory address that is inaccessible. For example, it could be that something is trying to access memory address for a hardware module within the SoC that is not turned on, or the memory is reserved for different cores, or the memory is not mapped. I see that the iommu devicetree node status is set to "disabled" and I do not see in the shared portion of devicetree a place that sets the status of iommu to "okay". So, it could be that the PCIe driver is trying to access some memory address of iommu hardware that is not enabled.

    Regards,

    Takuma

  • Hi 

    IOMMU node was not added before the kernel panic.

    A kernel panic occured duriong the cp command test, so i added iommu.

    IOMMU doesn't seem to exist in tda4vh linux sdk, so I tried it with reference to ti other sdk.

    In addition, I found the following improvement method through workaround.

    Kernel panic occurs when buffer/cache memory is used more than about 8GB.

    So, this is a method to clean by setting the buffer/cache threshold(< 8GB) through monitoring App.

    I think we'll have to use this method until we find a solution.

    Thanks, 

    Dongho

  • Hi Dongho,

    That is strange behavior that I have not seen before. The closest thing that I have observed in a past would be a memory leak leading to overloading the memory.

    So, I have 2 questions:

    • You mention in your first post "Made by New Board". Does this mean this is a custom board? And if yes, how much DDR is available on the board?
    • Has the memory map been changed?

    So far, I have not been able to reproduce the kernel panic on TI J784S4 EVM. I was using the 10.1 SDK instead of 9.2 SDK, so there are some differences, but I do not remember seeing kernel panic in 9.2 SDK so I assume behavior is same.

    Here is the information of the file I transferred from SD card to SSD card (it is 14GB SD card image I happened to have on my PC):

    root@j784s4-evm:/home/weston# ls -l
    total 13912068
    -rw-r--r-- 1 weston weston 14245953536 May 3 2024 tech_day_sktda4vm_robot_arm.img

    And I tried copying from both SD card to SSD card, as well as SSD card back to SSD card.

    Below is a screenshot of logs and some system information like "top" for CPU and running process info and "watch -n 1 cat /proc/meminfo" for monitoring the memory info:

    I used the "cp" command from commandline a couple of times, but it has not failed in all of my attempts. 

    Regards,

    Takuma

  • Hi Takuma

    It is designed as a custom board and uses the same memory size as the EVM. The memory map has not been modified.

    If you continue to copy 2Gbyte files from nvme0 using the cp command, you can see that the "Cached" memory continues to increase.

     - file '2' size is 2Gbyte.

     1) root@j784s4-evm:/run/media/nvme0n1p1/dongho# cp 2 222

     2root@j784s4-evm:/run/media/nvme0n1p1/dongho# cp 2 333

      

    And when "Cached" exceeds about 8Gbyte, Kernel Panic occurs.

     - root@j784s4-evm:/run/media/nvme0n1p1/dongho# cp 2 444

    Please try to continuously copy files using the cp command inside the SSD to see if the "cached" size increases and there are no issues.

    Thanks,

    Dongho

    -----------------------------------------------------------------------------------------

    * Kernel Log from "USB to COM"

    -----------------------------------------------------------------------------------------

    j784s4-evm login: [ 18.346250] cdns-mhdp8546 a000000.dp-bridge: invalid resource
    [ 18.352002] cdns-mhdp8546 a000000.dp-bridge: Failed to get SAPB memory resource, HDCP not supported
    [ 18.361044] cdns-mhdp8546 a000000.dp-bridge: no PHY configured
    [ 18.366919] platform main-r5fss-cpsw9g-virt-mac1: deferred probe pending
    [ 18.373611] platform main-r5fss-cpsw9g-virt-mac0: deferred probe pending
    [ 18.380303] platform a000000.dp-bridge: deferred probe pending
    [ 41.993729] kauditd_printk_skb: 2 callbacks suppressed
    [ 41.993739] audit: type=1334 audit(1746668057.098:12): prog-id=12 op=UNLOAD
    [ 42.006042] audit: type=1334 audit(1746668057.098:13): prog-id=11 op=UNLOAD
    [ 604.468961] Insufficient stack space to handle exception!
    [ 604.468961] Insufficient stack space to handle exception!
    [ 604.468970] ESR: 0x000000009a000000 -- SP Alignment
    [ 604.468975] FAR: 0x68223d736e6c80d8
    [ 604.468977] Task stack: [0xffff800009328000..0xffff80000932c000]
    [ 604.468979] IRQ stack: [0x68223d736e6c6d78..0x68223d736e6cad78]
    [ 604.468981] Overflow stack: [0xffff000f5c374300..0xffff000f5c375300]
    [ 604.468985] CPU: 1 PID: 1363 Comm: cp Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [ 604.468990] Hardware name: Texas Instruments J784S4 EVM (DT)
    [ 604.468993] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [ 604.468997] pc : el1h_64_sync+0x0/0x68
    [ 604.469006] lr : call_on_irq_stack+0x24/0x4c
    [ 604.469011] sp : 68223d736e6c80d8
    [ 604.469013] x29: ffff80000932b5e0 x28: ffff0008c6518000 x27: ffff000b585b2bc8
    [ 604.469019] x26: 0000000000001000 x25: 0000000000000001 x24: fffffc002d712440
    [ 604.469024] x23: 0000000020000005 x22: ffff80000829d720 x21: ffff80000932b790
    [ 604.469028] x20: 0000000000000000 x19: ffff800008ff46c8 x18: 000000000046c970
    [ 604.469032] x17: ffff800f53388000 x16: 68223d736e6c6d78 x15: 55746e6569646172
    [ 604.469036] x14: 672022293639342e x13: 55746e6569646172 x12: 672022293639342e
    [ 604.469040] x11: 6e65696461724772 x10: 61656e696c23223d x9 : 666572683a6b6e69
    [ 604.469044] x8 : ffff80000831feb0 x7 : 4f65636170537265 x6 : 000000007a4a2ac3
    [ 604.469048] x5 : ffff80000853f0c0 x4 : ffff80000932b5f1 x3 : ffff80000932b640
    [ 604.469052] x2 : ffff80000932c000 x1 : ffff80000853f0c0 x0 : ffff80000932b640
    [ 604.469057] Kernel panic - not syncing: kernel stack overflow
    [ 604.469060] CPU: 1 PID: 1363 Comm: cp Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [ 604.469063] Hardware name: Texas Instruments J784S4 EVM (DT)
    [ 604.469065] Call trace:
    [ 604.469067] dump_backtrace.part.0+0xdc/0xf0
    [ 604.469073] show_stack+0x18/0x30
    [ 604.469077] dump_stack_lvl+0x68/0x84
    [ 604.469084] dump_stack+0x18/0x34
    [ 604.469087] panic+0x188/0x348
    [ 604.469091] nmi_panic+0xac/0xb0
    [ 604.469097] panic_bad_stack+0x104/0x124
    [ 604.469102] handle_bad_stack+0x34/0xc0
    [ 604.469106] __bad_stack+0x78/0x7c
    [ 604.469109] el1h_64_sync+0x0/0x68
    [ 604.469111] do_interrupt_handler+0x80/0x8c
    [ 604.469115] el1_interrupt+0x34/0x70
    [ 604.469118] el1h_64_irq_handler+0x18/0x2c
    [ 604.469122] el1h_64_irq+0x64/0x68
    [ 604.469124] __block_commit_write.constprop.0.isra.0+0xa0/0x160
    [ 604.469133] generic_write_end+0x5c/0x194
    [ 604.469135] ext4_da_write_end+0xfc/0x1f4
    [ 604.469141] generic_perform_write+0x11c/0x1ec
    [ 604.469145] ext4_buffered_write_iter+0x80/0x130
    [ 604.469149] ext4_file_write_iter+0x5c/0x674
    [ 604.469152] do_iter_readv_writev+0xbc/0x150
    [ 604.469157] do_iter_write+0x90/0x200
    [ 604.469160] vfs_iter_write+0x1c/0x30
    [ 604.469163] iter_file_splice_write+0x248/0x3c0
    [ 604.469167] direct_splice_actor+0x30/0x4c
    [ 604.469169] splice_direct_to_actor+0xd8/0x280
    [ 604.469172] do_splice_direct+0x94/0xe4
    [ 604.469175] vfs_copy_file_range+0x13c/0x450
    [ 604.469179] __arm64_sys_copy_file_range+0x164/0x33c
    [ 604.469182] invoke_syscall+0x48/0x114
    [ 604.469187] el0_svc_common.constprop.0+0xd4/0xfc
    [ 604.469190] do_el0_svc+0x20/0x30
    [ 604.469194] el0_svc+0x28/0xa0
    [ 604.469197] el0t_64_sync_handler+0xbc/0x140
    [ 604.469201] el0t_64_sync+0x18c/0x190
    [ 604.469205] SMP: stopping secondary CPUs
    [ 604.469213] Insufficient stack space to handle exception!
    [ 604.469216] ESR: 0x000000009a000000 -- SP Alignment
    [ 604.469219] FAR: 0x68223d736e6c80d8
    [ 604.469220] Task stack: [0xffff800009140000..0xffff800009144000]
    [ 604.469219] Unable to handle kernel paging request at virtual address ffff800010d714a0
    [ 604.469220] Unable to handle kernel paging request at virtual address ffff8003ac201ea0
    [ 604.469223] IRQ stack: [0x68223d736e6c6d78..0x68223d736e6cad78]
    [ 604.469225] Mem abort info:
    [ 604.469225] Overflow stack: [0xffff000f5c360300..0xffff000f5c361300]
    [ 604.469226] ESR = 0x0000000096000007
    [ 604.469227] Unable to handle kernel paging request at virtual address ffff7ffffa2c85d8
    [ 604.469228] EC = 0x25: DABT (current EL), IL = 32 bits
    [ 604.469227] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [ 604.469229] Mem abort info:
    [ 604.469230] ESR = 0x0000000096000004
    [ 604.469231] SET = 0, FnV = 0
    [ 604.469231] Hardware name: Texas Instruments J784S4 EVM (DT)
    [ 604.469232] EA = 0, S1PTW = 0
    [ 604.469232] EC = 0x25: DABT (current EL), IL = 32 bits
    [ 604.469233] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [ 604.469234] FSC = 0x07: level 3 translation fault
    [ 604.469235] SET = 0, FnV = 0
    [ 604.469236] Data abort info:
    [ 604.469236] EA = 0, S1PTW = 0
    [ 604.469237] ISV = 0, ISS = 0x00000007
    [ 604.469237] FSC = 0x04: level 0 translation fault
    [ 604.469236] pc : el1h_64_sync+0x0/0x68
    [ 604.469239] Data abort info:
    [ 604.469239] CM = 0, WnR = 0
    [ 604.469240] ISV = 0, ISS = 0x00000004
    [ 604.469240] lr : call_on_irq_stack+0x24/0x4c
    [ 604.469242] CM = 0, WnR = 0
    [ 604.469241] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000082f40000
    [ 604.469243] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000082f40000
    [ 604.469244] sp : 68223d736e6c80d8
    [ 604.469244] [ffff800010d714a0] pgd=1000000ffffff003
    [ 604.469245] x29: ffff800009143bb0
    [ 604.469246] [ffff7ffffa2c85d8] pgd=0000000000000000
    [ 604.469246] , p4d=1000000ffffff003

  • Hi Dongho,

    I am still not seeing it. I tried multiple times and was able to get the cache up to 28GB. I tried stress testing by starting 3x cp commands in parallel but no kernel panic observed yet.

    I was trying initially on 10.1 SDK since that was the version I had flashed, but I also tried 9.2 SDK in case that was the cause of us seeing different behavior. However, in both cases, I cannot recreate the issue you are seeing. 

    So, a couple of different questions I have for you:

    • Can you try your experiment on a TI J784S4 EVM board to see if you can see the same issue on the TI board?
    • Are there any other applications running in parallel when you are doing the cp? 

    Regards,

    Takuma

  • Hi Takuma

    Sorry, I don't have an EVM board, and when "CP command" testing, no other apps were running at all.

    I think we need to review the System Memory Map. 

    Thanks,

    Dongho

  • Hi Dongho,

    Ok, understood.

    With current information, a bit hard to pinpoint what the issue could be. However, one hypothesis I can suggest is DDR. If your custom board design is similar to TI J784S4 EVM, then I assume there are 4 x 8GB DDR. If there is no DDR interleaving, then it could be that when 8GB is reached it accesses a new DDR instance, and there is some issues with the second DDR. For this, there is a memtester binary that we package in with the default SDK image. I am not too savvy with DDR testing, so I am not sure how aggressively the memtester tool checks DDR, but this is something you can try relatively easily to troubleshoot the issue.

    Additionally, as you mentioned, System Memory Map could also be causing issues, so reviewing it will be good as well.

    Regards,

    Takuma

  • Hi Takuma, a new week is beginning. 

    First at all the RAM design is the same as the EVM, LPDDR4(8G) x 4.

     - RAM Part Num : MT53E2G32D4DT-046

    If I can test DDR interleaving through memtester, I will do so and share it.

    Could you share how to use memtester? I think it needs to be run on uboot.

    Thanks, 

    Dongho

  • Hi 

    Additionally when I run memtester as below, a problem occurs.

    There is no problem up to 8G of memory area, but when testing by increasing the area to 9G, a problem occurs.

    HW engineer said that the Board is designed with 8GB x 4ea, which is the same as EVM, so I think I will have to try debugging the HW.

    But just in case, is there anything I should check first in SW?

    The memory settings in the k3-j784s4-evm.dts file are the same as in EVM.

       memory@80000000 {
             device_type = "memory";
             /* 32G RAM */
             reg = <0x00 0x80000000 0x00 0x80000000>,
             <0x08 0x80000000 0x07 0x80000000>;
      };

    ------------------------------------------------------------------------------------------

    Test Logs

    ------------------------------------------------------------------------------------------

    root@j784s4-evm:~# sudo memtester 8G 1
    memtester version 4.5.1 (64-bit)
    Copyright (C) 2001-2020 Charles Cazabon.
    Licensed under the GNU General Public License version 2 (only).

    pagesize is 4096
    pagesizemask is 0xfffffffffffff000
    want 8192MB (8589934592 bytes)
    got 8192MB (8589934592 bytes), trying mlock ...locked.
    Loop 1/1:
    Stuck Address : ok
    Random Value : ok
    Compare XOR : ok
    Compare SUB : ok
    Compare MUL : ok
    Compare DIV : ok
    Compare OR : ok
    Compare AND : ok
    Sequential Increment: ok
    Solid Bits : testing 7^C

    ------------------------------------------------------------------------------------------
    root@j784s4-evm:~#
    root@j784s4-evm:~#
    root@j784s4-evm:~# sudo memtester 9G 1
    memtester version 4.5.1 (64-bit)
    Copyright (C) 2001-2020 Charles Cazabon.
    Licensed under the GNU General Public License version 2 (only).

    pagesize is 4096
    pagesizemask is 0xfffffffffffff000
    want 9216MB (9663676416 bytes)
    got 9216MB (9663676416 bytes), trying mlock ...[13206.902256] Insufficient stack space to handle exception!
    [13206.902267] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902272] FAR: 0x0000000000000100
    [13206.902274] Task stack: [0xffff80000e410000..0xffff80000e414000]
    [13206.902274] Insufficient stack space to handle exception!
    [13206.902276] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902279] Overflow stack: [0xffff000f5c3b0300..0xffff000f5c3b1300]
    [13206.902280] ESR: 0x0000000096000045 -- DABT (current EL)
    [13206.902284] FAR: 0x0000000000000100
    [13206.902286] Task stack: [0xffff800009140000..0xffff800009144000]
    [13206.902283] CPU: 0 PID: 3407 Comm: memtester Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902288] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902290] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902290] Overflow stack: [0xffff000f5c360300..0xffff000f5c361300]
    [13206.902293] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902295] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902300] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902302] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902299] pc : el1h_64_sync+0x0/0x68
    [13206.902306] pc : el1h_64_sync+0x0/0x68
    [13206.902309] lr : call_on_irq_stack+0x24/0x4c
    [13206.902315] sp : 0000000000000100
    [13206.902313] lr : call_on_irq_stack+0x24/0x4c
    [13206.902318] x29: ffff80000e4138b0
    [13206.902319] sp : 0000000000000100
    [13206.902320] x28: ffff0008c13b0000
    [13206.902321] x29: ffff800009143bb0
    [13206.902323] x27: 0000000000000013 x28: ffff8000091529c0
    [13206.902325]
    [13206.902326] x26: ffff0008c6720cc0
    [13206.902327] x27: 00000000ffec880c
    [13206.902329] x25: ffff0008bdfb8700
    [13206.902331] x26: ffff8000091529c0
    [13206.902332] x24: 0000040000000000 x25: 0000000000000000
    [13206.902334]
    [13206.902336] x23: 0000000000000005
    [13206.902336] x24: 0000000000000000 x22: ffff800008b2a8bc
    [13206.902338]
    [13206.902340] x23: 0000000060000005
    [13206.902340] x21: ffff80000e413a60 x22: ffff800008b6c5bc
    [13206.902343]
    [13206.902345] x20: 0000000000000000
    [13206.902345] x21: ffff800009143d60
    [13206.902347]
    [13206.902347] x19: ffff800008ff46c8
    [13206.902349] x20: 0000000000000000
    [13206.902350] x18: 000000000046bc8a x19: ffff800008ff46c8
    [13206.902353]
    [13206.902354] x17: ffff800f533c4000
    [13206.902354] x18: 0000000000000000
    [13206.902357] x16: 0000000000000000
    [13206.902357] x17: ffff800f53374000 x15: 0000000000000000
    [13206.902360] x16: 0000000000000000
    [13206.902362] x15: 0000ffff6b2dd608
    [13206.902362] x14: 0000000000000001
    [13206.902365] x13: 0000000000000002
    [13206.902366] x14: 00000000000003f1 x12: ffff800009265728
    [13206.902368] x13: 0000000000000001
    [13206.902369]
    [13206.902371] x11: ffff800009265708
    [13206.902370] x12: 0000000000000000
    [13206.902373] x10: 0000000000000080
    [13206.902374] x11: 0000000000000001 x9 : ffff8000092656e8
    [13206.902376] x10: 00000000000009b0
    [13206.902377]
    [13206.902378] x9 : ffff800009143ce0
    [13206.902379] x8 : 0000000000000001
    [13206.902381]
    [13206.902381] x7 : ffff800f533c4000
    [13206.902383] x8 : ffff8000091533d0 x6 : 000004525edb76ee
    [13206.902385] x7 : 00000000000000c0
    [13206.902387]
    [13206.902388] x6 : 000004525edb76ee
    [13206.902388] x5 : ffff80000853f0c0
    [13206.902391] x4 : ffff80000e4138c1
    [13206.902392] x5 : ffff80000853f0c0 x3 : ffff80000e413910
    [13206.902394] x4 : ffff800009143bc1
    [13206.902396] x3 : ffff800009143c10
    [13206.902396] x2 : ffff80000e414000
    [13206.902399]
    [13206.902400] x1 : ffff80000853f0c0
    [13206.902400] x2 : ffff800009144000
    [13206.902402] x1 : ffff80000853f0c0
    [13206.902403] x0 : ffff80000e413910 x0 : ffff800009143c10
    [13206.902405]
    [13206.902406]
    [13206.902409] Kernel panic - not syncing: kernel stack overflow
    [13206.902412] CPU: 0 PID: 3407 Comm: memtester Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902416] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902418] Call trace:
    [13206.902419] dump_backtrace.part.0+0xdc/0xf0
    [13206.902426] show_stack+0x18/0x30
    [13206.902430] dump_stack_lvl+0x68/0x84
    [13206.902435] dump_stack+0x18/0x34
    [13206.902439] panic+0x188/0x348
    [13206.902443] nmi_panic+0xac/0xb0
    [13206.902448] panic_bad_stack+0x104/0x124
    [13206.902453] handle_bad_stack+0x34/0xc0
    [13206.902457] __bad_stack+0x78/0x7c
    [13206.902460] el1h_64_sync+0x0/0x68
    [13206.902462] do_interrupt_handler+0x80/0x8c
    [13206.902465] el1_interrupt+0x34/0x70
    [13206.902469] el1h_64_irq_handler+0x18/0x2c
    [13206.902474] el1h_64_irq+0x64/0x68
    [13206.902476] clear_page+0x18/0x4c
    [13206.902481] do_huge_pmd_anonymous_page+0x21c/0x7f0
    [13206.902487] __handle_mm_fault+0x430/0xc10
    [13206.902491] handle_mm_fault+0xec/0x280
    [13206.902494] __get_user_pages+0x200/0x3a0
    [13206.902499] populate_vma_page_range+0x58/0x74
    [13206.902504] __mm_populate+0xb4/0x190
    [13206.902508] do_mlock+0xd0/0x254
    [13206.902511] __arm64_sys_mlock+0x18/0x30
    [13206.902514] invoke_syscall+0x48/0x114
    [13206.902519] el0_svc_common.constprop.0+0xd4/0xfc
    [13206.902522] do_el0_svc+0x20/0x30
    [13206.902526] el0_svc+0x28/0xa0
    [13206.902529] el0t_64_sync_handler+0xbc/0x140
    [13206.902534] el0t_64_sync+0x18c/0x190
    [13206.902537] SMP: stopping secondary CPUs
    [13206.902544] Insufficient stack space to handle exception!
    [13206.902546] Insufficient stack space to handle exception!
    [13206.902546] Insufficient stack space to handle exception!
    [13206.902546] Insufficient stack space to handle exception!
    [13206.902547] Insufficient stack space to handle exception!
    [13206.902547] Insufficient stack space to handle exception!
    [13206.902548] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902550] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902551] FAR: 0x0000000000000100
    [13206.902551] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902551] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902551] ESR: 0x0000000096000046 -- DABT (current EL)
    [13206.902552] ESR: 0x0000000096000045 -- DABT (current EL)
    [13206.902554] FAR: 0x0000000000000100
    [13206.902554] Task stack: [0xffff8000093e8000..0xffff8000093ec000]
    [13206.902554] FAR: 0x0000000000000100
    [13206.902554] FAR: 0x0000000000000100
    [13206.902555] FAR: 0x0000000000000100
    [13206.902555] FAR: 0x0000000000000100
    [13206.902556] Task stack: [0xffff8000093d8000..0xffff8000093dc000]
    [13206.902557] Task stack: [0xffff800009618000..0xffff80000961c000]
    [13206.902557] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902557] Task stack: [0xffff800009610000..0xffff800009614000]
    [13206.902557] Task stack: [0xffff800009608000..0xffff80000960c000]
    [13206.902558] Task stack: [0xffff8000093e0000..0xffff8000093e4000]
    [13206.902559] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902560] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902560] Overflow stack: [0xffff000f5c39c300..0xffff000f5c39d300]
    [13206.902561] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902561] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902562] IRQ stack: [0x0000000000000000..0x0000000000004000]
    [13206.902562] Overflow stack: [0xffff000f5c3ec300..0xffff000f5c3ed300]
    [13206.902562] Overflow stack: [0xffff000f5c374300..0xffff000f5c375300]
    [13206.902563] Overflow stack: [0xffff000f5c3d8300..0xffff000f5c3d9300]
    [13206.902563] CPU: 0 PID: 0 Comm: swapper/3 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902565] Overflow stack: [0xffff000f5c3c4300..0xffff000f5c3c5300]
    [13206.902565] Overflow stack: [0xffff000f5c388300..0xffff000f5c389300]
    [13206.902566] CPU: 0 PID: 0 Comm: swapper/7 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902566] CPU: 0 PID: 0 Comm: swapper/1 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902568] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902567] CPU: 0 PID: 0 Comm: swapper/6 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902568] CPU: 0 PID: 0 Comm: swapper/2 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902570] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902569] CPU: 0 PID: 0 Comm: swapper/5 Tainted: G O 6.1.80-ti-g2e423244f8c0 #1
    [13206.902571] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902571] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902572] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902573] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902573] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902574] Hardware name: Texas Instruments J784S4 EVM (DT)
    [13206.902574] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902575] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902576] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902575] pc : el1h_64_sync+0x0/0x68
    [13206.902578] pstate: 200003c5 (nzCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
    [13206.902578] pc : el1h_64_sync+0x0/0x68
    [13206.902579] pc : el1h_64_sync+0x0/0x68
    [13206.902580] lr : call_on_irq_stack+0x24/0x4c
    [13206.902579] pc : el1h_64_sync+0x0/0x68
    [13206.902580] pc : el1h_64_sync+0x0/0x68
    [13206.902582] lr : call_on_irq_stack+0x24/0x4c
    [13206.902582] pc : el1h_64_sync+0x0/0x68
    [13206.902584] sp : 0000000000000100
    [13206.902584] lr : call_on_irq_stack+0x24/0x4c
    [13206.902584] lr : call_on_irq_stack+0x24/0x4c
    [13206.902585] lr : call_on_irq_stack+0x24/0x4c
    [13206.902586] x29: ffff8000093ebc40
    [13206.902587] sp : 0000000000000100
    [13206.902588] sp : 0000000000000100
    [13206.902587] lr : call_on_irq_stack+0x24/0x4c
    [13206.902589] x28: ffff0008bc2f0e40
    [13206.902589] sp : 0000000000000100
    [13206.902590] sp : 0000000000000100
    [13206.902590] x29: ffff8000093dbc40
    [13206.902590] x29: ffff80000961bc40
    [13206.902592] sp : 0000000000000100
    [13206.902592] x29: ffff8000093e3c40
    [13206.902592] x29: ffff800009613c40
    [13206.902592] x27: 0000000000000000
    [13206.902593] x28: ffff0008bc2de3c0
    [13206.902594] x28: ffff0008bc2f4740
    [13206.902595]
    [13206.902595] x29: ffff80000960bc40
    [13206.902595] x28: ffff0008bc2f3900
    [13206.902595] x28: ffff0008bc2f0000
    [13206.902596] x27: 0000000000000000
    [13206.902597] x27: 0000000000000000
    [13206.902598] x26: ffff0008bc2f0e40
    [13206.902599] x28: ffff0008bc2f2ac0
    [13206.902599]
    [13206.902599] x27: 0000000000000000
    [13206.902600] x27: 0000000000000000
    [13206.902600]
    [13206.902601] x25: 0000000000000000
    [13206.902602] x27: 0000000000000000
    [13206.902602] x26: ffff0008bc2de3c0
    [13206.902602]
    [13206.902602] x26: ffff0008bc2f4740
    [13206.902604] x24: 0000000000000000
    [13206.902604]
    [13206.902605] x25: 0000000000000000
    [13206.902605] x26: ffff0008bc2f0000
    [13206.902605]
    [13206.902606] x25: 0000000000000000
    [13206.902607]
    [13206.902607] x26: ffff0008bc2f3900
    [13206.902608] x24: 0000000000000000
    [13206.902608] x25: 0000000000000000
    [13206.902608] x26: ffff0008bc2f2ac0
    [13206.902610] x24: 0000000000000000
    [13206.902610] x23: 0000000060000005
    [13206.902611]
    [13206.902611] x25: 0000000000000000
    [13206.902611] x24: 0000000000000000
    [13206.902611] x25: 0000000000000000
    [13206.902612] x22: ffff800008b6c5bc
    [13206.902613]
    [13206.902613] x23: 0000000060000005
    [13206.902614] x24: 0000000000000000
    [13206.902614]
    [13206.902615] x23: 0000000060000005
    [13206.902615] x24: 0000000000000000
    [13206.902615] x21: ffff8000093ebdf0
    [13206.902618]
    [13206.902618]
    [13206.902617] x22: ffff800008b6c5bc
    [13206.902618] x23: 0000000060000005
    [13206.902619] x22: ffff800008b6c5bc
    [13206.902619]
    [13206.902620] x20: 0000000000000000
    [13206.902620] x23: 0000000060000005
    [13206.902622] x23: 0000000060000005
    [13206.902621] x22: ffff800008b6c5bc
    [13206.902621] x21: ffff8000093dbdf0
    [13206.902622] x21: ffff80000961bdf0
    [13206.902623] x22: ffff800008b6c5bc
    [13206.902623] x19: ffff800008ff46c8
    [13206.902624] x22: ffff800008b6c5bc
    [13206.902625]
    [13206.902625]
    [13206.902625] x21: ffff8000093e3df0
    [13206.902626] x18: 0000000000000000
    [13206.902627] x21: ffff800009613df0
    [13206.902627] x20: 0000000000000000
    [13206.902627] x20: 0000000000000000
    [13206.902627] x21: ffff80000960bdf0
    [13206.902628]
    [13206.902630]
    [13206.902629] x19: ffff800008ff46c8
    [13206.902630]
    [13206.902631] x20: 0000000000000000
    [13206.902631]
    [13206.902631] x19: ffff800008ff46c8
    [13206.902632] x17: ffff800f533b0000
    [13206.902633] x18: 0000000000000000
    [13206.902633] x20: 0000000000000000
    [13206.902633] x20: 0000000000000000
    [13206.902634] x19: ffff800008ff46c8
    [13206.902634] x18: 0000000000000000
    [13206.902634] x16: 0000000000000000
    [13206.902636]
    [13206.902636] x19: ffff800008ff46c8
    [13206.902636] x19: ffff800008ff46c8
    [13206.902637]
    [13206.902637] x18: 0000000000000000
    [13206.902638] x17: ffff800f53400000
    [13206.902638] x15: 00000000353f70d0
    [13206.902640]
    [13206.902639] x17: ffff800f53388000
    [13206.902639] x18: 0000000000000000
    [13206.902639] x18: 0000000000000000
    [13206.902641] x16: 0000000000000000
    [13206.902642]
    [13206.902643]
    [13206.902643] x17: ffff800f5339c000
    [13206.902643] x16: 0000000000000000
    [13206.902643]
    [13206.902644] x15: 0000000000000000
    [13206.902644] x14: 0000000000000264
    [13206.902645] x17: ffff800f533ec000
    [13206.902645] x17: ffff800f533d8000
    [13206.902646]
    [13206.902646] x15: 00000000353e9c40
    [13206.902646] x16: 0000000000000000
    [13206.902647] x13: 0000000000000001
    [13206.902647] x16: 0000000000000000
    [13206.902647] x16: 0000000000000000
    [13206.902649] x14: 00000000000003d0
    [13206.902650] x15: 00000006d999e3f6
    [13206.902650]
    [13206.902650] x15: 0000000000000000
    [13206.902650] x12: 0000000000000000
    [13206.902651] x15: 00000002ef8b188e
    [13206.902653]
    [13206.902653]
    [13206.902653] x14: 00000000000000e9
    [13206.902653]
    [13206.902653] x13: 0000000000000001
    [13206.902655]
    [13206.902656] x14: 00000000000003df
    [13206.902656] x11: 0000000000000001
    [13206.902656] x14: 0000000000000365
    [13206.902656] x13: 0000000000000001
    [13206.902656] x14: 00000bbf26e5eada
    [13206.902656] x12: 0000000000000000
    [13206.902659]
    [13206.902658] x10: 00000000000009b0
    [13206.902660] x12: 0000000000000000
    [13206.902658] x13: 0000000000000001
    [13206.902660] x13: 0000000000000001
    [13206.902660] x13: 0000000000000001
    [13206.902662] x11: 0000000000000001
    [13206.902662]
    [13206.902662] x9 : ffff8000093ebd70
    [13206.902663] x12: 0000000000000000
    [13206.902663] x12: 0000000000000001
    [13206.902664] x12: 0000000000000000
    [13206.902665] x11: 0000000000000001
    [13206.902665] x10: 00000000000009b0
    [13206.902666]
    [13206.902667]
    [13206.902667]
    [13206.902668]
    [13206.902668] x10: 00000000000009b0
    [13206.902668] x8 : ffff0008bc2f1850
    [13206.902669] x11: 0000000000000001
    [13206.902668] x9 : ffff80000961bd70
    [13206.902669] x11: 0000000000000001
    [13206.902670] x11: 0000000000000001
    [13206.902671] x7 : 00000000000000c0
    [13206.902672] x10: 00000000000009b0
    [13206.902672]
    [13206.902671] x9 : ffff8000093dbd70
    [13206.902672] x10: 00000000000009b0
    [13206.902672] x10: 00000000000009b0
    [13206.902674] x6 : 000004525edb76ee
    [13206.902675] x8 : ffff0008bc2f5150
    [13206.902676] x9 : ffff800009613d70
    [13206.902675] x9 : ffff8000093e3d70
    [13206.902676]
    [13206.902676] x9 : ffff80000960bd70
    [13206.902677]
    [13206.902678] x7 : 0000000000000000
    [13206.902679]
    [13206.902679]
    [13206.902680]
    [13206.902679] x8 : ffff0008bc2dedd0
    [13206.902679] x5 : ffff80000853f0c0
    [13206.902681] x8 : ffff0008bc2f4310
    [13206.902681] x6 : 000004525edb76ee
    [13206.902682] x8 : ffff0008bc2f0a10
    [13206.902683] x8 : ffff0008bc2f34d0
    [13206.902683] x7 : 0000000000000000
    [13206.902683] x4 : ffff8000093ebc51
    [13206.902685] x7 : 00000000000000c0
    [13206.902685]
    [13206.902685] x7 : 0000000000000000
    [13206.902686] x7 : 00000000000000c0
    [13206.902687] x6 : 000004525edb76ee
    [13206.902687] x5 : ffff80000853f0c0
    [13206.902687] x3 : ffff8000093ebca0
    [13206.902688] x6 : 000004525edb76ee
    [13206.902689] x6 : 000004525edb76ee
    [13206.902689] x6 : 000004525edb76ee
    [13206.902690]
    [13206.902691]
    [13206.902690] x4 : ffff80000961bc51
    [13206.902692]
    [13206.902692]
    [13206.902692] x5 : ffff80000853f0c0
    [13206.902693] x2 : ffff8000093ec000
    [13206.902692]
    [13206.902693] x3 : ffff80000961bca0
    [13206.902693] x5 : ffff80000853f0c0
    [13206.902694] x5 : ffff80000853f0c0
    [13206.902695] x4 : ffff8000093dbc51
    [13206.902696] x5 : ffff80000853f0c0
    [13206.902696] x1 : ffff80000853f0c0
    [13206.902696]
    [13206.902697] x4 : ffff800009613c51
    [13206.902697] x4 : ffff80000960bc51
    [13206.902698] x0 : ffff8000093ebca0
    [13206.902698] x3 : ffff8000093dbca0
    [13206.902699] x2 : ffff80000961c000
    [13206.902699] x4 : ffff8000093e3c51 x3 : ffff80000960bca0
    [13206.902700] x3 : ffff800009613ca0
    [13206.902702]
    [13206.902702] x3 : ffff8000093e3ca0
    [13206.902702]
    [13206.902703] x1 : ffff80000853f0c0
    [13206.902704]
    [13206.902704]
    [13206.902706] x2 : ffff800009614000
    [13206.902706] x2 : ffff80000960c000
    [13206.902706]
    [13206.902707] x2 : ffff8000093dc000
    [13206.902707] x0 : ffff80000961bca0
    [13206.902708] x1 : ffff80000853f0c0
    [13206.902709] x1 : ffff80000853f0c0
    [13206.902710]
    [13206.902709] x2 : ffff8000093e4000
    [13206.902710] x1 : ffff80000853f0c0
    [13206.902711] x0 : ffff800009613ca0
    [13206.902712] x0 : ffff80000960bca0
    [13206.902713]
    [13206.902712] x1 : ffff80000853f0c0
    [13206.902712] x0 : ffff8000093dbca0
    [13206.902714]
    [13206.902715]
    [13206.902715] x0 : ffff8000093e3ca0
    [13207.966039] SMP: failed to stop secondary CPUs 0-7
    [13207.966052] Kernel Offset: disabled
    [13207.966053] CPU features: 0x40000,20028084,0000420b
    [13207.966056] Memory Limit: none
    [13209.538412] ---[ end Kernel panic - not syncing: kernel stack overflow ]---

  • Hi Dongho,

    Could you create a new E2E forum post by clicking the "Ask a related question" button? And make the new post focused on memory instead of PCIe? This will route the new forum post to our memory experts within TI that will be able to help you out better.

    But from my limited knowledge with memory, there is a DDR config tool that is used to configure the DDR, and how to use this tool is in this application note: https://www.ti.com/lit/an/spracu8b/spracu8b.pdf?ts=1747054021791&ref_url=https%253A%252F%252Fwww.google.com%252F

    Perhaps there was a wrong configuration within the tool? Or, it could be hardware as you suspect. But in either case, it definitely looks to be an issue with DDR.

    Regards,

    Takuma

  • Hi Takuma

    OK ! Thank for you support. 

    Have a nice day! 

    Thanks,

    Dongho

  • Hi Dongho,

    I hope you have a nice day as well! 

    Best of luck with the debug!

    Regards,

    Takuma