• State Resolved
  • Locked Locked
  • Replies 19 replies
  • Subscribers 99 subscribers
  • Views 1016 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM-Q1: TDA4VM-Q1: startup failure

Tim JIANG
Prodigy 170 points

Part Number: TDA4VM-Q1

hello there,

in ourADAS  L2+ domian application, SOC MPN is TDA4VM88T5CALFRQ1, a return part from OEM with Zero-kilometer indicates that the SOC failed to startup. And we can reoccur this fault in our labaritory with 8/10 probability, can you support for this troubleshooting?

normal startup log.txt 

failure startup log.txt 

thank you!

  • 0
    TI__Guru**** 171020 points

    Hello,

    [ ***  ] (2 of 6) A start job is running for /dev/mmcblk0p9 (7s / 1min 30s)

    Are you using eMMC for rootfs? If yes can you try formatting and re-writing that?

    - Keerthy

  • 0 in reply to Keerthy J
    Prodigy 170 points

    Hello Keerthy,

    Sorry for the delay. Yes, we take eMMC as rootfs, and we try to reflash the whole software, and the failure still exists.

    (1) TI-SDK version is V8.6

    (2) kernel version is following

    Fullscreen during operation failure log_1125.txt Download 
    NOTICE:  BL31: v2.4(release):07.03.00.005-dirty
NOTICE:  BL31: Built : 00:15:40, Apr 10 2021
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.000000] cma: Failed to reserve 512 MiB
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.057195] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.063486] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.069821] debugfs: Directory '31010000.pat' with parent 'regmap' already present!
[    0.077803] debugfs: Directory '31011000.pat' with parent 'regmap' already present!
[    0.085733] debugfs: Directory '31012000.pat' with parent 'regmap' already present!
[    0.093657] debugfs: Directory '31013000.pat' with parent 'regmap' already present!
[    0.101593] debugfs: Directory '31014000.pat' with parent 'regmap' already present!
[    0.123727] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.130029] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.235383] am65-cpsw-nuss 46000000.ethernet: Failed to request tx dma channel -517
[    0.287575] davinci_gpio 42110000.gpio: IRQ index 0 not found
[    0.298694] davinci_gpio 601000.gpio: IRQ index 2 not found

Welcome to Linux!

[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Huge Pages File System...
         Starting Journal Service...
         Starting Create System Users...
         Starting udev Coldplug all Devices...
[  OK  ] Started Journal Service.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Started Create System Users.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
         Starting Network Service...
[  OK  ] Started Network Service.
[  OK  ] Started udev Coldplug all Devices.
[  OK  ] Reached target Network.
[    1.355459] platform 5c00000.r5f: configured R5F for remoteproc mode
[    1.405490] platform 5d00000.r5f: configured R5F for remoteproc mode
[    1.491264] remoteproc remoteproc4: request_firmware failed: -2
[    1.523608] platform 5e00000.r5f: configured R5F for IPC-only mode
[    1.579754] remoteproc remoteproc5: request_firmware failed: -2
[    1.591933] platform 5f00000.r5f: configured R5F for remoteproc mode
[    1.640312] remoteproc remoteproc6: request_firmware failed: -2
[  OK  ] Found device /dev/mmcblk0p11.
         Mounting iDC Recordfs Partition (/mnt)...
[  OK  ] Mounted iDC Recordfs Partition (/mnt).
[  OK  ] Found device /dev/mmcblk0p10.
[  OK  ] Found device /dev/mmcblk0p9.
[  OK  ] Found device /dev/mmcblk0p5.
         Mounting iDC APP Partition (/usr/ext)...
[  OK  ] Mounted iDC APP Partition (/usr/ext).
[  OK  ] Found device /dev/mmcblk0p7.
         Mounting iDC APP Partition (/opt/idc)...
[  OK  ] Mounted iDC APP Partition (/opt/idc).
         Mounting iDC usrdata Partition (/opt/idc/dat)...
         Mounting iDC sv3d Patition (/opt/idc/etc/sv3d)...
[  OK  ] Mounted iDC usrdata Partition (/opt/idc/dat).
[  OK  ] Mounted iDC sv3d Patition (/opt/idc/etc/sv3d).
[  OK  ] Found device /dev/mmcblk0p3.
         Mounting iDC pdm Patition (/opt/idc/pdm)...
[  OK  ] Mounted iDC pdm Patition (/opt/idc/pdm).
[  OK  ] Reached target Local File Systems.
         Starting Rebuild Dynamic Linker Cache...
[  OK  ] Started Rebuild Dynamic Linker Cache.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Daily rotation of log files.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Timers.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
[DEPEND] Dependency failed for Login Service.
[  OK  ] Listening on dropbear.socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
[  OK  ] Started irqbalance daemon.
[  OK  ] Started Serial Getty on ttyS2.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User System.
         Starting uart service of iDC...
         Starting OTA service of iDC...
[  OK  ] Started Service for iDC.
[  OK  ] Started OTA service of iDC.


iMotion Domain Computor 
http://www.imotion.ai

iDC 2024.01 iDC-mid ttyS2

iDC-mid login:     
iDC-mid login: 
iDC-mid login: 
iDC-mid login: 
iDC-mid login: 
iDC-mid login: 
iDC-mid login: 
iDC-mid login: root
root@iDC-mid:~# top
top - 08:05:57 up 0 min,  0 users,  load average: 0.36, 0.08, 0.03
Tasks: 101 total,   1 running, 100 sleeping,   0 stopped,   0 zombie
%Cpu(s): 22.8 us,  4.0 sy,  0.0 ni, 71.0 id,  0.0 wa,  2.0 hi,  0.2 si,  0.0 st
MiB Mem :   2208.1 total,    897.3 free,   1025.4 used,    285.3 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.    842.2 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                                                     
  818 root      20   0  504512  46656  39424 S  12.7   2.1   0:01.40 idc_ipark.bin                                                                                                                                                               
  820 root      20   0 1538752 316096  72192 S  12.7  14.0   0:05.85 idc_iuni.bin                                                                                                                                                                
  822 root      20   0 1685120 119104  46656 S   8.3   5.3   0:01.58 idc_vial.bin                                                                                                                                                                
  776 root      20   0  534336  48896  44032 S   7.3   2.2   0:01.89 idc_cpf.bin                                                                                                                                                                 
  821 root      20   0  673344  66560  42048 S   5.4   2.9   0:00.63 idc_viper.bin                                                                                                                                                               
  823 root      20   0  844352  40896  39360 S   3.4   1.8   0:00.46 idc_dalo_ser.bi                                                                                                                                                             
  819 root      20   0  766400  47232  38848 S   2.4   2.1   0:00.29 idc_iwalle.bin                                                                                                                                                              
  767 root      20   0  101888   4992   3840 S   1.0   0.2   0:00.10 idc_flasher.bin                                                                                                                                                             









root@iDC-mid:~# [   59.423431] Unable to handle kernel paging request at virtual address f7ff00081170d918
[   59.431340] Mem abort info:
[   59.434121]   ESR = 0x96000004
[   59.437162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   59.442455]   SET = 0, FnV = 0
[   59.445495]   EA = 0, S1PTW = 0
[   59.448621] Data abort info:
[   59.451487]   ISV = 0, ISS = 0x00000004
[   59.455308]   CM = 0, WnR = 0
[   59.458262] [f7ff00081170d918] address between user and kernel address ranges
[   59.465378] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[   59.470933] Modules linked in: can_raw can rpmsg_client_sample rpmsg_char irq_pruss_intc omap_rng rng_core crct10dif_ce ti_j721e_cpsw_virt_mac pvrsrvkm(O) ti_k3_r5_remoteproc pruss sa2ul sha512_generic authenc m_can_platform m_can cdns_dphy can_dev rti_wdt ti_k3_dsp_remoteproc virtio_rpmsg_bus ipv6
[   59.497312] CPU: 0 PID: 873 Comm: CodingMgr Tainted: G           O      5.4.106-g023faefa70 #32
[   59.505986] Hardware name: Texas Instruments K3 J721E SoC (DT)
[   59.511807] pstate: 20000085 (nzCv daIf -PAN -UAO)
[   59.516589] pc : timerqueue_add+0x44/0xc0
[   59.520585] lr : hrtimer_start_range_ns+0x1c8/0x2a8
[   59.525445] sp : ffff80001692fcb0
[   59.528745] x29: ffff80001692fcb0 x28: ffff00084d3a1c00 
[   59.534038] x27: 0000000000000040 x26: ffff800010d8db40 
[   59.539332] x25: ffff00087fa67f40 x24: ffff00084d3a1c00 
[   59.544625] x23: 0000000000000040 x22: ffff00087fa67f00 
[   59.549918] x21: 0000000000000001 x20: ffff00087fa67f40 
[   59.555211] x19: 0000000000000000 x18: 0000000000000000 
[   59.560504] x17: 0000000000000000 x16: 0000000000000000 
[   59.565798] x15: 0000000000000000 x14: 0000000000000000 
[   59.571091] x13: 0000000000000000 x12: 0000000000000000 
[   59.576385] x11: 0000000000000000 x10: 0000000000000000 
[   59.581682] x9 : 0000000000000000 x8 : 0000000005f5e100 
[   59.586976] x7 : 0000000000000000 x6 : 0000000ddb79a93a 
[   59.592268] x5 : ffff80001692fde0 x4 : ffff000812474408 
[   59.597564] x3 : f7ff00081170d900 x2 : f7ff00081170d900 
[   59.602864] x1 : ffff00087fa67f60 x0 : ffff80001692fde0 
[   59.608158] Call trace:
[   59.610593]  timerqueue_add+0x44/0xc0
[   59.614240]  hrtimer_start_range_ns+0x1c8/0x2a8
[   59.618754]  do_nanosleep+0x70/0x170
[   59.622314]  hrtimer_nanosleep+0xcc/0x1a0
[   59.626307]  __arm64_sys_nanosleep+0x74/0xa0
[   59.630564]  el0_svc_common.constprop.0+0x68/0x160
[   59.635338]  el0_svc_handler+0x20/0x80
[   59.639072]  el0_svc+0x8/0x204
[   59.642116] Code: f9400843 91004044 b4000123 aa0303e2 (f9400c43) 
[   59.648191] ---[ end trace 1db4ac0344fba8c7 ]---
[   59.652792] note: CodingMgr[873] exited with preempt_count 1
     
    Fullscreen failure startup log_1125.txt Download 
    NOTICE:  BL31: v2.4(release):07.03.00.005-dirty
NOTICE:  BL31: Built : 00:15:40, Apr 10 2021
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.000000] cma: Failed to reserve 512 MiB
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.057526] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.063862] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.070193] debugfs: Directory '31010000.pat' with parent 'regmap' already present!
[    0.078175] debugfs: Directory '31011000.pat' with parent 'regmap' already present!
[    0.086101] debugfs: Directory '31012000.pat' with parent 'regmap' already present!
[    0.094033] debugfs: Directory '31013000.pat' with parent 'regmap' already present!
[    0.101961] debugfs: Directory '31014000.pat' with parent 'regmap' already present!
[    0.124057] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.130375] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.235714] am65-cpsw-nuss 46000000.ethernet: Failed to request tx dma channel -517
[    0.283311] davinci_gpio 42110000.gpio: IRQ index 0 not found
[    0.293552] davinci_gpio 601000.gpio: IRQ index 2 not found

Welcome to Linux!

[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Huge Pages File System...
         Starting Journal Service...
         Starting Create System Users...
         Starting udev Coldplug all Devices...
[    0.927094] Unable to handle kernel paging request at virtual address dfff800010221fd8
[    0.939180] Mem abort info:
[    0.941961]   ESR = 0x86000004
[    0.945008]   EC = 0x21: IABT (current EL), IL = 32 bits
[    0.950306]   SET = 0, FnV = 0
[    0.953350]   EA = 0, S1PTW = 0
[    0.956481] [dfff800010221fd8] address between user and kernel address ranges
[    0.963600] Internal error: Oops: 86000004 [#1] PREEMPT SMP
[    0.969156] Modules linked in: ipv6
[    0.972633] CPU: 0 PID: 9 Comm: ksoftirqd/0 Not tainted 5.4.106-g023faefa70 #32
[    0.979919] Hardware name: Texas Instruments K3 J721E SoC (DT)
[    0.985733] pstate: a0000005 (NzCv daif -PAN -UAO)
[    0.990510] pc : 0xdfff800010221fd8
[    0.993988] lr : rcu_core+0x2b4/0x4b0
[    0.997635] sp : ffff80001112fcd0
[    1.000935] x29: ffff80001112fcd0 x28: 0000000000000000 
[    1.006230] x27: ffff800010d8b500 x26: 0000000000000000 
[    1.011524] x25: ffff0008101b7000 x24: 000000000000000a 
[    1.016819] x23: ffff80001112fd38 x22: ffff00087fa6b850 
[    1.022112] x21: ffff800010d8b680 x20: ffff0008101b7000 
[    1.027406] x19: ffff00087fa6b800 x18: 0000000000000000 
[    1.032700] x17: 0000000000000000 x16: 0000000000000000 
[    1.037993] x15: 0000000000000000 x14: 0000000000000000 
[    1.043287] x13: 0000000000000000 x12: 000000000000025e 
[    1.048581] x11: 0000000000000370 x10: 00000000000009e0 
[    1.053874] x9 : ffff80001112fd40 x8 : ffff0008101b7a40 
[    1.059169] x7 : ffff00087fa6ac40 x6 : ffff0008101b7528 
[    1.064463] x5 : 0000000000000001 x4 : ffffffe001e582a0 
[    1.069756] x3 : 0000000081000092 x2 : ffff0008160a6f00 
[    1.075050] x1 : dfff800010221fd8 x0 : ffff0008160a6e00 
[    1.080345] Call trace:
[    1.082778]  0xdfff800010221fd8
[    1.085906]  rcu_core_si+0xc/0x18
[    1.089208]  __do_softirq+0x120/0x25c
[    1.092856]  run_ksoftirqd+0x3c/0x50
[    1.096421]  smpboot_thread_fn+0x1e4/0x2a8
[    1.100502]  kthread+0x138/0x158
[    1.103717]  ret_from_fork+0x10/0x1c
[    1.107281] Code: bad PC value
[    1.110326] ---[ end trace 71a45338d4be0041 ]---
[    1.114926] Kernel panic - not syncing: Fatal exception in interrupt
[    1.121260] SMP: stopping secondary CPUs
[    1.125169] Kernel Offset: disabled
[    1.128643] CPU features: 0x0002,20006008
[    1.132636] Memory Limit: none
[    1.135678] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

    It looks like issue about kernel SMP PREEMPT, copy the failure log contents just for convenience: 

    root@iDC-mid:~# [ 59.423431] Unable to handle kernel paging request at virtual address f7ff00081170d918
    [ 59.431340] Mem abort info:
    [ 59.434121] ESR = 0x96000004
    [ 59.437162] EC = 0x25: DABT (current EL), IL = 32 bits
    [ 59.442455] SET = 0, FnV = 0
    [ 59.445495] EA = 0, S1PTW = 0
    [ 59.448621] Data abort info:
    [ 59.451487] ISV = 0, ISS = 0x00000004
    [ 59.455308] CM = 0, WnR = 0
    [ 59.458262] [f7ff00081170d918] address between user and kernel address ranges
    [ 59.465378] Internal error: Oops: 96000004 [#1] PREEMPT SMP
    [ 59.470933] Modules linked in: can_raw can rpmsg_client_sample rpmsg_char irq_pruss_intc omap_rng rng_core crct10dif_ce ti_j721e_cpsw_virt_mac pvrsrvkm(O) ti_k3_r5_remoteproc pruss sa2ul sha512_generic authenc m_can_platform m_can cdns_dphy can_dev rti_wdt ti_k3_dsp_remoteproc virtio_rpmsg_bus ipv6
    [ 59.497312] CPU: 0 PID: 873 Comm: CodingMgr Tainted: G O 5.4.106-g023faefa70 #32
    [ 59.505986] Hardware name: Texas Instruments K3 J721E SoC (DT)

  • 0 in reply to Tim JIANG
    TI__Guru**** 171020 points

    Hi,

    Thanks for the details & rewriting eMMC. Is this seen on all samples or this is the only failing sample?

    - Keerthy

  • 0 in reply to Keerthy J
    Prodigy 170 points

    Hi Keerthy,

    As so far, we delivered about 40k pcs, but only this one failing sample. 

  • 0 in reply to Tim JIANG
    TI__Guru**** 171020 points

    Hello Tim,

    On the failing sample is it possible to read the CPU voltage and core rail voltages and bump them to 0.85 volts?

    I believe you are using sbl to boot Linux right?

    Best Regards,

    Keerthy 

  • 0 in reply to Keerthy J
    Prodigy 170 points

    Hi Keerthy,

    Yes, we're using sbl to boot Linux. And in our application software, it has set VDD_CPU=0.9V, VDD_CORE=0.85V, VDD_RAM=0.85V, during sbl boot as following:

  • 0 in reply to Tim JIANG
    TI__Guru**** 171020 points

    Hi,

    So the failing sample still fails even when VDD_CPU is at .9V? Also any particular reason why VDD_CPU was bumped to .9V?

    - Keerthy

  • 0 in reply to Keerthy J
    Prodigy 170 points

    Hi,

    Yes, the failing sample still fails even when VDD_CPU is at 0.9V. ( Why did we set VDD_CPU at 0.9V? because we solved an startup failing issue at low temperatue stress with 0.9V )

  • 0 in reply to Tim JIANG
    TI__Guru**** 171020 points

    Tim,

    I will loop in our HW experts. Due to Thanks giving holidays in the US, we will respond early next week.

    Thanks,
    Keerthy

  • 0 in reply to Keerthy J
    TI__Guru 57697 points

    Hi Tim,

    Can you experiment with setting the various rails at their datasheet recommended values?  I.e., vdd_core = 0.80V, and vdd_cpu = AVS voltage level.

    Thanks,

    Kyle

  • 0 in reply to kcastille
    Prodigy 170 points

    Hi Kcastille,

    Ok, we're working on it, and will keep you informed ASAP.

    Thank you!

  • 0 in reply to kcastille
    Prodigy 170 points

    Hi Kcastille,

    We tried to set various rails at their recommended value, VDD_CPU=AVS voltage class-0, VDD_CORE=0.8V, the failure still exists. More details can be found following.

    Fullscreen startup fail log_20251203-1.txt Download 
    NOTICE:  BL31: v2.4(release):07.03.00.005-dirty
NOTICE:  BL31: Built : 00:15:40, Apr 10 2021
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.000000] cma: Failed to reserve 512 MiB
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.053999] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.060326] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.066657] debugfs: Directory '31010000.pat' with parent 'regmap' already present!
[    0.074630] debugfs: Directory '31011000.pat' with parent 'regmap' already present!
[    0.082553] debugfs: Directory '31012000.pat' with parent 'regmap' already present!
[    0.090527] debugfs: Directory '31013000.pat' with parent 'regmap' already present!
[    0.098459] debugfs: Directory '31014000.pat' with parent 'regmap' already present!
[    0.120813] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.127117] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.252619] am65-cpsw-nuss 46000000.ethernet: Failed to request tx dma channel -517
[    0.300228] davinci_gpio 42110000.gpio: IRQ index 0 not found
[    0.313590] davinci_gpio 601000.gpio: IRQ index 2 not found
[    0.370615] debugfs: Directory 'pd:242' with parent 'pm_genpd' already present!
[    0.378010] debugfs: Directory 'pd:241' with parent 'pm_genpd' already present!
[    0.385334] debugfs: Directory 'pd:240' with parent 'pm_genpd' already present!
[    0.392726] debugfs: Directory 'pd:239' with parent 'pm_genpd' already present!

Welcome to Linux!

[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on Network Service Netlink Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Huge Pages File System...
         Mounting POSIX Message Queue File System...
         Mounting /media/ram...
         Mounting /var/volatile...
         Starting Journal Service...
         Starting Create Static Device Nodes in /dev...
         Starting udev Coldplug all Devices...
[  OK  ] Started Journal Service.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted /media/ram.
[  OK  ] Mounted /var/volatile.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started udev Coldplug all Devices.
[    1.338644] Unable to handle kernel paging request at virtual address ffff800010c5b7c0
[    1.346608] Mem abort info:
[    1.349429]   ESR = 0x96000047
[    1.352485]   EC = 0x25: DABT (current EL), IL = 32 bits
[    1.357786]   SET = 0, FnV = 0
[    1.360831]   EA = 0, S1PTW = 0
[    1.363960] Data abort info:
[    1.366826]   ISV = 0, ISS = 0x00000047
[    1.370650]   CM = 0, WnR = 1
[    1.373610] swapper pgtable: 64k pages, 48-bit VAs, pgdp=0000000080bd0000
[    1.380389] [ffff800010c5b7c0] pgd=00000008ffff0003, pud=00000008ffff0003, pmd=00000008fffe0003, pte=0000000000000000
[    1.390989] Internal error: Oops: 96000047 [#1] PREEMPT SMP
[    1.396545] Modules linked in: ipv6
[    1.400023] CPU: 0 PID: 144 Comm: systemd-udevd Not tainted 5.4.106-g023faefa70 #1
[    1.407570] Hardware name: Texas Instruments K3 J721E SoC (DT)
[    1.413385] pstate: 80000005 (Nzcv daif -PAN -UAO)
[    1.418170] pc : queued_spin_lock_slowpath+0x268/0x3c8
[    1.423292] lr : _raw_spin_lock+0x5c/0x68
[    1.427285] sp : ffff80001252fb00
[    1.430585] x29: ffff80001252fb00 x28: 0000000000000040 
[    1.435878] x27: 0000000000000001 x26: ffff80001252fd10 
[    1.441171] x25: 0000000000000000 x24: ffff0008187e0dd0 
[    1.446465] x23: 0000000000008180 x22: 0000000000000000 
[    1.451759] x21: ffff000811a14800 x20: ffff00081182b9c0 
[    1.457053] x19: ffff00081182b980 x18: 0000000000000000 
[    1.462347] x17: 0000000000000000 x16: 0000000000000000 
[    1.467640] x15: 0000000000000000 x14: 0000000000000000 
[    1.472934] x13: ffff000816a03800 x12: 0000000000000000 
[    1.478229] x11: fefefefefefefeff x10: 0000000000000000 
[    1.483523] x9 : 0000000000000000 x8 : ffff00087fa6b7c0 
[    1.488818] x7 : ffff000816a03800 x6 : ffff800010c5b7c0 
[    1.494111] x5 : ffff00087fa6b7c0 x4 : ffff800010c5b7c0 
[    1.499406] x3 : ffff00081182b9c0 x2 : ffff800010c5b7c0 
[    1.504700] x1 : 0000000000040000 x0 : ffff00087fa6b7c8 
[    1.509993] Call trace:
[    1.512430]  queued_spin_lock_slowpath+0x268/0x3c8
[    1.517204]  _raw_spin_lock+0x5c/0x68
[    1.520853]  shmem_reserve_inode+0x34/0x70
[    1.524934]  shmem_get_inode+0x30/0x1c0
[    1.528754]  shmem_mknod+0x28/0xe8
[    1.532141]  shmem_create+0x14/0x20
[    1.535616]  path_openat+0x810/0x1130
[    1.539263]  do_filp_open+0x74/0xf8
[    1.542736]  do_sys_open+0x168/0x218
[    1.546297]  __arm64_sys_openat+0x20/0x28
[    1.550295]  el0_svc_common.constprop.0+0x68/0x160
[    1.555069]  el0_svc_handler+0x20/0x80
[    1.558803]  el0_svc+0x8/0x204
[    1.561848] Code: 91002100 8b0400c4 f862d94a aa0403e2 (f82a6848) 
[    1.567924] ---[ end trace 80fe7d60303b52f2 ]---
[    1.572550] note: systemd-udevd[144] exited with preempt_count 1
[   ***] (1 of 6) A start job is running for /dev/mmcblk0p11 (16s / 1min 30s)
    Fullscreen startup fail log_20251203-2.txt Download 
    NOTICE:  BL31: v2.4(release):07.03.00.005-dirty
NOTICE:  BL31: Built : 00:15:40, Apr 10 2021
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.000000] cma: Failed to reserve 512 MiB
ERROR:   GTC_CNTFID0 is 0! Assuming 200000000 Hz. Fix Bootloader
[    0.052904] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.059177] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.065508] debugfs: Directory '31010000.pat' with parent 'regmap' already present!
[    0.073485] debugfs: Directory '31011000.pat' with parent 'regmap' already present!
[    0.081484] debugfs: Directory '31012000.pat' with parent 'regmap' already present!
[    0.089409] debugfs: Directory '31013000.pat' with parent 'regmap' already present!
[    0.097339] debugfs: Directory '31014000.pat' with parent 'regmap' already present!
[    0.119584] k3-ringacc 2b800000.ringacc: Failed to get MSI domain
[    0.125886] k3-ringacc 3c000000.ringacc: Failed to get MSI domain
[    0.251364] am65-cpsw-nuss 46000000.ethernet: Failed to request tx dma channel -517
[    0.298993] davinci_gpio 42110000.gpio: IRQ index 0 not found
[    0.312413] davinci_gpio 601000.gpio: IRQ index 2 not found
[    0.369438] debugfs: Directory 'pd:242' with parent 'pm_genpd' already present!
[    0.376820] debugfs: Directory 'pd:241' with parent 'pm_genpd' already present!
[    0.384144] debugfs: Directory 'pd:240' with parent 'pm_genpd' already present!
[    0.391532] debugfs: Directory 'pd:239' with parent 'pm_genpd' already present!

Welcome to Linux!

[    0.666005] systemd[119]: /lib/systemd/system-generators/systemd-gpt-auto-generator terminated by signal ILL.
[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on Network Service Netlink Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Huge Pages File System...
         Mounting POSIX Message Queue File System...
         Mounting /media/ram...
         Mounting /var/volatile...
         Starting Journal Service...
         Starting Create Static Device Nodes in /dev...
         Starting udev Coldplug all Devices...
[  OK  ] Started Journal Service.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted /media/ram.
[  OK  ] Mounted /var/volatile.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started udev Coldplug all Devices.
[    1.617709] platform 5c00000.r5f: configured R5F for remoteproc mode
[    1.746997] platform 5d00000.r5f: configured R5F for remoteproc mode
[    1.765377] remoteproc remoteproc4: request_firmware failed: -2
[    1.787672] platform 5e00000.r5f: configured R5F for IPC-only mode
[    1.849354] remoteproc remoteproc5: request_firmware failed: -2
[    1.867193] platform 5f00000.r5f: configured R5F for remoteproc mode
[    1.890619] remoteproc remoteproc6: request_firmware failed: -2
[    2.041930] Unable to handle kernel paging request at virtual address f7ff000816f16f38
[    2.049898] Mem abort info:
[    2.052733]   ESR = 0x96000004
[    2.055818]   EC = 0x25: DABT (current EL), IL = 32 bits
[    2.060987] Unable to handle kernel paging request at virtual address f7ffffe001e67988
[    2.061135]   SET = 0, FnV = 0
[    2.069012] Mem abort info:
[    2.069014]   ESR = 0x96000044
[    2.069017]   EC = 0x25: DABT (current EL), IL = 32 bits
[    2.072057]   EA = 0, S1PTW = 0
[    2.074833]   SET = 0, FnV = 0
[    2.074834]   EA = 0, S1PTW = 0
[    2.074835] Data abort info:
[    2.074836]   ISV = 0, ISS = 0x00000044
[    2.074838]   CM = 0, WnR = 1
[    2.077876] Data abort info:
[    2.083168] [f7ffffe001e67988] address between user and kernel address ranges
[    2.083171] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[    2.083173] Modules linked in: crct10dif_ce ti_k3_r5_remoteproc pruss pvrsrvkm(O) sa2ul sha512_generic authenc m_can_platform m_can ti_k3_dsp_remoteproc can_dev virtio_rpmsg_bus cdns_dphy rti_wdt ti_j721e_cpsw_virt_mac ipv6
[    2.086297]   ISV = 0, ISS = 0x00000004
[    2.089339] CPU: 0 PID: 245 Comm: mount.sh Tainted: G           O      5.4.106-g023faefa70 #1
[    2.089340] Hardware name: Texas Instruments K3 J721E SoC (DT)
[    2.092467]   CM = 0, WnR = 0
[    2.095331] pstate: 40000085 (nZcv daIf -PAN -UAO)
[    2.095345] pc : release_pages+0x190/0x3a8
[    2.095348] lr : release_pages+0x104/0x3a8
[    2.099169] [f7ff000816f16f38] address between user and kernel address ranges
[    2.102118] sp : ffff80001290fb70
[    2.102120] x29: ffff80001290fb70 x28: 0000000000000004 
[    2.187124] x27: ffff00081c7a0050 x26: 0000000000001e00 
[    2.192419] x25: ffff800010dd6a00 x24: ffff00081723a800 
[    2.197712] x23: 0000000000000000 x22: ffff80001290fc08 
[    2.203007] x21: ffff800010d74560 x20: ffff00081c7a0160 
[    2.208301] x19: ffffffe001e65bc0 x18: 0000000000000000 
[    2.213594] x17: 0000000000000000 x16: 0000000000000000 
[    2.218888] x15: 0000000000000000 x14: 0000000000000000 
[    2.224183] x13: 0000000000000000 x12: ffff800010dd6a00 
[    2.229477] x11: ffffffffffffffff x10: 0000000000000002 
[    2.234770] x9 : ffff000815538400 x8 : 000000000000003a 
[    2.240064] x7 : 0000000000000002 x6 : 0000000000000001 
[    2.245357] x5 : 00000000000001aa x4 : fffffffffffffff7 
[    2.250651] x3 : 00000000ffffffff x2 : ffff800010dd6a00 
[    2.255946] x1 : ffffffe001e6a988 x0 : f7ffffe001e67988 
[    2.261240] Call trace:
[    2.263675]  release_pages+0x190/0x3a8
[    2.267411]  free_pages_and_swap_cache+0xf4/0x128
[    2.272100]  tlb_flush_mmu+0x3c/0x68
[    2.275661]  tlb_finish_mmu+0x44/0x118
[    2.279393]  exit_mmap+0xe4/0x168
[    2.282696]  mmput+0x74/0x148
[    2.285651]  do_exit+0x270/0x978
[    2.288864]  do_group_exit+0x40/0xa0
[    2.292425]  __arm64_sys_exit_group+0x14/0x18
[    2.296767]  el0_svc_common.constprop.0+0x68/0x160
[    2.301541]  el0_svc_handler+0x20/0x80
[    2.305275]  el0_svc+0x8/0x204
[    2.308318] Code: a9408261 f9000420 9280000b 2a0b03e3 (f9000001) 
[    2.314392] ---[ end trace 8f29a222f3deead7 ]---
[    2.318992] Internal error: Oops: 96000004 [#2] PREEMPT SMP
[    2.318994] note: mount.sh[245] exited with preempt_count 1
[    2.318995] Fixing recursive fault but reboot is needed!
[    2.324546] Modules linked in: crct10dif_ce ti_k3_r5_remoteproc pruss pvrsrvkm(O) sa2ul sha512_generic authenc m_can_platform m_can ti_k3_dsp_remoteproc can_dev virtio_rpmsg_bus cdns_dphy rti_wdt ti_j721e_cpsw_virt_mac ipv6
[    2.355157] CPU: 1 PID: 144 Comm: systemd-udevd Tainted: G      D    O      5.4.106-g023faefa70 #1
[    2.364091] Hardware name: Texas Instruments K3 J721E SoC (DT)
[    2.369905] pstate: 20000005 (nzCv daif -PAN -UAO)
[    2.374686] pc : f_dupfd+0xdc/0x168
[    2.378160] lr : dup_fd+0xe4/0x2d0
[    2.381546] sp : ffff80001254fc30
[    2.384845] x29: ffff80001254fc30 x28: ffff000817472a00 
[    2.390139] x27: ffff000818b85a00 x26: ffff0008174a0868 
[    2.395433] x25: ffff0008174a08c0 x24: ffff0008174a65e8 
[    2.400728] x23: ffff800010e21000 x22: ffff80001254fd5c 
[    2.406022] x21: ffff0008174a0840 x20: ffff0008174a65c0 
[    2.411315] x19: 0000000000000040 x18: 0000000000000000 
[    2.416609] x17: 0000000000000000 x16: 0000000000000000 
[    2.421904] x15: 0000000000000000 x14: 0000000000000000 
[    2.427197] x13: 0000000000000000 x12: 0000000000000000 
[    2.432489] x11: 0000000000000000 x10: ffff0008174a6660 
[    2.437784] x9 : 0000000000000001 x8 : 0000000000000040 
[    2.443077] x7 : ffff0008174a08e0 x6 : ffff0008174a6660 
[    2.448370] x5 : f7ff000816f16f38 x4 : f7ff000816f16f38 
[    2.453664] x3 : 000000000000000d x2 : ffff0008174a66c8 
[    2.458958] x1 : f7ff000816f16f00 x0 : 000000000000002f 
[    2.464251] Call trace:
[    2.466686]  f_dupfd+0xdc/0x168
[    2.469813]  copy_process+0x690/0x14a8
[    2.473546]  _do_fork+0x7c/0x328
[    2.476759]  __arm64_sys_clone+0x64/0x88
[    2.480668]  el0_svc_common.constprop.0+0x68/0x160
[    2.485442]  el0_svc_handler+0x20/0x80
[    2.489175]  el0_svc+0x8/0x204
[    2.492218] Code: c8077d46 35ffffa7 17fffbb3 f98000b1 (c85f7ca0) 
[    2.498293] ---[ end trace 8f29a222f3deead8 ]---
[    2.502905] note: systemd-udevd[144] exited with preempt_count 1

    thanks,

  • +1 in reply to Tim JIANG
    TI__Guru 57697 points

    Hi Tim,

    I recommend a) performing ABBA swap of the unit, and b) if ABBA fails then recommend returning the SoC to TI for retest.

    The return procedure is summarized here:

    https://www.ti.com/support-quality/additional-information/customer-returns.html

    Regards,

    Kyle

  • 0 in reply to kcastille
    Prodigy 170 points

    Hi Kyle,

    Here's an new analysis update for you, we reconfigured LPDDR4 bandwidth from 4266 to 3733, then the fault disappeared. The LPDDR4 component is from SK Hynix MPN= H54G56BYYQX046N in our board. Is there any way to adjust the configuration parameters of LPDDR4 with 4266Mbps?

    thank you!

    Tim

  • 0 in reply to Tim JIANG
    TI__Mastermind 44660 points

    Hi,

    Tim JIANG said:
    Is there any way to adjust the configuration parameters of LPDDR4 with 4266Mbps?

    A link to the DDR register configuration tool is below; however, I assume you needed to use this tool to adjust the data rate from 4266 MT/s to 3733 MT/s.

    https://dev.ti.com/sysconfig/?product=TDA4x_DRA8x_AM67x-AM69x_DDR_Config&device=J721E_DRA829_TDA4VM_AM752x 

    How are you currently configuring the DDR register configuration tool inputs? If you are not already, you could try enabling DBI to see if it has any impact on test results.

    Regards,
    Kevin

  • 0 in reply to Kevin S
    Prodigy 170 points

    Hi Kevin,

    In our software configuration, the DBI has been enabled. DDR register configuration inputs can be found following,

  • 0 in reply to Tim JIANG
    TI__Mastermind 44660 points

    Hi,

    Thanks for providing the snippets.

    A few initial comments.

    • tRFCab seems to be set to 240 ns, but I believe 280 ns is typical for the selected density size. However, I am not sure if the failure is related as I would expect the failure to follow the LPDDR4 memory or be systematic if this were related. Either way, you should confirm this value against the LPDDR4 memory datasheet.
    • DQ/ DQS drive strength on TDA4 side is 48 ohms, but DQ ODT on LPDDR4 side is 40 ohms. You may consider matching these.
    • CA drive strength is 34 ohms, CS drive strength is 80 ohms, and CA/CS ODT is 80 ohms. This may be OK, but you might also consider these options:
      • CA drive strength = 40 ohms, CS drive strength = 80 ohms, CA / CS ODT = 80 ohms
      • CA drive strength = 34 ohms, CS drive strength = 60 ohms, CA / CS ODT = 60 ohms

    Also, you mentioned that sometimes the board passes (2/10?). Have you tried running a memory test (ex: memtester) in boot attempts that pass?

    Regards,
    Kevin

  • +1 in reply to Kevin S
    Prodigy 170 points

    Hi Kevin,

    Sorry for reply delay, thanks a lot for your recommendation with details, and we took A-B-A analysis, then the fault follows the SoC, CPR process is on-going.

    Regards,

    Tim

  • +1 in reply to Tim JIANG
    TI__Mastermind 44660 points

    Hi Tim,

    Thanks for the update. We will close this ticket then. If further help is needed, feel free to respond back to this ticket or create a new one and reference this ticket.

    Thanks,
    Kevin