AM4372: PRU processor rpmsg device node not created and GPIO pin controlled by PRU is asserted

Another comment to do add to the main thread
- The issue of GPIO assertion is seen frequently now with our new firmware. The firmware does not have any kernel changes around this area.

Also, once this issue is seen, we also see several such crashes in the dmesg

bash-5.0# dmesg
[ 44.816336] ---[ end trace 5597c41047d0d1b1 ]---
[ 44.818077] ------------[ cut here ]------------
[ 44.818102] WARNING: CPU: 0 PID: 795 at drivers/bus/omap_l3_noc.c:141 l3_interrupt_handler+0x28c/0x384
[ 44.818109] 44000000.ocp:L3 Standard Error: MASTER ICSS0 TARGET GPMC (Read): At Address: 0x002E0014 : Data Access in Supervisor mode during Functional access
[ 44.818112] Modules linked in: gpio_mon(O) virtio_rpmsg_bus xt_state xt_comment ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_NFLOG xt_LOG nf_conntrack_tftp xt_conntrack nf_conntrack nfnetlink_log nf_defrag_ipv6 nfnetlink libcrc32c nf_defrag_ipv4 xt_multiport xt_tcpudp ip6table_filter ip6_tables iptable_filter ip_tables x_tables pru_rproc irq_pruss_intc pruss ti_emif_sram omap_wdt sch_fq_codel omap_sham omap_des libdes omap_aes_driver omap_crypto libaes crypto_engine
[ 44.818182] CPU: 0 PID: 795 Comm: check_redfish_s Tainted: G W O 5.4.28-MS_OCS_RM_1.1.36.254351-g6f3bf13d53 #1
[ 44.818185] Hardware name: Generic AM43 (Flattened Device Tree)
[ 44.818212] [<c010f1b8>] (unwind_backtrace) from [<c010b4d4>] (show_stack+0x10/0x14)
[ 44.818225] [<c010b4d4>] (show_stack) from [<c012ccd0>] (__warn+0xbc/0xd4)
[ 44.818234] [<c012ccd0>] (__warn) from [<c012cd80>] (warn_slowpath_fmt+0x98/0xc4)
[ 44.818242] [<c012cd80>] (warn_slowpath_fmt) from [<c0495f2c>] (l3_interrupt_handler+0x28c/0x384)
[ 44.818256] [<c0495f2c>] (l3_interrupt_handler) from [<c016f53c>] (__handle_irq_event_percpu+0x50/0x13c)
[ 44.818266] [<c016f53c>] (__handle_irq_event_percpu) from [<c016f658>] (handle_irq_event_percpu+0x30/0x8c)
[ 44.818272] [<c016f658>] (handle_irq_event_percpu) from [<c016f700>] (handle_irq_event+0x4c/0x8c)
[ 44.818283] [<c016f700>] (handle_irq_event) from [<c0173694>] (handle_fasteoi_irq+0xb4/0x184)
[ 44.818304] [<c0173694>] (handle_fasteoi_irq) from [<c016e710>] (generic_handle_irq+0x24/0x34)
[ 44.818313] [<c016e710>] (generic_handle_irq) from [<c016ee88>] (__handle_domain_irq+0x54/0xa4)
[ 44.818333] [<c016ee88>] (__handle_domain_irq) from [<c0494948>] (gic_handle_irq+0x3c/0x68)
[ 44.818345] [<c0494948>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0xa8)
[ 44.818350] Exception stack(0xed467f30 to 0xed467f78)
[ 44.818357] 7f20: ee773418 00000000 00000001 00000000
[ 44.818365] 7f40: ed467fb0 ffffe000 00000002 000000af c0101204 ed466000 00000020 004b8394
[ 44.818371] 7f60: 40000028 ed467f80 c0109f7c c01894d4 600e0013 ffffffff
[ 44.818388] [<c0101a8c>] (__irq_svc) from [<c01894d4>] (ktime_get_coarse_real_ts64+0x0/0x5c)
[ 44.818398] [<c01894d4>] (ktime_get_coarse_real_ts64) from [<000000af>] (0xaf)
[ 44.818402] ---[ end trace 5597c41047d0d1b2 ]---
[ 44.818996] ------------[ cut here ]------------
[ 44.819015] WARNING: CPU: 0 PID: 795 at drivers/bus/omap_l3_noc.c:141 l3_interrupt_handler+0x28c/0x384
[ 44.819021] 44000000.ocp:L3 Standard Error: MASTER ICSS0 TARGET GPMC (Read): At Address: 0x002E000C : Data Access in Supervisor mode during Functional access
[ 44.819023] Modules linked in: gpio_mon(O) virtio_rpmsg_bus xt_state xt_comment ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_NFLOG xt_LOG nf_conntrack_tftp xt_conntrack nf_conntrack nfnetlink_log nf_defrag_ipv6 nfnetlink libcrc32c nf_defrag_ipv4 xt_multiport xt_tcpudp ip6table_filter ip6_tables iptable_filter ip_tables x_tables pru_rproc irq_pruss_intc pruss ti_emif_sram omap_wdt sch_fq_codel omap_sham omap_des libdes omap_aes_driver omap_crypto libaes crypto_engine
[ 44.819082] CPU: 0 PID: 795 Comm: check_redfish_s Tainted: G W O 5.4.28-MS_OCS_RM_1.1.36.254351-g6f3bf13d53 #1
[ 44.819085] Hardware name: Generic AM43 (Flattened Device Tree)
[ 44.819109] [<c010f1b8>] (unwind_backtrace) from [<c010b4d4>] (show_stack+0x10/0x14)
[ 44.819121] [<c010b4d4>] (show_stack) from [<c012ccd0>] (__warn+0xbc/0xd4)
[ 44.819130] [<c012ccd0>] (__warn) from [<c012cd80>] (warn_slowpath_fmt+0x98/0xc4)
[ 44.819138] [<c012cd80>] (warn_slowpath_fmt) from [<c0495f2c>] (l3_interrupt_handler+0x28c/0x384)
[ 44.819148] [<c0495f2c>] (l3_interrupt_handler) from [<c016f53c>] (__handle_irq_event_percpu+0x50/0x13c)
[ 44.819157] [<c016f53c>] (__handle_irq_event_percpu) from [<c016f658>] (handle_irq_event_percpu+0x30/0x8c)
[ 44.819163] [<c016f658>] (handle_irq_event_percpu) from [<c016f700>] (handle_irq_event+0x4c/0x8c)
[ 44.819174] [<c016f700>] (handle_irq_event) from [<c0173694>] (handle_fasteoi_irq+0xb4/0x184)
[ 44.819192] [<c0173694>] (handle_fasteoi_irq) from [<c016e710>] (generic_handle_irq+0x24/0x34)
[ 44.819200] [<c016e710>] (generic_handle_irq) from [<c016ee88>] (__handle_domain_irq+0x54/0xa4)
[ 44.819216] [<c016ee88>] (__handle_domain_irq) from [<c0494948>] (gic_handle_irq+0x3c/0x68)
[ 44.819228] [<c0494948>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0xa8)
[ 44.819233] Exception stack(0xed467dd0 to 0xed467e18)
[ 44.819239] 7dc0: ed61d2a4 60070013 00000000 00000020
[ 44.819247] 7de0: cfc82600 ed61c660 ed61d2a0 cfc7ecc0 ed61d2a0 ed61d2a0 c0d6e380 ee7e9e1c
[ 44.819253] 7e00: 00000004 ed467e20 c02199d8 c084cb88 60070013 ffffffff
[ 44.819274] [<c0101a8c>] (__irq_svc) from [<c084cb88>] (down_write+0x0/0x48)
[ 44.819293] [<c084cb88>] (down_write) from [<c02199d8>] (anon_vma_clone+0x94/0x1d0)
[ 44.819303] [<c02199d8>] (anon_vma_clone) from [<c0219b38>] (anon_vma_fork+0x24/0x150)
[ 44.819312] [<c0219b38>] (anon_vma_fork) from [<c0129bec>] (dup_mm+0x2f4/0x4d4)
[ 44.819320] [<c0129bec>] (dup_mm) from [<c012b168>] (copy_process+0x1168/0x1654)
[ 44.819326] [<c012b168>] (copy_process) from [<c012b804>] (_do_fork+0x80/0x390)
[ 44.819334] [<c012b804>] (_do_fork) from [<c012bd20>] (sys_clone+0x70/0x94)
[ 44.819342] [<c012bd20>] (sys_clone) from [<c01011e0>] (__sys_trace_return+0x0/0x20)
[ 44.819346] Exception stack(0xed467fa8 to 0xed467ff0)
[ 44.819353] 7fa0: b6efde38 b6efe290 01200011 00000000 00000000 00000000
[ 44.819360] 7fc0: b6efde38 b6efe290 00000000 00000078 b6e9ce3c b6efddd0 00000005 00000001
[ 44.819365] 7fe0: 00000078 beb77698 b6e2fbd3 b6de3cf6
[ 44.819368] ---[ end trace 5597c41047d0d1b3 ]---
[ 44.819917] ------------[ cut here ]------------
[ 44.819937] WARNING: CPU: 0 PID: 795 at drivers/bus/omap_l3_noc.c:141 l3_interrupt_handler+0x28c/0x384
[ 44.819943] 44000000.ocp:L3 Standard Error: MASTER ICSS0 TARGET GPMC (Read): At Address: 0x002E0004 : Data Access in Supervisor mode during Functional access
[ 44.819945] Modules linked in: gpio_mon(O) virtio_rpmsg_bus xt_state xt_comment ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_NFLOG xt_LOG nf_conntrack_tftp xt_conntrack nf_conntrack nfnetlink_log nf_defrag_ipv6 nfnetlink libcrc32c nf_defrag_ipv4 xt_multiport xt_tcpudp ip6table_filter ip6_tables iptable_filter ip_tables x_tables pru_rproc irq_pruss_intc pruss ti_emif_sram omap_wdt sch_fq_codel omap_sham omap_des libdes omap_aes_driver omap_crypto libaes crypto_engine
[ 44.820000] CPU: 0 PID: 795 Comm: check_redfish_s Tainted: G W O 5.4.28-MS_OCS_RM_1.1.36.254351-g6f3bf13d53 #1
[ 44.820003] Hardware name: Generic AM43 (Flattened Device Tree)
[ 44.820023] [<c010f1b8>] (unwind_backtrace) from [<c010b4d4>] (show_stack+0x10/0x14)
[ 44.820035] [<c010b4d4>] (show_stack) from [<c012ccd0>] (__warn+0xbc/0xd4)
[ 44.820043] [<c012ccd0>] (__warn) from [<c012cd80>] (warn_slowpath_fmt+0x98/0xc4)
[ 44.820051] [<c012cd80>] (warn_slowpath_fmt) from [<c0495f2c>] (l3_interrupt_handler+0x28c/0x384)
[ 44.820061] [<c0495f2c>] (l3_interrupt_handler) from [<c016f53c>] (__handle_irq_event_percpu+0x50/0x13c)
[ 44.820070] [<c016f53c>] (__handle_irq_event_percpu) from [<c016f658>] (handle_irq_event_percpu+0x30/0x8c)
[ 44.820076] [<c016f658>] (handle_irq_event_percpu) from [<c016f700>] (handle_irq_event+0x4c/0x8c)
[ 44.820087] [<c016f700>] (handle_irq_event) from [<c0173694>] (handle_fasteoi_irq+0xb4/0x184)
[ 44.820104] [<c0173694>] (handle_fasteoi_irq) from [<c016e710>] (generic_handle_irq+0x24/0x34)
[ 44.820113] [<c016e710>] (generic_handle_irq) from [<c016ee88>] (__handle_domain_irq+0x54/0xa4)
[ 44.820129] [<c016ee88>] (__handle_domain_irq) from [<c0494948>] (gic_handle_irq+0x3c/0x68)
[ 44.820139] [<c0494948>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0xa8)
[ 44.820143] Exception stack(0xed467f28 to 0xed467f70)
[ 44.820150] 7f20: 00000001 00000000 ee77369c ee77369c ee773400 00000001
[ 44.820158] 7f40: 00000000 000000af c0101204 ed466000 000000af 00000001 00000000 ed467f78
[ 44.820163] 7f60: c010a068 c01be0a4 60070013 ffffffff
[ 44.820177] [<c0101a8c>] (__irq_svc) from [<c01be0a4>] (__audit_syscall_exit+0x40/0x250)
[ 44.820198] [<c01be0a4>] (__audit_syscall_exit) from [<c010a068>] (syscall_trace_exit+0xc4/0xf4)
[ 44.820207] [<c010a068>] (syscall_trace_exit) from [<c01011ec>] (__sys_trace_return+0xc/0x20)
[ 44.820210] Exception stack(0xed467fb0 to 0xed467ff8)
[ 44.820216] 7fa0: 00000000 beb7775c 00000000 00000008
[ 44.820223] 7fc0: 00000000 00000000 00000002 000000af beb7775c 004ce1f0 00000005 00000001
[ 44.820229] 7fe0: 000000af beb77628 b6df00e3 b6de3cf6 00070030 00000002
[ 44.820233] ---[ end trace 5597c41047d0d1b4 ]---

Hello Sumit,

Our Linux SDK for kernel 5.4 is more than 4 years old at this point. Are you planning on updating your design to a more recent kernel?

The reason I ask is that both GPIO and the Linux PRU remoteproc driver have changed significantly over the last 4 years. So if you are planning to update to a more recent kernel (like the upcoming kernel 6.12 release in another week or so), it would probably make sense to port first, and I can discuss with you how the PRU firmware needs to be updated to be loaded by the newer kernel.

Regards,

Nick

additional questions:

1) is there a specific timeframe by which the PRU cores must be initialized?

2) It looks like the PRU remoteproc driver is auto-booting the PRU cores during Linux boot time, instead of booting the PRU cores during Linux runtime, or writing a custom Linux driver that calls the PRU remoteproc driver when it is probed during boot. Is that correct? Any additional information I should know?

Regards,

Nick

We have started the work to update to latest TI SDK, but that activity will take some time. In the meantime, we have to understand this issue and fix if possible soon.

Over firmware update of ARM SoC, we do the warm reboot and let the flasher image kick off (boot up from eMMC and update the QSPI NOR partitions) and then eventually boot up from QSPI NOR. During this procedure, we do not touch PRU cores at all.

However, during boot up from QSPI NOR, we only do the following as part of init script (snippet below).

<TRUNCATED>

Hello Sumit,

Setting support expectations

I will do what I can to help, but I am limited in the support I can offer for Linux kernel 5.4. I can offer more support on later versions of Linux.

What happens during boot before you call "echo start"? 

First, Linux needs to know that the PRU cores even exist. It does that by probing the Linux devicetree nodes:

[   13.262687] remoteproc remoteproc1: 54434000.pru is available
[   13.284586] pru-rproc 54434000.pru: PRU rproc node /ocp@44000000/target-module@54400000/pruss@0/pru@34000 probed successfully
[   13.315688] remoteproc remoteproc2: 54438000.pru is available
[   13.321560] pru-rproc 54438000.pru: PRU rproc node /ocp@44000000/target-module@54400000/pruss@0/pru@38000 probed successfully

At this point, the PRU subsystem has not been initialized. There is no clock signal applied to the PRU cores.

Important note: PRU GPO signals are NOT deterministic before PRU cores are initialized! 

Before the PRU has been initialized, the values in the PRU registers are non-deterministic (i.e., each bit can be randomly 1 or 0).

If you apply pinmuxing settings that route a PRU GPO signal to a processor pin before the PRU core has been initialized, then the signal that appears on the pin will be RANDOM. If you need to make sure the signal is a fixed value, I would suggest following one of the suggestions in this e2e thread:  Linux/PROCESSOR-SDK-AM437X: GPO pin undefined until PRU starts 

What happens after "echo start"? Is there any useful documentation? 

After you call "echo start", then the PRU Remoteproc driver initializes the PRU core and sets up RPMsg communication (this happens by default in kernel 5.4, but it becomes optional in later Linux kernel releases).

The resource table and INTC configuration are one of the major changes from Linux kernel 5.4 and earlier, and Linux kernel 5.10 and later. Please refer to my documentation here: (I'll link to a later SDK version since I cannot remember exactly when I added this documentation to the SDK docs): https://software-dl.ti.com/processor-sdk-linux/esd/AM335X/09_03_05_02/exports/docs/linux/Foundational_Components/PRU-ICSS-Firmware-Development.html

Anything else? 

Off the top of my head, I have not seen the behavior you are reporting before. Here is another thread where the customer was doing something similar, not sure if it would be helpful:  PROCESSOR-SDK-AM437X: The remoteproc is starting very late during boot  

Regards,

Nick

Thanks Nick

Few more questions/comments

1) About the PRU subsystem initialization, as we do warm reboot of ARM SOC, I assume that PRU cores are still active and are running. Is that not the case?

If it is, then what impat does "echo start" do on PRU cores?

2) does setting bit1 of PRCM_PRM_RSTCTRL reboot both the PRU cores?

3) In a good v/s bad case (dmesg logs below). What can cause the failures/crash to happen in BAD case.

BAD case is seen once in few iterations which include back-to-back firmware update

Good

[ 31.661830] remoteproc remoteproc1: powering up 54434000.pru
[ 31.678925] remoteproc remoteproc1: Booting fw image am437x-pru1_0-fw, size 133492
[ 31.679014] remoteproc remoteproc1: remote processor 54434000.pru is now up
[ 31.696269] remoteproc remoteproc2: powering up 54438000.pru
[ 31.708704] remoteproc remoteproc2: Booting fw image am437x-pru1_1-fw, size 91484
[ 31.726321] pru-rproc 54438000.pru: configured system_events[63-0] = 00000000,000c0000
[ 31.726335] pru-rproc 54438000.pru: configured intr_channels = 0x0000000a host_intr = 0x0000000a
[ 31.738190] remoteproc2#vdev0buffer: registered virtio0 (type 7)
[ 31.738208] remoteproc remoteproc2: remote processor 54438000.pru is now up
[ 31.768160] virtio_rpmsg_bus virtio0: creating channel rpmsg-pru addr 0x1f
[ 31.771243] virtio_rpmsg_bus virtio0: rpmsg host is online

Bad

[ 24.806663] remoteproc remoteproc1: powering up 54434000.pru
[ 24.826845] remoteproc remoteproc1: Booting fw image am437x-pru1_0-fw, size 133624
[ 24.826933] remoteproc remoteproc1: remote processor 54434000.pru is now up
[ 24.827549] remoteproc remoteproc2: powering up 54438000.pru
[ 24.854061] remoteproc remoteproc2: Booting fw image am437x-pru1_1-fw, size 91528
[ 24.874836] pru-rproc 54438000.pru: configured system_events[63-0] = 00000000,000c0000
[ 24.874850] pru-rproc 54438000.pru: configured intr_channels = 0x0000000a host_intr = 0x0000000a
[ 24.883037] remoteproc2#vdev0buffer: registered virtio0 (type 7)
[ 24.883053] remoteproc remoteproc2: remote processor 54438000.pru is now up
[ 24.901282] ------------[ cut here ]------------
[ 24.901312] WARNING: CPU: 0 PID: 461 at drivers/bus/omap_l3_noc.c:141 l3_interrupt_handler+0x28c/0x384
[ 24.901320] 44000000.ocp:L3 Standard Error: MASTER ICSS1 TARGET GPMC (Read): At Address: 0x00090018 : Data Access in Supervisor mode during Functional access
[ 24.901323] Modules linked in: virtio_rpmsg_bus(+) xt_state xt_comment ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_NFLOG xt_LOG nf_conntrack_tftp xt_conntrack nf_conntrack nf_defrag_ipv6 libcrc32c nfnetlink_log nfnetlink nf_defrag_ipv4 xt_multiport xt_tcpudp ip6table_filter ip6_tables iptable_filter ip_tables x_tables pru_rproc irq_pruss_intc pruss ti_emif_sram omap_wdt sch_fq_codel omap_sham omap_des libdes omap_aes_driver omap_crypto libaes crypto_engine
[ 24.901393] CPU: 0 PID: 461 Comm: systemd-udevd Not tainted 5.4.28-MS_OCS_RM_1.1.37.254421-g6f3bf13d53 #1
[ 24.901396] Hardware name: Generic AM43 (Flattened Device Tree)
[ 24.901426] [<c010f1b8>] (unwind_backtrace) from [<c010b4d4>] (show_stack+0x10/0x14)
[ 24.901440] [<c010b4d4>] (show_stack) from [<c012ccd0>] (__warn+0xbc/0xd4)
[ 24.901449] [<c012ccd0>] (__warn) from [<c012cd80>] (warn_slowpath_fmt+0x98/0xc4)
[ 24.901457] [<c012cd80>] (warn_slowpath_fmt) from [<c0495f2c>] (l3_interrupt_handler+0x28c/0x384)
[ 24.901470] [<c0495f2c>] (l3_interrupt_handler) from [<c016f53c>] (__handle_irq_event_percpu+0x50/0x13c)
[ 24.901480] [<c016f53c>] (__handle_irq_event_percpu) from [<c016f658>] (handle_irq_event_percpu+0x30/0x8c)
[ 24.901486] [<c016f658>] (handle_irq_event_percpu) from [<c016f700>] (handle_irq_event+0x4c/0x8c)
[ 24.901499] [<c016f700>] (handle_irq_event) from [<c0173694>] (handle_fasteoi_irq+0xb4/0x184)
[ 24.901520] [<c0173694>] (handle_fasteoi_irq) from [<c016e710>] (generic_handle_irq+0x24/0x34)
[ 24.901530] [<c016e710>] (generic_handle_irq) from [<c016ee88>] (__handle_domain_irq+0x54/0xa4)
[ 24.901551] [<c016ee88>] (__handle_domain_irq) from [<c0494948>] (gic_handle_irq+0x3c/0x68)
[ 24.901564] [<c0494948>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0xa8)
[ 24.901569] Exception stack(0xcfc59bd8 to 0xcfc59c20)
[ 24.901575] 9bc0: cfb51000 60000013
[ 24.901583] 9be0: 00000001 f0a44000 00000001 00000000 00000001 00000000 00000cc0 f0c91000
[ 24.901590] 9c00: 00004000 f0c92e00 eec10ef4 cfc59c28 c016fbb8 c016f038 20000013 ffffffff
[ 24.901600] [<c0101a8c>] (__irq_svc) from [<c016f038>] (__irq_put_desc_unlock+0x10/0x7c)
[ 24.901608] [<c016f038>] (__irq_put_desc_unlock) from [<c016fbb8>] (irq_set_irqchip_state+0x88/0xc0)
[ 24.901649] [<c016fbb8>] (irq_set_irqchip_state) from [<bf056768>] (pru_rproc_kick+0x54/0x74 [pru_rproc])
[ 24.901684] [<bf056768>] (pru_rproc_kick [pru_rproc]) from [<c06b569c>] (rproc_virtio_notify+0x20/0x28)
[ 24.901710] [<c06b569c>] (rproc_virtio_notify) from [<c04fef4c>] (virtqueue_notify+0x1c/0x34)
[ 24.901753] [<c04fef4c>] (virtqueue_notify) from [<bf0f3b4c>] (rpmsg_probe+0x2a0/0x354 [virtio_rpmsg_bus])
[ 24.901789] [<bf0f3b4c>] (rpmsg_probe [virtio_rpmsg_bus]) from [<c04fedfc>] (virtio_dev_probe+0x1f4/0x2c0)
[ 24.901804] [<c04fedfc>] (virtio_dev_probe) from [<c055fe4c>] (really_probe+0x1e0/0x348)
[ 24.901815] [<c055fe4c>] (really_probe) from [<c0560128>] (driver_probe_device+0x60/0x170)
[ 24.901824] [<c0560128>] (driver_probe_device) from [<c05603d8>] (device_driver_attach+0x58/0x60)
[ 24.901833] [<c05603d8>] (device_driver_attach) from [<c0560438>] (__driver_attach+0x58/0xcc)
[ 24.901840] [<c0560438>] (__driver_attach) from [<c055e00c>] (bus_for_each_dev+0x78/0xc0)
[ 24.901848] [<c055e00c>] (bus_for_each_dev) from [<c055f1c8>] (bus_add_driver+0x15c/0x1e0)
[ 24.901858] [<c055f1c8>] (bus_add_driver) from [<c0560d14>] (driver_register+0x74/0x108)
[ 24.901873] [<c0560d14>] (driver_register) from [<bf027010>] (rpmsg_init+0x10/0x1000 [virtio_rpmsg_bus])
[ 24.901877] ---[ end trace 179cf74053c64539 ]---
[ 24.901923] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901941] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901971] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901980] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901988] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901998] virtio_rpmsg_bus virtio0: input:id 16 out of range
[ 24.902021] virtio_rpmsg_bus virtio0: rpmsg host is online

4) looks like (yet to understand the patch) more, we already have pinmuxing logic in Uboot which does pull down and GPO in output direction.

diff --git a/arch/arm/cpu/armv7/am33xx/board.c b/arch/arm/cpu/armv7/am33xx/board.c
index 8ca828f..2d71758 100644
--- a/arch/arm/cpu/armv7/am33xx/board.c
+++ b/arch/arm/cpu/armv7/am33xx/board.c
@@ -402,6 +402,24 @@ void s_init(void)
 #endif
 }
 
+/*
+ * Starts both PRU0 and PRU1 dual-cores for the PRU_ICSS1 processor.
+ * PRU_ICSS1 PRU1 Control register: at global memory-mapped location 0x54424000
+ * PRU_ICSS1 PRU0 Control register: at global memory-mapped location 0x54422000
+ *
+ * PRU Control Register definitions: (with value 0xB).
+ * bit 31-16: this field contains address where PRU will start executing code. (we will start at address zero)
+ * bit 3: if set, enable PRU cycle counters.
+ * bit 1: if set, enable PRU to fetch new instructions.
+ * bit 0: if set, no soft reset.
+*/
+void PRU_ICSS1_PRU0_and_1_Start(void)
+
+{
+	*((unsigned int*)0x54424000) = 0xB;
+	*((unsigned int*)0x54422000) = 0xB;
+}
+
 void early_system_init(void)
 {
 	/*
@@ -424,6 +442,22 @@ void early_system_init(void)
 	/* Enable RTC32K clock */
 	rtc32k_enable();
 #endif
+
+	/*
+	 * The following code must be done after the clock of
+	 * the PRU modules were enabled in the setup_early_clocks().
+	 */
+	if (warm_reset()) {
+		/*
+		 * Set GPIO pin 5 for BLADE_THROTTLE_N,
+		 * which will used by PRU0 to interrupt blade
+		 * for power throttle alert.
+		 */
+		set_dss_mux_conf();
+
+		/* restart PRU0 firmware */
+		PRU_ICSS1_PRU0_and_1_Start();
+	}
 }
 
 #ifdef CONFIG_SPL_BUILD
diff --git a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
index ce5b826..d2ffa6d 100644
--- a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
+++ b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
@@ -20,6 +20,8 @@ struct cm_perpll *const cmper = (struct cm_perpll *)CM_PER;
 struct cm_wkuppll *const cmwkup = (struct cm_wkuppll *)CM_WKUP;
 struct cm_dpll *const cmdpll = (struct cm_dpll *)CM_DPLL;
 
+#define PRCM_RM_PER_RSTCTRL	0x44DF0810
+
 const struct dpll_regs dpll_mpu_regs = {
 	.cm_clkmode_dpll	= CM_WKUP + 0x560,
 	.cm_idlest_dpll		= CM_WKUP + 0x564,
@@ -99,6 +101,8 @@ void setup_clocks_for_console_uart1(void)
 
 void enable_basic_clocks(void)
 {
+	u32 temp;
+
 	u32 *const clk_domains[] = {
 		&cmper->l3clkstctrl,
 		&cmper->l3sclkstctrl,
@@ -119,6 +123,9 @@ void enable_basic_clocks(void)
 		&cmwkup->wkctrlclkctrl,
 		&cmper->timer2clkctrl,
 		&cmper->gpmcclkctrl,
+		&cmper->pruclkctrl, 	// Enable PRUSS module and clock
+		&cmper->adc1clkctrl,	// Enable ADC clock
+		&cmwkup->wkadc0clkctrl,
 		&cmper->elmclkctrl,
 		&cmper->mmc0clkctrl,
 		&cmper->mmc1clkctrl,
@@ -145,6 +152,14 @@ void enable_basic_clocks(void)
 
 	/* For OPP100 the mac clock should be /5. */
 	writel(0x4, &cmdpll->clkselmacclk);
+
+	/* PRU-ICSS module local reset control */
+	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL register
+	temp |= 0x2;                		// Assert Reset for PRU-ICSS
+	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
+	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL
+	temp &= 0xFFFFFFFD;			// Clear Reset for PRU-ICSS
+	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
 }
 
 void rtc_only_enable_basic_clocks(void)
diff --git a/arch/arm/include/asm/arch-am33xx/cpu.h b/arch/arm/include/asm/arch-am33xx/cpu.h
index bacbee0..1c4852e 100644
--- a/arch/arm/include/asm/arch-am33xx/cpu.h
+++ b/arch/arm/include/asm/arch-am33xx/cpu.h
@@ -231,8 +231,10 @@ struct prm_device_inst {
 #else
 /* Encapsulating core pll registers */
 struct cm_wkuppll {
-	unsigned int resv0[136];
-	unsigned int wkl4wkclkctrl;	/* offset 0x220 */
+	unsigned int resv0[72];
+	unsigned int wkadc0clkctrl;	/* offset 0x120 */
+	unsigned int resv01[63];
+	unsigned int wkl4wkclkctrl; 	/* offset 0x220 */
 	unsigned int resv1[7];
 	unsigned int usbphy0clkctrl;	/* offset 0x240 */
 	unsigned int resv112;
@@ -313,7 +315,9 @@ struct cm_perpll {
 	unsigned int l3sclkstctrl;	/* offset 0x200 */
 	unsigned int resv8[7];
 	unsigned int gpmcclkctrl;	/* offset 0x220 */
-	unsigned int resv9[5];
+	unsigned int resv9[3];
+	unsigned int adc1clkctrl;   	/* offset 0x230 */
+	unsigned int resv91[1];
 	unsigned int mcasp0clkctrl;	/* offset 0x238 */
 	unsigned int resv10;
 	unsigned int mcasp1clkctrl;	/* offset 0x240 */
@@ -325,7 +329,11 @@ struct cm_perpll {
 	unsigned int usb0clkctrl;	/* offset 0x260 */
 	unsigned int resv122;
 	unsigned int usb1clkctrl;	/* offset 0x268 */
-	unsigned int resv13[101];
+	unsigned int resv13[37];
+	unsigned int pruclkstctrl;      /* offset 0x300 */
+	unsigned int resv131[7];
+	unsigned int pruclkctrl;        /* offset 0x320 */
+	unsigned int resv132[55];
 	unsigned int l4lsclkstctrl;	/* offset 0x400 */
 	unsigned int resv14[7];
 	unsigned int l4lsclkctrl;	/* offset 0x420 */
diff --git a/arch/arm/include/asm/arch-am33xx/sys_proto.h b/arch/arm/include/asm/arch-am33xx/sys_proto.h
index 2d34c96..ca94ce9 100644
--- a/arch/arm/include/asm/arch-am33xx/sys_proto.h
+++ b/arch/arm/include/asm/arch-am33xx/sys_proto.h
@@ -34,6 +34,7 @@ void enable_gpmc_cs_config(const u32 *gpmc_config, struct gpmc_cs *cs, u32 base,
 void omap_nand_switch_ecc(uint32_t, uint32_t);
 
 void set_uart_mux_conf(void);
+void set_dss_mux_conf(void);
 void set_mux_conf_regs(void);
 void sdram_init(void);
 u32 wait_on_value(u32, u32, void *, u32);
diff --git a/board/ti/am43xx_ocs/board.c b/board/ti/am43xx_ocs/board.c
index 2879987..5e0c317 100644
--- a/board/ti/am43xx_ocs/board.c
+++ b/board/ti/am43xx_ocs/board.c
@@ -459,6 +459,12 @@ void set_uart_mux_conf(void)
 	enable_uart1_pin_mux();
 }
 
+void set_dss_mux_conf(void)
+{
+	enable_dss_pin_mux();
+}
+
+
 void set_mux_conf_regs(void)
 {
 	enable_board_pin_mux();
diff --git a/board/ti/am43xx_ocs/board.h b/board/ti/am43xx_ocs/board.h
index a28a3ac..343c45f 100644
--- a/board/ti/am43xx_ocs/board.h
+++ b/board/ti/am43xx_ocs/board.h
@@ -21,4 +21,5 @@ void enable_uart0_pin_mux(void);
 void enable_uart1_pin_mux(void);
 void enable_board_pin_mux(void);
 void enable_i2c0_pin_mux(void);
+void enable_dss_pin_mux(void);
 #endif
diff --git a/board/ti/am43xx_ocs/mux.c b/board/ti/am43xx_ocs/mux.c
index b1a3a33..3625884 100644
--- a/board/ti/am43xx_ocs/mux.c
+++ b/board/ti/am43xx_ocs/mux.c
@@ -62,6 +62,12 @@ static struct module_pin_mux uart1_pin_mux[] = {
 	{-1},
 };
 
+/* This GPIO pin will be used by the ICSS PRU0 for BLADE_THROTTLE alert */
+static struct module_pin_mux dss_pin_mux[] = {
+	{OFFSET(lcd_data5), (MODE(5) | PULLDOWN_EN)},
+	{-1},
+};
+
 /*
 
 {OFFSET(uart1_ctsn), (MODE(0) | PULLUP_EN | RXACTIVE | SLEWCTRL)},
@@ -125,6 +131,11 @@ void enable_uart1_pin_mux(void)
 	configure_module_pin_mux(uart1_pin_mux);
 }
 
+void enable_dss_pin_mux(void)
+{
+	configure_module_pin_mux(dss_pin_mux);
+}
+
 void enable_board_pin_mux(void)
 {
 	configure_module_pin_mux(mdio_pin_mux);

Hello Sumit,

I am out of office until Wednesday. I am reassigning your thread to another team member to comment on the power questions. Please ping the thread if you do not get a response within a day or so.

Regards,

Nick

Hello Sumit, 

Thank you for your query !

Considering the following one of Nick's notes:

" Important note: PRU GPO signals are NOT deterministic before PRU cores are initialized! 

Before the PRU has been initialized, the values in the PRU registers are non-deterministic (i.e., each bit can be randomly 1 or 0).

If you apply pinmuxing settings that route a PRU GPO signal to a processor pin before the PRU core has been initialized, then the signal that appears on the pin will be RANDOM. If you need to make sure the signal is a fixed value, I would suggest following one of the suggestions in this e2e thread:

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/677952/linux-processor-sdk-am437x-gpo-pin-undefined-until-pru-starts 

"

from hardware perspective I have the following questions:

Q1. Do you use a custom PCB solution or the AM4372 EVM. If EVM which one ?

Q2. I understand that the focus of your query is the behavior of the pr1_pru0_gpo5 signal muxed on the AM4372 dss_data5 pin, correct  ?

Q3. What is the type of the Ethernet PHY that you use ? 

Q4. What is the level of the GPO pin when unexpectedly asserted - HIGH or LOW ?

Q5. Is the pr1_pru0_gpo5 or another PRU0 GPO signal related SoC pin connected to the reset input of the external Ethernet PHY that you use ?

Q6. Would you please confirm my understanding that the Cortex-A9 reboot event source is the global software warm reset triggered via the  PRCM_PRM_RSTCTRL[0] RST_GLOBAL_WARM_SW bit ?

Q7. Do you use an external pull-down resistor at the pr1_pru0_gpo5 output ?

Q8. According to the AM4372 Datasheet, TableNote6 of the Table, Pin Attributes (ZDN Package) continued, the dss_data5 pin also plays the role of the sysboot5 pin. How do you define the sysboot5 input level - via hardware buffered or non-buffered pull resistors solution ? 

The Table, Reset Sources of the  Section, Reset Characteristics of the AM437x TRM  specifies the following Pinmux and PRU GPO relevant impacts during a global warm reset event:

All the AM4372 IOs including the pr1_pruo_gpo5 pin will be tri-stated during a warm reset. The pin-mux logic will be reset ( and in particular the MuxMode reset to 0x7 GPIO0_13).

There is also a note in the Section, General-Purpose Outputs (R30): Enhanced PRU GP Module and Section, Direct Output of the Chapter, Programmable Real-Time Unit Subsystem and Industrial Communication Subsystem (PRU-ICSS) of the TRM:

Q9. Do you re-initialize the PRU0  R30 register after the reboot of the ARM before the pr1_pru0_gpo5 pin-mux register re-initialization ?

Q10. Do you have a similar issue upon PORz event or only during global software reset ?

Thanks for your clarification !

Kind Regards

Anastas Yordanov

Hi Anastas,

Sorry could not respond earlier to your queries. Please find responses for some of them as below

- We have our custom PCB board

- Actually, there are 2 issues. One is that the char device /dev/rpmsg_31 not created in some of the cases and the other one is GPIO assertion. For the GPIO assertion, we are internally investigating at the moment as our PRU firmware, dynamically set it. So, for now, I am mainly looking to investigate whey /dev/rpmsg_pru31 device does not get created during linux boot up?

[ 24.806663] remoteproc remoteproc1: powering up 54434000.pru
[ 24.826845] remoteproc remoteproc1: Booting fw image am437x-pru1_0-fw, size 133624
[ 24.826933] remoteproc remoteproc1: remote processor 54434000.pru is now up
[ 24.827549] remoteproc remoteproc2: powering up 54438000.pru
[ 24.854061] remoteproc remoteproc2: Booting fw image am437x-pru1_1-fw, size 91528
[ 24.874836] pru-rproc 54438000.pru: configured system_events[63-0] = 00000000,000c0000
[ 24.874850] pru-rproc 54438000.pru: configured intr_channels = 0x0000000a host_intr = 0x0000000a
[ 24.883037] remoteproc2#vdev0buffer: registered virtio0 (type 7)
[ 24.883053] remoteproc remoteproc2: remote processor 54438000.pru is now up
[ 24.901282] ------------[ cut here ]------------
[ 24.901312] WARNING: CPU: 0 PID: 461 at drivers/bus/omap_l3_noc.c:141 l3_interrupt_handler+0x28c/0x384
[ 24.901320] 44000000.ocp:L3 Standard Error: MASTER ICSS1 TARGET GPMC (Read): At Address: 0x00090018 : Data Access in Supervisor mode during Functional access
[ 24.901323] Modules linked in: virtio_rpmsg_bus(+) xt_state xt_comment ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_NFLOG xt_LOG nf_conntrack_tftp xt_conntrack nf_conntrack nf_defrag_ipv6 libcrc32c nfnetlink_log nfnetlink nf_defrag_ipv4 xt_multiport xt_tcpudp ip6table_filter ip6_tables iptable_filter ip_tables x_tables pru_rproc irq_pruss_intc pruss ti_emif_sram omap_wdt sch_fq_codel omap_sham omap_des libdes omap_aes_driver omap_crypto libaes crypto_engine
[ 24.901393] CPU: 0 PID: 461 Comm: systemd-udevd Not tainted 5.4.28-MS_OCS_RM_1.1.37.254421-g6f3bf13d53 #1
[ 24.901396] Hardware name: Generic AM43 (Flattened Device Tree)
[ 24.901426] [<c010f1b8>] (unwind_backtrace) from [<c010b4d4>] (show_stack+0x10/0x14)
[ 24.901440] [<c010b4d4>] (show_stack) from [<c012ccd0>] (__warn+0xbc/0xd4)
[ 24.901449] [<c012ccd0>] (__warn) from [<c012cd80>] (warn_slowpath_fmt+0x98/0xc4)
[ 24.901457] [<c012cd80>] (warn_slowpath_fmt) from [<c0495f2c>] (l3_interrupt_handler+0x28c/0x384)
[ 24.901470] [<c0495f2c>] (l3_interrupt_handler) from [<c016f53c>] (__handle_irq_event_percpu+0x50/0x13c)
[ 24.901480] [<c016f53c>] (__handle_irq_event_percpu) from [<c016f658>] (handle_irq_event_percpu+0x30/0x8c)
[ 24.901486] [<c016f658>] (handle_irq_event_percpu) from [<c016f700>] (handle_irq_event+0x4c/0x8c)
[ 24.901499] [<c016f700>] (handle_irq_event) from [<c0173694>] (handle_fasteoi_irq+0xb4/0x184)
[ 24.901520] [<c0173694>] (handle_fasteoi_irq) from [<c016e710>] (generic_handle_irq+0x24/0x34)
[ 24.901530] [<c016e710>] (generic_handle_irq) from [<c016ee88>] (__handle_domain_irq+0x54/0xa4)
[ 24.901551] [<c016ee88>] (__handle_domain_irq) from [<c0494948>] (gic_handle_irq+0x3c/0x68)
[ 24.901564] [<c0494948>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0xa8)
[ 24.901569] Exception stack(0xcfc59bd8 to 0xcfc59c20)
[ 24.901575] 9bc0: cfb51000 60000013
[ 24.901583] 9be0: 00000001 f0a44000 00000001 00000000 00000001 00000000 00000cc0 f0c91000
[ 24.901590] 9c00: 00004000 f0c92e00 eec10ef4 cfc59c28 c016fbb8 c016f038 20000013 ffffffff
[ 24.901600] [<c0101a8c>] (__irq_svc) from [<c016f038>] (__irq_put_desc_unlock+0x10/0x7c)
[ 24.901608] [<c016f038>] (__irq_put_desc_unlock) from [<c016fbb8>] (irq_set_irqchip_state+0x88/0xc0)
[ 24.901649] [<c016fbb8>] (irq_set_irqchip_state) from [<bf056768>] (pru_rproc_kick+0x54/0x74 [pru_rproc])
[ 24.901684] [<bf056768>] (pru_rproc_kick [pru_rproc]) from [<c06b569c>] (rproc_virtio_notify+0x20/0x28)
[ 24.901710] [<c06b569c>] (rproc_virtio_notify) from [<c04fef4c>] (virtqueue_notify+0x1c/0x34)
[ 24.901753] [<c04fef4c>] (virtqueue_notify) from [<bf0f3b4c>] (rpmsg_probe+0x2a0/0x354 [virtio_rpmsg_bus])
[ 24.901789] [<bf0f3b4c>] (rpmsg_probe [virtio_rpmsg_bus]) from [<c04fedfc>] (virtio_dev_probe+0x1f4/0x2c0)
[ 24.901804] [<c04fedfc>] (virtio_dev_probe) from [<c055fe4c>] (really_probe+0x1e0/0x348)
[ 24.901815] [<c055fe4c>] (really_probe) from [<c0560128>] (driver_probe_device+0x60/0x170)
[ 24.901824] [<c0560128>] (driver_probe_device) from [<c05603d8>] (device_driver_attach+0x58/0x60)
[ 24.901833] [<c05603d8>] (device_driver_attach) from [<c0560438>] (__driver_attach+0x58/0xcc)
[ 24.901840] [<c0560438>] (__driver_attach) from [<c055e00c>] (bus_for_each_dev+0x78/0xc0)
[ 24.901848] [<c055e00c>] (bus_for_each_dev) from [<c055f1c8>] (bus_add_driver+0x15c/0x1e0)
[ 24.901858] [<c055f1c8>] (bus_add_driver) from [<c0560d14>] (driver_register+0x74/0x108)
[ 24.901873] [<c0560d14>] (driver_register) from [<bf027010>] (rpmsg_init+0x10/0x1000 [virtio_rpmsg_bus])
[ 24.901877] ---[ end trace 179cf74053c64539 ]---
[ 24.901923] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901941] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901971] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901980] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901988] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 24.901998] virtio_rpmsg_bus virtio0: input:id 16 out of range
[ 24.902021] virtio_rpmsg_bus virtio0: rpmsg host is online

In some of the BAD cases, the entire dmesg is filled with below log completely

[ 24.901988] virtio_rpmsg_bus virtio0: msg received with no recipient

Q6: I need to check which register is internally used by the linux. We do the WARM reboot of the SoC using linux reboot command. Is there an easy way to find out from the user space?

Regards,

Sumit

Hello Sumit,

1) Your application is actually using the PRU RPMsg, right? If not, we do not actually need to set up PRU RPMsg communication on later Linux kernel versions (though I think it is still required on Linux kernel 5.4).

2) Can I get you to confirm that PRU is waiting for Linux to signal the Linux is ready before PRU starts creating the RPMsg channel? Your code should look like this (taken from PSSP tag 5.9, which was for kernel 5.4)
https://git.ti.com/cgit/pru-software-support-package/pru-software-support-package/tree/examples/am437x/PRU_RPMsg_Echo_Interrupt0_0/main.c?h=v5.9.0

	/* Enable the OCP master port of PRUSS1 to read external memories */
	PRUSS1_CFG_SYSCFG &= ~(STANDY_INIT_BIT);

	/* Clear the status of the PRU-ICSS system event that the ARM will use to 'kick' us */
	CT_INTC.SICR_bit.STATUS_CLR_INDEX = FROM_ARM_HOST;

	/* Make sure the Linux drivers are ready for RPMsg communication */
	status = &resourceTable.rpmsg_vdev.status;
	while (!(*status & VIRTIO_CONFIG_S_DRIVER_OK));

	/* Initialize the RPMsg transport structure */
	pru_rpmsg_init(&transport, &resourceTable.rpmsg_vring0, &resourceTable.rpmsg_vring1, TO_ARM_HOST, FROM_ARM_HOST);

	/* Create the RPMsg channel between the PRU and ARM user space using the transport structure. */
	while (pru_rpmsg_channel(RPMSG_NS_CREATE, &transport, CHAN_NAME, CHAN_DESC, CHAN_PORT) != PRU_RPMSG_SUCCESS);

Regards,

Nick

Thanks Nick.

1. Yes, we rely on PRU RPMsg on which ARM SoC talk to PRU cores. In the problem case, this device does not get created and we get several (virtio_rpmsg....) and eventually "rpmsg host is online". But still the device is not created.

What does "rpmsg host is online" means?

[ 40.657383] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 40.657391] virtio_rpmsg_bus virtio0: msg received with no recipient
[ 40.852618] systemd-journald[393]: /dev/kmsg buffer overrun, some messages lost.
[ 40.991504] virtio_rpmsg_bus virtio0: rpmsg host is online

2. Yes, our code is like this (same as what you pasted).

Hello Sumit,

I did not have time to dig into your thread today. Will give a response tomorrow.

Setting expectations, I can spend a few minutes skimming through kernel 5.4 drivers, but the PRU remoteproc driver has changed quite a bit between kernel 5.4 and the latest kernel 6.12 (SDK releasing within a week).

Regards,

Nick

Thanks Nick

Some more updates on this issue

Earlier in year 2019 we added below 2 uboot patches to enable support for PRU warm restart. Looks like this patch is now causing regression.

1) Not sure at the moment, why these patches were not causing this issue (aggressively) earlier but now. Any pointers you can give which we can check?

2) For now, we have disabled these patches, but we do have to support PRU warm restart. Can you give us the guidance that if there is anything incorrect with below patches and what changes we should do so that we do not come across this issue?

diff --git a/arch/arm/cpu/armv7/am33xx/board.c b/arch/arm/cpu/armv7/am33xx/board.c
index 8ca828f..2d71758 100644
--- a/arch/arm/cpu/armv7/am33xx/board.c
+++ b/arch/arm/cpu/armv7/am33xx/board.c
@@ -402,6 +402,24 @@ void s_init(void)
 #endif
 }
 
+/*
+ * Starts both PRU0 and PRU1 dual-cores for the PRU_ICSS1 processor.
+ * PRU_ICSS1 PRU1 Control register: at global memory-mapped location 0x54424000
+ * PRU_ICSS1 PRU0 Control register: at global memory-mapped location 0x54422000
+ *
+ * PRU Control Register definitions: (with value 0xB).
+ * bit 31-16: this field contains address where PRU will start executing code. (we will start at address zero)
+ * bit 3: if set, enable PRU cycle counters.
+ * bit 1: if set, enable PRU to fetch new instructions.
+ * bit 0: if set, no soft reset.
+*/
+void PRU_ICSS1_PRU0_and_1_Start(void)
+
+{
+	*((unsigned int*)0x54424000) = 0xB;
+	*((unsigned int*)0x54422000) = 0xB;
+}
+
 void early_system_init(void)
 {
 	/*
@@ -424,6 +442,22 @@ void early_system_init(void)
 	/* Enable RTC32K clock */
 	rtc32k_enable();
 #endif
+
+	/*
+	 * The following code must be done after the clock of
+	 * the PRU modules were enabled in the setup_early_clocks().
+	 */
+	if (warm_reset()) {
+		/*
+		 * Set GPIO pin 5 for BLADE_THROTTLE_N,
+		 * which will used by PRU0 to interrupt blade
+		 * for power throttle alert.
+		 */
+		set_dss_mux_conf();
+
+		/* restart PRU0 firmware */
+		PRU_ICSS1_PRU0_and_1_Start();
+	}
 }
 
 #ifdef CONFIG_SPL_BUILD
diff --git a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
index ce5b826..d2ffa6d 100644
--- a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
+++ b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
@@ -20,6 +20,8 @@ struct cm_perpll *const cmper = (struct cm_perpll *)CM_PER;
 struct cm_wkuppll *const cmwkup = (struct cm_wkuppll *)CM_WKUP;
 struct cm_dpll *const cmdpll = (struct cm_dpll *)CM_DPLL;
 
+#define PRCM_RM_PER_RSTCTRL	0x44DF0810
+
 const struct dpll_regs dpll_mpu_regs = {
 	.cm_clkmode_dpll	= CM_WKUP + 0x560,
 	.cm_idlest_dpll		= CM_WKUP + 0x564,
@@ -99,6 +101,8 @@ void setup_clocks_for_console_uart1(void)
 
 void enable_basic_clocks(void)
 {
+	u32 temp;
+
 	u32 *const clk_domains[] = {
 		&cmper->l3clkstctrl,
 		&cmper->l3sclkstctrl,
@@ -119,6 +123,9 @@ void enable_basic_clocks(void)
 		&cmwkup->wkctrlclkctrl,
 		&cmper->timer2clkctrl,
 		&cmper->gpmcclkctrl,
+		&cmper->pruclkctrl, 	// Enable PRUSS module and clock
+		&cmper->adc1clkctrl,	// Enable ADC clock
+		&cmwkup->wkadc0clkctrl,
 		&cmper->elmclkctrl,
 		&cmper->mmc0clkctrl,
 		&cmper->mmc1clkctrl,
@@ -145,6 +152,14 @@ void enable_basic_clocks(void)
 
 	/* For OPP100 the mac clock should be /5. */
 	writel(0x4, &cmdpll->clkselmacclk);
+
+	/* PRU-ICSS module local reset control */
+	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL register
+	temp |= 0x2;                		// Assert Reset for PRU-ICSS
+	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
+	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL
+	temp &= 0xFFFFFFFD;			// Clear Reset for PRU-ICSS
+	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
 }
 
 void rtc_only_enable_basic_clocks(void)
diff --git a/arch/arm/include/asm/arch-am33xx/cpu.h b/arch/arm/include/asm/arch-am33xx/cpu.h
index bacbee0..1c4852e 100644
--- a/arch/arm/include/asm/arch-am33xx/cpu.h
+++ b/arch/arm/include/asm/arch-am33xx/cpu.h
@@ -231,8 +231,10 @@ struct prm_device_inst {
 #else
 /* Encapsulating core pll registers */
 struct cm_wkuppll {
-	unsigned int resv0[136];
-	unsigned int wkl4wkclkctrl;	/* offset 0x220 */
+	unsigned int resv0[72];
+	unsigned int wkadc0clkctrl;	/* offset 0x120 */
+	unsigned int resv01[63];
+	unsigned int wkl4wkclkctrl; 	/* offset 0x220 */
 	unsigned int resv1[7];
 	unsigned int usbphy0clkctrl;	/* offset 0x240 */
 	unsigned int resv112;
@@ -313,7 +315,9 @@ struct cm_perpll {
 	unsigned int l3sclkstctrl;	/* offset 0x200 */
 	unsigned int resv8[7];
 	unsigned int gpmcclkctrl;	/* offset 0x220 */
-	unsigned int resv9[5];
+	unsigned int resv9[3];
+	unsigned int adc1clkctrl;   	/* offset 0x230 */
+	unsigned int resv91[1];
 	unsigned int mcasp0clkctrl;	/* offset 0x238 */
 	unsigned int resv10;
 	unsigned int mcasp1clkctrl;	/* offset 0x240 */
@@ -325,7 +329,11 @@ struct cm_perpll {
 	unsigned int usb0clkctrl;	/* offset 0x260 */
 	unsigned int resv122;
 	unsigned int usb1clkctrl;	/* offset 0x268 */
-	unsigned int resv13[101];
+	unsigned int resv13[37];
+	unsigned int pruclkstctrl;      /* offset 0x300 */
+	unsigned int resv131[7];
+	unsigned int pruclkctrl;        /* offset 0x320 */
+	unsigned int resv132[55];
 	unsigned int l4lsclkstctrl;	/* offset 0x400 */
 	unsigned int resv14[7];
 	unsigned int l4lsclkctrl;	/* offset 0x420 */
diff --git a/arch/arm/include/asm/arch-am33xx/sys_proto.h b/arch/arm/include/asm/arch-am33xx/sys_proto.h
index 2d34c96..ca94ce9 100644
--- a/arch/arm/include/asm/arch-am33xx/sys_proto.h
+++ b/arch/arm/include/asm/arch-am33xx/sys_proto.h
@@ -34,6 +34,7 @@ void enable_gpmc_cs_config(const u32 *gpmc_config, struct gpmc_cs *cs, u32 base,
 void omap_nand_switch_ecc(uint32_t, uint32_t);
 
 void set_uart_mux_conf(void);
+void set_dss_mux_conf(void);
 void set_mux_conf_regs(void);
 void sdram_init(void);
 u32 wait_on_value(u32, u32, void *, u32);
diff --git a/board/ti/am43xx_ocs/board.c b/board/ti/am43xx_ocs/board.c
index 2879987..5e0c317 100644
--- a/board/ti/am43xx_ocs/board.c
+++ b/board/ti/am43xx_ocs/board.c
@@ -459,6 +459,12 @@ void set_uart_mux_conf(void)
 	enable_uart1_pin_mux();
 }
 
+void set_dss_mux_conf(void)
+{
+	enable_dss_pin_mux();
+}
+
+
 void set_mux_conf_regs(void)
 {
 	enable_board_pin_mux();
diff --git a/board/ti/am43xx_ocs/board.h b/board/ti/am43xx_ocs/board.h
index a28a3ac..343c45f 100644
--- a/board/ti/am43xx_ocs/board.h
+++ b/board/ti/am43xx_ocs/board.h
@@ -21,4 +21,5 @@ void enable_uart0_pin_mux(void);
 void enable_uart1_pin_mux(void);
 void enable_board_pin_mux(void);
 void enable_i2c0_pin_mux(void);
+void enable_dss_pin_mux(void);
 #endif
diff --git a/board/ti/am43xx_ocs/mux.c b/board/ti/am43xx_ocs/mux.c
index b1a3a33..3625884 100644
--- a/board/ti/am43xx_ocs/mux.c
+++ b/board/ti/am43xx_ocs/mux.c
@@ -62,6 +62,12 @@ static struct module_pin_mux uart1_pin_mux[] = {
 	{-1},
 };
 
+/* This GPIO pin will be used by the ICSS PRU0 for BLADE_THROTTLE alert */
+static struct module_pin_mux dss_pin_mux[] = {
+	{OFFSET(lcd_data5), (MODE(5) | PULLDOWN_EN)},
+	{-1},
+};
+
 /*
 
 {OFFSET(uart1_ctsn), (MODE(0) | PULLUP_EN | RXACTIVE | SLEWCTRL)},
@@ -125,6 +131,11 @@ void enable_uart1_pin_mux(void)
 	configure_module_pin_mux(uart1_pin_mux);
 }
 
+void enable_dss_pin_mux(void)
+{
+	configure_module_pin_mux(dss_pin_mux);
+}
+
 void enable_board_pin_mux(void)
 {
 	configure_module_pin_mux(mdio_pin_mux);

Below patch on top of above

diff --git a/arch/arm/cpu/armv7/am33xx/board.c b/arch/arm/cpu/armv7/am33xx/board.c
index 2d71758..8ca828f 100644
--- a/arch/arm/cpu/armv7/am33xx/board.c
+++ b/arch/arm/cpu/armv7/am33xx/board.c
@@ -402,24 +402,6 @@ void s_init(void)
 #endif
 }
 
-/*
- * Starts both PRU0 and PRU1 dual-cores for the PRU_ICSS1 processor.
- * PRU_ICSS1 PRU1 Control register: at global memory-mapped location 0x54424000
- * PRU_ICSS1 PRU0 Control register: at global memory-mapped location 0x54422000
- *
- * PRU Control Register definitions: (with value 0xB).
- * bit 31-16: this field contains address where PRU will start executing code. (we will start at address zero)
- * bit 3: if set, enable PRU cycle counters.
- * bit 1: if set, enable PRU to fetch new instructions.
- * bit 0: if set, no soft reset.
-*/
-void PRU_ICSS1_PRU0_and_1_Start(void)
-
-{
-	*((unsigned int*)0x54424000) = 0xB;
-	*((unsigned int*)0x54422000) = 0xB;
-}
-
 void early_system_init(void)
 {
 	/*
@@ -442,22 +424,6 @@ void early_system_init(void)
 	/* Enable RTC32K clock */
 	rtc32k_enable();
 #endif
-
-	/*
-	 * The following code must be done after the clock of
-	 * the PRU modules were enabled in the setup_early_clocks().
-	 */
-	if (warm_reset()) {
-		/*
-		 * Set GPIO pin 5 for BLADE_THROTTLE_N,
-		 * which will used by PRU0 to interrupt blade
-		 * for power throttle alert.
-		 */
-		set_dss_mux_conf();
-
-		/* restart PRU0 firmware */
-		PRU_ICSS1_PRU0_and_1_Start();
-	}
 }
 
 #ifdef CONFIG_SPL_BUILD
diff --git a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
index d2ffa6d..8c97803 100644
--- a/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
+++ b/arch/arm/cpu/armv7/am33xx/clock_am43xx.c
@@ -20,8 +20,6 @@ struct cm_perpll *const cmper = (struct cm_perpll *)CM_PER;
 struct cm_wkuppll *const cmwkup = (struct cm_wkuppll *)CM_WKUP;
 struct cm_dpll *const cmdpll = (struct cm_dpll *)CM_DPLL;
 
-#define PRCM_RM_PER_RSTCTRL	0x44DF0810
-
 const struct dpll_regs dpll_mpu_regs = {
 	.cm_clkmode_dpll	= CM_WKUP + 0x560,
 	.cm_idlest_dpll		= CM_WKUP + 0x564,
@@ -101,7 +99,6 @@ void setup_clocks_for_console_uart1(void)
 
 void enable_basic_clocks(void)
 {
-	u32 temp;
 
 	u32 *const clk_domains[] = {
 		&cmper->l3clkstctrl,
@@ -123,8 +120,6 @@ void enable_basic_clocks(void)
 		&cmwkup->wkctrlclkctrl,
 		&cmper->timer2clkctrl,
 		&cmper->gpmcclkctrl,
-		&cmper->pruclkctrl, 	// Enable PRUSS module and clock
-		&cmper->adc1clkctrl,	// Enable ADC clock
 		&cmwkup->wkadc0clkctrl,
 		&cmper->elmclkctrl,
 		&cmper->mmc0clkctrl,
@@ -152,14 +147,6 @@ void enable_basic_clocks(void)
 
 	/* For OPP100 the mac clock should be /5. */
 	writel(0x4, &cmdpll->clkselmacclk);
-
-	/* PRU-ICSS module local reset control */
-	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL register
-	temp |= 0x2;                		// Assert Reset for PRU-ICSS
-	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
-	temp = readl(PRCM_RM_PER_RSTCTRL);	// Read PRCM_RM_PER_RSTCTRL
-	temp &= 0xFFFFFFFD;			// Clear Reset for PRU-ICSS
-	writel(temp, PRCM_RM_PER_RSTCTRL);	// Write PRCM_RM_PER_RSTCTRL
 }
 
 void rtc_only_enable_basic_clocks(void)
diff --git a/board/ti/am43xx_ocs/board.c b/board/ti/am43xx_ocs/board.c
index 5e0c317..bb25ef6 100644
--- a/board/ti/am43xx_ocs/board.c
+++ b/board/ti/am43xx_ocs/board.c
@@ -42,6 +42,7 @@ int ocs_i2c_eeprom_get_macid(uint8_t *mac0, uint8_t *mac1);
 
 #ifndef CONFIG_SKIP_LOWLEVEL_INIT
 #define NUM_OPPS	6
+#define PRCM_RM_PER_RSTCTRL     0x44DF0810
 
 const struct dpll_params dpll_mpu[NUM_CRYSTAL_FREQ][NUM_OPPS] = {
 	{	/* 19.2 MHz */
@@ -475,6 +476,61 @@ void sdram_init(void)
 	config_ddr(400, &ioregs_ddr3, NULL, NULL,
 		   &ddr3_msocs_emif_regs_400Mhz, 0);
 }
+
+/*
+ * Starts both PRU0 and PRU1 dual-cores for the PRU_ICSS1 processor.
+ * PRU_ICSS1 PRU1 Control register: at global memory-mapped location 0x54424000
+ * PRU_ICSS1 PRU0 Control register: at global memory-mapped location 0x54422000
+ *
+ * PRU Control Register definitions: (with value 0xB).
+ * bit 31-16: this field contains address where PRU will start executing code. (we will start at address zero)
+ * bit 3: if set, enable PRU cycle counters.
+ * bit 1: if set, enable PRU to fetch new instructions.
+ * bit 0: if set, no soft reset.
+*/
+void PRU_ICSS1_PRU0_and_1_Start(void)
+
+{
+	*((unsigned int*)0x54424000) = 0xB;
+	*((unsigned int*)0x54422000) = 0xB;
+}
+
+int pru_init(void)
+{
+
+	struct cm_perpll *const cmper = (struct cm_perpll *)CM_PER;
+
+	/*
+	 * The following code must be done after the clock of
+	 * the PRU modules were enabled in the setup_early_clocks().
+	 */
+	if (warm_reset()) {
+		u32 temp, i;
+		struct cm_perpll *const cmper = (struct cm_perpll *)CM_PER;
+
+		/* PRU-ICSS module local reset control */
+		temp = readl(PRCM_RM_PER_RSTCTRL);      // Read PRCM_RM_PER_RSTCTRL register
+		temp |= 0x2;                            // Assert Reset for PRU-ICSS
+		writel(temp, PRCM_RM_PER_RSTCTRL);      // Write PRCM_RM_PER_RSTCTRL
+		temp = readl(PRCM_RM_PER_RSTCTRL);      // Read PRCM_RM_PER_RSTCTRL
+		temp &= 0xFFFFFFFD;                     // Clear Reset for PRU-ICSS
+		writel(temp, PRCM_RM_PER_RSTCTRL);      // Write PRCM_RM_PER_RSTCTRL
+		writel(0x0002, &cmper->pruclkctrl);	// Enable PRUSS module and clock
+		writel(0x0002, &cmper->adc1clkctrl);	// Enable ADC clock
+
+		/*
+		 * Set GPIO pin 5 for BLADE_THROTTLE_N,
+		 * which will used by PRU0 to interrupt blade
+		 * for power throttle alert.
+		 */
+		set_dss_mux_conf();
+
+		/* restart PRU0 firmware */
+		PRU_ICSS1_PRU0_and_1_Start();
+	}
+	return 0;
+}
+
 #endif
 
 /* setup board specific PMIC */
diff --git a/common/board_f.c b/common/board_f.c
index 109025a..181935c 100644
--- a/common/board_f.c
+++ b/common/board_f.c
@@ -117,7 +117,7 @@ static int init_func_watchdog_init(void)
 # if defined(CONFIG_HW_WATCHDOG) && (defined(CONFIG_BLACKFIN) || \
 	defined(CONFIG_M68K) || defined(CONFIG_MICROBLAZE) || \
 	defined(CONFIG_SH) || defined(CONFIG_AT91SAM9_WATCHDOG) || \
-	defined(CONFIG_IMX_WATCHDOG))
+	defined(CONFIG_IMX_WATCHDOG) || defined(CONFIG_AM43XX))
 	hw_watchdog_init();
 # endif
 	puts("       Watchdog enabled\n");
diff --git a/common/board_r.c b/common/board_r.c
index ad02549..f1feca4 100644
--- a/common/board_r.c
+++ b/common/board_r.c
@@ -351,6 +351,11 @@ __weak int power_init_board(void)
 	return 0;
 }
 
+__weak int pru_init(void)
+{
+	return 0;
+}
+
 static int initr_announce(void)
 {
 	debug("Now running in RAM - U-Boot at: %08lx\n", gd->relocaddr);
@@ -841,6 +846,9 @@ init_fnc_t init_sequence_r[] = {
 	arch_early_init_r,
 #endif
 	power_init_board,
+#ifdef	CONFIG_AM43XX
+	pru_init,
+#endif
 #ifndef CONFIG_SYS_NO_FLASH
 	initr_flash,
 #endif

3) Is it possible that PRU warm restart enablement can cause the shared memory area to give random values in a bad case? We have few data stored at PRU shared memory area. Example below for one of the variable in that area.

[ 14.229931] remoteproc remoteproc1: 54434000.pru is available
[ 14.255653] pru-rproc 54434000.pru: PRU rproc node /ocp@44000000/target-module@54400000/pruss@0/pru@34000 probed successfully
[ 14.286956] remoteproc remoteproc2: 54438000.pru is available
[ 14.292833] pru-rproc 54438000.pru: PRU rproc node /ocp@44000000/target-module@54400000/pruss@0/pru@38000 probed successfully

For example, we have 

Hello,

It looks like your response got cut off (the last thing I see is "for example, we have").

Please note that the PRU register values, and the PRU DMEM and SMEM values, are not initialized to set values on power-on. (that is why you can get glitches on the PRU GPO signals during initialization if you do pinmuxing first, then power on the PRU subsystem. Because R30 will start driving signals high or low as soon as the PRU subsystem is powered on, but there will be a time between when the PRU subsystem is powered, and when the PRU firmware starts running and is able to zero out the R30 register).

For the Shared RAM / SMEM, I suggest initializing the memory to zero first, then writing and reading data to it.

For the U-Boot code, this is outside my area of expertise. However, we are working on adding support for U-Boot initializing PRU on AM62x & AM64x right now in kernel 6.12 (not AM437x unfortunately). If you want, I can make a note to point you to that U-Boot code after we publish it (in case it is helpful).

I will be on vacation for the rest of this week, but I will be back next week. Feel free to ping the thread if I have not replied within the first couple of days next week.

Regards,

Nick

Thanks Nick.

One quick question. Is it possible to read the value of R30 from devmem? How to find the exact register address of R30?

Nick Saulnier said:

sponse got cut off (the last thing I see is "for example, we have").

Hello Sumit,

I am catching back up from vacation, thank you for your patience.

R30 & R31 are internal registers in the PRU core. So you could write PRU code that copies the value of R30 into a memory location, and then use devmem2 to read the value of that memory location. However, you would not be able to read the register values directly from Linux.

You could see an example of interacting with R30/31 in C here:
https://git.ti.com/cgit/pru-software-support-package/pru-software-support-package/tree/examples/am335x/PRU_gpioToggle/PRU_gpioToggle.c

and the PRU Getting Started Labs have an example of reading and writing to known memory locations for Linux to access:
https://software-dl.ti.com/processor-sdk-linux/esd/AM437X/latest/exports/docs/common/PRU-ICSS/PRU-Getting-Started-Labs_Lab5.html

If you are debugging with CCS, you can use CCS to read the register values. See the Lab documentation above for more details.

Regards,

Nick