TMDS64EVM: PPS change frequency after re-enabling

Hello Jakub,

Jakub Grad said:

singal is measured on GPIO1_37 (V5) output

Does this V5 output route to a measurable jumper pin on the TMDS64EVM? Based on the k3-am642-evm.dts, V5 is pinmuxed to a signal for CPSW rgmii, so I'm wondering if you are changing the pinmuxing of CPSW rgmii pins in your dts in order to use this V5 pin?

Instead of V5, I would recommend routing the PPS signal to a jumper pin on the TMDS64EVM such as D18. You can use the instructions in the following thread as a guide (please note it is referencing SK-AM64B but the steps should be similar). https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1300940/am6442-1-pps-from-sk-am64b/

A couple of additional questions for you:

1. What Linux SDK version are you using?

2. You added TMDS64EVM as the part number but I want to double check: are you using the TMDS64EVM or a custom board with AM64x?

-Daolin

Hello Daolin,

I made mistake describing the problem: we measuring PPS signal from pin W1 (PRG0_PRU0_GPO19) and we have it configured in DT like that:

icssg0_iep0_pins_default: icssg0-iep0-pins-default {

    pinctrl-single,pins = <

		AM64X_IOPAD(0x01AC, PIN_OUTPUT, 2) /* (W1) PRG0_PRU0_GPO19.PRG0_IEP0_EDC_SYNC_OUT0 */
	>;

};
        
&icssg0_iep0 {

	pinctrl-names = "default";

	pinctrl-0 = <&icssg0_iep0_pins_default>;

	status = "okay";

};

Answering your questions:

1. We are using SDK 8 (meta-ti yocto layer)

2. Yes. we are using custom hardware based on AM6442, sorry I did't mentioned this at first.

I just want to mark that we have PPS signal form W1 pin, so I guess hardware is configured properly at least in some part. Our issue is that after disableing pps signal via kernel sysfs attribute (echo 0 > /sys/class/ptp/ptp0/pps_enable) and enabling it again, W1 output signal have wrong frequency (about 10 Hz, not 1 Hz). I don't have TMDS64EVM board on my desk currently. Are you able to reproduce this issue on evaluation board?

BR

Jakub

Hi Jakub,

Thanks for clarifying the details

Please allow me some time to see if I see the same issue on an TMD64EVM board. I will likely test on pin D18 since that breaks out a jumper that I can easily measure with an oscilloscope. 

I hope to get back you with an update tomorrow or Friday.

-Daolin

Update:

So, when I routed the 1PPS signal to D18 (SYNC0_out), I see 1hz rate both the first time starting 1PPS and after I disabled 1PPS and restarted again, see below for my exact sequence. 

1. Make changes to k3-am642-evm.dts to enabled D18 and SYNC0_out and build and copy dtb into SD card. https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/0876.k3_2D00_am642_2D00_evm.dts

main_ecap0_pins_default: main-ecap0-pins-default {
	pinctrl-single,pins = <
		AM64X_IOPAD(0x0270, PIN_INPUT, 1) /* (D18) ECAP0_IN_APWM_OUT ---> SYNC0_OUT*/
		AM64X_IOPAD(0x0274, PIN_INPUT, 0) /* (A19) EXT_REFCLK1 */
	>;
};

cpsw_cpts_pps: cpsw-cpts-pps {
	pinctrl-single,pins = <
		/* pps [cpts genf1] in22 -> out37 [cpts hw8_push] */
		TS_OFFSET(37, 22)
		/* pps [cpts genf1] in22 -> out24 [SYNC0_OUT pin], out26 [SYNC1_OUT pin] */
		TS_OFFSET(24, 22)
		>;
};

2. Run "echo 1 > /sys/class/ptp/ptp0/pps_enable" and saw 1Hz signal on oscilloscope at D18 (J12 on EVM)

3. Run "echo 0 > /sys/class/ptp/ptp0/pps_enable" and checked no signal on oscilloscope

4. Run "echo 1 > /sys/class/ptp/ptp0/pps_enable" and saw 1Hz signal on oscilloscope 

I noticed in your original dts snippet for mcu_cpts_pps, you used out25, however that corresponds to with "SYNC1_out" instead of the "PRG0_IEP0_EDC_SYNC_OUT0" you were trying to enable for pin W1.

See below TRM snippet from 10.3.2.2.1 TIMESYNC_INTRTR0 Integration.

I think you might have confused out25 with the list of inputs. If this is the case, I think you might have configured out25 based on the list of time sync router inputs. What you actually need to look at is the list of time sync router outputs.

For this reason, I would suggest using D18 or one of the other pins that can be pinmuxed to one of the time sync output signals. Is it possible for you to change to a different pin? Is there a particular reason why you are using W1?

-Daolin

Hi Daoilin

Thanks very much that you checked this. I was describing issue discovered by my team mate and looks like I misunderstood him at first. I reproduced issue on my custom hardware. PPS signal frequency changes if I disable and re-enable both PRU ethernet interfaces, not pps_enable sysfs attribute itself.

1. Checking which ethernet interface is PRU

# ethtool -i eth2
driver: icssg-prueth
version: 6.1.69-rt21
firmware-version:
expansion-rom-version:
bus-info: icssg0-eth
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
#
# ethtool -i eth3
driver: icssg-prueth
version: 6.1.69-rt21
firmware-version:
expansion-rom-version:
bus-info: icssg0-eth
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

2. ifconfig eth2 up / ifconfig eth3 up

3. PPS signal is correct

4. Disabling PRU interfaces: ifconifg eth2 down / ifconfig eth3 down

PPS signal not generated

5. Re-enabling PRU interfaces: PPS signal incorrect:

Could you check this scenario?

Regarding what you said:

1. What is the role of timesync_router DT node (I just copied it from EVM device tree). I done some test: commented out whole timesync router node and I have still PPS signal available. In kernel sources I see that driver responsible for PPS signal (am65-cpts,c) is enabled by CONFIG_TI_K3_AM65_CPTS. Does it mean that PPS can be delivered delivered multiple ways: by PRU ICSSG firmware or by Linux kernel driver?

2. Yes, we need to utilize W1 pin, because D18 i used for other purpose. It is possible to achieve what you described on W1 pin using timesync router?

Best regards
Jakub

Hi Jakub,

Thanks for clarifying the issue! I realized I probably should have first asked if you were sourcing the 1PPS from PRU ICSSG ethernet vs CPSW ethernet.

Please note that the CPTS (common platform time sync) is the module that handles timestamping for CPSW ethernet while IEP (Industrial Ethernet Peripheral) is the module that handles timestamping for PRU_ICSSG ethernet. Both are capable of 1PPS support via PTP but the former does it through CPSW and the latter does it through PRU ethernet.

(1) From my understanding, the timesync router (TSR) simply a module that is designed to route an input interrupt to multiple output interrupts. See this link for more information https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1061474/faq-am64x-what-is-the-time-sync-router-for-how-do-i-use-it.

The way I have previously tested 1PPS was to use the TSR to route the 1PPS signal from the CPTS genf1 module to a measurable jumper which happened to be connected to one of the TSR outputs (D18). Additionally, this CPTS genf1 was a subsystem of CPSW ethernet, so this method I was referencing was specifically for CPSW and not for PRU ICSSG ethernet.

I believe the timesync router actually is not necessary to route the 1PPS signal to a measurable pin as indicated by https://software-dl.ti.com/processor-sdk-linux-rt/esd/AM64X/latest/exports/docs/linux/Foundational_Components/PRU-ICSS/Linux_Drivers/PRU_ICSSG_Ethernet.html?highlight=pru_icssg#pps-pulse-per-second-support for PRU ICSSG ethernet. Based on this resource, I believe the 1PPS is delivered by Linux kernel driver but I will check with our sw developers.

(2) It should be possible to route the signal generated from W1 (PRG0_IEP0_EDC_SYNC_OUT0) as an input to the TSR and output to another TSR output connected to another pin. But as mentioned in (1) it's not necessary, doing this would only be useful if you wanted to route your signal from W1 to another SoC pin such as D18 in case there are any signal integrity issues with the physical pin W1 is connected to. My answer on this thread provides some details: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1339040/am6442-ptp-and-pps-via-pru-subsystem/5105883#5105883 

Apologies for the lengthy response, allow me some time to try and replicate your results. I will need to switch to 1PPS generated from IEP (PRU ethernet). I hope to respond with an update Monday or Tuesday next week. 

Questions for you:

1. If you disable pps_enable before bringing the PRU interfaces down, is the issue still observed?

2. If you re-enable pps_enable after bring the PRU interfaces back up, is the issue still observed?

2. Why does the scope show 15Hz for the signals measured both before the PRU interfaces down and after?

3. If I'm understanding the waveform correctly, it looks like the frequency of the 1PPS is same before and after the PRU interfaces down but the signal changes from low most of the time (with pulses up to high - what you expect) to high most of time (with pulses down to low)?

-Daolin

Update:

I've tested this out on SDK 09.02.01.09 and can reproduce what appears to be the same waveform as you (~100ms period). I've created a bug report to bring this to our developers' attention and will communicate with them on this. 

In my setup, I used W7 (pinned out to J18 on the EVM) without using the timesync router. Additionally, I used /usr/kernel-selftest/ptp/testptp -d /dev/ptp2 -P 1 to enable the 1PPS instead of through echo 1 > /sys/class/ptp/ptp2/pps_enable since this command did not work for ptp2, which is the ptp device associated with PRU ethernet interface (I think this is due to the SDK 8 (5.10) you were using kernel difference from SDK 09.02.01.09 (6.1) kernel I'm using). Is there a reason why you are sticking to SDK 8 as opposed to using the latest SDK? Just to clarify, are you currently using SDK 08.00.00.21?

root@am64xx-evm:/usr/kernel-selftest/ptp# ls -lt /sys/class/ptp 
total 0
lrwxrwxrwx 1 root root 0 May 29 08:50 ptp2 -> ../../devices/platform/bus@f4000/30080000.icssg/300ae000.iep/ptp/ptp2
lrwxrwxrwx 1 root root 0 Apr 28 17:42 ptp1 -> ../../devices/platform/bus@f4000/39000000.cpts/ptp/ptp1
lrwxrwxrwx 1 root root 0 Apr 28 17:42 ptp0 -> ../../devices/platform/bus@f4000/8000000.ethernet/ptp/ptp0 

I noticed that if you restart the 1PPS signal after bringing the PRU interface back up (run /usr/kernel-selftest/ptp/testptp -d /dev/ptp2 -P 1 again), the 1PPS is back to 1Hz. Can you check on this?

-Daolin

Hi Daolin

Thanks for comprehensive answer! That explain me a lot how it works internally.
Answering yo your questions:

Daolin Qiu said:

If you disable pps_enable before bringing the PRU interfaces down, is the issue still observed?

No, the signal is correct.

Daolin Qiu said:

If you re-enable pps_enable after bring the PRU interfaces back up, is the issue still observed?

No, the signal is correct. I looks like disabling PRU interfaces when "pps_enable" sysfs attribute has been written 1 causes the issue. But this question tracked me to workaround: maybe before switching off PRU interfaces we can disable PPS signal first. I must check this with rest of our custom boards.

Daolin Qiu said:

Why does the scope show 15Hz for the signals measured both before the PRU interfaces down and after?

This is just some dump setting enabled, sorry for misleading. 

Daolin Qiu said:

If I'm understanding the waveform correctly, it looks like the frequency of the 1PPS is same before and after the PRU interfaces down but the signal changes from low most of the time (with pulses up to high - what you expect) to high most of time (with pulses down to low)?

Hmm I don't agree with that. Please look at time base. It is 200 ms, same like on above picture, but the waveform is more "dense". Also oscilloscope measure about 10 Hz of the signal. 

Daolin Qiu said:

Is there a reason why you are sticking to SDK 8 as opposed to using the latest SDK? Just to clarify, are you currently using SDK 08.00.00.21?

We were testing newest SDK, especially kernel regarding to performance, but it looks like there is no difference from kernel 5.10. We plan to switch the newest SDK later.
I don’t know which version of SDK exactly we are using (how to check it?) but it is meta-ti kirkstone branch, commit: 8ad82f5e69c3ed739bbdd5d1a0afd75ab7608ec8
From May 1 2023

Daolin Qiu said:

I noticed that if you restart the 1PPS signal after bringing the PRU interface back up (run /usr/kernel-selftest/ptp/testptp -d /dev/ptp2 -P 1 again), the 1PPS is back to 1Hz. Can you check on this?

Yes, that is correct.
1. I enabled PPS (1HZ signal)
2. killed both PRU interfaces
3. Bring back PRU interfaces (PPS approximately 10 Hz)
4. disabled PPS signal via syfs attribute
5. enabled PPS signal (now is correct 1Hz)

Ok, thanks for reporting this as a bug. I will wait for update from your side.
BR
Jakub

Hi Jakub,

Jakub Grad said:

But this question tracked me to workaround: maybe before switching off PRU interfaces we can disable PPS signal first. I must check this with rest of our custom boards.

After discussing with a team member, it might be the case that by default, the IEP counter is configured to a period of ~100ms before 1PPS is even enabled and only configured to 1s after 1PPS is enabled and the pin-mux to measure 1PPS is "activated". Thus, afterwards when PRU ICSSG interfaces brought down and back up before disabled 1PPS, the pin to measure 1PPS will see the ~100ms possibly due to IEP counter not reconfigured correctly.

All of this sounds a bit abstract so a more specific way of checking this is to check the PRU IEP compare registers (TRM 6.4.14.9 PRU_IEP_IEP Registers) during the following cases:

1. Before 1PPS enable and after PRU ICSSG interfaces up

2. After 1PPS enable and before PRU ICSSG interfaces down (with the expected 1Hz 1PPS)

3. After PRU ICSSG interfaces down (without disabled 1PPS before bringing interfaces down) - during the ~10Hz 1PPS

4. After re-enabling 1PPS signal. (See the expected 1Hz 1PPS again)

We are looking to see if the IEP compare registers are similar/same between case 1 and 3. And if the register contents are similar between case 2 and 4.

Additional questions:

1. Does re-enabling PPS signal after interfaces down and back up work as a temporary workaround to unblock you from development? The reason I ask this is because it may take some time for our developers to dig into this specific issue.

2. Are you using the latest prueth (PRU ICSSG ethernet) firmwares? There have been significant changes in the firmware between SDK 8 and the current SDK 9.02.01. Although changing to latest firmware may not resolve this issue because I'm using the latest firmwares and I also see the same issue, I think in general it's best to use the latest to decrease any chances of other issues that might have been fixed with the latest firmwares.

>>>I don’t know which version of SDK exactly we are using (how to check it?) but it is meta-ti kirkstone branch, commit: 8ad82f5e69c3ed739bbdd5d1a0afd75ab7608ec8
From May 1 2023

Good question, I currently don't know either but I will check and get back to you on this.

-Daolin

>>>I don’t know which version of SDK exactly we are using (how to check it?) but it is meta-ti kirkstone branch

I currently don't have this branch cloned but it appears the way to check the tag is "git tag --contain <commit-sha>"

-Daolin

Hi Daolin,

Sorry for late response, I have some other issue with hardware now (unrelated to pps) that makes unable to me to test the workaround. I will respond as soon as I will deal with those problems.

Daolin Qiu said:

I currently don't have this branch cloned but it appears the way to check the tag is "git tag --contain <commit-sha>"

Looks like someone from team mates updated SDK version since I lastly checked. Now we are using SDK 9.0.

Hello Jakub,

We are working on fixing this issue for the next SDK release. I want to check with you regarding your use case for the PPS signal.

Is the expectation that if a link if brought down and brought back up, the PPS signal should resume expected signal without having to start the signal again with "echo 1 > /sys/class/ptp/ptp0/pps_enable" or "/usr/kernel-selftest/ptp/testptp -d /dev/ptp2 -P 1"? Or would manually restarting the signal suffice?

-Daolin