PROCESSOR-SDK-J784S4: Not able to get dynamic IP when DHCP is enabled

Doredla Sudheer Kumar Kindly please reply asap on the above query.

Thanks in advance

Hi,

Is Ping success with static IP configuration? check CPSW statistics while running ping.
Also, check whether packet sent out from Broadcom PHY to peer side by connecting to a PC and run wire-shark to capture packets.

Are you using Main CPSW2G here with Broadcom PHY using Native Linux Driver?

Best Regards,
Sudheer

Hi, 
Thanks for the immediate response.

Yes, the ping (pinging its own IP) got success with static IP configuration.  Attaching the snap below.

Is below is the command to check cpsw statistics?

ls /sys/class/net/eth1/statistics

Sure, will try to do using wire-shark and update here.

yes, I am using main cpsw2g and using native broadcom.c linux driver.

Hi,

Swedha R said:

Yes, the ping (pinging its own IP) got success with static IP configuration.  Attaching the snap below.

Thanks for the confirmation.

Swedha R said:

Is below is the command to check cpsw statistics?

ls /sys/class/net/eth1/statistics

No. You can get CPSW port statistics using ethtool.

# ethtool -S eth1

Also, can you check by running below by connecting port in DHCP network.

# dhclient eth1

If above is not working, can you check network configuration. Refer to below.
https://askubuntu.com/questions/891694/systemd-networkd-daemon-does-not-start-the-dhcp-client
https://unix.stackexchange.com/questions/319740/use-dhcp-on-eth0-using-command-line

Best Regards,
Sudheer

Thanks for your response.

Doredla Sudheer Kumar said:

# ethtool -S eth1

While running the above command we are not seeing the counters are increasing properly while we do ping test. Even we didn't run any ping test also the counter got increasing. So we are not sure how we can consider that values. Below is the output.
ethtool -S eth1 | grep -E 'tx|rx'
     p0_rx_good_frames: 550
     p0_rx_broadcast_frames: 485
     p0_rx_multicast_frames: 65
     p0_rx_crc_errors: 0
     p0_rx_oversized_frames: 0
     p0_rx_undersized_frames: 0
     p0_rx_octets: 39248
     p0_tx_good_frames: 1
     p0_tx_broadcast_frames: 1
     p0_tx_multicast_frames: 0
     p0_tx_octets: 64
     p0_tx_64B_frames: 486
     p0_tx_65_to_127B_frames: 40
     p0_tx_128_to_255B_frames: 22
     p0_tx_256_to_511B_frames: 3
     p0_tx_512_to_1023B_frames: 0
     p0_tx_1024B_frames: 0
     p0_rx_bottom_fifo_drop: 0
     p0_rx_port_mask_drop: 0
     p0_rx_top_fifo_drop: 0
     p0_tx_mem_protect_err: 0
     p0_tx_pri0: 0
     p0_tx_pri1: 0
     p0_tx_pri2: 0
     p0_tx_pri3: 0
     p0_tx_pri4: 0
     p0_tx_pri5: 0
     p0_tx_pri6: 0
     p0_tx_pri7: 0
     p0_tx_pri0_bcnt: 0
     p0_tx_pri1_bcnt: 0
     p0_tx_pri2_bcnt: 0
     p0_tx_pri3_bcnt: 0
     p0_tx_pri4_bcnt: 0
     p0_tx_pri5_bcnt: 0
     p0_tx_pri6_bcnt: 0
     p0_tx_pri7_bcnt: 0
     p0_tx_pri0_drop: 0
     p0_tx_pri1_drop: 0
     p0_tx_pri2_drop: 0
     p0_tx_pri3_drop: 0
     p0_tx_pri4_drop: 0
     p0_tx_pri5_drop: 0
     p0_tx_pri6_drop: 0
     p0_tx_pri7_drop: 0
     p0_tx_pri0_drop_bcnt: 0
     p0_tx_pri1_drop_bcnt: 0
     p0_tx_pri2_drop_bcnt: 0
     p0_tx_pri3_drop_bcnt: 0
     p0_tx_pri4_drop_bcnt: 0
     p0_tx_pri5_drop_bcnt: 0
     p0_tx_pri6_drop_bcnt: 0
     p0_tx_pri7_drop_bcnt: 0
     rx_good_frames: 8
     rx_broadcast_frames: 1
     rx_multicast_frames: 7
     rx_pause_frames: 0
     rx_crc_errors: 0
     rx_align_code_errors: 0
     rx_oversized_frames: 0
     rx_jabber_frames: 0
     rx_undersized_frames: 0
     rx_fragments: 1
     rx_octets: 3171
     tx_good_frames: 0
     tx_broadcast_frames: 0
     tx_multicast_frames: 0
     tx_pause_frames: 0
     tx_deferred_frames: 0
     tx_collision_frames: 0
     tx_single_coll_frames: 0
     tx_mult_coll_frames: 0
     tx_excessive_collisions: 0
     tx_late_collisions: 0
     rx_ipg_error: 0
     tx_carrier_sense_errors: 550
     tx_octets: 0
     tx_64B_frames: 1
     tx_65_to_127B_frames: 0
     tx_128_to_255B_frames: 0
     tx_256_to_511B_frames: 7
     tx_512_to_1023B_frames: 0
     tx_1024B_frames: 0
     rx_bottom_fifo_drop: 0
     rx_port_mask_drop: 7
     rx_top_fifo_drop: 0
     iet_rx_assembly_err: 0
     iet_rx_assembly_ok: 0
     iet_rx_smd_err: 137
     iet_rx_frag: 0
     iet_tx_hold: 0
     iet_tx_frag: 0
     tx_mem_protect_err: 0
     tx_pri0: 550
     tx_pri1: 0
     tx_pri2: 0
     tx_pri3: 0
     tx_pri4: 0
     tx_pri5: 0
     tx_pri6: 0
     tx_pri7: 0
     tx_pri0_bcnt: 39248
     tx_pri1_bcnt: 0
     tx_pri2_bcnt: 0
     tx_pri3_bcnt: 0
     tx_pri4_bcnt: 0
     tx_pri5_bcnt: 0
     tx_pri6_bcnt: 0
     tx_pri7_bcnt: 0
     tx_pri0_drop: 0
     tx_pri1_drop: 0
     tx_pri2_drop: 0
     tx_pri3_drop: 0
     tx_pri4_drop: 0
     tx_pri5_drop: 0
     tx_pri6_drop: 0
     tx_pri7_drop: 0
     tx_pri0_drop_bcnt: 0
     tx_pri1_drop_bcnt: 0
     tx_pri2_drop_bcnt: 0
     tx_pri3_drop_bcnt: 0
     tx_pri4_drop_bcnt: 0
     tx_pri5_drop_bcnt: 0
     tx_pri6_drop_bcnt: 0
     tx_pri7_drop_bcnt: 0
root@j784s4-evm:/opt/edgeai-gst-apps# ping 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=0.094 ms
64 bytes from 192.168.0.2: seq=1 ttl=64 time=0.043 ms
64 bytes from 192.168.0.2: seq=2 ttl=64 time=0.039 ms
^C
--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.039/0.058/0.094 ms
root@j784s4-evm:/opt/edgeai-gst-apps# ethtool -S eth1 | grep -E 'tx|rx'
     p0_rx_good_frames: 646
     p0_rx_broadcast_frames: 581
     p0_rx_multicast_frames: 65
     p0_rx_crc_errors: 0
     p0_rx_oversized_frames: 0
     p0_rx_undersized_frames: 0
     p0_rx_octets: 45392
     p0_tx_good_frames: 2
     p0_tx_broadcast_frames: 2
     p0_tx_multicast_frames: 0
     p0_tx_octets: 128
     p0_tx_64B_frames: 583
     p0_tx_65_to_127B_frames: 40
     p0_tx_128_to_255B_frames: 22
     p0_tx_256_to_511B_frames: 3
     p0_tx_512_to_1023B_frames: 0
     p0_tx_1024B_frames: 0
     p0_rx_bottom_fifo_drop: 0
     p0_rx_port_mask_drop: 0
     p0_rx_top_fifo_drop: 0
     p0_tx_mem_protect_err: 0
     p0_tx_pri0: 0
     p0_tx_pri1: 0
     p0_tx_pri2: 0
     p0_tx_pri3: 0
     p0_tx_pri4: 0
     p0_tx_pri5: 0
     p0_tx_pri6: 0
     p0_tx_pri7: 0
     p0_tx_pri0_bcnt: 0
     p0_tx_pri1_bcnt: 0
     p0_tx_pri2_bcnt: 0
     p0_tx_pri3_bcnt: 0
     p0_tx_pri4_bcnt: 0
     p0_tx_pri5_bcnt: 0
     p0_tx_pri6_bcnt: 0
     p0_tx_pri7_bcnt: 0
     p0_tx_pri0_drop: 0
     p0_tx_pri1_drop: 0
     p0_tx_pri2_drop: 0
     p0_tx_pri3_drop: 0
     p0_tx_pri4_drop: 0
     p0_tx_pri5_drop: 0
     p0_tx_pri6_drop: 0
     p0_tx_pri7_drop: 0
     p0_tx_pri0_drop_bcnt: 0
     p0_tx_pri1_drop_bcnt: 0
     p0_tx_pri2_drop_bcnt: 0
     p0_tx_pri3_drop_bcnt: 0
     p0_tx_pri4_drop_bcnt: 0
     p0_tx_pri5_drop_bcnt: 0
     p0_tx_pri6_drop_bcnt: 0
     p0_tx_pri7_drop_bcnt: 0
     rx_good_frames: 19
     rx_broadcast_frames: 2
     rx_multicast_frames: 17
     rx_pause_frames: 0
     rx_crc_errors: 0
     rx_align_code_errors: 0
     rx_oversized_frames: 0
     rx_jabber_frames: 0
     rx_undersized_frames: 0
     rx_fragments: 1
     rx_octets: 7735
     tx_good_frames: 0
     tx_broadcast_frames: 0
     tx_multicast_frames: 0
     tx_pause_frames: 0
     tx_deferred_frames: 0
     tx_collision_frames: 0
     tx_single_coll_frames: 0
     tx_mult_coll_frames: 0
     tx_excessive_collisions: 0
     tx_late_collisions: 0
     rx_ipg_error: 0
     tx_carrier_sense_errors: 646
     tx_octets: 0
     tx_64B_frames: 2
     tx_65_to_127B_frames: 0
     tx_128_to_255B_frames: 0
     tx_256_to_511B_frames: 17
     tx_512_to_1023B_frames: 0
     tx_1024B_frames: 0
     rx_bottom_fifo_drop: 0
     rx_port_mask_drop: 17
     rx_top_fifo_drop: 0
     iet_rx_assembly_err: 0
     iet_rx_assembly_ok: 0
     iet_rx_smd_err: 137
     iet_rx_frag: 0
     iet_tx_hold: 0
     iet_tx_frag: 0
     tx_mem_protect_err: 0
     tx_pri0: 646
     tx_pri1: 0
     tx_pri2: 0
     tx_pri3: 0
     tx_pri4: 0
     tx_pri5: 0
     tx_pri6: 0
     tx_pri7: 0
     tx_pri0_bcnt: 45392
     tx_pri1_bcnt: 0
     tx_pri2_bcnt: 0
     tx_pri3_bcnt: 0
     tx_pri4_bcnt: 0
     tx_pri5_bcnt: 0
     tx_pri6_bcnt: 0
     tx_pri7_bcnt: 0
     tx_pri0_drop: 0
     tx_pri1_drop: 0
     tx_pri2_drop: 0
     tx_pri3_drop: 0
     tx_pri4_drop: 0
     tx_pri5_drop: 0
     tx_pri6_drop: 0
     tx_pri7_drop: 0
     tx_pri0_drop_bcnt: 0
     tx_pri1_drop_bcnt: 0
     tx_pri2_drop_bcnt: 0
     tx_pri3_drop_bcnt: 0
     tx_pri4_drop_bcnt: 0
     tx_pri5_drop_bcnt: 0
     tx_pri6_drop_bcnt: 0
     tx_pri7_drop_bcnt: 0

Doredla Sudheer Kumar said:

# dhclient eth1

 The dhclient command was not present in our board. May be any configurations need to enable in the defconfig. But not sure on it. 

Doredla Sudheer Kumar said:

If above is not working, can you check network configuration. Refer to below.
https://askubuntu.com/questions/891694/systemd-networkd-daemon-does-not-start-the-dhcp-client
https://unix.stackexchange.com/questions/319740/use-dhcp-on-eth0-using-command-line

Tried the ways mentioned in the above links but nothing is working.

But could you please clairfy by below query?

I have added the broadcom phy driver to the linux.
Broadcom driver loaded.
Eth1 interface created
Link Detected status 



Apart from the above, is there any software configuration needed in any layer of the ethernet stack that I need to do to resolve this  issue?

Thanks in advance,
Swedha R

Hi@Doredla Sudheer Kumar 
Could you reply on the above query ?

Thanks in advance,

Swedha R

Hi,

From the CPSW Statistics dump I can see packet received on external Port are Broadcast & Multicast.
Multicast packets are dropped due to ALE. It means there no ALE entry matching to Multicast Address of received packet.
Only Broadcast packet is sent to Host Port and Host Port transferred to Application.


Also, I could see errors in port transmission as carrier sense errors. It seems like issue in MAC to PHY signal detection.

It looks like you are pinging local IP. It will be always success as Network stack it self will respond for the request.

root@j784s4-evm:/opt/edgeai-gst-apps# ping 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=0.094 ms
64 bytes from 192.168.0.2: seq=1 ttl=64 time=0.043 ms
64 bytes from 192.168.0.2: seq=2 ttl=64 time=0.039 ms


Can you please confirm RMGII Pin Mux same as TI SDK or not?

Also, RGMII delays are configured properly or not, refer to below FAQ for more details. Also check with your HW team about the same.
e2e.ti.com/.../faq-tda4vm-how-to-configure-rgmii-clock-delay-on-j7-devices

Also, can you check whether your schematics are reviewed by TI. If not, please get it reviewed by TI H/W team.

Best Regards,
Sudheer

Doredla Sudheer Kumar said:

Also, check whether packet sent out from Broadcom PHY to peer side by connecting to a PC and run wire-shark to capture packets.

Hi Sudheer, 
I have checked with wireshark also. I have set a static IP to my board as I have mentioned before it was not able to get the IP when the DHCP is enabled. I have ran the wireshark tool in my PC and tried to ping the board. But it was not happend. One thing we noted is, we not even able to ping the gateway of the board from the board itself. The below is the output from the board,

tcpdump -i eth1 arp
[  109.789656] device eth1 entered promiscuous mode
[  109.794321] kauditd_printk_skb: 6 callbacks suppressed
[  109.794325] audit: type=1700 audit(1651169166.180:35): dev=eth1 prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295
[  109.810849] audit: type=1300 audit(1651169166.180:35): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=107 a2=1 a3=ffffd7860bb0 items=0 ppid=1346 pid=1463 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS2 ses=4294967295 comm="tcpdump" exe="/usr/bin/tcpdump" key=(null)
[  109.837743] audit: type=1327 audit(1651169166.180:35): proctitle=74637064756D70002D69006574683100617270
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:06:07.055256 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:08.079247 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:09.103385 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:10.127244 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:11.151246 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:12.175362 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:12.982092 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:12.982125 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:13.199244 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:14.043731 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:14.043743 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:14.223243 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:15.067687 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:15.067697 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:15.247347 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:16.091638 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:16.091645 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:16.271243 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:25.307471 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:25.307484 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:25.487247 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:26.331434 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:26.331442 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:26.511246 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:27.355334 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:27.355342 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:27.535410 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:28.379313 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:28.379319 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:28.559242 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:29.403322 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:29.403354 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:29.583251 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:30.427256 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:30.427272 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:30.607426 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:31.451198 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:31.451207 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:31.631241 ARP, Request who-has _gateway tell j784s4-evm, length 28
^C
39 packets captured
61 packets received by filter
22 packets[  135.671573] device eth1 left promiscuous mode
 dropped by kernel
[  135.681305] audit: type=1700 audit(1651169192.072:36): dev=eth1 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295

And one other thing to be noted is below,
arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.107            ether   34:60:f9:ce:36:ae   C                     eth1
192.168.1.1                      (incomplete)                              eth1 

It was not able to get its MAC address.

Could you please give your thoughts on this Doredla Sudheer Kumar?

Thanks in advacne,
Swedha Raja

Hi Sudheer,

Thanks for the response.

Doredla Sudheer Kumar said:

From the CPSW Statistics dump I can see packet received on external Port are Broadcast & Multicast.
Multicast packets are dropped due to ALE. It means there no ALE entry matching to Multicast Address of received packet.
Only Broadcast packet is sent to Host Port and Host Port transferred to Application.

How we can see ALE entries? Because I have tried to locate it from the directory /sys/kernel/debug/..but there is no related entries or folders. 
And how you are confirming broadcast packets are forwarding fine? And one ther doubt I have is why even we didn't do anything the counter values are getting increased (for eg, ti carrier sense errors) when we tried to just read the statistics.

Doredla Sudheer Kumar said:

It seems like issue in MAC to PHY signal detection.

Is there I need to do any additional software configurations in CPSW driver or any other configuration is needed to resolve this issue?

Doredla Sudheer Kumar said:

t looks like you are pinging local IP. It will be always success as Network stack it self will respond for the request.

Thanks for the clarfication on this.

Doredla Sudheer Kumar said:

Can you please confirm RMGII Pin Mux same as TI SDK or not?

Yes I have cross confirmed. Everything is same as TI SDK.

Doredla Sudheer Kumar said:

Also, RGMII delays are configured properly or not, refer to below FAQ for more details. Also check with your HW team about the same.

I went through the FAQ, as we configured phy-mode as rgmii-rxid in our device tree. But we didn't add eny rx-internal delay node in it. As we believe by enabling this following bit itself RGMII RXD to RXC skew itself will handle the delay (timing adjustment). Is my understanding is correct or wrong. Because I was not able to see any specific register to configure the RX internal delay or TX internal delay.

And In MAC side how I can configure for TX delay. Because in the FAQ, it was mentioned for RTOS SDK and DP83867 PHY. But we are using LINUX SDK. Any other method is there to do this?


Awaiting for your reply Doredla Sudheer Kumar Because we are not sure on our next steps .

Thanks in advance,
Swedha Raja.

Hi,

Swedha R said:

How we can see ALE entries? Because I have tried to locate it from the directory /sys/kernel/debug/..but there is no related entries or folders. 
And how you are confirming broadcast packets are forwarding fine? And one ther doubt I have is why even we didn't do anything the counter values are getting increased (for eg, ti carrier sense errors) when we tried to just read the statistics.

Please refer to below FAQ for dumping ALE entries.
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1204394/faq-tda4vm-how-to-print-the-ale-table-for-cpsw-in-tda4-dra8-devices

By default Network stack will send few packets when interface is Link Up.
From statistics, it was observed that carrier sense errors are seen and no good_tx_frames as well.

Swedha R said:

Doredla Sudheer Kumar said:

It seems like issue in MAC to PHY signal detection.

Is there I need to do any additional software configurations in CPSW driver or any other configuration is needed to resolve this issue?

I don't think any specific configuration required at driver side. It will be based on device tree.
We can configure RGMII Tx Delay on MAC side from device tree, if required.

Swedha R said:

Doredla Sudheer Kumar said:

Also, RGMII delays are configured properly or not, refer to below FAQ for more details. Also check with your HW team about the same.

I went through the FAQ, as we configured phy-mode as rgmii-rxid in our device tree. But we didn't add eny rx-internal delay node in it. As we believe by enabling this following bit itself RGMII RXD to RXC skew itself will handle the delay (timing adjustment). Is my understanding is correct or wrong. Because I was not able to see any specific register to configure the RX internal delay or TX internal delay.

Delay at PHY side is based on PHY Driver implementation, It may accept delay value from device tree configuration.
MAC side it was from device tree, that to we can enable only tx delay.

Swedha R said:

And In MAC side how I can configure for TX delay. Because in the FAQ, it was mentioned for RTOS SDK and DP83867 PHY. But we are using LINUX SDK. Any other method is there to do this?

FAQ talks about RGMII Tx delay enable at MAC side in Linux SDK via device tree phy-mode configuration.

Also, can you please check with your H/W team about carrier sense error in Transmission, and RGMII delay.
Also, check whether your schematics are reviewed by TI H/W expert or not?

Best Regards,
Sudheer

Hi,
Thanks for the reply.

Doredla Sudheer Kumar said:

Please refer to below FAQ for dumping ALE entries.
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1204394/faq-tda4vm-how-to-print-the-ale-table-for-cpsw-in-tda4-dra8-devices

By default Network stack will send few packets when interface is Link Up.
From statistics, it was observed that carrier sense errors are seen and no good_tx_frames as well.

Sure I will look into the FAQ for ALE entries. Thanks for the clarification on statistics.

Doredla Sudheer Kumar said:

Delay at PHY side is based on PHY Driver implementation, It may accept delay value from device tree configuration.
MAC side it was from device tree, that to we can enable only tx delay.

So my understanding is wrong right? (As we believe by enabling this following bit itself RGMII RXD to RXC skew itself will handle the delay (timing adjustment).) So I need to add rx-internal-node in the device tree by confirming the delay of RGMII with the hardware team. Is that correct? Because there is no other mentions for these delays in the datasheet.

Doredla Sudheer Kumar said:

MAC side it was from device tree, that to we can enable only tx delay.

How we can enable only tx delay for PHY?

Doredla Sudheer Kumar said:

FAQ talks about RGMII Tx delay enable at MAC side in Linux SDK via device tree phy-mode configuration.

But the below registers are for DP83867 phy . There is no register kind of thing like the below for our Broadcom phy (BCM54810). 

• Reg addr 0x32 - RGMII Control Register (RGMIICTL)
• Reg addr 0x86 - RGMII Delay Control Register (RGMIIDCTL)

 I have one doubt, the phy-mode = rgmii-rxid itself will take care of MAC side TX delay?

Thanks in advance,
Swedha R

Swedha R said:

Hi Sudheer, 
I have checked with wireshark also. I have set a static IP to my board as I have mentioned before it was not able to get the IP when the DHCP is enabled. I have ran the wireshark tool in my PC and tried to ping the board. But it was not happend. One thing we noted is, we not even able to ping the gateway of the board from the board itself. The below is the output from the board,

Doredla Sudheer Kumar Could you please give your thoughts  on this wireshark testing also?

Hi,

Swedha R said:

Doredla Sudheer Kumar said:

Delay at PHY side is based on PHY Driver implementation, It may accept delay value from device tree configuration.
MAC side it was from device tree, that to we can enable only tx delay.

So my understanding is wrong right? (As we believe by enabling this following bit itself RGMII RXD to RXC skew itself will handle the delay (timing adjustment).) So I need to add rx-internal-node in the device tree by confirming the delay of RGMII with the hardware team. Is that correct? Because there is no other mentions for these delays in the datasheet.

Yes, RGMII delay can be configured either from MAC or PHY or both sides based on support from MAC and PHY.
TI SoCs support only Tx delay from MAC side, Rx can be configured by PHY or PCB Schematic traces.
Check whether PHY side delay support model, and default any delays are enabled or not.

Ideally these delay will matter at higher speed like 1Gpbs. but, based on schematic design we might need at lower speed as well. 

Swedha R said:

Doredla Sudheer Kumar said:

MAC side it was from device tree, that to we can enable only tx delay.

How we can enable only tx delay for PHY?

Tx can be configured from MAC, related to PHY side please check with PHY Vendor.

Swedha R said:

 I have one doubt, the phy-mode = rgmii-rxid itself will take care of MAC side TX delay?

Yes, phy-mode "rgmii-rxid" itself enables MAC side Tx delay.

Doredla Sudheer Kumar said:

Also, can you please check with your H/W team about carrier sense error in Transmission, and RGMII delay.
Also, check whether your schematics are reviewed by TI H/W expert or not?

Please check above points as well.

 Best Regards,
Sudheer

Hi,

Swedha R said:

Swedha R said:

Hi Sudheer, 
I have checked with wireshark also. I have set a static IP to my board as I have mentioned before it was not able to get the IP when the DHCP is enabled. I have ran the wireshark tool in my PC and tried to ping the board. But it was not happend. One thing we noted is, we not even able to ping the gateway of the board from the board itself. The below is the output from the board,

Doredla Sudheer Kumar Could you please give your thoughts  on this wireshark testing also?

You can connect the CPSW External Port to PC side, and run Wireshark on PC and check whether any packets received or not?
If packet are received to Wireshark means Tx from MAC side is working fine.
For Rx send some packets from PC to CPSW and check statistics whether any Rx counters are increasing or not?
If Rx counters are increasing, but packets not reaching to Host Application (run (tcpdump -i <eth1> -XX) then those might be dropped due to ALE. This can be found from CPSW statistics.

Best Regards,
Sudheer

Doredla Sudheer Kumar said:

Yes, phy-mode "rgmii-rxid" itself enables MAC side Tx delay.

Thanks for the immediate response.

Doredla Sudheer Kumar said:

Also, can you please check with your H/W team about carrier sense error in Transmission, and RGMII delay.
Also, check whether your schematics are reviewed by TI H/W expert or not?

Sure.

Doredla Sudheer Kumar said:

You can connect the CPSW External Port to PC side, and run Wireshark on PC and check whether any packets received or not?
If packet are received to Wireshark means Tx from MAC side is working fine.
For Rx send some packets from PC to CPSW and check statistics whether any Rx counters are increasing or not?
If Rx counters are increasing, but packets not reaching to Host Application (run (tcpdump -i <eth1> -XX) then those might be dropped due to ALE. This can be found from CPSW statistics.

Yeah, I have done the above scenario too but didn't checked the RX counter.

Just for your need, I am copy pasting from the past message. In this message, I have clearly explained on how we did and what we got.

I have checked with wireshark also. I have set a static IP to my board as I have mentioned before it was not able to get the IP when the DHCP is enabled. I have ran the wireshark tool in my PC and tried to ping the board. But it was not happend. One thing we noted is, we not even able to ping the gateway of the board from the board itself. The below is the output from the board,

tcpdump -i eth1 arp
[  109.789656] device eth1 entered promiscuous mode
[  109.794321] kauditd_printk_skb: 6 callbacks suppressed
[  109.794325] audit: type=1700 audit(1651169166.180:35): dev=eth1 prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295
[  109.810849] audit: type=1300 audit(1651169166.180:35): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=107 a2=1 a3=ffffd7860bb0 items=0 ppid=1346 pid=1463 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS2 ses=4294967295 comm="tcpdump" exe="/usr/bin/tcpdump" key=(null)
[  109.837743] audit: type=1327 audit(1651169166.180:35): proctitle=74637064756D70002D69006574683100617270
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:06:07.055256 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:08.079247 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:09.103385 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:10.127244 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:11.151246 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:12.175362 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:12.982092 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:12.982125 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:13.199244 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:14.043731 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:14.043743 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:14.223243 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:15.067687 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:15.067697 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:15.247347 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:16.091638 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:16.091645 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:16.271243 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:25.307471 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:25.307484 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:25.487247 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:26.331434 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:26.331442 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:26.511246 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:27.355334 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:27.355342 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:27.535410 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:28.379313 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:28.379319 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:28.559242 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:29.403322 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:29.403354 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:29.583251 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:30.427256 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:30.427272 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:30.607426 ARP, Request who-has _gateway tell j784s4-evm, length 28
18:06:31.451198 ARP, Request who-has j784s4-evm tell 192.168.1.107, length 46
18:06:31.451207 ARP, Reply j784s4-evm is-at ea:c7:85:95:36:b2 (oui Unknown), length 28
18:06:31.631241 ARP, Request who-has _gateway tell j784s4-evm, length 28
^C
39 packets captured
61 packets received by filter
22 packets[  135.671573] device eth1 left promiscuous mode
 dropped by kernel
[  135.681305] audit: type=1700 audit(1651169192.072:36): dev=eth1 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295

And one other thing to be noted is below,
arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.107            ether   34:60:f9:ce:36:ae   C                     eth1
192.168.1.1                      (incomplete)                              eth1 

It was not able to get its MAC address.

Hi,

Swedha R said:

One thing we noted is, we not even able to ping the gateway of the board from the board itself. The below is the output from the board,

How you are pining to gate-way.

I have tried locally from Linux machine, configured static IP 192.168.5.5.
I could not able to ping 192.168.5.1 or 192.168.5.255, but I can ping only 192.168.5.5 as it was self IP.

Can you please try the same thing in your test PC and check.

Can you please check from RGMII delay and Schematic point of view with your H/W team why Carrier sense error is observed in Tx path.

Best Regards,
Sudheer

Hi Doredla Sudheer Kumar,
Thanks for the prompt response and sorry for the late response we were stuck in some other work.

Doredla Sudheer Kumar said:

I have tried locally from Linux machine, configured static IP 192.168.5.5.
I could not able to ping 192.168.5.1 or 192.168.5.255, but I can ping only 192.168.5.5 as it was self IP.

Can you please try the same thing in your test PC and check.

Yes, I have tried in my local pc. I have connected my PC to the router and manually I have configured IP for my PC. I have did self test and also I have pinged the gateway. Able to ping it.

 And one more thing, I have mentioned in the initial as we are not able to get the IP from the DHCP server for Broadcom alone right, with the tcpdump -i eth1 command I have checked, the DHCP discover request is going but we are not sure whether the signal is coming out of board and it was transmitting. But one thing we are sure the router is not having any issue. Because it was able to assign IP for my PC and other board having ADIN PHY.

 
And still for the gateway we are not able to resolve MAC address.

arp -n 
Address                HWtype HWaddress         Flags Mask          Iface
192.168.1.1                    (incomplete)                                      eth1

I have tried setting manually by running the following command,
sudo ip neigh add 192.168.1.1 lladdr xx:xx:xx:01:26:65 dev eth1

It was added, but not able to ping the same issue happened again. Communication is not working.
ip neigh
192.168.1.1 dev eth1 lladdr xx:xx:xx:01:26:65 PERMANENT

But one thing I have is, while running the wireshark, the MAC is showing for this gateway. How I have confirmed means, we are using the same router for the boards having ADIN and BROADCOM PHY separately. 

And also we analysed the RGMII timing diagram and trying to probe the signals from the RGMII pins.

Could you please suggest where we are missing? Or any software modifications additionally required to do this? Or any specific driver we need to understand ? 

Kindly request you to help us on this.

Hi,

Swedha R said:

Doredla Sudheer Kumar said:

Also, can you please check with your H/W team about carrier sense error in Transmission, and RGMII delay.
Also, check whether your schematics are reviewed by TI H/W expert or not?

As I have informed above, packet will not be sent out due to carrier sense error.
Can you please check with your H/W expert once.

Best Regards,
Sudheer

Thanks for the response. Will get back to you on this.

I have one doubt, you have mentioned, if we gave phy-mode = rgmii-rxid means, MAC side tx delay will be taken care. But how we know how much delay actually  it is taking and via software how we can read and also from PHY side we need to check with our hardware team whether they enabled in the hardware itself if not we need to add the rx delay from phy side in the device tree, Is that right? Because, in RGMII signal diagram the setup delay and  holdtime delay only they have mentioned. So when we probe the signals, we can check these setup and holdtime delays. Is that itself enough?
 
Regards,
Swedha R

Hi,

Swedha R said:

I have one doubt, you have mentioned, if we gave phy-mode = rgmii-rxid means, MAC side tx delay will be taken care. But how we know how much delay actually  it is taking and via software how we can read

Yes, if we configure rgmii-rxid MAC side tx delay enabled. The delay on MAC side is fixed 2nsec it is not configurable value.
This can be verified from ENET CTRL register from CTRL MMR module (CTRL_MMR_CFG0_CPSW2_ENET1_CTRL)
CPSW2_ENET1_CTRL_RGMII_ID_MODE : 0 means Tx delay enabled, 1 means Tx delay disabled.

Swedha R said:

also from PHY side we need to check with our hardware team whether they enabled in the hardware itself if not we need to add the rx delay from phy side in the device tree, Is that right?

Yes, Once check with PHY Driver if it takes delay from device tree or not and use accordingly.

Swedha R said:

ecause, in RGMII signal diagram the setup delay and  holdtime delay only they have mentioned. So when we probe the signals, we can check these setup and holdtime delays. Is that itself enough?

Yes, You can capture the data and analyze signals.

Best Regards,
Sudheer

Doredla Sudheer Kumar said:

Yes, You can capture the data and analyze signals.

Hi Doredla Sudheer Kumar,
We have captured the signals, in that the tx_ctl line is not going high when I see data being transferred on the tx data lines. What was the issue? How we can debug this?

Doredla Sudheer Kumar said:

Yes, if we configure rgmii-rxid MAC side tx delay enabled. The delay on MAC side is fixed 2nsec it is not configurable value.
This can be verified from ENET CTRL register from CTRL MMR module (CTRL_MMR_CFG0_CPSW2_ENET1_CTRL)
CPSW2_ENET1_CTRL_RGMII_ID_MODE : 0 means Tx delay enabled, 1 means Tx delay disabled.

And one other doubt is, here the MAC side fixed delay is 2nsec. But in our case, in the delayed mode it was mentioned minimum 2.9 as input hold time delay.


Is could be the issue? 

Thanks in advance

Swedha R

Hi,

Can you please check the RGMII TX_CLK ?

Also, can you please confirm your PHY supports in-band signaling? 

Also, can you please re-confirm ping with static IP also not working right?

Best  Regards,
Sudheer

Hi ,

Doredla Sudheer Kumar said:

Can you please check the RGMII TX_CLK ?

The clock period is coming as 800 picoseconds as per the Broadcom Phy datasheet I have shared in the previous query. 

Doredla Sudheer Kumar said:

Also, can you please confirm your PHY supports in-band signaling? 

Is that you mean SGMII, QSGMII as in-band signaling? If yes, our phy won't support in-band signaling. MAC and PHY are communicating via RGMII interfaces only.
I saw the below part of code from cpsw.c driver code mentioning about in-band mode.

Doredla Sudheer Kumar said:

Also, can you please re-confirm ping with static IP also not working right?

If I set static IP, able to ping its own IP. But not able to ping external device in the same subnet.

Thanks in advance,
Swedha R

Hi,

Swedha R said:

Doredla Sudheer Kumar said:

Can you please check the RGMII TX_CLK ?

The clock period is coming as 800 picoseconds as per the Broadcom Phy datasheet I have shared in the previous query. 

For 10Mbps RGMII, the clock should be 2.5MHz.
800 picoseconds might be noise, check the peak-peak to identify whether it is noise or actual signal.

Swedha R said:

Doredla Sudheer Kumar said:

Also, can you please confirm your PHY supports in-band signaling? 

Is that you mean SGMII, QSGMII as in-band signaling? If yes, our phy won't support in-band signaling. MAC and PHY are communicating via RGMII interfaces only.
I saw the below part of code from cpsw.c driver code mentioning about in-band mode.

yes, even RGMII also can have in-band signalling.
If In-band signal is not supported from PHY, then 10Mbps RGMII won't work.

Swedha R said:

Doredla Sudheer Kumar said:

Also, can you please re-confirm ping with static IP also not working right?

If I set static IP, able to ping its own IP. But not able to ping external device in the same subnet.

As I have mentioned above in-band signal is required for 10Mbps RGMII.

Best Regards,
Sudheer

Hi Doredla Sudheer Kumar,
Thanks for the response.

Doredla Sudheer Kumar said:

For 10Mbps RGMII, the clock should be 2.5MHz.
800 picoseconds might be noise, check the peak-peak to identify whether it is noise or actual signal.

Ok will check and get back here.

Doredla Sudheer Kumar said:

yes, even RGMII also can have in-band signalling.
If In-band signal is not supported from PHY, then 10Mbps RGMII won't work.

If it entered into the above else if statement means, shall we confirm in-band signaling is enabled or not? Because I am not seeing about in-band signaling in phy datasheet.

Thanks in advance,
Swedha R

Hi,

Swedha R said:

If it entered into the above else if statement means, shall we confirm in-band signaling is enabled or not? Because I am not seeing about in-band signaling in phy datasheet.

If it enters into else if, in-band is enabled at MAC side.
PHY also should support it, if not link up will not up and communication will not start and MAC can't configure the clock required for link.

Best Regards,
Sudheer

Hi Doredla Sudheer Kumar,

Doredla Sudheer Kumar said:

PHY also should support it, if not link up will not up and communication will not start and MAC can't configure the clock required for link.

But already the link is UP. Communication only not happening.

MAC and PHY are configured with same speed, mode etc.,

Could you please let me know in the ti-sdk which files or function is responsible to receive packets from PHY by CPSW(PHY->CPSW)? I am already going through the cpsw.c,  am65-cpsw-nuss.c files and found CPSW->CPPDMA point. 


Thanks,
Swedha R.

Hi,

Swedha R said:

Doredla Sudheer Kumar said:

PHY also should support it, if not link up will not up and communication will not start and MAC can't configure the clock required for link.

But already the link is UP. Communication only not happening.

Link here is reading from PHY, it is Link with PHY and link partner.

MAC to PHY communication requires proper RGMII clock, MAC will generate TXC, PHY will generate MAC RXC.

Swedha R said:

MAC and PHY are configured with same speed, mode etc.,

It would be as Link up at PHY side.

Swedha R said:

Could you please let me know in the ti-sdk which files or function is responsible to receive packets from PHY by CPSW(PHY->CPSW)? I am already going through the cpsw.c,  am65-cpsw-nuss.c files and found CPSW->CPPDMA point. 

As informed above, you can check CPSW statistics for good rx packet count.
Before driver or application, packet has to be accepted by MAC H/W.

Best Regards,
Sudheer