Debugging networking issues after reboots.

Philipp Schrader

Background: Custom board with DM8147 and DVR RDK 04.00.00.03 using the cpsw driver.

So we're having issues with our network interface not coming up properly after a reboot. Clean power cycles work fine, but after a reboot, the processor has a chance of booting into a state where it won't recognize incoming traffic. In this state, traffic gets sent out just fine, but any replies are either ignored or discarded or something else. I'm trying to figure out what it's doing and why it's ignoring or discarding those.

Has anyone encountered something like this?

In the meantime, I wanted to take a look at the MII registers to see if it sees something wrong. However, it seems that there aren't really any utilities from user space that let you read the MII regs. Similarly, the driver doesn't support the SIOCGMIIREG ioctl.

I've taken a look at this post here: http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/322269.aspx

But it seems that there's no answer yet. Has anyone implemented this functionality?

Thanks,
Philipp

over 10 years ago

0 Hongfeng Wang over 10 years ago

Genius 3925 points

what PHY chip are you using, is its reset signal tied to dm8148 reset output?

0 Philipp Schrader over 10 years ago in reply to Hongfeng Wang

Intellectual 560 points

We're using a LAN8720 PHY. The reset signal is indeed tied to the same dm8148 reset. U-boot has no trouble doing any ethernet communication. It's only when it boots into Linux that it can get into this weird state.

I managed to add SIOCGMIIREG functionality to the cpsw driver so I can dump the MII registers from userspace now. Unfortunately, they all seem OK. There seems to be nothing strange about them.

0 Hongfeng Wang over 10 years ago in reply to Philipp Schrader

Genius 3925 points

There's no difference between POR and warm reset from software's perspective, so it gets to be something related to hardware. I still suspect the reset signal is the culprit, can you check the reset signal of phy chip on a scope, see if it meets the timing requirement of the datasheet?

BTW: how did you perform the reset, a reset button or from linux command? Are you sure the RST of phy is connected to the 8148's reset OUTPUT, pin K6?

0 Philipp Schrader over 9 years ago in reply to Hongfeng Wang

Intellectual 560 points

So the resets are actually not connected. However, the kernel does perform a physical reset on the PHY as it's booting up (we have a GPIO connected to the PHY's reset line). The SMSC driver also performs a soft-reset as part of the initialization.

Both of those things appear to not help.

Our investigation eventually lead us to uboot being the culprit. We are using uboot to perform some initialization. When uboot uses the network, it doesn't shut down the peripheral properly and so something goes wrong between when uboot shuts down the peripheral and the kernel starts using the peripheral again.

I should have mentioned in my original post that this happens only on networks with heavy traffic. I've been testing with 80 Mbit iperf traffic on a 100 Mbit network.

I'm attaching a uboot patch that fixed the problem for us.

5700.reset-cpsw-slave-ports.txt

diff --git a/drivers/net/cpsw.c b/drivers/net/cpsw.c
index d43687d..2e8aa2a 100644
--- a/drivers/net/cpsw.c
+++ b/drivers/net/cpsw.c
@@ -620,6 +620,11 @@ static inline u32 cpsw_get_slave_port(struct cpsw_priv *priv, u32 slave_num)
 		return slave_num;
 }
 
+static void cpsw_slave_reset(struct cpsw_slave *slave, struct cpsw_priv *priv)
+{
+	soft_reset(&slave->sliver->soft_reset);
+}
+
 static void cpsw_slave_init(struct cpsw_slave *slave, struct cpsw_priv *priv)
 {
 	u32	slave_port;
@@ -831,6 +836,9 @@ static void cpsw_halt(struct eth_device *dev)
 	/* clear dma state */
 	soft_reset(priv->dma_regs + CPDMA_SOFTRESET);
 
+	/* reset the slave ports as well */
+	for_each_slave(priv, cpsw_slave_reset, priv);
+
 	priv->data.control(0);
 }

EDIT: Sorry for the long delay in the post, I was on vacation for a few weeks.

0 Philipp Schrader over 9 years ago in reply to Philipp Schrader

Intellectual 560 points

Also, if anyone's interested in reading MII registers using the cpsw driver in some of TI's development kits, I've hacked something together here

6082.siocgmiireg_cpsw.txt

diff --git a/misc/read_mii/read_mii.c b/misc/read_mii/read_mii.c
new file mode 100644
index 0000000..603c8f1
--- /dev/null
+++ b/misc/read_mii/read_mii.c
@@ -0,0 +1,41 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <netinet/in.h>
+#include <string.h>
+#include <linux/mii.h>
+#include <linux/sockios.h>
+
+int main(int argc, char** argv)
+{
+    if (argc < 3) {
+        exit(1);
+    }
+
+    struct ifreq ifr;
+    struct mii_ioctl_data* mii = (struct mii_ioctl_data*)(&ifr.ifr_data);
+
+    memset(&ifr, 0, sizeof(ifr));
+    strcpy(ifr.ifr_name, "eth0"); //set to whatever your ethernet device is
+
+    mii->phy_id = atoi(argv[1]); //set to your phy's ID
+    mii->reg_num = atoi(argv[2]); //the register you want to read
+
+    int fd = socket(AF_INET, SOCK_DGRAM, 0);
+    if (fd < 0) {
+        perror("Couldn't open socket");
+        exit(2);
+    }
+
+    int err = ioctl(fd, SIOCGMIIREG, &ifr);
+    if (err < 0) {
+        perror("Couldn't get MII registers");
+        exit(3);
+    }
+
+    printf("<<< PHY dev %d reg %2d: 0x%04x\n", (int)mii->phy_id,
+            (int)mii->reg_num, (int)mii->val_out);
+
+    return 0;
+}
diff --git a/ti_tools/linux_lsp/kernel/linux-dvr-rdk/drivers/net/cpsw.c b/ti_tools/linux_lsp/kernel/linux-dvr-rdk/drivers/net/cpsw.c
index 8242ac6..2e293eb 100755
--- a/ti_tools/linux_lsp/kernel/linux-dvr-rdk/drivers/net/cpsw.c
+++ b/ti_tools/linux_lsp/kernel/linux-dvr-rdk/drivers/net/cpsw.c
@@ -2438,6 +2438,26 @@ static int cpsw_ndo_do_ioctl(struct net_device *ndev, struct ifreq *ifrq,
 		return cpsw_switch_config_ioctl(ndev, ifrq, cmd);
 #endif /* CONFIG_TI_CPSW_DUAL_EMAC */
 
+	case SIOCGMIIREG: {
+		int ret;
+		struct cpsw_priv *priv = netdev_priv(ndev);
+		struct mii_ioctl_data* mii = (struct mii_ioctl_data*)&ifrq->ifr_data;
+		int slave_no = cpsw_slave_phy_index(priv);
+		struct phy_device *phy = priv->slaves[slave_no].phy;
+		struct mii_bus *bus = phy->bus;
+
+		if (!phy || !bus || !bus->read) {
+			return -EOPNOTSUPP;
+		}
+
+		ret = bus->read(bus, mii->phy_id, mii->reg_num);
+		if (ret < 0)
+			return ret;
+
+		mii->val_out = (u16)ret;
+		break;
+	}
+
 	default:
 		return -EOPNOTSUPP;
 	}

0 Srini over 9 years ago in reply to Philipp Schrader

Genius 5250 points

Am using ti am335x cpsw interface to DP83848 chip

Am trying to read back external phy chip registers, does the above patch works for reading external phy chip registers????

If it is used for reading external phy chip registers?? am unable to read extended chip regsiters of the external phy chip

Could anybody please suggest any inputs w.r.t the above queries??

Kindly do the needful as early as possible

Am trying this from many days

Many Thanks in advance

Processors

Processors forum

Debugging networking issues after reboots.