RTOS/AM3358: Ethernet storm attack issue

Hi Schuyler,

Thanks. We are using TI SDK Version 03.01.00. Actually, 4 of our products are using the same SDK version using the chip AM3358 and we have a long term support for the same. Changing to the newer SDK involves a very huge process and effort from our side. Is it possible to provide a patch for this cpsw driver change to resolve the storm attack in this SDK version 03.01.00 ?

Thank you.

With Regards,

N.Raghu Raman.

Hi Schuyler,

I have integrated the patch and tested with the same. I see that with the patch and with coalescing and multicast and broadcast rate limiting,  the patch does not work in 100mbps mode. 

Note that we are using dual emac mode and not switch mode and I adapted the patch above for dual emac mode of operation.

Please find the register dumps of CPSW_ALE REGISTERS below and I have coalesced using a value of 500 and multicast and broadcast rate limits are set to 255:

/Flash/AppData/A/App# ./devmem2 0x4A100D00
/dev/mem opened.
Memory mapped at address 0xb6f9e000.
Value at address 0x4A100D00 (0xb6f9ed00): 0x290104
/Flash/AppData/A/App# ./devmem2 0x4A100D08
/dev/mem opened.
Memory mapped at address 0xb6fe8000.
Value at address 0x4A100D08 (0xb6fe8d08): 0x80000005
/Flash/AppData/A/App# ./devmem2 0x4A100D10
/dev/mem opened.
Memory mapped at address 0xb6f77000.
Value at address 0x4A100D10 (0xb6f77d10): 0x2FDEE
/Flash/AppData/A/App# ./devmem2 0x4A100D18
/dev/mem opened.
Memory mapped at address 0xb6f57000.
Value at address 0x4A100D18 (0xb6f57d18): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D20
/dev/mem opened.
Memory mapped at address 0xb6feb000.
Value at address 0x4A100D20 (0xb6febd20): 0x4
/Flash/AppData/A/App# ./devmem2 0x4A100D34
/dev/mem opened.
Memory mapped at address 0xb6f9e000.
Value at address 0x4A100D34 (0xb6f9ed34): 0x14
/Flash/AppData/A/App# ./devmem2 0x4A100D38
/dev/mem opened.
Memory mapped at address 0xb6f91000.
Value at address 0x4A100D38 (0xb6f91d38): 0x30010100
/Flash/AppData/A/App# ./devmem2 0x4A100D3c
/dev/mem opened.
Memory mapped at address 0xb6ff7000.
Value at address 0x4A100D3C (0xb6ff7d3c): 0x5E000001
/Flash/AppData/A/App# ./devmem2 0x4A100D40
/dev/mem opened.
Memory mapped at address 0xb6f0c000.
Value at address 0x4A100D40 (0xb6f0cd40): 0x3
/Flash/AppData/A/App# ./devmem2 0x4A100D44
/dev/mem opened.
Memory mapped at address 0xb6f19000.
Value at address 0x4A100D44 (0xb6f19d44): 0x1010000
/Flash/AppData/A/App# ./devmem2 0x4A100D48
/dev/mem opened.
Memory mapped at address 0xb6f66000.
Value at address 0x4A100D48 (0xb6f66d48): 0x1010003
/Flash/AppData/A/App# ./devmem2 0x4A100D4c
/dev/mem opened.
Memory mapped at address 0xb6fbf000.
Value at address 0x4A100D4C (0xb6fbfd4c): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D50
/dev/mem opened.
Memory mapped at address 0xb6f58000.
Value at address 0x4A100D50 (0xb6f58d50): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D54
/dev/mem opened.
Memory mapped at address 0xb6f18000.
Value at address 0x4A100D54 (0xb6f18d54): 0x0

/Flash/AppData/A/App# ethtool -c eth1
Coalesce parameters for eth1:
Adaptive RX: off TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 500
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 0
tx-frames: 0
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
rx-max-mcast: 255
rx-max-bcast: 255

/Flash/AppData/A/App#

Please confirm via the register settings whether the patch is applied correctly and help in rate limiting of storms like ethernet, arp, ip, tcp and udp - unicast multicast and broadcast storms for 100 Mbps mode. However the patch seem to work in the 10Mbps mode of operation.

Thank you.

With Regards,
N.Raghu Raman.

Hi Schuyler,

Please find attached the patch that I have used. I have not used the switch-config tool. I had adapted to use the ethtool to test. I will follow your previous steps to use switch config tool and provide you the results. 

Thanks.

With Regards,

N.Raghu Raman.

diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 760ccc7..37b367c 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -1008,6 +1008,7 @@ static int cpsw_set_coalesce_mcast(struct net_device *ndev,
 {
 	struct cpsw_priv *priv = netdev_priv(ndev);
 	int port;
+	int ret = -EINVAL;
 
 	priv->rx_max_mcast = coal->rx_max_mcast;
 
@@ -1016,9 +1017,25 @@ static int cpsw_set_coalesce_mcast(struct net_device *ndev,
 	else
 		port = priv->data.active_slave;
 
-	cpsw_ale_control_set(priv->ale, port, ALE_PORT_MCAST_LIMIT,
-			     coal->rx_max_mcast);
-
+        ret = cpsw_ale_set_ratelimit(priv->ale, 
+                                     priv->bus_freq_mhz * 1000000,
+                                     port,
+                                     coal->rx_max_bcast,
+                                     coal->rx_max_mcast,
+                                     0);
+        if (ret)
+            dev_err(priv->dev, "CPSW_ALE set ratelimit failed");
+
+        ret = cpsw_ale_set_ratelimit(priv->ale, 
+                                     priv->bus_freq_mhz * 1000000,
+                                     port,
+                                     coal->rx_max_bcast,
+                                     coal->rx_max_mcast,
+                                     1);
+        if (ret)
+            dev_err(priv->dev, "CPSW_ALE set ratelimit failed");
+
+                                     
 	dev_dbg(priv->dev, "rx_max_mcast set to %d\n", priv->rx_max_mcast);
 	return 0;
 }
@@ -1028,6 +1045,7 @@ static int cpsw_set_coalesce_bcast(struct net_device *ndev,
 {
 	struct cpsw_priv *priv = netdev_priv(ndev);
 	int port;
+	int ret = -EINVAL;
 
 	priv->rx_max_bcast = coal->rx_max_bcast;
 
@@ -1036,8 +1054,23 @@ static int cpsw_set_coalesce_bcast(struct net_device *ndev,
 	else
 		port = priv->data.active_slave + 1;
 
-	cpsw_ale_control_set(priv->ale, port, ALE_PORT_BCAST_LIMIT,
-			     coal->rx_max_bcast);
+        ret = cpsw_ale_set_ratelimit(priv->ale, 
+                                     priv->bus_freq_mhz * 1000000,
+                                     port,
+                                     coal->rx_max_bcast,
+                                     coal->rx_max_mcast,
+                                     0);
+        if (ret)
+            dev_err(priv->dev, "CPSW_ALE set ratelimit failed");
+
+        ret = cpsw_ale_set_ratelimit(priv->ale, 
+                                     priv->bus_freq_mhz * 1000000,
+                                     port,
+                                     coal->rx_max_bcast,
+                                     coal->rx_max_mcast,
+                                     1);
+        if (ret)
+            dev_err(priv->dev, "CPSW_ALE set ratelimit failed");
 
 	dev_dbg(priv->dev, "rx_max_mcast set to %d\n", priv->rx_max_bcast);
 	return 0;
@@ -1911,26 +1944,15 @@ static int cpsw_switch_config_ioctl(struct net_device *ndev,
 			break;
 		}
 
-		ret = cpsw_ale_control_set(priv->ale, 0, ALE_RATE_LIMIT_TX,
-					   !!config.direction);
-		if (ret) {
-			dev_err(priv->dev, "CPSW_ALE control set failed");
-			break;
-		}
-
-		ret = cpsw_ale_control_set(priv->ale, config.port,
-					   ALE_PORT_BCAST_LIMIT,
-					   config.bcast_rate_limit);
-		if (ret) {
-			dev_err(priv->dev, "CPSW_ALE control set failed");
-			break;
-		}
+                ret = cpsw_ale_set_ratelimit(priv->ale,
+                                             priv->bus_freq_mhz * 1000000,
+                                             config.port,
+                                             config.bcast_rate_limit,
+                                             config.mcast_rate_limit,
+                                             !!config.direction);
 
-		ret = cpsw_ale_control_set(priv->ale, config.port,
-					   ALE_PORT_MCAST_LIMIT,
-					   config.mcast_rate_limit);
 		if (ret)
-			dev_err(priv->dev, "CPSW_ALE control set failed");
+			dev_err(priv->dev, "CPSW_ALE control set ratelimit");
 		break;
 	}
 
diff --git a/drivers/net/ethernet/ti/cpsw_ale.c b/drivers/net/ethernet/ti/cpsw_ale.c
index 44fb14d..27fe2af 100644
--- a/drivers/net/ethernet/ti/cpsw_ale.c
+++ b/drivers/net/ethernet/ti/cpsw_ale.c
@@ -826,6 +826,52 @@ int cpsw_ale_control_get(struct cpsw_ale *ale, int port, int control)
 }
 EXPORT_SYMBOL_GPL(cpsw_ale_control_get);
 
+int cpsw_ale_set_ratelimit(struct cpsw_ale *ale, unsigned long freq, int port,
+                          unsigned int bcast_rate_limit,
+                          unsigned int mcast_rate_limit,
+                          bool direction)
+
+{
+       unsigned int rate_limit;
+       unsigned long ale_prescale;
+
+       if (!bcast_rate_limit && !mcast_rate_limit) {
+               /* disable rate limit */
+               cpsw_ale_control_set(ale, 0, ALE_RATE_LIMIT, 0);
+               cpsw_ale_control_set(ale, port, ALE_PORT_BCAST_LIMIT, 0);
+               cpsw_ale_control_set(ale, port, ALE_PORT_MCAST_LIMIT, 0);
+               writel(0, ale->params.ale_regs + ALE_PRESCALE);
+               return 0;
+       }
+
+       /* configure Broadcast and Multicast Rate Limit
+        * number_of_packets = (Fclk / ALE_PRESCALE) * port.BCASTMCAST/_LIMIT
+        * ALE_PRESCALE width is 19bit and min value 0x10
+        * with Fclk = 125MHz and port.BCASTMCAST/_LIMIT = 1
+        *
+        * max number_of_packets = (125MHz / 0x10) * 1 = 7812500
+        * min number_of_packets = (125MHz / 0xFFFFF) * 1 = 119
+        *
+        * above values are more than enough (with higher Fclk they will be
+        * just better), so port.BCASTMCAST/_LIMIT can be selected to be 1
+        * while ALE_PRESCALE is calculated as:
+        *  ALE_PRESCALE = Fclk / number_of_packets
+        */
+       rate_limit = max_t(unsigned int, bcast_rate_limit, mcast_rate_limit);
+       ale_prescale = freq / rate_limit;
+       if (ale_prescale & (~0xfffff))
+               return -EINVAL;
+
+       cpsw_ale_control_set(ale, 0, ALE_RATE_LIMIT_TX, direction);
+       cpsw_ale_control_set(ale, port, ALE_PORT_BCAST_LIMIT, 1);
+       cpsw_ale_control_set(ale, port, ALE_PORT_MCAST_LIMIT, 1);
+       writel((u32)ale_prescale, ale->params.ale_regs + ALE_PRESCALE);
+       cpsw_ale_control_set(ale, 0, ALE_RATE_LIMIT, 1);
+
+       return 0;
+}
+EXPORT_SYMBOL_GPL(cpsw_ale_set_ratelimit);
+
 static int cpsw_ale_dump_mcast(struct cpsw_ale *ale, u32 *ale_entry, char *buf,
 			       int len)
 {
diff --git a/drivers/net/ethernet/ti/cpsw_ale.h b/drivers/net/ethernet/ti/cpsw_ale.h
index b165ba9..e6d08f1 100644
--- a/drivers/net/ethernet/ti/cpsw_ale.h
+++ b/drivers/net/ethernet/ti/cpsw_ale.h
@@ -130,6 +130,10 @@ int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
 			int reg_mcast, int unreg_mcast);
 int cpsw_ale_del_vlan(struct cpsw_ale *ale, u16 vid, int port);
 void cpsw_ale_set_allmulti(struct cpsw_ale *ale, int allmulti);
+int cpsw_ale_set_ratelimit(struct cpsw_ale *ale, unsigned long freq, int port,
+                          unsigned int bcast_rate_limit,
+                          unsigned int mcast_rate_limit,
+                          bool direction);
 
 int cpsw_ale_control_get(struct cpsw_ale *ale, int port, int control);
 int cpsw_ale_control_set(struct cpsw_ale *ale, int port,

Switch config command used is :

./switch-config -l -n 1 -B 1000 -L 1000

/Flash/AppData/A/App# ./devmem2 0x4A100D00
/dev/mem opened.
Memory mapped at address 0xb6f2a000.
Value at address 0x4A100D00 (0xb6f2ad00): 0x290104
/Flash/AppData/A/App# ./devmem2 0x4A100D08
/dev/mem opened.
Memory mapped at address 0xb6f83000.
Value at address 0x4A100D08 (0xb6f83d08): 0x80000004
/Flash/AppData/A/App# ./devmem2 0x4A100D10
/dev/mem opened.
Memory mapped at address 0xb6f20000.
Value at address 0x4A100D10 (0xb6f20d10): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D18
/dev/mem opened.
Memory mapped at address 0xb6f23000.
Value at address 0x4A100D18 (0xb6f23d18): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D20
/dev/mem opened.
Memory mapped at address 0xb6f86000.
Value at address 0x4A100D20 (0xb6f86d20): 0x4
/Flash/AppData/A/App# ./devmem2 0x4A100D34
/dev/mem opened.
Memory mapped at address 0xb6ff2000.
Value at address 0x4A100D34 (0xb6ff2d34): 0x14
/Flash/AppData/A/App# ./devmem2 0x4A100D38
/dev/mem opened.
Memory mapped at address 0xb6f7a000.
Value at address 0x4A100D38 (0xb6f7ad38): 0x30010100
/Flash/AppData/A/App# ./devmem2 0x4A100D3c
/dev/mem opened.
Memory mapped at address 0xb6f30000.
Value at address 0x4A100D3C (0xb6f30d3c): 0x5E000001
/Flash/AppData/A/App# ./devmem2 0x4A100D40
/dev/mem opened.
Memory mapped at address 0xb6fd2000.
Value at address 0x4A100D40 (0xb6fd2d40): 0x3
/Flash/AppData/A/App# ./devmem2 0x4A100D44
/dev/mem opened.
Memory mapped at address 0xb6feb000.
Value at address 0x4A100D44 (0xb6febd44): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D48
/dev/mem opened.
Memory mapped at address 0xb6f44000.
Value at address 0x4A100D48 (0xb6f44d48): 0x3
/Flash/AppData/A/App# ./devmem2 0x4A100D4c
/dev/mem opened.
Memory mapped at address 0xb6f87000.
Value at address 0x4A100D4C (0xb6f87d4c): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D50
/dev/mem opened.
Memory mapped at address 0xb6f76000.
Value at address 0x4A100D50 (0xb6f76d50): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D54
/dev/mem opened.
Memory mapped at address 0xb6f9a000.
Value at address 0x4A100D54 (0xb6f9ad54): 0x0
/Flash/AppData/A/App#

We have a proprietary tool for storm attacks which is based on python which we use to do an IP, MAC and port based storm attacks. We use this tool to generate ethernet, ip, icmp, arp, udp, tcp based storms for different types like unicast multicast and broadcast.

Hi,

With the mentioned changes, I see that the global broadcast storm of UDP, IP and ICMP works i.e., broadcast packets with destination as 255.255.255.255. However, the network broadcast of UDP, IP, ICMP, for example, with destination address 192.168.0.255 if the eth1 ip address is 192.168.0.240/255.255.255.0 does not work. Ethernet broadcast and the multicast rate limiting also does not work. Please note that I am testing broadcast and multicast of UDP, IP, ICMP types and ethernet broadcast packets for the storm attacks.

Please find below the devmem2 output:

/Flash/AppData/A/App# ./devmem2 0x4A100D00
/dev/mem opened.
Memory mapped at address 0xb6f56000.
Value at address 0x4A100D00 (0xb6f56d00): 0x290104
/Flash/AppData/A/App# ./devmem2 0x4A100D08
/dev/mem opened.
Memory mapped at address 0xb6f68000.
Value at address 0x4A100D08 (0xb6f68d08): 0x80000005
/Flash/AppData/A/App# ./devmem2 0x4A100D10
/dev/mem opened.
Memory mapped at address 0xb6fe9000.
Value at address 0x4A100D10 (0xb6fe9d10): 0x3D090
/Flash/AppData/A/App# ./devmem2 0x4A100D18
/dev/mem opened.
Memory mapped at address 0xb6f9f000.
Value at address 0x4A100D18 (0xb6f9fd18): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D20
/dev/mem opened.
Memory mapped at address 0xb6f2b000.
Value at address 0x4A100D20 (0xb6f2bd20): 0x3FF
/Flash/AppData/A/App# ./devmem2 0x4A100D34
/dev/mem opened.
Memory mapped at address 0xb6fe6000.
Value at address 0x4A100D34 (0xb6fe6d34): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D38
/dev/mem opened.
Memory mapped at address 0xb6f27000.
Value at address 0x4A100D38 (0xb6f27d38): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D3c
/dev/mem opened.
Memory mapped at address 0xb6fda000.
Value at address 0x4A100D3C (0xb6fdad3c): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D40
/dev/mem opened.
Memory mapped at address 0xb6f1d000.
Value at address 0x4A100D40 (0xb6f1dd40): 0x3
/Flash/AppData/A/App# ./devmem2 0x4A100D44
/dev/mem opened.
Memory mapped at address 0xb6f18000.
Value at address 0x4A100D44 (0xb6f18d44): 0x1010000
/Flash/AppData/A/App# ./devmem2 0x4A100D48
/dev/mem opened.
Memory mapped at address 0xb6fea000.
Value at address 0x4A100D48 (0xb6fead48): 0x3
/Flash/AppData/A/App# ./devmem2 0x4A100D4c
/dev/mem opened.
Memory mapped at address 0xb6f10000.
Value at address 0x4A100D4C (0xb6f10d4c): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D50
/dev/mem opened.
Memory mapped at address 0xb6f26000.
Value at address 0x4A100D50 (0xb6f26d50): 0x0
/Flash/AppData/A/App# ./devmem2 0x4A100D54
/dev/mem opened.
Memory mapped at address 0xb6ff6000.
Value at address 0x4A100D54 (0xb6ff6d54): 0x0
/Flash/AppData/A/App#

Please help in resolving this issue.

Thanks.

With Regards,

N.Raghu Raman.

packet_captures.zip

Hi Grygorii,

Please find attached the UDP broadcast and ICMP multicast packet captures attached.

Also, please share your UDP broadcast and ICMP multicast packet capture where rate limiting is working with which I can also compare.

Thanks.

With Regards,

N.Raghu Raman.

Hi,

Can you please let me know how to enable napi in the cpsw so that I can see whether napi has an improvement in behavior to the storm attack along with these rate limiting options?

Thank you.

With Regards,

N.Raghu Raman.