HI all:

We also refer to the below document

processors.wiki.ti.com/.../Linux_Core_CPSW_User's_Guide

My custom board is dual standalone emac mode and their CPDMA and skb buffers are common for both eth interfaces

So we doubt whether it will have effect

Thanks 

regards

HI Schuyler:

Thanks for your reply;

1 Our application try to do stream forwarding, So it need high network bandwidth.

2 Are the two ports bridged? 

   I think the two ports is not bridged, because my custom board is dual standalone emac mode。

3  Are the two ports on independent subnets?

Yes there are in different subnets and eth0 is 192.168.0.200,  eth1 is 192.168.1.200

4 Are you using iperf or are using a custom application to send recieve traffic? 

We use my custom application to test the eth0 and use iperf to test the eth1.

5 For the linux kernel could please post the results of uname -a?

My software is Processor SDK 4.2 release，the uname -a is :

Linux am57xx-evm 4.9.59-ga75d8e9305 #5 SMP PREEMPT Fri Jan 26 11:07:32 CST 2018 armv7l GNU/Linux

6 I would like which kernel you are using. Are you using a custom kernel configuration?

we are using the config file arch/arm/configs/tisdk_am57xx-evm_defconfig and the parameter has not been modified

7 Is this issue occurring on a custom board or a TI EVM? 

The issue occur on my custom board and we don't know whether the TI EVM will occur.

My hardware is a custom board, the am5728 chip version is ES2.0。

8  The last few days， we have make some test.

We referd to the below document

processors.wiki.ti.com/.../Linux_Core_CPSW_User's_Guide

We changed the ti_cpsw.descs_pool_size=4096 and we get some new test result:

We use eth0 to do stream forwarding and use eth1(tcp methord) to iperf testing

generally when the eth0 get to 400Mb/s, the eth1’s bindwidth will decrease to 320Mb/s.

when the eth0 get to 300Mb/s, the eth1’s bindwidth will increase to 420Mb/s

That is to say the total bindwidth for eth0 plus eth1 is 720Mb/s, if one’s bindwidth increased then anther one will decrease the bindwidth.

So my question is how to increase the total bindwidth to 900Mb/s?

Thanks

regards

HI Schuyler

Thanks for your reply;

1 How are forwarding the traffic between the two ports?

We use my custom application to test the eth0 and use iperf to test the eth1.

My custom application is used to do stream forwarding.

My custom application receive web camera stream(8Mb/s) and then copy it  to lots of stream, finally send these stream to PC.

We use iperf to test eth1, my iperf is a TCP client which will send stream to the iperf server.

Generally when the eth0's send bindwidth get to 400Mb/s, the eth1’s bindwidth will decrease to 320Mb/s.

when the eth0'send bindwidth get to 300Mb/s, the eth1’s bindwidth will increase to 420Mb/s

That is to say the total send bindwidth for eth0 plus eth1 is 720Mb/s, if one’s bindwidth increased then anther one will decrease the bindwidth.

2  Are you setting up the route table?

No, We don't set up the route table.

3 Are you running the TI SDK rootfs? 

Yes, We used the TI SDK rootfs and the filesystem has not been modified

4 Will your final application need the systemD type user space? 

We just use the systemd to setup the boot mode

HI Schuyler

Thanks for your reply;

1 Is the custom app running on eth0 always just the streaming app and is it using TCP?

Yes, My custom app use the TCP to do streaming and it is always the stream app on eth0;

We have enabled the interrupt pacing by the ethtool, but there is no use.

2 Do you know to boot to a shell? 

Just now we start the streaming app and iperf  manually after linux bootup.

and later we will use systemd to setup the streaming app.

3 We are so confusing about the CPSW.

My custom board is dual standalone emac mode and their CPDMA and skb buffers are common for both eth interfaces

So we doubt whether it will have effect.

Thanks

regards

Hi,

I forgot to attach the patch:

Best Regards,

Schuyler

From bff1236f42c224ee854d4f4bbc2a053c566149f1 Mon Sep 17 00:00:00 2001
From: Grygorii Strashko <grygorii.strashko@ti.com>
Date: Fri, 2 Feb 2018 10:23:58 -0600
Subject: [PATCH] [debug] net watchdog timeout fix

Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 drivers/net/ethernet/ti/cpsw.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 39c39d6..f6c64a6 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -1683,6 +1683,7 @@ static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
 		q_idx = q_idx % cpsw->tx_ch_num;
 
 	txch = cpsw->txv[q_idx].ch;
+	txq = netdev_get_tx_queue(ndev, q_idx);
 	ret = cpsw_tx_packet_submit(priv, skb, txch);
 	if (unlikely(ret != 0)) {
 		cpsw_err(priv, tx_err, "desc submit failed\n");
@@ -1693,15 +1694,26 @@ static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
 	 * tell the kernel to stop sending us tx frames.
 	 */
 	if (unlikely(!cpdma_check_free_tx_desc(txch))) {
-		txq = netdev_get_tx_queue(ndev, q_idx);
 		netif_tx_stop_queue(txq);
+
+		/* Barrier, so that stop_queue visible to other cpus */
+		smp_mb();
+
+		if (cpdma_check_free_tx_desc(txch))
+			netif_tx_wake_queue(txq);
 	}
 
 	return NETDEV_TX_OK;
 fail:
 	ndev->stats.tx_dropped++;
-	txq = netdev_get_tx_queue(ndev, skb_get_queue_mapping(skb));
 	netif_tx_stop_queue(txq);
+
+	/* Barrier, so that stop_queue visible to other cpus */
+	smp_mb();
+
+	if (cpdma_check_free_tx_desc(txch))
+		netif_tx_wake_queue(txq);
+
 	return NETDEV_TX_BUSY;
 }
 
-- 
2.10.5

HI Schuyler

Tnanks for your reply;

Just now the total bindwidth of the eth0 and eth1 is 720Mb/s and we have test over 100 hours The stability is OK。

we will test the patch and then tell you the resoult.

My custom TCP application receive web camera stream(8Mb/s) and then copy it  to lots of stream (All the stream can be up to 400Mb/s), finally send these stream to (Windows)PC.

We use iperf to test eth1, my iperf is a TCP client which send stream to the iperf server, the iperf server is a AM5728 board.

So we mainly test the sending performance of the eth0 and eth1.

My ethtool -C that set interrupt pacing is like below:

ethtool -C eth0 rx-usecs 500

 The iperf command that we are using is like below:

iperf client:

iperf -B 192.168.0.27 -c 192.168.0.77 -i 10 -t 999999 &

iperf server:

iperf -B 192.168.0.77 -s -i 1 &

About detect the bandwidth of the two ethernet ports? 

As you know the iperf can detect the bindwidth of eth1, because it will print the bindwidth real-time

My custom sending stream to the windows PC and The windows PC can print the receive bindwidth real-time

Thanks

regards

xixi

Dear Schuyler

Very thanks!

1 Please try a rx-usecs value of 150 or 100 for interrupt pacing

We have tried this parameters and the result is the same as before.

 2 About my custom application

My custom use the eth0 and receive a 8Mb/s web camera stream and then coping the stream. lastly sending to Windows PC that are connected to the eth0 subnet.

Just now only one windows PC to receive these stream.

In the feature these stream will send to several  devices which is unicast addresses.

3 Right now the traffic on the two ports is de-coupled. Do you plan to synchronize them in the future?

No, we will not synchronize them in the future.

4 The below picture is my system load:

Dear Schuyler

Do you have any other advice for me about this issue.

Thanks

regards

HI Schuyler

I'm very sorry to reply you so late because of the Chinese Spring Festival holidays.

we are interested in 900Mbps total send bandwith.

We used one iperf to test the eth0 performance.

The result is like below:

My server command is :

iperf -B 192.168.0.77 -s -i 1

We also used two iperf process to test eth0 and eth1.

The result is like below:

My server command is :

iperf -B 192.168.0.77 -s -i 1 &

iperf -B 192.168.1.77 -s -i 1 &

According to the above test, eth0 is about 400Mb/s, eth1 is about 400Mb/s. the total bindwidth of eth0 and eth1 is 800Mb/s

So how can we improve the bindwidth to 900Mb/s

Thanks

regards 

HI Schuyler：

Can you give us an other advice about  this issue.

Thanks

Regards

Dear Schuyler

Thanks for your reply

According to the below website, the ethernet eth0 TCP performance can get to 1108.8 Mb/s, But why my board can not get to this bitrate