Other Parts Discussed in Thread: AM5728
Hello,
we use a am5726 with two ethernet interfaces and we want to priorize traffic for a real time communication protocol (Profinet) over other traffic. We implemented this as decribed in Documentation/networking/ti-cpsw.txt in the Linux source code with the second example. (I copied this to the end) We put all the commands for configuration into a script which is executed once in initd imediately after Linux has booted.
If we have a physical connection from each port to a switch at boot time, the configuration seemed to work.
But if we don't have plugged eth0 at boot time, several steps of the configuration don't work. Exspecially when having the ehternet on eth0 unplugged but eth1 connected and writing to
/sys/class/net/eth0/queues/tx-*/tx_maxrate
eth1 does not come up properly. If I plug in eth0 minutes later eth1 also comes up.
Also when running the cbs configuration in step 8 without having eth0 plugged there is a warning that offloading to hardware is not possible.
So my questions are:
- What are the prerequesites to execute each configuration step? For step 4 and step 8 it seems to me, that there must be a physical connection already.
- How/where should I integrate the configuration in Linux? Do I need to wait for an event on plugging or unplugging the the physical ehternet interface
and run a reconfiguration. Is there and example in the linux SDK for the am5728 evm or Beagleboard X15.
Then I have another question on ingress. In the file /proc/net/vlan/eth1.100 there is also an ingress mapping reported. Would it also be possible to let the hardware sort the
incomming packet into different queues so that the packets for real time communication (Profinet) will be handled with iptables, tc prior to other packets and be available at the socket for receive
with minimul latency?
*********************************************************************
*********************************************************************
*********************************************************************
Example 2: Two port tx AVB configuration scheme for target board
----------------------------------------------------------------------
(prints and scheme for AM572x evm, for dual emac boards only)
+------------------------------------------------------------------+ u
| +----------+ +----------+ +------+ +----------+ +----------+ | s
| | | | | | | | | | | | e
| | App 1 | | App 2 | | Apps | | App 3 | | App 4 | | r
| | Class A | | Class B | | Rest | | Class B | | Class A | |
| | Eth0 | | Eth0 | | | | | Eth1 | | Eth1 | | s
| | VLAN100 | | VLAN100 | | | | | VLAN100 | | VLAN100 | | p
| | 40 Mb/s | | 20 Mb/s | | | | | 10 Mb/s | | 30 Mb/s | | a
| | SO_PRI=3 | | SO_PRI=2 | | | | | SO_PRI=3 | | SO_PRI=2 | | c
| | | | | | | | | | | | | | | | | e
| +---|------+ +---|------+ +---|--+ +---|------+ +---|------+ |
+-----|-------------|-------------|---------|-------------|--------+
+-+ +-------+ | +----------+ +----+
| | +-------+------+ | |
| | | | | |
+---|-------|-------------|--------------|-------------|-------|---+
| +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ |
| | p3 | | p2 | | p1 | | p0 | | p0 | | p1 | | p2 | | p3 | | k
| \ / \ / \ / \ / \ / \ / \ / \ / | e
| \ / \ / \ / \ / \ / \ / \ / \ / | r
| \/ \/ \/ \/ \/ \/ \/ \/ | n
| | | | | | | | e
| | | +----+ +----+ | | | l
| | | | | | | |
| +----+ +----+ +----+ +----+ +----+ +----+ | s
| |tc0 | |tc1 | |tc2 | |tc2 | |tc1 | |tc0 | | p
| \ / \ / \ / \ / \ / \ / | a
| \ / \ / \ / \ / \ / \ / | c
| \/ \/ \/ \/ \/ \/ | e
| | | +-----+ +-----+ | | |
| | | | | | | | | |
| | | | | | | | | |
| | | | | E E | | | | |
| +----+ +----+ +----+ +----+ t t +----+ +----+ +----+ +----+ |
| |txq0| |txq1| |txq4| |txq5| h h |txq6| |txq7| |txq3| |txq2| |
| \ / \ / \ / \ / 0 1 \ / \ / \ / \ / |
| \ / \ / \ / \ / . . \ / \ / \ / \ / |
| \/ \/ \/ \/ 1 1 \/ \/ \/ \/ |
| +-|------|------|------|--+ 0 0 +-|------|------|------|--+ |
| | | | | | | 0 0 | | | | | | |
+---|------|------|------|---------------|------|------|------|----+
| | | | | | | |
p p p p p p p p
3 2 0-1, 4-7 <-L2 pri-> 0-1, 4-7 2 3
| | | | | | | |
| | | | | | | |
+---|------|------|------|---------------|------|------|------|----+
| | | | | | | | | |
| +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ |
| |dma7| |dma6| |dma3| |dma2| |dma1| |dma0| |dma4| |dma5| |
| \ / \ / \ / \ / \ / \ / \ / \ / | c
| \S / \S / \ / \ / \ / \ / \S / \S / | p
| \/ \/ \/ \/ \/ \/ \/ \/ | s
| | | | +----- | | | | | w
| | | | | +----+ | | | |
| | | | | | | | | | d
| +----+ +----+ +----+p p+----+ +----+ +----+ | r
| | | | | | |o o| | | | | | | i
| | f3 | | f2 | | f0 |r CPSW r| f3 | | f2 | | f0 | | v
| |tc0 | |tc1 | |tc2 |t t|tc0 | |tc1 | |tc2 | | e
| \CBS / \CBS / \CBS /1 2\CBS / \CBS / \CBS / | r
| \S / \S / \ / \S / \S / \ / |
| \/ \/ \/ \/ \/ \/ |
+------------------------------------------------------------------+
========================================Eth==========================>
1)
// Add 8 tx queues, for interface Eth0, but they are common, so are accessed
// by two interfaces Eth0 and Eth1.
$ ethtool -L eth1 rx 1 tx 8
rx unmodified, ignoring
2)
// Check if num of queues is set correctly:
$ ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX: 8
TX: 8
Other: 0
Combined: 0
Current hardware settings:
RX: 1
TX: 8
Other: 0
Combined: 0
3)
// TX queues must be rated starting from 0, so set bws for tx0 and tx1 for Eth0
// and for tx2 and tx3 for Eth1. That is, rates 40 and 20 Mb/s appropriately
// for Eth0 and 30 and 10 Mb/s for Eth1.
// Real speed can differ a bit due to discreetness
// Leave last 4 tx queues as not rated
$ echo 40 > /sys/class/net/eth0/queues/tx-0/tx_maxrate
$ echo 20 > /sys/class/net/eth0/queues/tx-1/tx_maxrate
$ echo 30 > /sys/class/net/eth1/queues/tx-2/tx_maxrate
$ echo 10 > /sys/class/net/eth1/queues/tx-3/tx_maxrate
4)
// Check maximum rate of tx (cpdma) queues:
$ cat /sys/class/net/eth0/queues/tx-*/tx_maxrate
40
20
30
10
0
0
0
0
5)
// Map skb->priority to traffic class for Eth0:
// 3pri -> tc0, 2pri -> tc1, (0,1,4-7)pri -> tc2
// Map traffic class to transmit queue:
// tc0 -> txq0, tc1 -> txq1, tc2 -> (txq4, txq5)
$ tc qdisc replace dev eth0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 queues 1@0 1@1 2@4 hw 1
6)
// Check classes settings
$ tc -g class show dev eth0
+---(100:ffe2) mqprio
| +---(100:5) mqprio
| +---(100:6) mqprio
|
+---(100:ffe1) mqprio
| +---(100:2) mqprio
|
+---(100:ffe0) mqprio
+---(100:1) mqprio
7)
// Set rate for class A - 41 Mbit (tc0, txq0) using CBS Qdisc for Eth0
// here only idle slope is important, others ignored
// Real speed can differ a bit due to discreetness
$ tc qdisc add dev eth0 parent 100:1 cbs locredit -1470 \
hicredit 62 sendslope -959000 idleslope 41000 offload 1
net eth0: set FIFO3 bw = 50
8)
// Set rate for class B - 21 Mbit (tc1, txq1) using CBS Qdisc for Eth0
$ tc qdisc add dev eth0 parent 100:2 cbs locredit -1470 \
hicredit 65 sendslope -979000 idleslope 21000 offload 1
net eth0: set FIFO2 bw = 30
9)
// Create vlan 100 to map sk->priority to vlan qos for Eth0
$ ip link add link eth0 name eth0.100 type vlan id 100
net eth0: Adding vlanid 100 to vlan filter
10)
// Map skb->priority to L2 prio for Eth0.100, one to one
$ ip link set eth0.100 type vlan \
egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
11)
// Check egress map for vlan 100
$ cat /proc/net/vlan/eth0.100
[...]
INGRESS priority mappings: 0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0
EGRESS priority mappings: 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
12)
// Map skb->priority to traffic class for Eth1:
// 3pri -> tc0, 2pri -> tc1, (0,1,4-7)pri -> tc2
// Map traffic class to transmit queue:
// tc0 -> txq2, tc1 -> txq3, tc2 -> (txq6, txq7)
$ tc qdisc replace dev eth1 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 queues 1@2 1@3 2@6 hw 1
13)
// Check classes settings
$ tc -g class show dev eth1
+---(100:ffe2) mqprio
| +---(100:7) mqprio
| +---(100:8) mqprio
|
+---(100:ffe1) mqprio
| +---(100:4) mqprio
|
+---(100:ffe0) mqprio
+---(100:3) mqprio
14)
// Set rate for class A - 31 Mbit (tc0, txq2) using CBS Qdisc for Eth1
// here only idle slope is important, others ignored, but calculated
// for interface speed - 100Mb for eth1 port.
// Set it +1 Mb for reserve (important!)
$ tc qdisc add dev eth1 parent 100:3 cbs locredit -1035 \
hicredit 465 sendslope -69000 idleslope 31000 offload 1
net eth1: set FIFO3 bw = 31
15)
// Set rate for class B - 11 Mbit (tc1, txq3) using CBS Qdisc for Eth1
// Set it +1 Mb for reserve (important!)
$ tc qdisc add dev eth1 parent 100:4 cbs locredit -1335 \
hicredit 405 sendslope -89000 idleslope 11000 offload 1
net eth1: set FIFO2 bw = 11
16)
// Create vlan 100 to map sk->priority to vlan qos for Eth1
$ ip link add link eth1 name eth1.100 type vlan id 100
net eth1: Adding vlanid 100 to vlan filter
17)
// Map skb->priority to L2 prio for Eth1.100, one to one
$ ip link set eth1.100 type vlan \
egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
18)
// Check egress map for vlan 100
$ cat /proc/net/vlan/eth1.100
[...]
INGRESS priority mappings: 0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0
EGRESS priority mappings: 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
19)
// Run appropriate tools with socket option "SO_PRIORITY" to 3
// for class A and to 2 for class B. For both interfaces
./tsn_talker -d 18:03:73:66:87:42 -i eth0.100 -p2 -s 1500&
./tsn_talker -d 18:03:73:66:87:42 -i eth0.100 -p3 -s 1500&
./tsn_talker -d 20:cf:30:85:7d:fd -i eth1.100 -p2 -s 1500&
./tsn_talker -d 20:cf:30:85:7d:fd -i eth1.100 -p3 -s 1500&
20)
// run your listener on workstation (should be in same vlan)
// (I took at www.spinics.net/.../msg460869.html)
./tsn_listener -d 18:03:73:66:87:42 -i enp5s0 -s 1500
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39012 kbps
Receiving data rate: 39000 kbps
21)
// Restore default configuration if needed
$ ip link del eth1.100
$ ip link del eth0.100
$ tc qdisc del dev eth1 root
net eth1: Prev FIFO2 is shaped
net eth1: set FIFO3 bw = 0
net eth1: set FIFO2 bw = 0
$ tc qdisc del dev eth0 root
net eth0: Prev FIFO2 is shaped
net eth0: set FIFO3 bw = 0
net eth0: set FIFO2 bw = 0
$ ethtool -L eth0 rx 1 tx 1
