how to implement direct forwarding on 6678?

anybody can answer this question?

Murat,

There is no test code as per your requirement. But you can find the some test code for packet forwarding through PA using PA LLD at this path: C:\ti\pdk_C6678_1_1_2_6\packages\ti\drv\pa

Here the CPSW (EMAC) Single core example demonstrates the use of CPSW CSL APIs to configure the Ethernet switch on target.  It further demonstrates the use of CPPI/QMSS/PA LLDs to send/receive data through the Ethernet switch using Host descriptors on any given DSP core.

I hope, you take this as a reference code and develop for packet forwarding between SRIO and NETCP without DSP core intervention.

Refer the SRIO LLD and transport files for SRIO module.

C:\ti\pdk_C6678_1_1_2_6\packages\ti\drv\srio

C:\ti\pdk_C6678_1_1_2_6\packages\ti\transport\ipc\srio\transports

In addition that, see the section "Daisy Chain Operation and Packet Forwarding" in the SRIO user guide for this implementation. The srio expert will give more info If you need further details on SRIO.

Thanks! Pubesh.

Sorry to hear that there is no sample code. 

In your reply, you said I can refer to "Daisy Chain Operation and Packet Forwarding". After reading

that, I think it is related to only SRIO (connecting SRIO device in daisy chain), not including netcp.

is it understanding right?

Murat,

Yes, Your understanding is correct.

Murat,

I hope, you are clear now on this. You can also comeback If you have any doubts on this. If not, please help to close this thread.

Murat,

For further clarification, are you wanting the forwarding to be bidirectional? Meaning, that packets received at the Ethernet peripheral are all transmitted out of the SRIO peripheral and all packets received at the SRIO peripheral are all transmitted out of the Ethernet peripheral? Or will this be one way traffic?

Also, will your packetized data be conforming to any Ethernet or SRIO protocols (header data) that will prevent the data from easily being forwarded out of the other peripheral?

Thanks,

Jason Reeder

Thanks Reeder for replying.

For further clarification, are you wanting the forwarding to be bidirectional? Meaning, that packets received at the Ethernet peripheral are all transmitted out of the SRIO peripheral and all packets received at the SRIO peripheral are all transmitted out of the Ethernet peripheral? Or will this be one way traffic?

The traffic is bidirection.

Also, will your packetized data be conforming to any Ethernet or SRIO protocols (header data) that will prevent the data from easily being forwarded out of the other peripheral?

I am not clear about your meaning on this question. The original data comes from srio which was encapsulated

in ethernet packet format. And I want this data to be directly forwarded to the ethernet port (SGMII1).  After doing this, I will implement the data forwarding from ethernet to srio.

Murat,

I'm not sure how familiar you are with the SRIO and NetCP peripherals but I can describe a high level overview of how data is received and transmitted on these two peripherals and hopefully looking into examples provided in the MCSDK can fill in any gaps in understanding.

• Data receive (from the external world into the 6678 device)
  • Data arrives at the peripheral (either SRIO or NetCP) from the external world and makes its way to the PKTDMA in the peripheral (either the SRIO PKTDMA or the NetCP PKTDMA)
  • The PKTDMA uses the information stored in its RX flows to determine which queue to pop a descriptor from in order to copy the data into the device (this queue is usually called a RX free descriptor queue and is set up only for the purpose of bringing data into the device at this peripheral. You will probably want to have separate RX free descriptor queues for both the SRIO and the NetCP)
  • The PKTDMA also uses information from the RX flows to determine which queue to push the newly filled descriptor into. This queue is normally called the RX destination queue. 
  • In summary: data arrives at the peripheral, the PKTDMA pops a free descriptor from the RX free descriptor queue, the PKTDMA transfers the received data into this descriptor buffer, the PKTDMA then pushes the descriptor onto the RX destination queue.
• Data transmit (from the 6678 device toward the external world)
  • A descriptor is pushed onto a dedicated TX queue for a peripheral (either the SRIO or the NetCP TX queue)
    • For the dedicated queue mappings see Table 5-1 in section 5.1 of the KeyStone Architecture Multicore Navigator User's Guide (http://www.ti.com/lit/ug/sprugr9g/sprugr9g.pdf)
    • For the 6678 device the dedicated TX queues for SRIO are 672-687 and the dedicated TX queues for NetCP are 640-648
  • Once a descriptor arrives on the dedicated TX queue the PKTDMA of the peripheral begins transferring data from the descriptor buffer into the peripheral destined for the external world
  • After the PKTDMA has completed transferring the data then the PKTDMA is responsible for returning the descriptor to the queue specified as the return queue in the descriptor header
  • In summary: a descriptor is pushed onto a dedicated TX queue, the PKTDMA of the peripheral transfers the data from the descriptor buffer into the peripheral, the peripheral continues the process of transferring the data out of the device, and the PKTDMA returns the descriptor to the return queue.

For your purposes I would suggest using queue 648 to transmit data out of the NetCP peripheral. This queue bypasses the PA in the egress direction and sends data directly from the NetCP PKTDMA to the GbE Switch. Also, if you would like to send all Ethernet data out of SGMII1 (external port 2) then I would suggest using a directed packet by setting the TO_PORT value to 2 in the descriptor of the packet that you place on queue 648. 

Knowing all of this, it should be possible set up your RX flow at the SRIO PKTDMA to push received packets (descriptors) onto the NetCP TX queue, 648 (and vice versa where the NetCP RX flow pushes received packets (descriptors) onto the SRIO TX queue, 672). Since all packets received at SRIO will be destined for NetCP you will need to make sure that the descriptors in the SRIO RX free descriptor queue are initialized for NetCP transmit (TO_PORT = 2, etc.) and also that all descriptors in the NetCP RX free descriptor queue are initialized for SRIO transmit.

To my knowledge there are no examples provided that do what you are trying to accomplish. However, you can look at examples provided in the MCSDK for both the SRIO peripheral and the NetCP peripheral to see how to initialize the peripherals (including setting up RX flows, RX DMA channels, TX DMA channels, descriptors, queues, etc.)

For more information on the PKTDMAs and RX Flows also see the KeyStone Architecture Multicore Navigator User's Guide (http://www.ti.com/lit/ug/sprugr9g/sprugr9g.pdf).

For more information on directed packets see table 2-3 in section 2.3.1.3 of the KeyStone Architecture Gigabit Ethernet (GbE) Switch Subsystem User Guide (http://www.ti.com/lit/ug/sprugv9d/sprugv9d.pdf).

Thanks,

Jason Reeder

Thanks, Reeder:

What I did several days ago is almost the same as you mentioned in your reply.

The descriptor put in srio rx free queue is pushed to Q648 as I expect, after 

netcp automatically tx the buffer linked with this descriptor, the descriptor is

then returned to the queue I assigned (e.g. Q712). 

But the PC directly connected to the 6678's ethernet port can't receive any 

packet. 

1. Would you give me some hints on how to debug this? 

    When I recycle the descriptor returned to Q712,  I print the content

    of the buffer linked with this descriptor, the content is what I expect.

2. In your reply, you mention the packet can be forwarded to the ethernet port

    without PA, are you sure?

Murat,

When you push descriptors onto Q648 of the 6678 device the NetCP PKTDMA transfers the data from the descriptor directly to host port 0 in the 3 port GbE switch, bypassing the PA completely. The PKTDMA then returns the descriptor to its return queue (Q712 in your case).

If the packets are never coming out of the 6678 device and are not being received by your PC then the only place that the packets could be getting dropped are in the 3 port GbE switch.

In order to determine why the packets are dropped in the switch we will need to look at the switch statistics that are defined in section 2.3.4 and section 3.5.4 of the Keystone Architecture Gigabit Ethernet (GbE) Switch Subsystem User Guide (http://www.ti.com/lit/ug/sprugv9d/sprugv9d.pdf). STATSA records statistics and errors for host port 0 and STATSB records statistics and errors for both external ports 1 and 2 combined.

To enable statistics gathering you will need to use the STAT_PORT_EN register. If you would only like to read statistics from host port 0 and external port 2 (which I would recommend for your use case) then you should enable everything in the STAT_PORT_EN register EXCEPT for P1_STAT_EN.

Keep in mind that all statistics are from the point of view of the center of the 3 port GbE switch. So, in your example where you are sending packets from SRIO out of the Ethernet toward the external world you should see STATSA (host port 0) RXGOODFRAMES incrementing and you should see STATSB (external port 2) TXGOODFRAMES incrementing. 

For the 6678 device the STATSA registers begin at address 0x02090B00 and the addresses and offsets from there are listed in Table 3-106 of section 3.5.4 in the User Guide mentioned above.

STATSB registers begin at address 0x02090C00 and the offsets from there are the same and are listed in Table 3-106.

Thanks,

Jason Reeder

hi, Reeder:

Thank you for detailed reply and your patience.

I read the statistics you mentioned. The RXGOODFRAMES on 

PORTA is the number I send from SRIO, that is, when I send

3 packets from SRIO, RXGOODFRAMES on PORTA is 3.

But the count on PORTB is 0. I read all the register starting from

02090c00 ~ 02090c8c, they are all 0.

In addition, I've set PS flags in the SRIO's descriptor to be 2.

So you see, the packets are recieved on CPSW port 0, but they

are not sent out from CPSW.

I can't figure it out.

continued:

When I check the descriptor that will be recycled in the netcp transmit side,

I found the PS flags is not set to 2, but 0.

I tried to set this flag in several places:

srio send side descriptor

srio receive side descriptor

it has no effect, it is always 0.

What place to set this flag?

Thanks, Reeder:

I did set the GbE TO_PORT PS flags during initialization for the descriptors, e.g. 2. I set a general purpose queue

to be the place to hold the received SRIO data and its descriptor. Then I print the descriptor's packet info in the general

purpose queue. At this time, the PS flags becomes 0, not the expected 2. I am sure the descriptor is the one I init its

descriptor to be 2.

I noticed in the multicore navigator's user guide, the PS flags can be overwritten by the pktdma. Any other suggestion?

regards

Murat

Murat,

If the SRIO PKTDMA is clearing the bits where TO_PORT is stored then we will have to configure the switch in such a way that it forwards your packets to the desired external port. There are generally two ways to do this:

1. Use the table entries in the ALE (Address Lookup Engine) to decide which port to forward packets to
   1. This is usually used when both external ports are being used and learning is enabled. Source MAC addresses are learned, along with the port number that they entered through, on switch ingress and destination MAC addresses are used for lookup to determine which port to send the packet to during switch egress. This allows the switch to actually perform its switching function based on address.
2. Bypass the ALE in the Ethernet ingress direction so that all packets that arrive at an external port are forwarded to the host port. As well as, flood the two external ports in the Ethernet egress direction so that any packet entering the switch at host port zero gets duplicated and sent out of both external ports.
   1. This is normally used when only one external port is connected and all traffic sent from inside the device toward the switch is destined for the one external port. This is more of a pass through method when no switching is desired.

Regardless of which option you choose, all packets will go through the exact same switch ingress filtering, switch lookup, and switch egress processes described in section 2.3.6.5 of the KeyStone Architecture Gigabit Ethernet (GbE) Switch Subsystem User Guide (http://www.ti.com/lit/ug/sprugv9d/sprugv9d.pdf).

I would suggest using option 2 above in order to have all packets from external port 2 forwarded to host port 0 and all packets sent to host port 0 forwarded to external port 2. To achieve this you will need to configure the switch as follows:

• Enable the ALE
• Put the ports into no learn mode
• Clear the ALE table (clearing the ALE table and putting the ports into no learn mode will ensure that the ALE table remains empty and that there will be no table matches on any incoming data) 
• Enable ALE bypass mode (this will make all packets that arrive at external Ethernet ports be forwarded to the host port)
• Disable ALE VLAN aware
• Disable Switch VLAN aware (putting the ALE and Switch into VLAN unaware mode will cause the VLAN_MEMBER_LIST to default to a flooding port mask as shown in the ingress filtering process mentioned above)
• Disable port 1 and put ports 0 and 2 into the forwarding state

Taking these steps should ensure that the packets will use the ALE VLAN Unaware Lookup Process from the document and that there will also be no table matches. If you look at the table this should make every packet that arrives at port 0 be forwarded to port 2.

Take a look at existing examples that use the switch from the MCSDK. There are functions provided by the CSL to perform most, if not all, of the configurations listed above. The CSL functions for the switch configurations are prefaced by 'CSL_CPSW_3GF_'. (e.g. CSL_CPSW_3GF_disableVlanAware();)

I would suggest using a DSP core to source the packets and watch the switch statistics to make sure that the packets are making it out of the device before bringing the SRIO back into the equation.

Thanks,

Jason Reeder

Murat,

Any progress here? Were you able to get data sent out of the NetCP peripheral and see the switch statistics increment for both the internal host port and external Ethernet port?

Jason Reeder

hi, Reeder:

thanks for attention.

I'm now reading the pa_unittest example under ccs hoping for a solution.

now I have one question: when the program calls Pa_addSrio, then it will

send command to PA. The command contains information  of routing

packets when the packet matches, it also has a routing info when the

packet doesn't  match, it has the same setting as the match one, why

set the second mismatch routing info.  when match and mismatch

has the same setting, I think all packets will have the same routing,

the LUT will be useless. 

Would you mind explaining this?

thanks again