• State Resolved
  • Replies 8 replies
  • Subscribers 99 subscribers
  • Views 205 views
  • Users 0 members are here
Support feedback
Options
Options
Related

AM623:   AM623：CPSW3G: How to assign respective RX Flow IDs to the two ports of CPSW3G in Independent MAC Mode？

shungang wang
Prodigy 10 points
Part Number: AM623

Tool/software:

Hi

We're developing a system using the AM623 processor and need to configure the CPSW3G Ethernet subsystem to direct network packets from specific ports to dedicated RX flow IDs. Here's our detailed  requirements:

1. Current Understanding

      Per the Linux 6.12 kernel driver and AM62x TRM documentation:

  • The CPSW3G in AM62x series has 8 TX DMA channels and 1 RX DMA channel divided into 8 RX flows (0-7) .
  • Each RX flow can generate independent interrupts, enabling per-flow CPU core assignment for improved performance

2. Application Requirements

We need to implement the following:
  • All packets from Port 1 are exclusively received by RX Flow 0.
  • All packets from Port 2 are exclusively received by RX Flow 1.
This separation will enable:
  • Dedicated interrupt handling for each port.
  • CPU core isolation for packet processing.
  • Avoidance of shared buffer contention and interrupt storms.

3. Requested Assistance

We have encountered the following challenges with documentation and hardware differences:
  • Documentation Gap: While the SDK mentions flow-based interrupts, it provides no clear guidance on configuring port-to-flow mapping.
  • Hardware Difference: The CPSW3G in the AM623 appears to differ from that in the AM335x—it lacks the P0_CPDMA_RX_CH_MAP register, which is present in the AM335x’s CPSW3G.
We urgently need clear guidance on mapping ingress ports to flows for the CPSW3G in the AM623. Thank you for your help.

Regards

ShunGang

 

  • 0
    TI__Prodigy 180 points

    Hi,

    Could you please confirm whether you are using Linux or MCU+,RTOS, Bare Metal domain ?

  • 0 in reply to Varun C
    Prodigy 10 points

    Hi,

    Thank you for following up. To confirm our development environment in detail:
    We are working with the Linux domain—specifically using SDK version 11.01.05.03 and running the Linux 6.12 kernel for our AM623-based system.


  • 0 in reply to shungang wang
    TI__Genius 16475 points

    Hello Shungang, 

    Apologies for the delayed response as I was/am out of office this week.

    A couple of initial questions

    1. Are you developing on a custom board featuring AM623 processor?

    shungang wang said:
    Documentation Gap: While the SDK mentions flow-based interrupts, it provides no clear guidance on configuring port-to-flow mapping.

    2. Can you share where in the SDK you have found the mention of "flow-based interrupts"? I assume it is in the SDK release notes but I would like the specific part of the release notes or from the TRM if that is where you are looking.

    Below are some steps of using RX DMA flows that I personally haven't gotten the chance to try myself but could be helpful for your effort to implement packet interrupt separation per port. Please let me know if you encounter issues/have follow-up questions.

    • Up to 8 RX DMA Flows can be enabled using the ethtool -L command in the following manner:
      ethtool -L eth0 rx 8
    • Note that all of the CPSW3G interfaces in Linux should be brought down using ifconfig ethX down before running the above command.
    • The interrupts associated with each of the RX DMA flows will be visible in the output of the cat /proc/interrupts command.
    • After identifying the IRQ number corresponding to the RX DMA Flows, it is possible to bind the interrupt to one or more CPUs using SMP Affinity. The process is documented in the Linux Documentation at:
      https://docs.kernel.org/core-api/irq/irq-affinity.html
    • The irqbalance binary should be removed/disabled for SMP Affinity to work. It can be done using the following command:
      systemctl stop irqbalance.service
    • Can use a port's MAC address to map traffic from the port's MAC address to a specific RX DMA flow by using a specific ethtool command (example below)
      • ethtool -N eth0 flow-type ether dst aa:bb:cc:dd:ee:ff action 5
      • Example will assign RX DMA Flow number 5 to traffic having a Destination MAC Address of aa:bb:cc:dd:ee:ff

    Example to set RX DMA flows to 8:

    root@am64xx-evm:~# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:       8
TX:       8
Other:     n/a
Combined:    n/a
Current hardware settings:
RX:       1
TX:       8
Other:     n/a
Combined:    n/a
root@am64xx-evm:~# ip link set dev eth0 down
[ 1030.925065] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
root@am64xx-evm:~# ethtool -L eth0 rx 8
[ 1042.528606] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 16
root@am64xx-evm:~# ip link set dev eth0 up
[ 1055.718707] am65-cpsw-nuss 8000000.ethernet eth0: PHY [8000f00.mdio:00] driver [TI DP83867] (irq)
[ 1055.718739] am65-cpsw-nuss 8000000.ethernet eth0: configuring for phy/rgmii-rxid link mode
root@am64xx-evm:~# [ 1058.789119] am65-cpsw-nuss 8000000.ethernet eth0: Link is Up - 1Gbps/Full - fx

root@am64xx-evm:~# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:       8
TX:       8
Other:     n/a
Combined:    n/a
Current hardware settings:
RX:       8
TX:       8
Other:     n/a
Combined:    n/a
root@am64xx-evm:~# uname -a
Linux am64xx-evm 6.12.35-gf462b01e9ac7-dirty #7 SMP PREEMPT_RT Fri Aug 22 14:20:37 CDT 2025 aarch64x
root@am64xx-evm:~#

    shungang wang said:
    Hardware Difference: The CPSW3G in the AM623 appears to differ from that in the AM335x—it lacks the P0_CPDMA_RX_CH_MAP register, which is present in the AM335x’s CPSW3G.

    I need to check with colleague about this and if the CPSW3G features on AM335x should be different from AM623 or not.

    -Daolin

  • 0 in reply to Daolin Qiu
    Prodigy 10 points

    Hello Daolin,  

    Thank you for your follow-up. To address your second query:

    We confirm that each RX flow can generate independent interrupts, which we've verified through two means:

    1. Our review of the `am65-cpsw-nuss` driver code shows that the driver explicitly registers interrupts for each individual RX flow.
    2. The output of `cat /proc/interrupts` provides concrete confirmation, as it clearly displays separate interrupt entries corresponding to each RX flow.

    We appreciate the suggestions you provided, which initially seemed like they would resolve our issue. Unfortunately, when we attempted to implement them, we received an "unsupported" error. Further investigation revealed that the command requires the `struct ethtool_ops.set_rxnfc()` interface to be implemented in the network card driver. However, this interface is not implemented in `am65-cpsw-ethtool.c`.
    root@am62xx-evm:~# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:             8
TX:             8
Other:          n/a
Combined:       n/a
Current hardware settings:
RX:             2
TX:             8
Other:          n/a
Combined:       n/a
root@am62xx-evm:~# ethtool -N eth0 flow-type ether dst f4:84:4c:fc:1c:dc action 1
rxclass: Cannot get RX class rule count: Operation not supported
Cannot insert classification rule

    As such, we're seeking explicit guidance on register-level configurations that would allow us to implement this functionality ourselves. Specifically, we need detailed information on how to map specific ingress ports to dedicated RX flows at the register level in the AM623 CPSW3G.

    Any documentation references, register addresses, bit definitions, or configuration sequences related to port-to-flow mapping would be extremely helpful for our implementation efforts.

    Thank you for your continued assistance.

    Best regards,
    Shungang

  • 0 in reply to Daolin Qiu
    TI__Genius 16475 points

    Hi Shungang,

    Daolin Qiu said:
    I need to check with colleague about this and if the CPSW3G features on AM335x should be different from AM623 or not.

    AM335x has similar CPSW3G hardware but it is not the same version so how things might have been handled in AM335x will differ in AM623.

    shungang wang said:
    Further investigation revealed that the command requires the `struct ethtool_ops.set_rxnfc()` interface to be implemented in the network card driver. However, this interface is not implemented in `am65-cpsw-ethtool.c`.
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    root@am62xx-evm:~# ethtool -l eth0
    Channel parameters for eth0:
    Pre-set maximums:
    RX: 8
    TX: 8
    Other: n/a
    Combined: n/a
    Current hardware settings:
    RX: 2
    TX: 8
    Other: n/a
    Combined: n/a
    root@am62xx-evm:~# ethtool -N eth0 flow-type ether dst f4:84:4c:fc:1c:dc action 1
    rxclass: Cannot get RX class rule count: Operation not supported
    Cannot insert classification rule
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Thanks for informing us that this operation does not seem to be supported. Upon further checking with "ethtool -i eth0", the "supports-rxnfc" parameter also doesn't show up, indicating the CPSW Linux driver hasn't implement RX flow classification as you mentioned. 

    Below are some references to RX flow from the AM62x TRM that I hope can be helpful. Unfortunately, we just don't seem to have a good guide step-by-step guide on how to use/setup the RX flows (between the TRM and SDK documentation). A colleague has mentioned using RX flows have been configured in EthFW before so I'm waiting to get more information about that.

    • 12.3.1.4.6.13 Ethernet Port Flow Control
    • 12.3.1.4.6.13.1 Ethernet Receive Flow Control
    • 11.1.4.3 PKTDMA/BCDMA - Output Event Generation
    • 11.1.4.12 PKTDMA - Receive Operation
    • 14.7.3.2.1 PKTDMA_RFLOW_RFLOW_RFA_J_J_Register (DMA Controller Registers)
    shungang wang said:
    As such, we're seeking explicit guidance on register-level configurations that would allow us to implement this functionality ourselves.

    Please let me know if these references are somewhat helpful or specifically what other information you would need to implement the functionality.

    -Daolin

  • +1 in reply to Daolin Qiu
    TI__Genius 16475 points

    Update:

    I recently learned that there is a patch series, not yet fully upstreamed to Linux mainline, that implements support for the missing operation (including the set_rxnfc op that is missing): https://lore.kernel.org/all/20250514-am65-cpsw-rx-class-v4-0-5202d8119241@kernel.org/ 

    Since the patch series is not yet upstreamed, there may be more changes going forward, but at least it will give some starting point.

    -Daolin

  • 0 in reply to Daolin Qiu
    Prodigy 10 points

    Hi Daolin,

    Thank you so much for sharing this critical update—this patch series is exactly the starting point we’ve been looking for!
    Best regards,
    Shungang
  • 0 in reply to shungang wang
    TI__Genius 16475 points

    Hi Shunwang, 

    No problem. If you find a breakthrough in this please do let us know. My understanding is that you are looking to implement RX flow mapping based on port to separate out EtherCAT traffic and non-EtherCAT traffic?

    -Daolin