TMS320DM814x CPSW TX_PRI[N]_RATE register field information

In addition to the above, I am seeing something strange when enabling rate limiting mode and setting TX_PRI[N]_RATE. For instance, setting the register to 0x00011000 (or 0x00010100, but these are not the only values for which it happens) I see frames being sent on the wire but the OWNER bit in the TX buffer descriptor is never cleared by the EMAC, the EMAC does not signal any error condition; the software will eventually run out of buffer descriptors because it does not have a way of knowing they have been processed by the EMAC.

Has anyone seen this?

Thanks
--
Delio Brignoli
AudioScience Inc

Hi Delio,

I'm currently escalating your question. I believe the 'clock' you mention for the equation is the G/MII ref clock (for 1000 Mbps is 125 MHz), but I will verify that. Could you give a little more information on what you're writing to the TX_PRI[N]_RATE register? Are you describing the value you set to the register in hex or binary? Keep in mind the TX_PRI[N]_RATE registers are 32 bits in length and have 4 reserved bits per the TRM (Section 9.4.1.51).

Best,

Keegan

Hi Delio,

There is info about the CPSW rate limiting mode here :

http://processors.wiki.ti.com/index.php/TI81XX_PSP_Ethernet_Switch_User_Guide#CONFIG_SWITCH_RATELIMIT

The transmit rate limit source code is in the Ethernet Switch Driver :

ti-ezsdk_dm814x-evm_5_04_00_11/board-support/linux-2.6.37-psp04.04.00.01/drivers/net/cpsw.c

Here is an extract of some code:

static int cpsw_switch_config_ioctl(struct net_device *ndev,
        struct ifreq *ifrq, int cmd)
{

..........

case CONFIG_SWITCH_RATELIMIT:
        if ((portcmd(switch_config).port <= 2) &&
                (portcmd(switch_config).enable == 0))
            ret = cpsw_ale_control_set(priv->ale,
                portcmd(switch_config).port,
                ALE_RATE_LIMIT, 0);
        else if ((portcmd(switch_config).port <= 2) &&
                ((portcmd(switch_config).addr_type
                    == ADDR_TYPE_BROADCAST) ||
                (portcmd(switch_config).addr_type
                    == ADDR_TYPE_MULTICAST)) &&
                (portcmd(switch_config).limit <= 255)) {
            ret = cpsw_rate_limit(priv->ale,
                portcmd(switch_config).enable,
                portcmd(switch_config).direction,
                portcmd(switch_config).port,
                portcmd(switch_config).addr_type,
                portcmd(switch_config).limit);
        } else {
            printk(KERN_ERR "Invalid Arguments\n");
            ret = -EFAULT;
        }
        break;

..........

}

/**
 *    cpsw_rate_limit_tx
 *    Limits the number of Tx packets on a port for a 100ms time period.
 *    Rate limit can be done either on Tx or Rx for all the ports due to
 *    hardware limitation
 */
static int cpsw_rate_limit_tx(struct cpsw_ale *ale, u32 enable, u32 port,
                u32 addr_type, u32 limit)
{
    if (!enable) {
        cpsw_ale_control_set(ale, port, ALE_RATE_LIMIT, 0);
        return 0;
    }

    /* ALE prescale is set for 100 ms time */
    if (addr_type == ADDR_TYPE_BROADCAST)
        cpsw_ale_control_set(ale, port, ALE_PORT_BCAST_LIMIT, limit);
    else if (addr_type == ADDR_TYPE_MULTICAST)
        cpsw_ale_control_set(ale, port, ALE_PORT_MCAST_LIMIT, limit);
    else
        return -EFAULT;

    /* Enable the bit for rate limiting transmission */
    cpsw_ale_control_set(ale, port, ALE_RATE_LIMIT_TX, 1);
    cpsw_ale_control_set(ale, port, ALE_RATE_LIMIT, 1);

    return 0;
}

int cpsw_rate_limit(struct cpsw_ale *ale, u32 enable, u32 direction, u32 port,
            u32 packet_type, u32 limit)
{

    if (port == 0) {
        /* For host port transmit/receive terminlogy is inverse
            of cpgmac port */
        if (direction)
            cpsw_rate_limit_rx(ale, enable, port,
                    packet_type, limit);
        else
            cpsw_rate_limit_tx(ale, enable, port,
                    packet_type, limit);
    } else {
        if (direction)
            cpsw_rate_limit_tx(ale, enable, port,
                    packet_type, limit);
        else
            cpsw_rate_limit_rx(ale, enable, port,
                    packet_type, limit);
    }
    return 0;
}

The TX_PRI[N]_RATE registers looks to me that are not implemented in the CPSW driver. 

I am still searching answers about the TX_PRI[N]_RATE and CLK description/details.

Best Regards,

Pavel

Hi Delio,

I received a feedback that rate limiting is supported on DM814X devices. The CPDMA rate limiting registers are in the CPDMA register address space.

The equation below defines the megabits per second transferred on each rate limited transmit DMA channel. The frequency is the OCP (interconnect) clock frequency which is an input to the module.  IDLE counts idle clocks for the priority.  SEND counts send clocks for the priority. However, those statements don't really matter.  The transfer rate holds by the formula. I will add that in general it's better to have lower IDLE register values than higher IDLE register values.  If you want 100 Mbps on a transmit channel total then IDLE_CNT = 1 and SEND_CNT = 111 (at 350Mhz).

Mbit/s = ((PRI_IDLE_CNT/(PRI_IDLE_CNT + PRI_SEND_CNT)) × frequency × 32

BR,

Pavel

Hi Pavel and Keegan,

Sorry for not replying earlier but I was out of the office. I will reply to all your messages below.

Keegan: I am describing the values in hex as indicated by the 0x prefix in the values, I am aware of the reserved bits and not touching them.

Pavel: Your first reply referencing code from cpsw.c delas with a different level of rate limiting that happens elsewhere in CPSW. Your second reply seems to refer to the CPDMA rate limiting I was looking for. However, you say:

IDLE counts idle clocks for the priority.  SEND counts send clocks for the priority. However, those statements don't really matter.  The transfer rate holds by the formula. I will add that in general it's better to have lower IDLE register values than higher IDLE register values.  If you want 100 Mbps on a transmit channel total then IDLE_CNT = 1 and SEND_CNT = 111 (at 350Mhz).

Mbit/s = ((PRI_IDLE_CNT/(PRI_IDLE_CNT + PRI_SEND_CNT)) × frequency × 32

I think knowing what IDLE and SEND count is important and may affect cases in which rate limiting is enabled on more than one channel at a time. I imagine the amount of data transmitted for each SEND clock is 32 bits, can you confirm? Must there be a certain relationship between IDLE count of (say) the channel with top priority and the SEND counts of channels with lower priority?

To find out the OCP clock frequency I need to figure out where it is derived from, I looked in the TRM and grep'ed linux's sysfs clock tree but didn't find any mention of the OCP clock. Linux's sysfs clock tree shows:

/sys/kernel/debug/clock/osc0_clkin_ck/l3_dpll_clkin_ck/l3_dpll_ck/sysclk4_ck/sysclk5_ck/cpsw_ick/rate ==> 200000000

Is this the clock you mentioned?

Thanks
--
Delio Brignoli
AudioScience Inc

Delio,

The above is correct.  Each send count has 32-bits of data because the CPDMA has a 32-bit bus.

Yes, OCP clock is cpsw_ick (200MHz).

BR,

Pavel

Thank you Pavel. I was also asking:

Must there be a certain relationship between (say) IDLE count of the channel with top priority and the SEND counts of channels with lower priority?
In other words, are IDLE and SEND values of a channel dependent on the values of other channels or not?

I realize you said to just use the equation to figure the rate but, as I stated in a previous post, after a while CPDMA gets into a state where one or more channels stall (no data transmitted, no error indication), so I think there maybe more to it; without some documentation on CPDMA (a block diagram for instance) it's difficult to work out a mental model of how this works and ask the right questions, let alone understand/fix the issue. Thank you for all your help so far.

Kind Regards
--
Delio Brignoli
AudioScience Inc

Hello,

Rate-limiting (or shaping) has some basic assumptions.  The first is that the port receiving the traffic is not over-subscribed.  That is, everything sending to a particular port is rate limited. There is no policing on ingress from an Ethernet port.  It must be shaped from the sending port on any rate-limited priority.  For the CPDMA, each rate limited channel is independent from the others, but the rates must be such that all the rules are obeyed and the bus bandwidth can be
supplied.

Best Regards,

Pavel

Pavel,

I switched threads!

Seems to me the above equation is incorrect (or the text is). If PRI_SEND_CNT is the send (pkt) clock counts, the equation should read

Mbits/s = PRI_SEND_CNT/(PRI_SEND_CNT + PRI_IDLE_CNT) x OCPfreq x 32

What am I missing?

- Andrew

Further to what I wrote above, suppose we want put an AVTP packet of 94 bytes (752 bits) on the wire every 125usec (8 kHz rate), how does one derive the SEND and IDLE counts?

Is the observation interval in sec = (PRI_SEND_CNT + PRI_IDLE_CNT)/OCP freq      ?

OCP freq = 200 000 000, internal = 0.000125, then

PRI_SEND+CTL + PRI_IDLE_CNT = 25000

Is the above thinking correct, or totally off base? Is there a state machine drawing or anything that would clarify the meaning of PRI_SEND_CTL and PRI_IDLE_CNT?

Thanks,

Andrew