TMS570 EMAC flow control

Emiliano,

I have forwarded your question to our EMAC expert. He will get back to you shortly.

Thanks and regards,

Zhaohong

Emiliano,

The MAC supports flow control.  See sections 29.2.10.1.3 (RX) and 29.2.10.2.6 (TX) TRM.

For the PAUSE frames you need to be in fullduplex and you can chose to enable transmission and/or reception of pause frames - there are separate bits for each.

I believe you can advertise both since you could enable or disable either one or both.  There is an an Auto-Negotiation Advertisement Register (ANAR) register in the DP86340 that appears to let you do this.   You might want to post to the Ethernet forum to get the perspective of the team that supports the DP83640 phy. 

Emiliano,

We currently don't have any EMAC examples that use flow control. I'll see if I can get something up and running using it.

In the mean time, could you post more details of the error you're facing? We might be able to help fix it and it'll help others in the future.

Also, as Anthony mentioned, you can( and should) advertise PAUSE capabilities using the Auto-Negotiation Advertisement Register of the PHY, though you might get a better and more elaborate answer from the guys who support the DP83640.

Hope this helps.

Thanks and Regards,

Chaitanya

Hi,

Thanks for your help. Sure I can give you more details. I have set up flow control with the following code:

#define EMAC_0_BASE                ( 0xFCF78000 )

EMACNumFreeBufSet( EMAC_0_BASE, 0, 16 );
HWREG(EMAC_0_BASE + EMAC_RXFLOWTHRESH(0)) = 1;
HWREG(EMAC_0_BASE + EMAC_MACCONTROL ) |= EMAC_MACCONTROL_RXBUFFERFLOWEN;
HWREG(EMAC_0_BASE + EMAC_MACCONTROL ) |= EMAC_MACCONTROL_TXFLOWEN;

Then, I am creating a TCP socket with the NODELAY option, and sending pairs of consecutive TCP frames, every 100ms, with a small payload from the PC. I.e.:

1. send() 7 bytes via TCP

2. send() 14 bytes via TCP

3. wait 100ms

4. goto 1.

Intermittently, I see that the TX collision register is incremented, and the 14-byte packet is lost. I suspect that the outgoing ACK to the 7-byte packet is colliding with the incoming 14-byte packet. The time when this occurs is not fully deterministic but, after it has once, it will keep happening every time until I reset the system.

If I disable flow control, I get no collisions and no packet loss.

Regards,

-Emiliano

Emiliano,

Can you check what mode of communication-- half-duplex/full-duplex -- is enabled? In case of half-duplex, the flow control mechanism itself is collision based. The EMAC generates collisions in order to prevent frame reception. So if the collision register is incremented, I'm guessing that's what is supposed to happen in this case.

Also, even if it is full-duplex, you answered your own question. Maybe because of transmitting 2 consecutive frames, there are collisions happening since the receiver hasn't had enough time to acknowledged the first frame. I think  adding a delay between the 2 transmissions might help.

Let me know if either of these answers help.

Thanks and Regards,

Chaitanya

Hi Chaitanya,

Thanks for your answers and I am sorry I haven't been able to reply earlier. I have been doing a lot of work on this. Your answer pointed me to the right direction.

It turned out that the initial autonegotiation occasionally failed, and the following read of the link partner ability register returned 0, which was read as half duplex. I have never seen collisions on a full duplex link (I am using only 1 EMAC channel), so that made me suspicious.

However, flow control is still not working. If I understand correctly, every time a RX descriptor is used, RXFREEBUFFER is automatically decremented by 1, and I the software must re-increment it when it releases the descriptor, by calling EMACNumFreeBufSet(<emac base>, 0, 1). Is that correct?

That seems to work if I just ping the system: I can see RXFREEBUFFER decrementing and then re-incrementing when I call EMACNumFreeBufSet(). However, after hundreds of packets, even if I pause the communications, the value of RXFREEBUFFER is lower than it should be: eventually it reaches 1 and I start getting pause frames; it never goes back up to the initial value, even if I checked that I am calling EMACNumFreeBufSet() the right number of times.

Could there be some race condition between the DMA and the core? Is EMACNumFreeBufSet() the right function to call?

Thanks,

-Emiliano

Hello,

I have come to the conclusion that there is a race condition on register RXnFREEBUFFER. On my project, the Rx ISR notifies that a new packet has been received, and the application 'unloads' the descriptors and then calls EMACNumFreeBufSet().

As I explained on my previous message, RXnFREEBUFFER occasionally misses increments, and it eventually ends up to 1 (which causes a neverending storm of pause frames).

If I disable Rx before incrementing RXnFREEBUFFER and re-enable it immediately afterwards, RXnFREEBUFFER never misses an increment:

<disable interrupts>

HWREG( emac_base + EMAC_RXCONTROL )= 0;
EMACNumFreeBufSet( emac_base, EMAC_CHANNEL, 1 );
HWREG( emac_base + EMAC_RXCONTROL ) = 1;

<enable interrupts>

Do you agree that this proves the point? I would have thought that the point of having RXnFREEBUFFER as a  write-to-increment register was indeed to access it atomically, but, as it stands, this doesn't seem to be the case.

As a result, since I am not sure whether disabling and re-enabling Rx is actually safe (in theory I could miss a frame anyway), it doesn't look like I can use flow control... any comments?

Thanks,

-Emiliano

Emiliano,

Thanks for this little piece of information. I wasn't aware of this earlier and I'm sure it'll help a lot of other people as well.

If there is a race condition, using a critical section seems to be the only solution as you've mentioned above. Although the conventional method of implementing a critical section would be turn off interrupts, rather than turning off Rx entirely, since here the packets will still be received. Though I'm not entirely sure if this would help since all the packet processing is still interrupt based, but maybe a workaround could fix this, since the packets are received but there is no interrupt. I suggest you look at this approach as well, in case it helps.

Apart from that, I think you're right in the sense that using flow control might be a risk, especially if data is received at a high rate. I'll look into possible solutions for this, I'm sure this must've come up in other situations that use flow control.

Do let me know how it goes.

Regards,

Chaitanya

Hi,

I am disabling interrupts just because I am also updating one or two volatile counters there, but the key part is having the EMAC RX module disabled while I am incrementing the RXnFREEBUFFER register. In fact, masking the EMAC RX pulse interrupt won't prevent the race condition, because RXnFREEBUFFER is decremented by the DMA module, not by the ISR.

That's what worries me: if I mask the interrupt for a few instructions, it will still be serviced when I unmask, but I don't know what happens if a frame is received while the EMAC RX is disabled: will it be lost? And that is the main question: if I may lose frames anyway, the whole point of using flow control falls apart, doesn't it?

To be fair, I have been sending lots of frames to the TMS with this workaround, and no one has been lost, but I would like to know that theory is also on my side, if you know what I mean.

As far as the 'official' software is concerned, for the time being RX flow control will be disabled.

Please let me know if there are any news!

Thanks a lot,

-Emiliano

Hi,

Just wondering if there were any news on this on your side.

Regards,

-Emiliano

Emiliano,

I asked a couple of people who've worked on EMAC and specifically on flow control, but they said they've never faced something like this, at least on the software side of things, and that a deeper look at the hardware side might be required. I'm trying to get hold of someone who knows that much about EMAC and I'll let you know if I find something.

Apologies for the delay in replying.

Regards,

Chaitanya