NDK transmitting fragmeneted UDP

Lech,

    This is a known bug that's been tracked in our system. The work-around can be found in this post

Let me know if you have other questions.

Thanks,

Moses

Hi Moses,

Tnak's for your answer. I'm not sure, if the post you suggest to read describe my problem. In my programm sendto() returns -1 and the fdError is EMSGSIZE.. Of course the SPRU524H give the information "For a datagram type socket, the send operation always copies one packet's worth of
data. If the buffer size is too large to be transmitted in a single packet, an error code of
EMSGSIZE is returned." But id does'nt explain if it can  fragment the packet .

I analyse the udp.c code and see i the function UdpOutput() line mss = (INT32)IFGetMTU( hIFTx ); and following

    if( size > mss )
    {
        error = EMSGSIZE;
        goto UDP_EXIT;
    }

The error is prodused by this code. But I don't know the NDK as deep and I can't find the answer in code how to force the NDK to fragment large UDP packets.

The ARP table is fulfilled  before the transmisson because of preceding short messages I send to host.

So, I don't know if my problem can be solve or not.

Lech

Hi,

I analysed more specially the code of udp.c and ipout.c from the NDK catalogue, especially the functions  UdpOut() and IPtxPacket(). The UdpOutput analyse the length of incoming data in line 132:

if( size > mss )
{
        error = EMSGSIZE;
        goto UDP_EXIT;
}

Of course if the value of mss is 1473 the function always return error EMSGSIZE. But the analyse of  IPtxPacket() shows in line 344 the loop:

/* Send out a bunch of fragmented packets */
for( off=0; off<(UINT16)Valid-hlen; off+=w )
{
..
}

Below there is the copy of data and header (lines 368 and 369). The operation repeats for all fragmented parts of message. Probably the actual value of mss should be larger, but I dont't know how to do it.

So, the question is, why it doesn't work? I linked library of the NDK to my project, so I can't debug it and investigate full call stack.

Lech

Lech,

Can you add some debug print messages into the UdpOutput function to help debug this?

First, please try copying the file

C:/ti/ndk_2_23_02_03/packages/ti/ndk/stack/udp/udp.c

into your project.  Then rebuild.  Can you successfully rebuild your app with the local copy of udp.c as part of your project?  If so, you can modify and debug that local udp.c.

If it does not build, then you would need to update the udp.c that's in the stack, which will require rebuilding the NDK libraries.

Rebuilding the NDK libraries is not too difficult, you can see instructions in the NDK's release notes on how to do that (it will redirect you to a wiki).

Once you figure out which of the above methods works best for you, please add these prints (note that some extra curly braces are needed, too):

int UdpOutput( HANDLE hSock, UINT8 *buf, INT32 size, INT32 *pRetSize )
{
...
    printf("UdpOutput: enter\n");

    /* Bound the size to something we can handle */
    if (pSock->TxBufSize == 0) {
        printf("UdpOutput: A\n");

        /* If send buffer size (SO_SNDBUF) is not set: */
        /* We'll try and get the MTU from the route, then the egress IF, */
        /* or finally we just assume a default. */
        if ((hRoute = SockGetRoute( hSock ))) {
            mss = (INT32)RtGetMTU( hRoute );
            printf("UdpOutput: B (mss = %d\n", mss);
            hIFTx = 0;
        }
        else if ((hIFTx = SockGetIFTx( hSock ))) {
            mss = (INT32)IFGetMTU( hIFTx );
            printf("UdpOutput: C (mss = %d\n", mss);
        }

        else {
            mss = UDP_MSS_DEFAULT;
            printf("UdpOutput: D (mss = %d\n", mss);
        }
    }
    else {
        mss = pSock->TxBufSize;
        printf("UdpOutput: E (mss = %d\n", mss);
        hRoute = SockGetRoute (hSock);
        if( hRoute != 0 )
            hIFTx = 0;
        else
            hIFTx = SockGetIFTx (hSock);
    }
    printf("UdpOutput: F (IP hdr size = %d, UDP hdr size = %d\n",
             SockGetIpHdrSize(hSock), UDPHDR_SIZE);
    mss -= SockGetIpHdrSize(hSock) + UDPHDR_SIZE;   /* Sub off IpHdr & UdpHdr */

    if( size > mss )
    {
        printf("UdpOutput: G (error: size (%d) > mss (%d) \n", size, mss);
        error = EMSGSIZE;
        goto UDP_EXIT;
    }

Steve

Hi Steve,
I made the changes you suggested. The stack returns following messages:
UdpOutput: enter
UdpOutput: A
UdpOutput: B (mss = 1500
UdpOutput: F (IP hdr size = 20, UDP hdr size = 8
UdpOutput: G (error: size (8280) > mss (1472)

It confirms earlier observations. Do you have some idea, how to solve the problem?
Regarrds,

lech

Hi Steve,

In the meantime I analysed call stack from sendto() toUdpoutput(). It's clear for me how it works, but I still don't know when the value pSock->TxBufSize is set. So, I can't answer the question, what setting of NDK is wrong. If you could look at the app.cfg file1374.app.rar, maybye it becomes clear.

Regards,
Lech

lech,

Can you try increasing the size of the send buffer for your socket (that you are trying to send UDP data on)?

The code would look similar to the following:

    /* change below size of 8192 to the size that's most appropriate for your app */
    int i = 8192;
    setsockopt(mysocket, SOL_SOCKET, SO_SNDBUF, &i, sizeof(int));

Steve

Hi,
The situation has changed, now I have error 55 = ENOBUFS.

The maximum UDP buffer size I can send with sendto() is 3016 bytes and not depends on the size of buffer I set with the code you sent. Better, but not enough :(

Lech

Lech,

Yes, you are now hitting the limitation which requires jumbo frames and will need to use jumbo frame to get past that.  Also, you may run into another issue, #2 below, but fortunately a fix for that will be available soon.

There are a few different limitations you will encounter with calling sendto() with a size which requires fragmentation.

1. the socket buffer size:

- must be large enough to hold the data you are passing to sendto()

- this is the issue you originally ran into, which I'm guessing that the above suggested call to setsockopt() fixed for you

- results in error 'EMSGSIZE' (40, "message too long")

2. A design bug in the ARP layer due to queue length of 1

- causes all but the last fragment sent to be lost (when there is no ARP table entry for the destination host you are trying to send data to) when sending out ARP requests to determine IP <--> MAC address mapping.

- This issue is covered by the following bug and will fixed very soon in an NDK patch release to version 2.24.x:

SDOCM00088612 (Child) ARP Resolution Queue Len is 1

- In this case, sendto() will return the full number of bytes sent and fdError() will not report any error (it returns 0), making the problem elusive.

3. Allocation limit for non-jumbo frames

- Frames are currently limited to the following constant:

    #define MMALLOC_MAXSIZE 3068

- Any allocation greater than this size will require jumbo frame support

- Furthermore, as you have discovered, this maximum size translates to a maximum value of 3,016 bytes that can be passed in sendto().

- This breaks down as follows, as the 3016 accounts for just the payload size:

3068: -20 (nimu_mcb hdr_size) - 4 (nimu_mcb trailer_size) - 20 (IP hdr size) - 8 (UDP hdr size) = 3016

- So, passing a value of 3,017 or higher will result in the error "ENOBUFS" (55,  No buffer space available)

- Note that the "nimu_mcb" values may vary by device (and hence result in an actual limitation value that differs from 3016).  I got these values from the evmOMAPL138 device.

Steve

Hi Steve,

Thank you for the explanation. I have only one additional question. What about the sendnc() function which works on user's buffer? It's not documented in User's guide. I think, that the MAC controller uses memory descriptors rather than FIFO, so, potentially there is the possibility of sending larger messages without allocating jumbo buffers, but user has to save the space for IP and UDP headers. So, I'm afraid that it is not a way to solve the problem. In conclusion, I have to settle for 3kB UDP buffer :(

Regards,

Lech

Lech,

Let me look into that a little more ... but I have a feeling you may hit the same issue.

In any case, sendnc is documented in the API guide (spru524i.pdf) and is meant for raw ethernet sockets.  Please see the API guide for details on that.

Steve

Hi Steve,
You have absolutely right. The sendnc() is described in spru524i.pdf. Unfortunately it force me to prepare all headers "manually". So, I think about the sendto() once again. Maybe including jumbo support in conjunction with increasing mss variable as you described earlier give the desired behaviour? In this case there should be no error of mmalloc size. Please tell me what command I have include in ndk.bld to do it. Wiki doesn't explain this option, but I'm sure that the jumbo_pbm.c is not included to the ndk library.

Lech

Another thing, it may be necessary to increase the size(s) of the PBM pool(s):

http://e2e.ti.com/support/embedded/tirtos/f/355/t/169168.aspx

Steve