This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TivaC + TI-RTOS + NDK and 3 TCP/IP sockets question

Other Parts Discussed in Thread: TM4C1294NCPDT

I am working  on a new board that has a TM4C1294NCPDT TivaC CPU  and the application will need three TCP/IP sockets to send and receive data asynchronously.  I will be using CCS 6.01 with TI RTOS and I am evaluating the integrated NDK (TI’s TCP/IP stack) so I am at the beginning of the learning curve on the NDK. 

I want some advice on how to implement / integrate the sockets with the RTOS.  I would like to use the sockets in blocking mode so there is no polling and wasting CPU cycles etc.  In some of the bigger operating systems where resources are less limited, I would use two tasks for each socket.  With two tasks for each TCP/IP socket, one task would receive TCP/IP data and the other task will send the TCP/IP data  Lets call the tasks rcvTask_port1() sendTask_port2() for now.  

            The rcvTask_port1() task would block on the recv() and will only receive data and or messages from the TCP/IP socket and handle the data or message.

            The sendTask_port1(). task would block on the mailbox message and will wake up and send any data over TCP/IP.   Other tasks in the system would send a mailbox message to this task if they need to send data over the socket.

 

With this model, each socket would have two tasks that block till there is work to do so there is no polling.  This assumes that the TCP/IP stack is "re entrant" or thread safe since the messages being received and sent are asynchronous.    If my assumption is not true and the stack is not thread safe, the software may work but randomly crash or do other odd things if this design is used.

 

I have NDK version 2_23_01_01, TI-RTOS version 2_01_00_06, and CCS 6.0.1.00040.

I have been reading the two documents found in the folder

 

\tirtos_tivac_2_01_00_03\products\ndk_2_23_01_01\docs

 I have only seen real simple examples of the NDK using sockets like the tcpEcho example provided with some of the TivaC boards.  Would using two tasks for each sockets be the recommend approach for the NDK stack or is there a better way this should be implemented?

 

Thanks,

Doug

  • Hi Doug,

    I'm the lead developer for the NDK.  Your design of having a separate task for sending and receiving sounds good, I don't see any issue there.  One thing you will need to do, however, is use the fdShare() function IFF the two tasks will use the same socket.  If you try to share the same socket between tasks without that call, you will get an error of -1 when using socket APIs, due to the file descriptor environment not being found.  You can see some example code of the fdShare()API in the ndk tree, in the file ti/ndk/tools/console/contest.c (look for the TcpShare function).

    The TivaC board has a relatively large amount of RAM, so I don't think you should have stack size issues with having multiple tasks like you mentioned in your design.


    Regarding reentrancy, the NDK uses priority exclusion, so the priority levels you use for your sockets/NDK tasks will be important.  You can find more info on that in the NDK User's Guide and/or API guide, if you haven't already.


    Steve

  • Steve,

    Thanks for the info.  I have been prototyping a design with two tasks for each socket and I have a couple of questions. For now I only have one socket so I am just learning how to integrate the OS and NDK.

    1)      The  document “TI Network Developers Kit NDK v 2.221 SPRU523H section 4.2.1 is Scheduling Overview.  Quote:  “Although the NDK provides a reentrant environment, the core of the stack is not reentrant.  Portions of the code must be protected from access by reentrant calls.”    I don’t know if this implies that if I have three sockets each with two tasks for a total of 6 tasks that I would run a risk of a crash or other problems doe to reentrancy.  Is the “portions of the code that need protection” something I need to do in my case?

    2)      I created a new  project in CCS 6.0.1.00040 for the TM4C1294 eval board and have a socket working with a pair of tasks.  I did run into what you described where I need to call fdShare() in each task.   The odd thing I found was if the calling task is too high of a priority, fdShare() will not return a success.  I assumed the task running at too high of a priority and not allow other tasks to run so I did the following (see code clip below). Basically this code work fine at priority 3 but not priority 5.  This probably has something to do with the scheduling of the NDK but I don’t know how the explain it and if it is expected.  I would have thought if the priority was too high and fdShare() was failing to return success, the task would block on the Task_sleep(5) and let other tasks run and return after the time has expired and then fdShare() could be called again and return a success but it never did.

    void receiveTask()

    {

                    long dataReceived, errorCounter = 0;

                    int fdOpenResult = 0;

     

                    // this didn't work when the task priority was 5????  Why works at priority 3.

                    fdOpenResult = fdOpenSession(TaskSelf());

                    while(fdOpenResult == 0)

                    {

                                    printf ("receiveTask() fdOpenSession failed\n");

                                    Task_sleep(5);

                    }

     

                    printf ("receiveTask() fdOpenSession OK\n");

     

    3)      The socket on the TM4C1294 board will need to be able to be closed and re opened from the PC on the other side of the wire.  The prototype code I have been experimenting with has a TCP/IP socket server that listens on one port for a connection.  When there is a connection on the port, the application on the TM4C1294 board will start sending a 256 byte message periodically with a count starting from zero and it increases by one each time a message is sent.  I use static memory in the application so the application is real simple.  When the application is running, and the socket is sending data from the TM4C1294 board to the PC, I also have the PC in a ping loop to constantly ping the TM4C1294 board.  I can connect to the TM4C1294 board from the PC and the TM4C1294 will start sending data.  I can close the connection and re open the connection and the TM4C1294 will resume sending data.  I can do this only a 4 times and then the 5th attempt to connect, the connection will fail.

    Any ideas? 

    Doug

  • FYI, to answer #2 question above, it was missing a line of code where it calls the fdOpen again in the loop.  it works correctly with the following code:

        fdOpenResult = fdOpenSession(TaskSelf());
        while(fdOpenResult == 0)
        {
            fdOpenResult = fdOpenSession(TaskSelf());
            printf ("receiveTask() fdOpenSession failed\n");
            Task_sleep(5);
        }

        printf ("receiveTask() fdOpenSession OK\n");

  • Hi Doug,

    Doug Baker said:
    I can do this only a 4 times and then the 5th attempt to connect, the connection will fail.

    You might.  I believe that section is talking about llEnter/llExit, which is to enter/exit NDK's kernel mode.  Here's some guidelines for calling llEnter/llExit:

    The following are good general guidelines:
    • Always call llEnter() before calling a stack function, and llExit() when done calling stack functions.
    • Try and keep all code that requires llEnter() and llExit() in a single module. They are only required for
    system maintenance.
    • Do not call a normal user function (like a socket function) between an llEnter()/llExit() pair.
    • Never call llEnter() or llExit() from an ISR.

    Also, make sure that any global variable access is protected, especially if the same function is used for multiple Tasks if that Task is reading/writing to any kind of global data.

    Doug Baker said:
    Is the “portions of the code that need protection” something I need to do in my case?

    Are you creating a socket for each new incoming connection via accept()?  If so, are you calling close on each socket once communication on that socket is finished.  You might be hitting memory allocation issues, as each socket() call allocates a lot of memory by default (large TCP send and receive buffers).

    Steve

  • The sequence I use is to statically create two tasks, a send task and a receive task.  The socket is a TCP/IP server.

    The receive task will create a TCP/IP socket and post a semaphore after the socket is created and connected.  The semaphore will signal the transmit task that the socket is created and ready.

    The transmit task will do the fdOpenSession() and pend on the semaphore.  The semaphore prevents the transmit task from running if the socket is not ready.  For now, we can ignore the transmit task (I think) .  I did read that fdclose() has to be done in both tasks but I don’ think this is the problem since if I don’t release the semaphore to allow the transmit task to run I still have a memory leak when I disconnect and connect.

    On the receive task I do:

                    1 Call function debugSocInit() to create the socket

                                    debugSocInit() does the following                          

                                                    fdOpenSession()

                                                    create socket with socket()

                                                    Set the socket option with function setSocketOptions()

                                                    bind()

                                                    listen()

                                                    accept() (blocking)

                    2 while (1)

                                    receive()

                                    if receive returns zero I assume the socket is closed on other end so I do fclose()  then a fdCloseSession.  After the socket is closed,  I then call the function debugSocInit() again to create a new socket.

     

    I can run the code and the socket works as expected but if I disconnect and re connect a few times I get a crash.  I can confirm using the ROV that I seem to leak 0x1010  bytes on each connect / disconnect til the memory leak / crash.  I’m not sure what I need to do when the socket is closed on the other end to free up the memory.  In my application, the remote host can close and re open any time.

     

    here is a receive task:

    void debugSocTask_RX(void)

    {

           long dataReceived;

           int result;

           CmdMsgObj cmDMsg;

     

           Task_sleep(10);

     

           debugSocInit(); // init the socket

     

     

           //////////////////////////////////////

           Semaphore_post(debugSocketStartSema);

           //////////////////////////////////////

     

     

           while(1)

           {

                  if(debugSoc.connected)

                  {

                         memset(recBufferDebugSoc, 0x0, sizeof(recBufferDebugSoc));

                         dataReceived  = recv(debugSoc.clientfd, (char *)recBufferDebugSoc, sizeof(recBufferDebugSoc), 0);

                         if(dataReceived > 0)

                         {

                               if(++bufferStruct.idx >= NUM_BUFFERS)

                               {

                                      bufferStruct.idx = 0;

                               }

     

                               memcpy(bufferStruct.globalStringBufArray[bufferStruct.idx], &recBufferDebugSoc[0], EACH_BUFFER_SIZE);

                               printf ("debugSocTask_RX() Received bytes %d\n", dataReceived);

                               cmDMsg.strPointer = bufferStruct.globalStringBufArray[bufferStruct.idx];

                               cmDMsg.id = socket_ID;

                               Mailbox_post(cmd_mbx, &cmDMsg, 12);

                         }

                         else if(dataReceived == 0)

                         {

     

                               if(debugSoc.lSocket != (socket) -1)

                               {

                                      fdClose(debugSoc.lSocket);

                                      printf ("debugSocTask_RX fdClose() called\n");

                               }

                               debugSoc.connected = FALSE;

     

                               fdCloseSession(TaskSelf());

                               printf ("debugSocTask_RX socket receieved 0 bytes, fdClose\n");

     

                         }

                         else

                         {      // see C:\ccs6p0b\tirtos_tivac_2_01_00_03\products\ndk_2_23_01_01\packages\ti\ndk\inc\serrno.h

                               result =  fdError();

                               printf ("debugSocTask_RX Receive error Code %d\n", result); // 57 is not connected

                               if(result == 57)

                               {

                                      shutdown(debugSoc.clientfd, 2);

                               }

                               Task_sleep(1000);

                         }

                  }

                  else // not connected, look for a new connection

                  {

                  //debugSoc.clientfd = accept(debugSoc.lSocket, (struct sockaddr*)&client_addr, &addrlen);

                  //debugSoc.connected = TRUE;

                  //GPIO_write(Board_LED1, Board_LED_ON);

                         debugSocInit();

                         GPIO_write(Board_LED1, Board_LED_ON);

                  //fdClose(debugSoc.lSocket);

                  }

           } // end connected

    } //  end void debugSocTask_RX(void)

     

    void debugSocInit(void)

    {

           int fdOpenResult = 0;

           int errorCount = 0;

           static int connectCounter = 0;

     

           fdOpenResult = fdOpenSession(TaskSelf());

           while(fdOpenResult == 0)

           {

                  Task_sleep(1000);

                  fdOpenResult = fdOpenSession(TaskSelf());

                  printf ("debugSocTask_RX() fdOpenSession failed %d times\n", ++errorCount);

           }

     

           printf ("debugSocTask_RX() fdOpenSession OK\n");

     

        debugSoc.connected = FALSE;

        debugSoc.lSocket = 0;

        debugSoc.lSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);

        if (debugSoc.lSocket < 0)

        {

            System_printf("tcpHandler: socket failed\n");

            Task_exit();

            return;

        }

     

        status = 0;

        memset((char *)&sLocalAddr, 0, sizeof(sLocalAddr));

        sLocalAddr.sin_family = AF_INET;

        sLocalAddr.sin_len = sizeof(sLocalAddr);

        sLocalAddr.sin_addr.s_addr = htonl(INADDR_ANY);

        sLocalAddr.sin_port = htons(5060);

     

        setSocketOptions(debugSoc);

     

        status = bind(debugSoc.lSocket, (struct sockaddr *)&sLocalAddr, sizeof(sLocalAddr));

        if (status < 0)

        {

            System_printf("tcpHandler: bind failed\n");

            fdClose(debugSoc.lSocket);

            Task_exit();

            return;

        }

     

        if (listen(debugSoc.lSocket, NUMTCPWORKERS) != 0)

        {

            System_printf("tcpHandler: listen failed\n");

            fdClose(debugSoc.lSocket);

            Task_exit();

            return;

        }

     

        debugSoc.clientfd = accept(debugSoc.lSocket, (struct sockaddr*)&client_addr, &addrlen);

        debugSoc.connected = TRUE;

        printf("Connected %d times\n", ++connectCounter);

    } // end void debugSocTask_TX()

     

     

    void setSocketOptions(SOCKET_INFO socketID)

    {

        int optval;

        int optlen = sizeof(optval);

        unsigned long rc;

     

        struct linger lingerConfig;

     

        rc = getsockopt(socketID.lSocket, SOL_SOCKET, TCP_NODELAY, &optval, &optlen);

        if(rc)

        {

           printf ("Error, getsockopt 'no delay' error Code %d\n", fdError());

        }

        else

        {

           ;

        }

     

     

        optlen = sizeof(struct linger);

        memset(&lingerConfig, 0, optlen);

     

        rc = getsockopt(socketID.lSocket, SOL_SOCKET, SO_LINGER, &lingerConfig, &optlen);

        if(rc)

        {  // see serrno.h

           printf ("Error getsockopt 'linger' error Code %d\n", fdError());      }

        else

        {

           ;

        }

        optlen = sizeof(struct linger);

        memset(&lingerConfig, 0, optlen);

        lingerConfig.l_onoff = 1;

        lingerConfig.l_linger = 100;

        setsockopt(socketID.lSocket, SOL_SOCKET, SO_LINGER , &lingerConfig, optlen);

     

     

     

     

        rc = getsockopt(socketID.lSocket, SOL_SOCKET, SO_REUSEADDR, &optval, &optlen);

        if(rc)

        {

           printf ("pre getsockopt 'reuseadddr' error Code %d\n", fdError()); 

        }

        else

        {

           ;

        }

     

     

        optval = 1;

        if (setsockopt(socketID.lSocket, SOL_SOCKET, TCP_NODELAY, &optval, optlen) < 0)

        {

            printf ("tcpHandler: setsockopt failed error Code %d\n", fdError());

        }

     

        optval = 1;

        if (setsockopt(socketID.lSocket, SOL_SOCKET, SO_BLOCKING, &optval, optlen) < 0)

        {

            printf ("tcpHandler: setsockopt failed error Code %d\n", fdError());

        }

     

     

        optval = 1;

        if (setsockopt(socketID.lSocket, SOL_SOCKET, SO_REUSEADDR, &optval, optlen) < 0)

        {

            printf ("tcpHandler: setsockopt failed error Code %d\n", fdError());

        }

     

        rc = getsockopt(socketID.lSocket, SOL_SOCKET, TCP_NODELAY, &optval, &optlen);

        if(rc)

        {

           printf ("post getsockopt error Code %d\n", fdError()); 

        }

        else

        {

           ;

        }

     

        rc = getsockopt(socketID.lSocket, SOL_SOCKET, SO_REUSEADDR, &optval, &optlen);

        if(rc)

        {

           printf ("post getsockopt error Code %d\n", fdError()); 

        }

        else

        {

           ;

        }

    } // end void setSo