NDK. Problem with closing TCP connection from Dsp side.

Du Fu

Other Parts Discussed in Thread: OMAPL138

Hi all,

We plan to use NDK for data communication between Dsp and host PC. In some cases TCP connection can be closed from Dsp side.

How to do this in correct way?

We slightly changed example application code and tested it on EVM 6457L.

There is no any problem when connection is closed from host PC, but when Dsp finish connection, closing socket using fdClose(s) NDK call, connection doesn't closed, but preserved in FINWAIT2 state, without notifying host, so it continue sending data through connection.

This connection state kept even after processing connection Dsp task ended with fdCloseSession(TaskSelf()) NDK call before exiting task.

Calling explicitly shutdown(stcp, SHUT_RDWR) doesn't help also, as it should be called inside fdClose.

Connection closed actually only after we close it from host PC side.

We tried to do more deep investigation rebuilding NDK with debug information, but it is quite difficult to debug it as debugger intervention break some timing inside TCP stack.

Configuration, we are using now:

TMDS EVM 6457L

NDK : 2.20.6.35

CCS : 5.1.1.00033

Thanks in advance.

PS. We used for testing on host PC very simple application derived from

http://msdn.microsoft.com/en-us/library/windows/desktop/ms737591%28v=vs.85%29.aspx

over 13 years ago

0 Tom Kopriva over 13 years ago

TI__Mastermind 20480 points

Hi Dmitry,

I have to check back with our NDK expert. Which example did you change so I can get up to speed in the mean time? Did you get the example from the NSP?

0 Du Fu over 13 years ago in reply to Tom Kopriva

Intellectual 380 points

Hi Tom.

We faced with this problem first, when we make some changes in hpdspua project source code, which was the part of demo applications shipped with einfochips EVM6457L.

In order to do more deep investigation of this problem we rebuilt ndk_evm6748_coff_client demo project from NSP, adopting it to EVM6457L platform, we used for testing.

Unfortunately, NSP (1.10.0.3) doesn't provide support nether for EVM6457L, nether EVM6678L.

I attached some files, including changed sources, and test host application we run on Windows PC, here.

Please have a look into newservers.c file, which we included into this projects. It was originated from ndk/src/tools/servers/.

We just changed function int dtask_tcp_nullsrv( SOCKET s, UINT32 unused ), removing endless loop and close socket with fdClose( s );

Hope it helps.

ndk_client_files.7z

0 Steven Connell over 13 years ago in reply to Du Fu

TI__Mastermind 45025 points

Hi Dmitry,

It sounds like you are using the MCSDK product, is that correct? If so, this product actually does ship with the NDK client example, already updated for the 6457l for you. You should have it in your installation. For example, it's here on my machine:

C:\Program Files\Texas Instruments\mcsdk_1_00_00_08\packages\ti\platform\nimu\test\evm6457l\client

Can you try out that example? Do you experience the same issue?

Thanks,

Steve

0 Du Fu over 13 years ago in reply to Steven Connell

Intellectual 380 points

Dear Steve.

Yes, I have MSDK 1.0.0.08 installed.

Thank you for point me to the updated NDK client example.

I rebuild this NDK client demo project (it was located at /home/ti/mcsdk_1_00_00_08/packages/ti/platform/nimu/test/evm6457l/client on my computer).

I've got several warning about different versions while doing that.

I've got also several linker warnings like:

warning: library
"/home/ti/pdk_c64x_1_00_00_06/packages/ti/csl/C6457/csl_c6457/lib/csl_c6457.lib" contains ELF object files which are incompatible with the TI-COFF output file. Ensure you are using the proper library.

I'm always getting this while building any project with NDK or CSL. Probably I have to rebuild all libraries, but can't do this time due to some CCS issue (I reported it here: http://e2e.ti.com/support/dsp/tms320c6000_high_performance_dsps/f/112/t/176360.aspx)

My build system configuration was:

CCS 5.1.1.00033

CGT 7.3.3

MSDK 1.0.0.08

C64x PDK 1.0.0.06

NDK 2.20.6.35

I was not able to load and start output client_evm6457l.out I've got first. CCS window always closed silently right after loading it using onboard XDS emulator without any kind of message about error or even warning.

Fortunately, this was fixed after I changed build profile in CCS RTSC settings from whole_program to debug. Program was loaded OK and worked fine.

After that I included newservers.c file with changes, I'd described in first post (you can find source here http://e2e.ti.com/cfs-file.ashx/__key/CommunityServer-Components-PostAttachments/00-00-63-59-20/ndk_5F00_client_5F00_files.7z) into project, rebuilt it, loaded and started.

The identical issue, as I reported still exist.

Hope it helps.

Thank you.

Dmitry

0 Steven Connell over 13 years ago

TI__Mastermind 45025 points

A couple of things after re-reading the description of your problem...

Dmitry Froloff said:
There is no any problem when connection is closed from host PC, but when Dsp finish connection, closing socket using fdClose(s) NDK call, connection doesn't closed, but preserved in FINWAIT2 state, without notifying host, so it continue sending data through connection.

I found the following description at http://www.tcpipguide.com/free/t_TCPOperationalOverviewandtheTCPFiniteStateMachineF-2.htm

"FIN-WAIT-2 A device in this state has received an ACK for its request to terminate the connection and is now waiting for a matching FIN from the other device."

So, it seems that the NDK stack at this point has sent a termination request to the PC, and has at this point also received an ACK from the PC. It is now waiting for the PC to send FIN.

Dmitry Froloff said:
Connection closed actually only after we close it from host PC side.

After this is done, the connection is closed on the NDK side, correct? So it seems that when you closed it from the PC side, then perhaps this is when the FIN was sent that the NDK was waiting for.

Steve

0 Du Fu over 13 years ago in reply to Steven Connell

Intellectual 380 points

Hi Steve.

Thank you for getting some time to have a look into this problem.

I made more deep investigation, using old OS/2 box, which allows to get more clear picture (no random LAN activity, like on Windows) and then repeat the same on Windows host also.

I check first, how those hosts perform in same situation. What I found that unlike NDK, which send <FIN>, OS/2 and Windows both send <RST> when closesocket() called on server side. It is not supposed to receive any data on the socket after closesocket(), the only processing which TCP stack should perform gracefully is to send the rest outgoing data from it's internal buffer, but this is not our case, as we only receive data from external host.

This could be NDK TCP stack implementation specific, (I didn't find strict requirement regarding this in RFC), but definitely NDK should respond with <RST> on further incoming data in this connection, but I didn't observe that in real communication. It continue <ACK> incoming data instead.

I also tried to play with SO_LINGER socket option.

According to the SO_LINGER specification (W. Richard Stevens, "UNIX Network Programming" vol.1) is

when linger->l_onoff is non-zero and linger->l_linger is zero:

a close() should returns immediately. The underlying stack discards any unsent data, and, in the case of connection-oriented protocols such as TCP, sends a <RST> (reset) to the peer (this is termed a hard or abortive close). All subsequent attempts by the peer's application to read()/recv() data will result in an ECONNRESET.

But again I didn't observe <RST> flag triggered in NDK answers, it <ACK> all data sent after fdClose(). Connection again freeze in the FINWAIT2 state.

I can provide corresponding TCP trace logs if necessary.

Thanks in advance.

0 Steven Connell over 13 years ago in reply to Du Fu

TI__Mastermind 45025 points

Dmitry,

Ok, I'm trying to reproduce all of this here on my end. Here's where I'm at with it...

My set up is as follows:

A. NDK client example running on the OMAPL138 (IP address = 146.252.161.157)

B. Linux machine running NDK test application "echoc.x86U" (probably "echoc.exe" in the NDK release you have). (IP address = 146.252.160.81)

C. PC with Wireshark and OMAPL138 board hooked into a hub which also connects to the LAN (and Linux machine)

D. Wireshark capturning packets on my PC via hub connection

If I run the echoc program, it sends data to the OMAPL138 board and the echo server running within the NDK echos that same data back. If I Control-C the echoc program to kill the app, I can cause an "abortive release" and see that a RST is sent from the Linux machine:

However, if I change the echoc.c file to just send and receive the echo data once, then to call the closesocket() API. With this case, I don't see any RST; the connection appears to end as an "orderly release":

Now, if I make the NDK exit the echo server after one transfer of data, then I see:

Is this the same that you're seeing on your side?

(also ... I'll continue on this tomorrow...)

Steve

0 Steven Connell over 13 years ago in reply to Steven Connell

TI__Mastermind 45025 points

Dmitry,

I am able to cause the NDK to send a RST upon connection close (via call to fdClose(s) ). I did this by enabling linger and setting the time to 0:

My setup is the same as mentioned above (regarding IP addresses mentioned and machines used).

I have modified the Client example's echo server (function dtask_tcp_echo() ) which is a daemon task that runs when the echoc.exe program is run to connect to the target DSP.

To set the linger option, I ran the following code within the dtask_tcp_echo() function:

int dtask_tcp_echo( SOCKET s, UINT32 unused )
{

...

struct linger lingerOpts;

    i = sizeof(struct linger);
    lingerOpts.l_onoff = 1;
    lingerOpts.l_linger = 0;
    setsockopt(s, SOL_SOCKET, SO_LINGER, &lingerOpts, i);

...

    for(;;)
    {
        i = (int)recvnc( s, (void **)&pBuf, 0, &hBuffer );

        // If we read data, echo it back
        if( i > 0 )
        {
            if( send( s, pBuf, i, 0 ) < 0 )
                break;
            recvncfree( hBuffer );
        }
        // If the connection got an error or disconnect, close
        else
            break;

        goto done; // <-- force call to close the socket
    }

done:
    fdClose( s );

    // Return "0" since we closed the socket
    return(0);
}

As shown in the Wireshark output, this causes the connection to terminate with a RST, as you previously requested. Can you try this on your side?

(I've attached the updated file "newservers.c" which contains the full code changes listed above)

0 Du Fu over 13 years ago in reply to Steven Connell

Intellectual 380 points

Hi Steve.

Thank you for looking more deep inside.

It seems that there is no problem when we have data exchange with NDK DSP, e.g. we send some data to DSP, then receive some data from DSP.

In this case we'll get zero return value from recv() socket call on the host PC when NDK DSP close socket from it's side, so we can close socket peacefully, which cause expected <FIN> exchange procedure.

The problem, I faced, was when NDK DSP only received data and didn't send anything, or sent data very rare.

If I simulate this case with some server application, which only receive data, running on Windows or OS/2, then this server PC send <RST>, when the server application close socket on it's side and this cause client application on another side receive error code from next send() call. While in the same scenario, when server is running on NDK, DSP send <FIN>, when DSP server application close socket, while client application on another side running on Windows or OS/2 (didn't check on Linux yet), don't get any notice that connection is closed by the partner, so the host can continue sending data to the DSP, while NDK socket will be frozen in FINWAIT2 state.

So the only reliable way to close NDK socket on the DSP side, irrespectively to the other party behavior, is to activate SO_LINGER option.

Thank you.

Processors

Processors forum

NDK. Problem with closing TCP connection from Dsp side.