NDK winapps tools send and recv throughput issue

Prasad Jondhale

Hello,

I am using NDK send and recv utilities from winapps for Ethernet benchmarking.

I am seeing the throughput shown in the log of these tools is not showing max. supported by driver/HW as CPU is not fully loaded (~10%). I have set he update counter to very high value.

I tried changing the values of NDK receive and transmit buffers in .cfg file still see no big improvement.

NdkConfigGlobal.pktNumFrameBufs = 256*8;
NdkConfigGlobal.memRawPageCount = 16*8;
/* change the size of our receive buffers */
Tcp.transmitBufSize = 128*1024;
Tcp.receiveBufSize = 128*1024;
Tcp.receiveBufLimit = 128*1024;
Udp.receiveBufSize = 128*1024;

NDK version 2.24.2.31.

Please suggest way so i can send/recv with full supported bandwidth so as to benchmark my driver correctly. Thanks.

over 9 years ago

0 Steven Connell over 9 years ago

TI__Mastermind 45025 points

Prasad,

What throughput values are you seeing?

What h/w platform are you on? And which NIMU driver (NSP)?

What is your network topology for your tests?

Steve

0 Prasad Jondhale over 9 years ago in reply to Steven Connell

TI__Mastermind 27743 points

Hello Steve,

I am using TDA2x EVM and VSDK NSP.

With above mentioned packet/buffer configuration I am seeing 480Mbps throughput for send with CPU load ~50%. Also for recv is not exceeding 80Mbps (which is critical for us)

0 Steven Connell over 9 years ago in reply to Prasad Jondhale

TI__Mastermind 45025 points

Hi Prasad,

Which core/speed is the NDK running on on the TDA2x again?

Are you using no copy sockets?

Have you tried increasing the number of TX/RX frames in the driver? I.e. I think this is the code that governs that (ti/nsp/drv/ndk2nsp/ndk2nsp.h, tag "release/4_14_00_00"):

#if defined (TFDTP_STACK_ENABLED)
#define NDK2NSP_RX_PACKETS   (128U)
#else
#define NDK2NSP_RX_PACKETS   (64U)
#endif
#define NDK2NSP_TX_PACKETS   (64U)
#define NDK_RX_PACKETS (NDK2NSP_RX_PACKETS * (uint32_t)2U)

(This looks like a lot already but it could be a bottleneck)

Steve

0 Steven Connell over 9 years ago in reply to Steven Connell

TI__Mastermind 45025 points

Also, are where are the NDK buffers located? Are they in cache (or is cache available)?

0 Prasad Jondhale over 9 years ago in reply to Steven Connell

TI__Mastermind 27743 points

Hello Steve,

NDK is running on A15 (configured at 750MHz).

As I am using ndk send/recv utility and ndk client application, it guess i am using predefine daemon server dtask_tcp_datasrv. Not sure if it uses nocopy sockets.

NDK buffers are in DDR for which cache is enabled.

As CPU load in only 7% and there are no packet drops, cache and no. of rx/tx frames should not matter. I believe we will have to configure TCP daemon to increase buffers/window size etc.

0 Steven Connell over 9 years ago in reply to Prasad Jondhale

TI__Mastermind 45025 points

Prasad Jondhale said:
As CPU load in only 7% and there are no packet drops, cache and no. of rx/tx frames should not matter.

So just to confirm, you are not seeing anything interesting in this struct? In particular, the bolded fields?

typedef struct NDK2NSP_STATS_
{
    NDK2NSP_PacketTaskStats rxStats;
#if defined(NSP_INSTRUMENTATION_ENABLED)
    uint32_t rxReadyPBMPacketEnqueue;
    uint32_t rxUnderFlowCount;
    uint32_t rxReadyPBMPacketDequeue;
#if defined TFDTP_STACK_ENABLED
    uint32_t rxReadyTFDTPPacketEnqueue;
    uint32_t rxReadyTFDTPPacketDequeue;
    uint32_t rxFreeTFDTPPBMPacketEnqueue;
    uint32_t rxFreeTFDTPPBMPacketDequeue;
#else
    /* If TFDTP stack is enabled, we use TFDTP RX Packet Q for packet allocation */
    uint32_t rxFreePBMPacketEnqueue;
    uint32_t rxFreePBMPacketDequeue;
#endif
    uint32_t ndkStackNotifyCount;
#endif

    NDK2NSP_PacketTaskStats txStats;
#if defined(NSP_INSTRUMENTATION_ENABLED)
    uint32_t txReadyPBMPacketEnqueue;
    uint32_t txReadyPBMPacketDequeue;
    uint32_t txFreeCPDMAPacketEnqueue;
    uint32_t txFreeCPDMAPacketDequeue;
    uint32_t txDroppedPacketCount;
#endif
    uint32_t gCpuLoad[HISTORY_CNT];
    uint32_t gHwiLoad[HISTORY_CNT];
}
NDK2NSP_Stats;

Prasad Jondhale said:
I believe we will have to configure TCP daemon to increase buffers/window size etc.

Have you taken a wireshark caputure? Are you seeing the window go to zero? Can you please attach a Wireshark capture?

Steve

0 Steven Connell over 9 years ago in reply to Steven Connell

TI__Mastermind 45025 points

(I'll try running this on my side, too)

0 Prasad Jondhale over 9 years ago in reply to Steven Connell

TI__Mastermind 27743 points

Hello Steve,

The contents of bold fields(packet underflow and drop count) are zero. I am allocating enough PBM buffers during initialization.

Also the HW receive underflow register is zero so i guess no packet drops are happening from HW too.

You can run Client application from NSP package on target a15 core and NDK recv utility on PC. Let me know any set up details are needed. Thanks.

Processors

Processors forum

NDK winapps tools send and recv throughput issue