This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM572x NDK (TCP/IP) Performance Improvement Points

Hi,Expert

The customer's product specification requires 320kB to be transferred in 25msec cycle by TCP/IP communication.
However, it currently takes about 37.5msec. However, it currently takes about 37.5msec, which does not meet the specification.

As a result of capturing and analyzing the transmission data with Wireshark, we have found that the transmission time is about 37.5ms.
We have some questions about this.
The attached document explains in detail, but I will briefly describe the contents of my question.
The following is a brief description of the question.
Please check the attached document for details.


NDK Performance Results.pptx

Q1: Why is the window size of the PC side 64240?
  Why does it send the data in units of 8192 bytes with PSH=1?
  Can you make it every 16384?
  For example, is it possible to make it every 16384 bytes?

   After receiving the ACK signal from the PC, the time until the next data transmission
   starts is very long (about 1.0msec on average).
Q2: What is the purpose of this time?

Q3: Is it possible to change this time to a shorter time (e.g., half) or to a fixed time?
  
  We believe that if this time is not shortened, we will not be able to meet the specifications.
  If there is another way, please let me know.
  
Best Regards,
Hidekazu

  • Hi,

    1. Which SDK version is this ? I cannot comment until I reproduce the problem.
    2. Can you share changes done on top of the RTOS application and the command executed on PC side to execute this ?
    3. Low cycle time can be caused by a variety of issues.
      1. Is your system running another application or servicing an interrupt ?
      2. How did you compile the application ? Have you checked the build flags and made sure that it's optimized for performance ?
      3. What is the priority of the task used to send the TCP messages ? Have you tried increasing it's priority ?
    4. The RTOS applications are only for demo purpose and are not really tuned for performance. I do know that in RT Linux we can achieve quite a high throughput with iperf so the hardware itself is quite capable of meeting your goal.

    Regards

    Vineet

  • Hi,Vineet

    Thank you for Reply.

    1.Which SDK version is this ? I cannot comment until I reproduce the problem.

    Ans1.)

    As described in the NDK Performance Results.ppt attached to this thread.
    I am using the following version
    bios:bios_6_76_03_01
    sdk:processor_sdk_rtos_am57xx_6_03_00_106
    ndk:ndk_3_61_01_01
    pdk:pdk_am57xx_1_0_17


    2.Can you share changes done on top of the RTOS application and the command executed on PC side to execute this ?

    Ans2.)

    The NDK Performance Results.ppt attached to this thread shows the modifications and test summary.

    3.Low cycle time can be caused by a variety of issues.
    Is your system running another application or servicing an interrupt ?
    How did you compile the application ? Have you checked the build flags and made sure that it's optimized for performance ?
    What is the priority of the task used to send the TCP messages ? Have you tried increasing it's priority ?

    Ans3.)

    1. As described in the NDK Performance Results.ppt attached to this thread.
    2. This is a customized versionbased on NIMU_FtpExample_idkAM574x_armExampleproject().

    Major changes:
        1: Program to send 64kBx5 (=320kB) when accessed from PC
                {
                       bytes = send(ioh->socket, (void*)0x90000000, 8192*8, 0);
                       bytes = send(ioh->socket, (void*)0x90000000, 8192*8, 0);
                       bytes = send(ioh->socket, (void*)0x90000000, 8192*8, 0);
          bytes = send(ioh->socket, (void*)0x90000000, 8192*8, 0);
                       bytes = send(ioh->socket, (void*)0x90000000, 8192*8, 0);
                }
                  0x9000 0000-0x9001 0000 (Stores values between 0 and 1)
                 2:Change Send Buffer from 16384 to 65536 ( Set by cfg file )
             nimu_idk.cfg Tcp.transmitBufSize=65536
             Software on PC : Original software created in C#
             Connection form: TMDSEVM572X <-> PC (direct connection)

        We have not made any changes to the priority.


    Best Regards,
    Hidekazu

  • Hi,Vineet

    I have disclosed the required information but have not received any comments.
    Our customers are having a hard time, so can you please provide us with
    information that will help us improve our performance immediately?

    Best Regards,
    Hidekazu

  • Hi Hidekazu,

    Sorry for the delay.

    I am going to run this test and let you know, please give me 2-3 days to get back to you.

    Regards

    Vineet

  • Hi,Vineet

    The scheduled deadline for responses has passed significantly,
    please provide test results and comments as soon as possible?

    Best Regards,
    Hidekazu

  • Hi Hidekazu,

    I am sorry for the delay. I was facing an issue with CPSW ports on my AM572 IDK and I am now running the FTP example application on a new AM574 IDK.

    Here is the command line output I see

    wget user:password@192.168.1.4/test --no-passive
    --2021-08-23 19:55:00--  user:*password*@192.168.1.4/test
               => ‘test.5’
    Connecting to 192.168.1.4:21... connected.
    Logging in as user ... Logged in!
    ==> SYST ... done.    ==> PWD ... done.
    ==> TYPE I ... done.  ==> CWD not needed.
    ==> SIZE test ... done.

    ==> PORT ... done.    ==> RETR test ... done.

    test.5                                                 [ <=>                                                                                                            ]   2.44M  --.-KB/s    in 0.1s    

    2021-08-23 19:55:00 (20.7 MB/s) - ‘test.5’ saved [2560000]

    Here is my Wireshark output

    I am running the test you mentioned in your PPT and I will update this thread tomorrow

    Regards

    Vineet

  • Hi,

    I am unable to understand the latency issue with the FTP application as there are additional tasks and functions in the application which are redundant.

    I am trying to simulate it with the simpler NIMU Basic example, will post an update soon.

    Regards

    Vineet

  • Hi Vineet,

    Thank you for your response.

    Currently, it seems that the cause of the delay is still unclear.
    What were the results of the evaluation with the new different approach?

    Best Regards,
    Hidekazu 

  • Hi,

    Sorry for the delay. 

    I have simulated the scenario with NIMU_BasicExample_idkAM574x_armExampleproject on an AM574IDK. And here are my test results:

    IDK to PC, 327680 Bytes in 6.321ms

    PC to IDK, 327680 Bytes in 4.356ms

    This is much lower than your expected values of 25ms and proves the hardware capability.

    I have attahed the modified {PDK_DIrectory}/packages/ti/transport/ndk/nimu/example/src/main_AM57xx.c for your reference.

    /*
     * Copyright (C) 2017 - 2018 Texas Instruments Incorporated - http://www.ti.com/
     *
     *
     *  Redistribution and use in source and binary forms, with or without
     *  modification, are permitted provided that the following conditions
     *  are met:
     *
     *    Redistributions of source code must retain the above copyright
     *    notice, this list of conditions and the following disclaimer.
     *
     *    Redistributions in binary form must reproduce the above copyright
     *    notice, this list of conditions and the following disclaimer in the
     *    documentation and/or other materials provided with the
     *    distribution.
     *
     *    Neither the name of Texas Instruments Incorporated nor the names of
     *    its contributors may be used to endorse or promote products derived
     *    from this software without specific prior written permission.
     *
     *  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
     *  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
     *  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
     *  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
     *  OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
     *  SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
     *  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
     *  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
     *  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
     *  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
     *  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     *
    */
    
    /* ========================================================================== */
    /*                             Include Files                                  */
    /* ========================================================================== */
    
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <xdc/std.h>
    #include <xdc/runtime/Error.h>
    #include <xdc/runtime/System.h>
    #include <ti/sysbios/BIOS.h>
    #include <ti/sysbios/knl/Task.h>
    
    #include "ti/transport/ndk/nimu/example/ftpApp/modules.h"
    #include <ti/csl/csl_error.h>
    #include <ti/fs/fatfs/ff.h>
    
    
    #include <ti/csl/soc.h>
    #include <ti/csl/cslr_device.h>
    #if defined (SOC_AM574x)
    #include <ti/csl/soc/am574x/src/csl_device_xbar.h>
    #else
    #include <ti/csl/soc/am572x/src/csl_device_xbar.h>
    #endif
    
    #include <ti/ndk/inc/netmain.h>
    #include <ti/ndk/inc/stkmain.h>
    #include "ti/ndk/inc/os/osif.h"
    
    #include <ti/board/board.h>
    
    #include <ti/drv/uart/UART.h>
    #include <ti/drv/uart/UART_stdio.h>
    
    #include <ti/drv/emac/emac_drv.h>
    #include <ti/drv/emac/src/v4/emac_drv_v4.h>
    
    #ifdef NIMU_FTP_APP
    #include <ti/transport/ndk/nimu/example/ftpApp/ftpserver/ftpserver.h>
    #endif
    
    #include <ti/transport/ndk/nimu/example/ftpApp/ftpserver/ftpserver.h>
    
    extern char *LocalIPAddr;
    extern void app_stats(UArg arg0, UArg arg1);
    #ifdef NIMU_DUAL_MAC_MODE
        /* Static IP Address settings for interface 1 */
        char *ipAddr      = "192.168.2.4";
        char *ipMask      = "255.255.255.0";
        char *ipGateway   = "192.168.2.1";
    #endif
    
    /* ========================================================================== */
    /*                             Macros                                  */
    /* ========================================================================== */
    /* Enable the below macro to have prints on the IO Console */
    
    //#define IO_CONSOLE
    
    #ifndef IO_CONSOLE
    #define NIMU_log                UART_printf
    #else
    #define NIMU_log                printf
    #endif
    
    #define MAX_TABLE_ENTRIES   3
    #define SIZE32KB 320*1024
    /* ========================================================================== */
    /*                            Global Variables                                */
    /* ========================================================================== */
    
    /**Task handle for EIP*/
    Task_Handle main_task;
    Task_Handle tcp_task;
    
    static int nimu_device_index = 0U;
    
    
    NIMU_DEVICE_TABLE_ENTRY NIMUDeviceTable[MAX_TABLE_ENTRIES];
    
    void TaskFxn(UArg a0, UArg a1);
    extern int CpswEmacInit (STKEVENT_Handle hEvent);
    void tcp_server_start (UArg a0, UArg a1);
    
    
    /**
     *  \name main
     *  \brief Main Function
     *  \param none
     *  \return none
     *
     */
    int main()
    {
        /* Call board init functions */
        Board_initCfg boardCfg;
        Task_Params taskParams;
    
        boardCfg =  BOARD_INIT_PINMUX_CONFIG |
            BOARD_INIT_MODULE_CLOCK | BOARD_INIT_UART_STDIO;
    #if defined (SOC_AM574x) && defined(NIMU_DUAL_MAC_MODE)
        boardCfg |= BOARD_INIT_ETH_PHY;
    #endif
        Board_init(boardCfg);
    
    #ifdef _TMS320C6X
        CSL_xbarDspIrqConfigure(1,CSL_XBAR_INST_DSP1_IRQ_75, CSL_XBAR_GMAC_SW_IRQ_RX_PULSE);
        CSL_xbarDspIrqConfigure(1,CSL_XBAR_INST_DSP1_IRQ_76, CSL_XBAR_GMAC_SW_IRQ_TX_PULSE);
    #elif defined (__TI_ARM_V7M4__)
        CSL_xbarIpuIrqConfigure(1, CSL_XBAR_INST_IPU1_IRQ_60, CSL_XBAR_GMAC_SW_IRQ_RX_PULSE);
        CSL_xbarIpuIrqConfigure(1, CSL_XBAR_INST_IPU1_IRQ_61, CSL_XBAR_GMAC_SW_IRQ_TX_PULSE);
    #elif defined(__ARM_ARCH_7A__)
        CSL_xbarMpuIrqConfigure(CSL_XBAR_INST_MPU_IRQ_92, CSL_XBAR_GMAC_SW_IRQ_RX_PULSE);
        CSL_xbarMpuIrqConfigure(CSL_XBAR_INST_MPU_IRQ_93, CSL_XBAR_GMAC_SW_IRQ_TX_PULSE);
    #endif
        /* Select RGMII 2 ports GMIIx_SEL = 2 for RGMII*/
        CSL_FINS (((CSL_control_coreRegs *) CSL_MPU_CTRL_MODULE_CORE_CORE_REGISTERS_REGS)->CONTROL_IO_1,
              CONTROL_CORE_CONTROL_IO_1_GMII1_SEL, 2U);
        CSL_FINS (((CSL_control_coreRegs *) CSL_MPU_CTRL_MODULE_CORE_CORE_REGISTERS_REGS)->CONTROL_IO_1,
              CONTROL_CORE_CONTROL_IO_1_GMII2_SEL, 2U);
    
        /*GMAC RESET ISOLATION Enable*/
        CSL_FINS (((CSL_control_coreRegs *) CSL_MPU_CTRL_MODULE_CORE_CORE_REGISTERS_REGS)->CONTROL_IO_2,
              CONTROL_CORE_CONTROL_IO_2_GMAC_RESET_ISOLATION_ENABLE, 0U);
        CSL_FINS (((CSL_control_coreRegs *) CSL_MPU_CTRL_MODULE_CORE_CORE_REGISTERS_REGS)->CONTROL_IO_2,
              CONTROL_CORE_CONTROL_IO_2_GMAC_RESET_ISOLATION_ENABLE, 1U);
    
    #ifdef evmAM572x
        EMAC_HwAttrs_V4 cfg;
        EMAC_socGetInitCfg(0, &cfg);
        cfg.port[0].phy_addr = GMAC_PORT1_ETHERNET_PHY_ADRESS;
        cfg.port[1].phy_addr = GMAC_PORT2_ETHERNET_PHY_ADRESS;
        EMAC_socSetInitCfg(0, &cfg);
    #endif
        Task_Params_init(&taskParams);
        taskParams.priority = 1;
        taskParams.stackSize = 0x1400;
        main_task = Task_create (TaskFxn, &taskParams, NULL);
    
    #if !defined (NIMU_DUAL_MAC_MODE) && !defined (NIMU_FTP_APP)
        Task_Params_init(&taskParams);
        taskParams.priority = 9;
        taskParams.instance->name = "EmacStats";
        Task_create(app_stats, &taskParams, NULL);
    #endif
    
        NIMUDeviceTable[nimu_device_index++].init =  &CpswEmacInit;
    
    #ifdef NIMU_DUAL_MAC_MODE
        NIMUDeviceTable[nimu_device_index++].init =  &CpswEmacInit;
    #endif
        NIMUDeviceTable[nimu_device_index].init =  NULL;
    
        BIOS_start();
    
        return -1;
    }
    
    /**
     *  \name TaskFxn
     *  \brief Task Fxn
     *  \param a0
     *  \param a1
     *  \return none
     *
     */
    void TaskFxn(UArg a0, UArg a1)
    {
        Task_Params params;
        Error_Block eb;
    
    #ifdef NIMU_DUAL_MAC_MODE
        NIMU_log("\n\rSYS/BIOS Ethernet/IP (CPSW) Sample application, EVM IP address I/F 1: %s\n\r", ipAddr);
        NIMU_log("\n\rSYS/BIOS Ethernet/IP (CPSW) Sample application, EVM IP address I/F 2: %s\n\r", LocalIPAddr);
    #else
        NIMU_log("\n\rSYS/BIOS Ethernet/IP (CPSW) Sample application, EVM IP address I/F 1: %s\n\r", LocalIPAddr);
    #ifndef NIMU_FTP_APP
        /* Basic Example completes here */
        UART_printStatus("\n All tests have passed.\n");
    #endif
    #endif
    #ifdef NIMU_FTP_APP
        ftpserver_init();
    #endif
    
        Task_Params_init(&params);
        Error_init(&eb);
    
        params.priority = 5;
        params.stackSize = 4096;
        params.arg0 = 0;
        params.arg1 = 0;
    
        tcp_task = Task_create(tcp_server_start, &params, &eb);
        if (tcp_task == NULL)
        {
            UART_printf("Task : TCP Task, creation failed for ftpserver_thread!, Error %s\n", Error_getMsg(&eb));
            return;
        }
    
    
    }
    
    
    void tcp_server_start (UArg a0, UArg a1){
    
        struct sockaddr_in server_addr, client_addr;
        SOCKET listen_sock, client_sock;
        uint16_t port = 5005;
        int len = sizeof(struct sockaddr_in);
        struct timeval timeout = {10000, 0};
        uint32_t n = 0;
        int32_t opt = 1;
    
        memset(&server_addr, 0, sizeof(struct sockaddr_in));
        server_addr.sin_family = AF_INET;
        server_addr.sin_addr.s_addr = NDK_htonl(INADDR_ANY);
        server_addr.sin_port = NDK_htons(port);
    
        uint8_t * buf = malloc(65536);
        char temp[3] = "ACK";
    
        for(uint32_t i = 0; i<65536; i++) buf[i] = 55;
    
    
        fdOpenSession (TaskSelf ());
    
        if ((listen_sock = socket (AF_INET, SOCK_STREAM, IPPROTO_TCP)) == INVALID_SOCKET)
        {
            return;
        }
    
        UART_printf("\n\n MY PRINT : SOCKET CREATION SUCCESSFULL!!!");
    
        memset(&server_addr, 0, sizeof(struct sockaddr_in));
        server_addr.sin_family = AF_INET;
        server_addr.sin_addr.s_addr = inet_addr("0.0.0.0");
        server_addr.sin_port = NDK_htons(port);
    
        if (bind(listen_sock, (struct sockaddr*)&server_addr, sizeof(struct sockaddr_in)) == SOCKET_ERROR)
        {
            fdClose(listen_sock);
            return;
        }
    
        if (listen(listen_sock, 10) == SOCKET_ERROR)
        {
            fdClose(listen_sock);
            return;
        }
    
        /* Set listen timeout to make watchdog work */
        if (setsockopt(listen_sock, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout)) == SOCKET_ERROR)
        {
            fdClose(listen_sock);
            return;
        }
        UART_printStatus("\n All tests have passed.\n");
    
        for (;;)
        {
            for(uint32_t i = 0; i<65536; i++) buf[i] = 55;
    
            if( (client_sock = accept(listen_sock, (struct sockaddr*)&client_addr, &len)) == INVALID_SOCKET)
            {
                /* Timeout, Wait a short time and try again */
                fdClose (listen_sock);
                Task_sleep (100);
                break;
            }
            setsockopt(client_sock, IPPROTO_TCP, NDK_TCP_NODELAY, &opt, sizeof(opt));
            n = 0;
            while(n<SIZE32KB){
                n += send(client_sock, buf, 65536, 0);
            }
            UART_printf("\n DATA SENT\t\t Total data sent = %dB.\n", n);
            n = 0;
            while(n<SIZE32KB){
                n += recv(client_sock, buf, 65536, 0);
            }
            if ((int)n == -1) UART_printf("Error occured while receiveing, errno : %d", fdError());
            UART_printf("\n DATA RECEIVED\t\t Total data received = %dB.\n", n);
            n = 0;
            n = send(client_sock, temp, strlen(temp), 0);
    
        }
    
        // Close the file session
        fdCloseSession(TaskSelf());
    
    }
    
    /*
     * ======== stackInitHook ========
     * hook called from ti_nkd_config_Global_stackThread() to run user setup code
     */
    void stackInitHook(void* hCfg)
    {
    #ifdef NIMU_DUAL_MAC_MODE
        CI_IPNET NA;
        CI_ROUTE RT;
    
    
        /* Add IP address for interface 1 */
        NA.IPAddr = inet_addr(ipAddr);
        NA.IPMask = inet_addr(ipMask);
        CfgAddEntry(hCfg, CFGTAG_IPNET, 1, 0, sizeof(CI_IPNET),
                    (uint8_t *)&NA, 0);
    
        /*
         * Add gateway for interface 1
         * --> I *think* the below should work but you may need different values ...
         */
        bzero(&RT, sizeof(RT));
        RT.IPDestAddr = 0;
        RT.IPDestMask = 0;
        RT.IPGateAddr = inet_addr(ipGateway);
    
        CfgAddEntry(hCfg, CFGTAG_ROUTE, 0, 0,
                    sizeof(CI_ROUTE), (uint8_t *)&RT, 0);
    #endif
    }
    

    I am also attching the PC client application which I used for testing on my ubuntu 18.04.

    #include <sys/socket.h>
    #include <sys/types.h>
    #include <sys/time.h>
    #include <netinet/in.h>
    #include <netdb.h>
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <errno.h>
    #include <arpa/inet.h> 
    
    #define SIZE32KB 320*1024
    
    int main(int argc, char *argv[])
    {
        int sockfd = 0;
        char recvBuff[65536];
        struct sockaddr_in serv_addr; 
        uint32_t n = 0;
        char cmp[3] = "ACK";
        struct timeval start, stop;
    
        if(argc != 2)
        {
            printf("\n Usage: %s <ip of server> \n",argv[0]);
            return 1;
        } 
    
        memset(recvBuff, '0',sizeof(recvBuff));
        if((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
        {
            printf("\n Error : Could not create socket \n");
            return 1;
        } 
    
        memset(&serv_addr, '0', sizeof(serv_addr)); 
    
        serv_addr.sin_family = AF_INET;
        serv_addr.sin_port = htons(5005); 
        serv_addr.sin_addr.s_addr = inet_addr(argv[1]);
    
    
        if( connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0)
        {
           printf("\n Error : Connect Failed \n");
           return 1;
        } 
        n = 0;    
        gettimeofday(&start, 0);
        while(n < SIZE32KB){
            n += recv(sockfd, recvBuff, 65536, 0);
        }
        gettimeofday(&stop, 0);
        printf("DATA RECEIVED IS %d Bytes\n", n);
        printf("Time to receive data is %ldus \n", stop.tv_usec-start.tv_usec);
        n = 0;
        for(int i = 0; i<65536; i++) recvBuff[i] = 57;
    
        gettimeofday(&start, 0);
        while(n < SIZE32KB){
            n += send(sockfd, recvBuff, 65536, 0);
        }
        gettimeofday(&stop, 0);
        printf("DATA SENT IS %d Bytes\n", n);
        printf("Time to send data is %ldus \n", stop.tv_usec-start.tv_usec);
        
        n = recv(sockfd, recvBuff, 65536,0);
        recvBuff[n]=0;
        if(strcmp(cmp, recvBuff)) {
            printf("EVERYTHING SUCCESSFULL!!!\n");
        }
    
        if(n < 0)
        {
            printf("\n Read error \n");
        } 
    
        return 0;
    };

    I am also attching the wireshark log for the test.

    https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/trace2_5F00_proper_5F00_successfull.pcapng

    Regards,

    Tanmay