Network TX speed in TDAx processors

Yes. NDK performance on M4 is limited by CPU speed due to CPU copy inside the NDK.
To overcome this performance issue, we have developed a customized protocol called TFDTP based on UDP in Vision SDK.
It is documented in VisionSDK_UserGuide_NetworkTools.pdf and how to use it.
With TFDTP, you should be able to send 720p YUV420 SP at 30 fps on TDA3.
If you are saving the video frames to hard drive, we have noticed that Windows filesystem has much higher latency than Linux PC.

On TDA2, NDK can achieve sufficient performance since A15 is much faster.

Regards,
Stanley

Hi Stanley,

Thanks for your response. Is there any use-case developed for "ISS_capture -> ISS_m2msimcop ->network_tx" ? In other words, we want to capture OV10640 image raw (1280x720) at 20 fps, use ISS for WDR processing and then transmit the YUV420 continously over Ethernet from the TDA3x ?

In the example usecases, there is "ISS_capture->ISS_m2msimcop -> display". I am trying to add network link to this chain. Any suggestions on where to start hacking this usecase to achieve our goal?

Thanks,
Keerthi

Hi Keerthi,

You can reuse ISS_Capture->ISS_M2MSIMCOP->Display. You can either replace Display Link with NetworkTx Link or add Dup and NetworkTx Links if you want to see the camera output on the display.

IssCapture -> IssM2mIsp -> IssM2mSimcop -> Dup -> Display_Video
Dup -> NetworkTx (IPU1_1)

You also need to add the following to the cfg.mk.
NDK_PROC_TO_USE=ipu1_1
NSP_TFDTP_INCLUDE=yes

In the use case, you can refer to ~/usecases/network_rx_tx/chains_networkTxCapture.c to see how to configure NetworkTx Link.

Regards,
Stanley

Hi Stanley,

I managed to develop the usecase as below on TDA3x Starter Kit :

UseCase: chains_issIspSimcop_NetworkTx

IssCapture -> IssM2mIsp -> IssM2mSimcop -> Dup -> Display_Video

IssM2mIsp -> Alg_IssAewb

IssM2mIsp -> IssM2mResizer

IssM2mResizer -> Display_VideoRszB

Dup -> Null (IPU1_1)

GrpxSrc -> Display_Grpx

In the network receive tool Only thing I have changed is frame size since I want to capture 1280x720 image with YUV420.

params.frameSize = 1280*720*1.5; // From (1920*1080*2)

When I run the network receive tool on Linux Ubuntu 14.04 with the command below, I get the errors shown below. Not sure if the problem is on the PC side or if the Target isn't configured correctly for TFDTP. I followed the userguide on network tools and I think I have done all the steps necessary.

./network_rx.out -host_ip 192.168.1.1 --target_ip 192.168.1.200 --port 29172 --usetfdtp --verbose --files cam_capture.yuv

# Connecting to server 192.168.1.200:29172 ...
# NETWORK: Connected to Server (192.168.1.200:29172)!!!
# Using TFDTP for Receive
TFDTP: Throughput = 500, frameSize = 1382400
# NETWORK: TFDTP_open() Done (192.168.1.200:29174)!!!
Failed to get Adapter MAC address
# ERROR: NETWORK: TFDTP_setChConfig() Failed (-1)!!!
# NETWORK: Sent TFDTP channel params!!!
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# Network Rx ERROR: TFDTP read failed after retry
ERROR : Network read error
# INFO: DATA: CH0: Frame0: 1382400 bytes
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete

Even with the above errors, the receive tool can write to the file correctly (correct number of bytes), but not sure if the data values are correct. The Target side doesn't print any errors and the PC side tool doesn't crash or stop. 

Any inputs/ideas?

Thanks,

Keerthi

Hi Keerthi,

Are you sure your PC IP address is "192.168.1.1" as set in command line "-host_ip 192.168.1.1"?

You shouldn't see the below error messages.

Failed to get Adapter MAC address
# ERROR: NETWORK: TFDTP_setChConfig() Failed (-1)!!!

How did you connect Ethernet cable between PC and Starter Kit? thru a router, switch, or directly?
Did you build the code with static IP or DHCP enabled?

Regards,
Stanley

Hi Stanley,

Yes, the host ethernet IP address is 192.168.1.1. Output of "ifconfig" is below:

eth2      Link encap:Ethernet  HWaddr 88:d7:f6:d5:08:7c  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::8ad7:f6ff:fed5:87c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:5788725745 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1020873893 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:8762283639837 (8.7 TB)  TX bytes:67836836899 (67.8 GB)

The connection is a direct one between the host PC and target, without any switches or routers. I can ping the target without issues. What do you mean by 

"Did you build the code with static IP or DHCP enabled?"

Is it the target code or the host receiver code? I know that host ethernet interface is assigned a static IP. 

Thanks,

Keerthi

In ~\vision_sdk\links_fw\src\rtos\bios_app_common\tda3xx\cfg\NDK_config.cfg,
Is "enableStaticIP" set to 0 or 1?

Stanley,

I think the MAC address detection failure was because of my mistake. The command line argument I was passing just had "-host_ip" instead of "--host_ip". So, the IP address it parsed from the command line was incorrect. Now I have the following set of errors printing:

# Connecting to server 192.168.1.200:29172 ...
# NETWORK: Connected to Server (192.168.1.200:29172)!!!
# Using TFDTP for Receive 
TFDTP: Throughput = 500, frameSize = 1382400
# NETWORK: TFDTP_open() Done (192.168.1.200:29174)!!!
TFDTP: Adapter Name: eth2
TFDTP: MAC Address: FFFFFF88-FFFFFFD7-FFFFFFF6-FFFFFFD5-08-7C
TFDTP: MAC address Size 16
TFDTP: Send TFDTP Channel Config Params, IP 0xc0a80101
# NETWORK: Sent TFDTP channel params!!!
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# Network Rx ERROR: TFDTP read failed after retry
ERROR : Network read error 
# INFO: DATA: CH0: Frame0: 1382400 bytes
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# ERROR: TFDTP_recv failed/not complete
# Network Rx ERROR: TFDTP read failed after retry
ERROR : Network read error 

The "enableStaticIP" is set to 1.

It seems like the receiver is using both 29172 and 29174 ports. Is this expected?

It is expected.
If you let the test continue to run, do you see always "Network read error"?
You can also rebuild network_rx with "#define DEBUG_PRINT 1" in ~\vision_sdk\apps\tools\network_tools\common\src\tfdtp_api.c to get more info about the error from the test.
Sometimes you may see error at the beginning due to packet drop from network but it will stabilize later.
The data save in the file is the correctly received output frames so you can still use the data.

Regards,
Stanley

The network read program continues but it prints errors always. I debugged a little more and found out that in the following snippet in "TFDTP_recv()" function, the "byte_read" is always -1 and it seems like PC side is not able to read from the socket correctly. It retries for 2 more times and still doesn't receive the data. However, the caller functions higher up doesn't catch this and still writes zeros to the file. It file gets bigger in size, but they are all zeros, which I verified by reading it in binary mode.

/* Blocking call until data is received or RX timeout reached */

       byte_read = recvfrom(pObj->udpSocket, (char *)&rxPacket, recvSize, 0,(struct sockaddr *) &senderAddr, &senderLen);

       if (byte_read <= 0)

       {

           status = -1;

           done = 1;

           printf("# STATUS: -1 \n Byte Read =%d\n", byte_read);

       }

BTW, I just noticed that the eth2 MTU size is set to 9000 on PC side. Could you try changing that to 1500?

I changed ethernet MTU size to 1500 and that didn't help. I used wireshark to to see if there is traffic. I could see that one TCP packet goes from PC to Target and then nothing happens.

I debugged a little more and found out that in the snippet below in function "NullLink_networkTxSendData()", we are getting a failure with ack:

if(status==SYSTEM_LINK_STATUS_SOK)

       {

           status = NullLink_networkTxWriteHeader(pObj, pNetTxObj, &cmdHeader);

#if (defined(NSP_TFDTP_INCLUDE))

           if (pObj->createArgs.dumpDataType == NULL_LINK_COPY_TYPE_NETWORK_TFDTP)

           {

               if(status==SYSTEM_LINK_STATUS_SOK)

               {

                   ack = Network_tfdtpWaitAck(&pNetTxObj->sockObj);

                   if (ack != TFDTP_TRANSMIT_OK)

                   {

                       status = SYSTEM_LINK_STATUS_EFAIL;

                       Vps_printf(" NULL:Status = SYSTEM_LINK_STATUS_EFAIL\n");

                   }

               }

           }

#endif

       }

Does this mean the issue is with the target or the host?

In your use case code on TDA3, did you call ChainsCommon_networkTxSetMode()?

Yes, "ChainsCommon_networkTxSetMode()" is called already.

Weirdly enough, the host side started receiving frames now, without any modifications to the target code, and minor debug print statements enabled in the host receiver code. I tried to make it break again so that I can debug it, but I haven't changed anything major, so not sure what was the problem. To double check, I restarted my PC as well as the target processor to see if it breaks, but it seems to work to a decent extent.

Although, there are sometimes frame drops seen and I get messages such as this (for around 3-4% of the frames) and the application still continues to receive the data further:

TFDTP: Frame [77]: No. Packets received[902] for Total [955]

# ERROR: TFDTP_recv failed/not complete

I tried with both MTU = 1500 and MTU = 9000. In both cases, around 3-4% of frames drop.

Any ideas ?

For our application, we actually don't want the data to be retransmited since we want the video frames to be received in real-time. I ran the application for 64 hours and we had 0.35% of frames drop when sending 1280x720 at 30 FPS. I will try and debug more later. For now, the network works to a satisfactory extent. So I am closing this thread. Thanks for your help, Stanley.

- Keerthi