RTOS/66AK2G12: NDK initialization issue

Thank you so much Michael.

Hello Champs,

Please let me know if the procedures are not clear or you don't see the same issue at your end.

Many thanks,
YW

Hi,

I did a very simple test of ping 100 packets at 1 second interval. The test is done after the C66x core fully running. I tried several times and I saw 5-10% packet loss rate. Let me know if you saw the same before moving to UDP test (and sending packet before/after NDK initialization).

Regards, Eric

Hello Eric,

Our ping test shows 0% packet loss rate unless we send the packets too early, i.e., before NDK initialization is done, which we don't know how to determine. In the example we consider NDK is fully initialized after NetworkIPAddr() is called because NetworkOpen() will not be called unless the first packet arrives.

If you agree NDK is fully initialized after NetworkIPAddr() is called, then please try step 7 with a small modification: send packets right after "Network Added: ..." is showed. The result will be the same.

To be short, we are looking for a flag/status/function that tell us when we can start sending packets to DSP safely.

Thanks,
YW

Hi,

Thanks for the info! Then we have different issues. The periodical ping failure on my setup prevented me from reproducing your issue. I tried ping continuously from the host PC to the EVM and tried to correlate when the EVM started to respond with different NDK code stages. Then I found ping failure on my setup, I need to fix this first.

Regards, Eric

Hi,

As you used the C66x NIMU example, I tried the same with both PDK 5.0 and PDK 5.1 releases and tried two K2G EVM boards, I don’t have ping worked reliably for a basic setup. I have ping loss if I ping dozens of packets at 1 packet/second rate.

I used the same setup and run same test on A15 core (NIMU_BasicExample_evmK2G_armExampleproject). I don’t have any issue to ping it continuously.

So my rest comment are based on A15 NIMU test, using a tool called Ostinato to send UDP packets (with controlled MAC, IP, UDP and packet rate) to K2G EVM, before the code recvncfrom() or after it. Or, I sent the packet to EVM directly even before it starts to run. My goal is to see if packets received before NDK setup can block the packet reply. I didn’t see any issue on A15, that is, I saw packet replied once the NDK runs. Imaging a K2G processor connected to a live network, you can’t control when the remote side sends packet to you, even there are many broadcast packets or multicast packets reached the K2G EMAC port. It doesn’t make sense that K2G NDK will die if it received packets before its NDK ready. For the C66x I can’t run the same Ostinato test because the above ping failure.

I would think this is an issue on C66x side and there is no flag to tell the remote side “I am ready, please start to send packet to me”. Can you confirm that K2G ARM NIMU test worked well regardless when it started to receive packet? If yes, then our focus will be on debug the C66x side.

Regards, Eric

Hi Eric,

I just tested ARM using the same UDP hello world example. It behaves the same as DSP -- 0 % packet loss rate over 300+ packets, and same packet blocking problem if I start sending packets immediately after "Network Added: If-......." is displayed. You said you can still get echo packet even you send packets to EVM directly before it starts to run. Did you "continuously" send UDP packets from desktop at an 0.5 sec interval or less?

What network configuration did you use for DSP/ARM? We changed the subnet to 192.168.192.0 to make sure the transmission is not messed up by the IP Table in our system. We were able to do that because our desktop has two Ethernet ports and we use one dedicated for the K2G EVM. If you only have one Ethernet port, you can try the ping test when K2G EVM is shutdown. If you can receive any ping response then your earlier ping test with DSP was affected by others. Following is our configuration in helloWorld_k2g.c:

char *HostName = "tidsp";
char *LocalIPAddr = "192.168.192.11";
char *LocalIPMask = "255.255.255.0"; // Not used when using DHCP
char *GatewayIP = "192.168.192.1"; // Not used when using DHCP
char *DomainName = "demo.net"; // Not used when using DHCP
char *DNSServer = "0.0.0.0"; // Used when set to anything but zero

You mentioned "Imaging a K2G processor connected to a live network, you can’t control when the remote side sends packet to you, even there are many broadcast packets or multicast packets reached the K2G EMAC port. It doesn’t make sense that K2G NDK will die if it received packets before its NDK ready." This is what we are worrying about, although we haven't received our final product to confirm that. However, our network is actually used internally (i.e. inside a product) and not connected to the outside world. This gives us a chance to control all kinds of broadcast/multicast packets. But we need something like a flag to tell the rest of the system "I am ready, please start to send packet to me”. Does that make sense to you?

Thanks,
YW

Hi YW,

Then we need to find out the difference. My setup is:
- PC has two Ethernet cards. one card is connected to network, the other is directly connected to the K2G EVM. This card has a static IP 192.168.1.11
- The EVM I also used a static IP. In helloWorld_k2g.c line 110: bool useDhcp = FALSE;
char *HostName = "tidsp";
char *LocalIPAddr = "192.168.1.4";
char *LocalIPMask = "255.255.255.0"; // Not used when using DHCP
char *GatewayIP = "192.168.1.1"; // Not used when using DHCP
char *DomainName = "demo.net"; // Not used when using DHCP
char *DNSServer = "0.0.0.0"; // Used when set to anything but zero
So the EVM has a static IP 192.168.1.4.

The PC test tool sends UDP packets to port 7 of EVM continuously at 2 packets/second interval. I didn't add any debug print before/after recvncfrom(). I started sending packet before EVM starts. I didn't see any problem for the response.

Can you do a simple test:
- Power off K2G EVM
- From your PC ping the EVM continuously with -t (this is 1 packet/second, ICMP doesn't go through UDP port 7)
- Remove your debugging message before and after recvncfrom()
- Used static IP, build
- Power on EVM, load and run

Will you see the EVM starts to respond ping? Or never respond?

Regards, Eric

Hi Eric,

I believe you are still using ARM so I also tested with ARM following your procedures. I reverted all code changes in the example and then only modified the following three lines:
bool useDhcp = FALSE;
char *LocalIPAddr = "192.168.192.11";
char *GatewayIP = "192.168.192.1"; // Not used when using DHCP
Also, during the ping testing, I started ping after I see the "Network Added: ..." message. The result is that EVM will respond ping with 0% packet loss rate AFTER missing tens of ping packets in the beginning. ("Request timed out.")
When I ping EVM with
ping 192.168.192.11 -t
then around 12 packets were missing in the beginning.
However when I ping EVM with
ping 192.168.192.11 -t -w 1
then around 42 packets were missing in the beginning. Please note that although the above command use 1ms timeout, the real timeout is still around 1 second.
When I send any other packets to 192.168.192.11 along with the ping (doens't matter what's the destination port) in a much shorter interval (for example 100ms), then the EVM never respond ping.

Besides, 42 missing packets is about 40 seconds wait which I won't consider acceptable for a system startup.

Thanks,
YW

Hi,

Yes, I used ARM for testing. I thought you reported two issues:

1) can you use the "Network Added: ..." message as the indication as the network is ready so you started to ping and expect immediate response.====>No, this is just a message printed from the application level. The PHY negotiation needs to be complicate before you can send packets, so it is safe you wait for a while.

2) Ping no response when you have another traffic in background.=========>No, I can't reproduce this issue. Attached is a wireshark capture, 192.168.1.11 is my PC and 192.16.1.4 is my EVM running NDK. Ping is every 1 second (length is 74 bytes), the background traffic is 10 packets/second, length is 96 bytes. There is no any loss.

ping_k2g.pcapng

Regards, Eric

Hi Eric,

That's weird.

Could you please use the same test but with K2G EVM shutdown? I'm guessing maybe it's another device in your network which responds the ping of your PC.

In your second last reply you mentioned "The PC test tool sends UDP packets to port 7 of EVM continuously at 2 packets/second interval. I didn't add any debug print before/after recvncfrom(). I started sending packet before EVM starts. I didn't see any problem for the response."
Could you please help do a quick test with the UDP sending tools only, and start sending UDP to port 7 of EVM continuously BEFORE EVM starts? In that case can you really see all UDP ECHO packets without packet loss?

Many thanks,
YW

Hi,

There is no any other device in my network. It is a local network with EVM directly connected to the PC Ethernet port.

I just re-did the test this morning:

1. I powered off the EVM

2. I started Wireshark on the PC and used  the tool sending the packets continuously to UDP port 7 at 10 packets/second

3. I didn't see any packets capture on the Wireshark because the link is not active

4. I powered on the EVM, I started to see the packet from PC to EVM in the Wireshark (of course no packet from EVM to PC)

5. I connected the ARM core and load the NIMU example, and ran

6. In wireshark I saw the response from EVM to PC

Attached is the capture. Do you have a dedicated connection between PC and EVM? Or, if any router/switch in between you can remove?

k2g_udp.pcapng

Regards, Eric

Hi Eric,

We do have have a dedicated connection between PC and EVM like your setup.
Could you please pack your CCS project (including the linked c files) for me to test? And what are the version of tools you use? Our NDK version is 2.26.00.08, TI-RTOS is 05.00.00.15 and CCS is 8.1.0.00011.
Also could you please help check your Ethernet card model?

Thank you,
YW

Hi,

I think an easier way is for me to give you a binary worked for me to test on your setup, to understand if this is a build/code issue or test setup issue.

If I give you a NIMU A15 binary using a hard coded IP 192.168.1.4, are you able to change your Ethernet card IP and subnet mask to 192.168.1.x for the test?

Regards, Eric

Hi,

Yes, I can temporarily unplug the other Ethernet cable which is connected to our intranet.

By the way following are some hardware numbers on our K2G EVM:
FY160626000104
SN: 15164P540011
ASSY REV: D1.3

Thanks,
YW

2677.NIMU_BasicExample_evmK2G_armExampleproject.outAttached is the binary I used for testing:

1. Put the EVM in no-boot mode

2. Connect the A15 core with GEL to initialize the board

3. Load and run

Regards, Eric

Hi Eric,

We've tested it with Wireshark logging.

When we unplug the other Ethernet cable which is connected to our intranet, the ping and UDP packets are always sent to EVM and get the response.

When the PC is connected to the intranet, in the beginning of EVM boot up, the PC always sends the packets through the intranet interface instead of the one to EVM, until the EVM replies the ARP to the PC. Before the PC get the reply of ARP it takes tens of seconds.

Could you please help paste your routing table here (by windows command "route print -4")? We haven't figure out why is that.

Many thanks,
YW

Hi,

13...68 05 ca 28 a2 20 ......TwinCAT-Intel PCI Ethernet Adapter (Gigabit) is the one connected to K2G EVM.

Regards, Eric

Hi Eric,

Sorry for the late reply. Yes we agree this is a pure PC OS setup issue. In our final product we will be able to control the network switch without dealing with the black box in Windows. Therefore we can close this issue now. Thank you so much for your time helping us debugging this issue. We should have verified the OS packets flow before reporting this problem. I apologize for that.

Many thanks,

YW