RTOS/EK-TM4C1294XL: tcpEchoIPv6

Hi Claus,

XDCtools 3.50 does not work with TI-RTOS for TivaC 2.1x versions. Which version did you use to run the application?

Todd

Good Day Todd,
as reported above, I used the 3.32.2.25_core version as well as the 3.32.0.06_core version with the same result. As long as the demo worked (for about a minute), a ping on the IPv4 address was successful.
But the tcpSendReceive.exe didn't work, neither with the IPv4 nor with the IPv6 address. It echoed back a " connect failed 111" and "connect failed 101".
Can You give me a hint where to look after the infos shown on the console about the exception?

Hi Claus,

Here's a good reference for debugging exceptions: training.ti.com/debugging-common-application-issues-ti-rtos . Note: a common cause for an exception is a blown stack (which is covered in this presentation also).

Did you make any modifications to the tcpEchoIPv6 example?

Another good thing to look at is to use Wireshark on your PC to look at the traffic between the PC and target. What was the network traffic to/from the device like before the exception occurred?

Todd

G'Day Todd,

thanks for the training hint about those exceptions. Will take a look...

I didn't do any modifications to the tcpEchoIPv6. I also tried the 3.31.3.43_core tools. It is the same. I also used Wireshark. Nothing unusual afaik. Even without any network activities, the exceptions occurs after one minute. I guess, the issue is something within the RTOS. By the way, the IPv4 tcpEcho example works perfectly without any exception.

Best regards from Bavaria. Claus

Claus,

I tried to reproduce the tcpEchoIPv6 failure you are observing, but the program works for me. I tried both the TI Network example and the GNU Network example. Which example are you using?

However, I did notice that some of the task stack sizes are close to reaching their maximum size. In particular, the tcpWorker, tcpHandler, and Idle_loop tasks. Would you try increasing the stack sizes for these tasks and see if it helps.

Here are my results:

tcpEchoIPv6 - CCS

HwiStackSize        HwiStackPeak
----------------------------------
2048                416

Fxn                                 StackSize       StackPeak
---------------------------------------------------------------
ti_ndk_config_Global_stackThread    1536            1072
ti_sysbios_knl_Idle_loop__E         512             384
dhcpState                           1024            648
tcpHandler                          1024            960
tcpWorker                           1280            948

tcpEchoIPv6 - GCC

HwiStackSize        HwiStackPeak
----------------------------------
2048                456

Fxn                                 StackSize       StackPeak
---------------------------------------------------------------
ti_ndk_config_Global_stackThread    1536            1080
ti_sysbios_knl_Idle_loop__E         512             472
dhcpState                           1024            616
tcpHandler                          1024            804
tcpWorker                           1280            980

Also, I checked the stack sizes over a period of several minutes and they are not growing. The stack peak usage remains the same. Would you check the stack peak values after about 5 seconds and then again around 40 seconds (before the program crashes). Make a note to see if the peak is slowly growing in size.

One note. I found that sometimes the tcpSendReceive program would loose the connection when I halt the target to check the stack sizes. This is expected. If this happens, just restart both the target and the tcpSendReceive program to gather your data point.

Were you able to get any analysis of the exception after viewing the debug training video?

~Ramsey

Ramsey, great, You found some time trying to help.

Alas, I cannot spend all my time on this interesting issue. But I've now had a look on the training video. Great tools for debugging.
But there were no stack overflows the ROV reported. This wasn't the reason for the exceptions. I've used the ROV hwi exception tab outputs and copied them. But even when my IPv6 system on my Linux machine here is not working correctly, the RTOS TCPIP stack should not throw an exception. But maybe I'm wrong and it's something else. Please advice.....

[
    {
        "module": "ti.sysbios.family.arm.m3.Hwi",
        "view": "Exception"
    },
    {
        "Decoded exception": {
            "Decoded": "Hard Fault: FORCED: BUSFAULT: IMPRECISERR"
        },
        "Exception context": {
            "$addr": "0x2000def8",
            "$type": "ti.sysbios.family.arm.m3.Hwi.ExcContext",
            "threadType": {},
            "threadHandle": "0x2000f320",
            "threadStack": "0x2000dc68",
            "threadStackSize": "1536",
            "r0": "0x0",
            "r1": "0x86",
            "r2": "0x1",
            "r3": "0x0",
            "r4": "0x0",
            "r5": "0x0",
            "r6": "0x86",
            "r7": "0x0",
            "r8": "0x2000bab6",
            "r9": "0x2000f158",
            "r10": "0x2000da74",
            "r11": "0x0",
            "r12": "0x0",
            "sp": "0x2000dfd0",
            "lr": "0x183f3",
            "pc": "0x1bf20",
            "psr": "0x81000000",
            "ICSR": "0x803",
            "MMFSR": "0x0",
            "BFSR": "0x4",
            "UFSR": "0x0",
            "HFSR": "0x40000000",
            "DFSR": "0xb",
            "MMAR": "0xe000ed34",
            "BFAR": "0xe000ed38",
            "AFSR": "0x0"
        },
        "Exception call stack": {
            "0    LLI6IsRouterStatMachine at lli6.c:217 :": "PC=0x0001BF20",
            "1    LLI6Update at lli6.c:458 :": "PC=0x000183F2",
            "2    Rt6Update at route6.c:1215 :": "PC=0x0001BF9E",
            "3    ICMPv6RecvRA at icmpv6_ndisc.c:894 :": "PC=0x000046E6",
            "4    ICMPv6RxPacket at icmpv6in.c:585 :": "PC=0x00008C82",
            "5    IPv6ParseExtnHeaders at ipv6_exthdrs.c:430 :": "PC=0x00012206",
            "6    IPv6RxPacket at ipv6in.c:163 :": "PC=0x0000FA02",
            "7    NIMUReceivePacket at nimu.c:304 :": "PC=0x000108C4",
            "8    EMACSnow_pkt_service at EMACSnow.c:748 :": "PC=0x0001C4B4",
            "9    NIMUPacketService at nimu.c:169 :": "PC=0x0001C1CA",
            "10    NetScheduler at netctrl.c:429 :": "PC=0x0000F126",
            "11    NC_NetStart at netctrl.c:211 :": "PC=0x0000A166",
            "12    ti_ndk_config_Global_stackThread at tcpEchoIPv6_pem4f.c:2595 :": "PC=0x0000D8BA",
            "13    ti_sysbios_knl_Task_exit__E at Task.c:414 :": "PC=0x00012BF0"
        }
    }
]

Claus,

The stack back trace is very helpful. We are analyzing the results. Thank you for taking the time to send this.

You are correct. Our stack should not raise an exception even when the remote end sends us invalid packets. It will be good to understand what is happening here.

From your original description, I'm thinking that after about 60 second of connection time, the remote end sends us a packet which causes us trouble. It would be helpful to know what type of machine we are connected to. Is it Linux, Windows? What version of the OS?

Thanks,
~Ramsey

Hi Ramsey,

I'm using 
Kernel         : Linux 3.16.0-5-amd64 (x86_64)
Compiled    : #1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08)
Distribution : LMDE 2 Betsy
and a Fritz!box Router for DHCP.   I've captured all traffic on the network with WireShark. But didn't find a way how to attach the output file. Please advice.
Here is a screen copy of the last packets:

At the 61st packet the exception occurs.   Claus

Well, I've managed to get ion at least a screenshot of the Wireshark output. I still don't see a way how to attach the complete *.pcapng. If helpful, please let me know, where to send it.

Best regards Claus

Claus,

Many thanks for your continued debugging effort. The Wireshark screen capture above indicates a possible issue in the NDK when receiving an IPv6 route advertisement. We indeed have a bug on file which might be related. It would be good to have the capture file to confirm this.

Files are attached to a post. Please start a new post by clicking the Reply button. If you enter the basic text widget, click on the 'Insert Code, Attach Files and more...' link (lower right corner). This will take you to the advanced text editor. Look on the toolbar for the paper clip. Click the paper clip to open a new dialog. Click the browse button to select the file for uploading.

Also, would you confirm the MAC address on which the NDK stack is running. It looks like TexasIns_03:3d:97.

Thanks,
~Ramsey

Ramsey, G'Day,

I've compressed the Wireshark File and attached it:sceExceptionCapture.pcapng.zip

Claus,

Thank you for attaching the Wireshark data file. This helped us confirm the issue you are seeing. We have filed a report and passed it on to the development team for resolution.

~Ramsey

Thanks Ramsey,

alas, I didn't found that old forum post. This is definitely the same problem. I've changed the file route.c in the folder ti/tirtos_tivac_2_16_00_08/products/ndk_2_25_00_09/packages/ti/ndk/stack/route6/ accordingly.
But now I've to rebuild the ndk. When looking into the tirtos.mak file in ti/tirtos_tivac_2_16_00_08/ there are a lot of environment variables that I have to modify. I'm afraid I've missed something...

as I can't compile the ndk:

mate@mars ~/ti/tirtos_tivac_2_16_00_08 $ make --file=tirtos.mak ndk
building ndk ...
gmake[1]: Entering directory `/home/mate/ti/tirtos_tivac_2_16_00_08/products/ndk_2_25_00_09'
building ndk packages ...
making all: Tue 15 May 13:54:51 CEST 2018 ...
======== .interfaces [./packages/ti/ndk] ========
making package.mak (because of package.bld) ...
js: "/home/mate/ti/tirtos_tivac_2_16_00_08/products/ndk_2_25_00_09/ndk.bld", line 123: xdc.services.global.XDCException: xdc.PACKAGE_NOT_FOUND: can't locate the package 'ti.targets.arm.elf' along the path: '~/ti/tirtos_tivac_2_16_00_08/products/bios_6_45_01_29/packages;/home/mate/ti/xdctools_3_32_00_06_core/packages;../..;'. Ensure that the package path is set correctly.
gmake[1]: *** No rule to make target `package.mak', needed by `.interfaces'.  Stop.
gmake: *** [packages/ti/ndk,.interfaces] Error 2
gmake[1]: *** [all] Error 2
gmake[1]: Leaving directory `/home/mate/ti/tirtos_tivac_2_16_00_08/products/ndk_2_25_00_09'
tirtos.mak:212: recipe for target 'ndk' failed
make: *** [ndk] Error 2

In the file tirtos.mak I've changed:

#  sce: edit DEFAULT_INSTALL_DIR for linux; and ccsv8 (from ccsv6) 5/18
DEFAULT_INSTALL_DIR      ?= ~/ti
CCS_COMPILERS_DIR        ?= $(DEFAULT_INSTALL_DIR)/ccsv8/tools/compiler

#
# Enable TI-RTOS to build for CCS.
# Set CCS_BUILD to true and modify path to toolchain
# sce: set toolchain ti-cgt-arm_18.1.1.LTS instead of ti-cgt-arm_5.2.5
CCS_BUILD ?= true
TI_INSTALL_DIR           ?= $(CCS_COMPILERS_DIR)
ti.targets.arm.elf.M4F   ?= $(TI_INSTALL_DIR)/ti-cgt-arm_18.1.1.LTS

Can someone give me a hint where I made a mistake?

Claus,

I'm not sure, but I think the you need to expand '~' in the make variable DEFAULT_INSTALL_DIR. Try the following:

edit tirtos.mak

DEFAULT_INSTALL_DIR = /home/mate/ti

Your other edits look okay. Also, the conditional assignment operator (?=) will not work if the variable already has a value. Make sure DEFAULT_INSTALL_DIR is not already defined or use the regular assignment operator (=) as I've done above.

If this still does not work, add the '-n' option to your make command and post the results for me.

make -n --file=tirtos.mak ndk

This will show the command that would have been executed.

~Ramsey

Great. Thanks for letting us know.
~Ramsey