Hello,
while working with NDK 1.94 on C64+ platform I've stumbled upon problem - I think - related to this library.
My test device is using DHCP and joins multicast group (IP_ADD_MEMBERSHIP) on startup. When address from DHCP changes (either router was changed or IP assignment on same router after lease timeout) I want to shut down part of application and relaunch it. Unfortunately when I'm trying to close socket that previously joined multicast group some task at priority 9 (I believe one of NDK tasks) hangs up consuming all CPU time indefinitely and starving all lower priority tasks.
This happens only with multicast-enabled socket. If I drop multicast membership before network removed/network added events occur than everything works fine. Task halts when socket is closed or when IP_DROP_MEMBERSHIP option is set after DHCP renew. It happens even if delay between DHCP renew and and socket closing is long and in my test even if renewed address is the same as previous (although I don't want normally to reopen/rebind socket in this case).
I would be glad if someone could confirm problem or send any suggestions.
Edit: after next few tries I'm not sure if this can happen when renewed address is the same as previous, but changing router seems to be reliable method to get this effect. Task at priority 9 seems to be NDK kernel in my configuration (default NDK priorities). Halting of this task may be just random effect - at the same time I'm observing using TSK_stat strange (either 100% or some seemingly random, out of range number) stack usage of two other tasks that are using NDK - that may suggest some incorrect memory access.
Hi tomeko,
Which version of BIOS and CCS are you using?
This certainly could be due to a task stack overflow and this is the easiest thing to try first. Can you try increasing the stack size of the tasks that you see having close to 100% stack usage?
Also, as a side note, there have been several NDK releases since 1.94. I checked the release notes but did not see any bugs related to this listed as being fixed. But, it's possible that something causing this issue was fixed but not tracked in the release notes.
But before going the update route, let's see what the results are with increased stack size.
Steve
Thanks for answer.
Stack overflow seemed unlikely - mentioned tasks have stack margins of 8 kB (or little more) out of 16 kB. After removing from code single setsockopt with IP_ADD_MEMBERSHIP and repeating test this margin is kept, so stack overflow would be possible only if using multicast would consume more than 7 kB (taking DSP/BIOS required stack margin into account). If router was not swapped (and DHCP lease time expired) between IP_ADD_MEMBERSHIP and fdClose() or IP_DROP_MEMBERSHIP or if multicast was not enabled than any significant stack usage increase is not observed.
I'm using DSP/BIOS 5.33.03, cgtools 6.1.7, CCS 3.3. Unfortunately any updates would not be a viable option (at least not just because of this problem) due to project size and risk involved. If I won't find solution I would probably just block functionality that requires multicast when DHCP is configured.
Edit: I've tested with 256 kB stacks and effect is the same.
Also I would suppose that IP_DROP_MEMBERSHIP action might be source of the problem - it seems to be executed when calling fdClose(). IGMP Leave message is sent when socket is closed under normal conditions (DHCP router not changed), I didn't checked if it actually sent in this situation.
Very glad that you did and didn't find any stack overflow issues ... by the way:
tomekoat the same time I'm observing using TSK_stat strange (either 100% or some seemingly random, out of range number) stack usage of two other tasks that are using NDK - that may suggest some incorrect memory access.
... this is the only reason I suggested that you check the stacks.
I looked at the code of setsockopts() for IP_DROP_MEMBERSHIP but nothing stood out immediately.
Do you have the NDK sources with your 1.94 release? Can you try adding the file igmp.c into your project and rebuilding? This should allow you to set a break point into the function:
int IGMPLeave (HANDLE hSock, struct ip_mreq* ptr_ipmreq)
Hopefully you can check to see which part of the code is run for this failure case and then we can go from there.
No, I don't have NDK sources (at least not stack.lib sources, just usual 1.94 package). I've tried to link with own (empty) IGMPLeave, but it gives me linker conflict.
tomeko,
Ok, yes, the NDK didn't ship sources until version 2.0.0. I see that the igmp.c file is the same between 1.94 and 2.0.0. I've attached it for you. Can you try again with the attached file?
6786.igmp.c
Thanks, I didn't know that NDK 2.x comes with source code for stack.lib.
I think I see the problem. Here is log fragment:
Link Status: 100Mb/s Full Duplex on PHY 15Network Added: If-1:10.0.0.125Service Status: DHCPC : Enabled : Running : 017DHCP Server 1 = '10.0.0.1'Router 1 = '10.0.0.1'Mask 1 = '255.255.255.0'IGMP: Join hSock = -2121356932, ptr_ipmreq = 80403064IGMP: JoinHostGroupIGMP: joinedIGMP: Timer...IGMP: Timer// eth detachedLink Status: No Link on PHY 15// second router attachedLink Status: 100Mb/s Half Duplex on PHY 15Network Removed: If-1:10.0.0.125Service Status: DHCPC : Enabled : Running : 018Network Added: If-1:192.168.0.202Service Status: DHCPC : Enabled : Running : 017DHCP Server 1 = '192.168.1.6'Router 1 = '192.168.1.1'Mask 1 = '255.255.254.0'// closing socket after Network Added eventIGMP: Leave hSock = -2121356932, ptr_ipmreq = 818e3d14IGMP: EINVAL bindIGMP: Leave hSock = -2121356932, ptr_ipmreq = 818e3d14IGMP: EINVAL bindIGMP: Leave hSock = -2121356932, ptr_ipmreq = 818e3d14IGMP: EINVAL bind...
In SockClose():
/* If the socket is being closed; we need to ensure that all the multicast group * this socket has joined are left. */ ptr_mcast_rec = (MCAST_SOCK_REC *)list_get_head ((LIST_NODE**)&ps->pMcastList); while (ptr_mcast_rec != NULL) { /* Leave the multicast group */ IGMPLeave (h, &ptr_mcast_rec->mreq); /* Get the head; since the IGMPLeave will have deleted the entry from the list. */ ptr_mcast_rec = (MCAST_SOCK_REC *)list_get_head ((LIST_NODE**)&ps->pMcastList); }And IGMPLeave() in my test is leaving immediately without touching multicast list:
hIf = BindIPHost2IF ((IPN)ptr_ipmreq->imr_interface.s_addr); if (!hIf) { return (EINVAL); }In effect SockClose() is stuck in loop.I'm not sure if my previous report with setsockopt + IP_DROP_MEMBERSHIP was correct - looking into source code I don't see problem there (it would not leave group but it would stuck in loop) and I've done this test only once, so I may have missed that it stuck inside fdClose() not inside setsockopt().
Anyway, I'll probably have to leave this problem for a moment or two and I'll just deny IGMP in DHCP configuration in next software release.