66AK2G12: system crashes when enough network packets are sent to the assigned interface during NDK initialization

Hi,

From the call stack and runtime error, it is showing that there is a NULL handle passed to SemaphoreP_post() API.

It appears to me there may be some kind of memory corruption happening corrupting the handle in the application.

Possibly, to catch/debug this issue, you can change SEMOSAL_Assert implementation which is Osal_DebugP_assert(). (in ti\osal\src\tirtos\utils_tirtos.c file, around line number 98)

/*
* ======== Osal_DebugP_assert ========
*/
void Osal_DebugP_assert(int32_t expression, const char *file, int32_t line)
{
    if (expression != 0) {
        xdc_runtime_Assert_raise__I(Module__MID, file, line, Assert_E_assertFailed); --> Replace this with while (1) loop and a print that it hit this case.. This would help to debug the which function is passing the NULL pointer as the handle, (you can get it from call stack).
    }
}

/*
* ======== SemaphoreP_post ========
*/
SemaphoreP_Status SemaphoreP_post(SemaphoreP_Handle handle)
{
SEMOSAL_Assert((handle == NULL_PTR));  <------------------  The error is happening here.
SemaphoreP_tiRtos *semaphore = (SemaphoreP_tiRtos *)handle;

Semaphore_post((Semaphore_Handle)&semaphore->sem);
return (SemaphoreP_OK);
}

Hi Aravind,

Our debugging already found out in this NDK initialization scenario, the system failed in the NDK API call NC_NetStart() of C:\ti\ndk_3_61_01_01\packages\ti\ndk\netctrl\netctrl.c, which is the original TI NDK source code. Could you please try reproduce the issue on your end?

Thank you,
YW

Hi YW,

I tried to reproduce the issue at my end.

The issue showed up only one or two times and later on it is not happening at my setup. I even tried filling up the entire MSMC, DDR, L2 memory with a known pattern and then reload the program and run.

It did not happen.. However, I think, I agree with you that there is some issue to be addressed. I need to investigate this further.

How frequent is this at your side? Can you please share the "ASCII" field with 1024 ascii characters, that you are using in your test?

Hi Aravind,

Thanks for your effort and taking this issue seriously.

I actually tried to upload the ascii file which can be loaded by the "Load File" button in Packet Sender. But this forum doesn't seem to be able to take such file by either "Insert File" or "Insert Media". However you can create my file content by yourself. It's 64 repeated "0123456789abcdef" sequence without EOF in the file. But the UDP payload content really doesn't matter in this test, as long as the payload is long enough to produce enough traffic and mess up the internal memory/buffer for some reason.

The frequency of this issue on my end is "always", given the right sequence and timing of step 1 and 2 in the procedures. Basically, you can try letting the packet sender keep resending with 0.1 delay and never stop it. Then I start "Launch Selected Configuration" of K2GEVM.ccxml in CCS, launch c66x core, load the example binary to the core, and run it.

One possibility of your inconsistent results which don't have NDK crash is that your OS didn't send the UDP packets to the target K2GEVM. This may be caused by multiple network interfaces in the OS and can be checked by Wireshark. For more details you can refer to the whole discussion of this post:
e2e.ti.com/.../2766061

Thank you,
YW

Hi YW,

Thanks for the ASCII file input data that you are using.

I noticed that the Event combiner group interrupts are not set in the helloWorld.cfg file for this example.

Can you try updating the cfg file and rebuild the project with enabling the Ecm event group interrupts as below and let me know your observations?

update ti/nimu/example/helloworld/k2g/c66/bios/helloworld.cfg file:

/*
* Enable Event Groups here and registering of ISR for specific GEM INTC is done
* using EventCombiner_dispatchPlug() and Hwi_eventMap() APIs
*/

Ecm.eventGroupHwiNum[0] = 7;
Ecm.eventGroupHwiNum[1] = 8;
Ecm.eventGroupHwiNum[2] = 9;
Ecm.eventGroupHwiNum[3] = 10;

Hi Aravind,

Sorry for the late reply.

It's the same on my end, and was actually used in my main project (where I first found the system crashing during NDK initialization).
However, I have some new findings regarding the steps to reproduce this issue. I'm still testing them and will get back to you as soon as I can.

Regards,
YW