We are using networking in five different tasks which were getting configured and setup before we call BIOS_start(). Those tasks would run before the main NDK task gets a chance to run.
We call fdOpenSession() at the beginning of each of those tasks and each would get an OOM (out of memory) error. Putting in a massive amount of time in a task_sleep() call in those tasks solved the issue.
The better solution is to force the NDK to run as the only task in the system. Here is the sequence:
1) Init all hardware in main() while the system is still in "bare metal" mode. Configure all the pins and enable all of the GPIO subsystems you will use
2) Configure the NDK to hook network start. We use a function called netOpenHook()
3) Call BIOS_start() which will kick off the RTOS with the NDK task as the only useful task running.
4) in netOpenHook() configure all of your real tasks for the system and start them running. That made our tasks work without having any task_sleep() calls before fdOpenSession(). Remember to click the obscure radio button in the BIOS configuration that lets you dynamically create tasks. If you do not turn on dynamic task creation using the XDC system, you will get incomprehensible error messages as your first task dies a horrible death for no apparent reason.
This ensures that the race conditions from the NDK setting up all of its data structures as well as the hardware initialization that happens "under the covers" will happen before any of your tasks start using those resources.
I would appreciate any words of wisdom if this solution has any issues from other hidden sequences.