AM5728: DSPs and IPUs are not loaded on boot on am5728 based custom board

Part Number: AM5728

We are using a custom board based on the TI AM5728 SoC, currently running TI Processor SDK 6.03. We intend to use the DSPs, IPUs, and IVAHD for our multimedia application, and we understand that Processor SDK 6.03 provides support for the required multimedia framework.

During board boot, we run a shell script to replace the default DSP and IPU firmware with our custom firmware. The following commands are executed:

cd /sys/bus/platform/drivers/omap-rproc/

# DSP
echo 40800000.dsp > unbind

sleep 1

rm /lib/firmware/dra7-dsp1-fw.xe66
ln -s /home/root/cfg/dra7xx-c66x-dsp.xe66 \
    /lib/firmware/dra7-dsp1-fw.xe66

echo 40800000.dsp > bind


# IPU
echo 55020000.ipu > unbind

sleep 1

rm /lib/firmware/dra7-ipu2-fw.xem4
ln -s /home/root/cfg/dra7-ipu2-fw.xem4 \
    /lib/firmware/dra7-ipu2-fw.xem4

echo 55020000.ipu > bind

After the above script has completed, our application uses libdce to communicate with the DSPs and IPUs. However, we occasionally encounter an issue where the application is unable to access or communicate with one or more of the remote processors.

We would like to understand the possible reasons for this intermittent failure.

In particular, we have the following questions:

1. Remote processor initialization:
   After executing the unbind/bind sequence, how can we ensure that the DSP and IPU firmware has been completely loaded and that the respective remote processors are fully initialized before our application starts using libdce?

2. Possible race condition:
   Could there be a race condition between executing:

    

   echo <device> > bind

    

   and starting our application? In other words, does the bind operation return before the DSP/IPU firmware, RPMsg services, and other required components are fully initialized?

3. IPU state showing suspended:
   Sometimes, when we check the IPU state using:

    

   cat /sys/class/remoteproc/remoteproc1/state

    

   the state is reported as:

    

   suspended

    

   Could this suspended state be the reason why our application is unable to communicate with the IPU?

   Also, why does the IPU state sometimes show suspended, even though the firmware has already been loaded using the unbind/bind sequence? Does binding the omap-rproc driver guarantee that the remote processor firmware is running, or can the processor subsequently enter a suspended state due to runtime power management?

4. DSP firmware crash/error:
   We have also observed another intermittent issue when checking the DSP trace using:

    

   cat /sys/kernel/debug/remoteproc/remoteproc2/trace0

    

   In some cases, the DSP trace shows the following error:

    

   [      0.000] 21 Resource entries at 0xfeb00000
   [      0.000] registering rpmsg-proto:rpmsg-proto service on 61 with HOST
   [      0.000] [t=0x0015fd55] ti.sysbios.knl.Semaphore: ERROR: line 215:
   assertion failure: A_badContext: bad calling context.
   Must be called from a Task.
   
   [      0.000] ti.sysbios.knl.Semaphore: line 215:
   assertion failure: A_badContext: bad calling context.
   Must be called from a Task.
   
   [      0.000] xdc.runtime.Error.raise: terminating execution

    

   Could this DSP-side assertion failure be related to the application's inability to communicate with the DSP?

   What could cause the SYS/BIOS Semaphore API to be called from an invalid context instead of a Task? Could this be related to the remote processor startup sequence, RPMsg initialization, firmware loading, or an issue in the DSP firmware itself?

Overall, we would like to understand the correct startup and synchronization mechanism for the DSP and IPU so that our application starts only after all required firmware, RPMsg/OMAPRPC services, and libdce components are fully initialized and ready for communication.