• State Resolved
  • Locked Locked
  • Replies 11 replies
  • Answers 3 answers
  • Subscribers 102 subscribers
  • Views 736 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5726: System hangs when stopping the IPU

Alexander Danzer
Prodigy 60 points
Part Number: AM5726
Other Parts Discussed in Thread: SYSBIOS

Tool/software: Linux

Hi,

When I stop the IPU of the AM572x, the Linux system freezes.
The problem has already been reported and marked as resolved, but without describing a solution:
e2e.ti.com/.../757568
e2e.ti.com/.../755141

Steps to reproduce:
- Boot the Linux image of the processor SDK "am57xx-evm-linux-05.03.00.07.img"
- run "ln -sfn /usr/bin/ipc/examples/ex02_messageq/release/server_ipu1.xem4 /lib/firmware/dra7-ipu1-fw.xem4"
- run "echo 58820000.ipu > /sys/bus/platform/drivers/omap-rproc/bind" and "echo 58820000.ipu > /sys/bus/platform/drivers/omap-rproc/unbind" repeatedly, until the system hangs. This happens usually within 10 repetitions.

Some more details:
The problem might be associated with the MessageQ IPC mechanism, as it does not seem to appear with the "opencl-monitor" firmware included in the SDK image, but does also happen with my custom TI-RTOS application firmware which makes use of MessageQ.
I have encountered this issue with all other Linux kernel versions I tried: 4.14.98, 4.14.108 and 4.19.94, all built with yocto from the meta-ti layer.

I have also tried restarting the IPU with other commands:
echo start > /sys/class/remoteproc/remoteproc0/state
echo stop > /sys/class/remoteproc/remoteproc0/state
The result is the same, however.

Thanks in advance,
Alexander

  • 0
    TI__Guru 50170 points
    Hi, Alexander,

    Please read this recent thread which gave the reason, and try MessageQBench in the file system instead of ex02_messageq.
    e2e.ti.com/.../798468

    Rex
  • 0 in reply to Rex Chang
    Prodigy 60 points

    Hi Rex,

    Thanks for your quick reply.

    I can only find a MessageQBench binary for the Linux host, not for the IPU. When running ex02_messageq, it is enough to run the IPU firmware in order to crash the system, so I do not run the Linux host application.

    From the other thread, I read that building the application with PM and DEH might solve the problem.

    I tried adding the following lines from the messageq_single test to my app.cfg which is based on the ex02_messageq example:

    /* Modules used in Power Management */

    xdc.loadPackage('ti.pm');

    var Power = xdc.useModule('ti.sysbios.family.arm.ducati.smp.Power');

    Power.loadSegment = "PM_DATA";

    DEH was already configured. The firmware compiles fine, but the behavior remains unchanged, unfortunately.

    I need the possibility to shut down the IPU properly in order to perform software updates without the need to reboot the whole system.

    Alexander

  • 0 in reply to Alexander Danzer
    TI__Guru 50170 points
    Hi, Alexander,

    please take a look at how it is implemented in Dsp_vayu_power.cfg which is used by test code messageq_single.

    Rex
  • 0 in reply to Rex Chang
    Prodigy 60 points

    Hi Rex,

    Thanks for your reply. I took a look at said file, the comments seem to refer to a really similar problem. So I added the following lines to my app.cfg:

    Idle.addFunc('&IpcPower_idle'); /* IpcPower_idle must be at the end */
    Program.sectMap[".text:IpcPower_callIdle"] = "L2_RAM";

    Note that I changed "L2SRAM" from the file Dsp_vayu_power.cfg to "L2_RAM" in order to make it compile for the IPU.


    Unfortunately, the system still hangs on stopping the IPU.

    Alexander

  • 0 in reply to Alexander Danzer
    TI__Guru 50170 points
    Alexander,

    Please try the power module in Dsp_vayu_power.cfg. Specifically, the following lines

    /* Modules used in Power Management */
    xdc.loadPackage('ti.pm');
    var Power = xdc.useModule('ti.sysbios.family.c66.vayu.Power');
    Power.loadSegment = "PM_DATA";

    Rex
  • 0 in reply to Rex Chang
    Prodigy 60 points
    Rex,

    The lines you suggested work for the DSP, not the IPU. I have the following in place:

    xdc.loadPackage('ti.pm');
    var Power = xdc.useModule('ti.sysbios.family.arm.ducati.smp.Power');
    Power.loadSegment = "PM_DATA";

    Alexander
  • 0 in reply to Alexander Danzer
    TI__Guru 50170 points
    Hi, Alexander,

    It doesn't look right to me by using arm,smp,power on IPU. The difference between ex02_messageq and messageq_single is the latter includes the power module inside the Dsp_vayu_power.cfg.

    I'll have DSP expert to follow up on this.

    Rex
  • 0 in reply to Alexander Danzer
    TI__Mastermind 27355 points

    Hi Alexander,

    Alexander Danzer said:
    xdc.loadPackage('ti.pm');
    var Power = xdc.useModule('ti.sysbios.family.arm.ducati.smp.Power');
    Power.loadSegment = "PM_DATA";

    This is correct for the IPUs in SMP mode.

    Please try adding the following lines of code to the end of the *.cfg file to prevent the hang when going idle:

    /* Idle Power Management functions for each core */

Idle.addCoreFunc('&IpcPower_idle', 0); /* IpcPower_idle must be at the end */

Idle.addCoreFunc('&IpcPower_idle', 1); /* IpcPower_idle must be at the end */

    Regards,
    Sahin

  • 0 in reply to Sahin Okur
    Prodigy 60 points

    Hi Sahin,

    Unfortunately, the problem still persists after adding these lines.

    Alexander

  • 0 in reply to Alexander Danzer
    TI__Guru 50170 points
    Hi, Alexander,

    Instead of debugging why ex02 not working for unbind/bind, I suggest you follow the messageq_single.xe66 test program as the base and go from there. ex02 is an example showing how messageq works and not a production code.

    Rex
  • 0 in reply to Rex Chang
    Prodigy 60 points

    Hi Rex,

    Thank you very much, your advice enabled me to solve the problem.
    It turns out that adding the lines as suggested by Sahin was enough to solve the problem.

    Thanks again,
    Alexander