• State
  • Locked Locked
  • Replies 8 replies
  • Subscribers 102 subscribers
  • Views 221 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PROCESSOR-SDK-AM64X: R5F Graceful shutdown issue

Joao Lima
Prodigy 95 points
Part Number: PROCESSOR-SDK-AM64X

Tool/software:

Hello,

Refer to PROCESSOR-SDK-AM64X: Issue to run TI's pcie_buf_transfer_rc example on core R5F - Processors forum - Processors - TI E2E support forums

When I stop the core r5f 0 and relaunch the firmware, the board crashes. Only with a power cycle will the board work again.
Any suggestions about why this is happening?

I am stopping and the relaunching the firmware from linux, by using the commands:

echo stop > /sys/class/remoteproc/remoteproc0/state
echo start > /sys/class/remoteproc/remoteproc0/state


, can you help with this, please?

Compiler version
ti-cgt-armllvm_3.2.1.LTS

Sdk version
mcu_plus_sdk_am64x_09_02_01_05

Linux version
Linux am64xx-evm 6.6.32-ti-g8086474caf5d-dirty #1 SMP PREEMPT Fri Jun 14 07:13:33 UTC 2024 aarch64 GNU/Linux

Thanks.

Best Regard,
Joao Lima

 

  • 0
    TI__Guru* 83305 points

    Hello Joao,

    Hypothesis: the remote core is not releasing its resources before it gets shut off. Thus, it is unable to reclaim those resources when it turns back on.

    For more information on that hypothesis, refer to the AM64x academy, multicore module, page "Graceful shutdown", section "Why is graceful shutdown needed" https://dev.ti.com/tirex/explore/node?a=7qm9DIS__LATEST&node=A__AVt2qZLTY3BgKr3D4YIv8w__AM64-ACADEMY__WI1KRXP__LATEST

    Additional notes

    First off, please note that kernel 6.6 is not supported yet. Our developers are still working to port code to kernel 6.6 for SDK 10.0, which will release later this summer. It is possible that the remoteproc code that handles shutting down and restarting remote cores is not fully ported to kernel 6.6 yet.

    I would start by seeing if things work as expected on the latest SDK 9.2 release (kernel 6.1).

    If you have specifically added the graceful shutdown code to your R5F project and you are still able to turn off the remote core during Linux, that tells me that the kernel version you are using has not finished porting the remoteproc changes yet. If the Linux driver supports graceful shutdown, but the remote core firmware does NOT include the graceful shutdown code, then Linux should time out and refuse to shut down the remote core.

    Setting expectations for my next response 

    I will be on vacation until the week of July 15. We can continue our conversation then as needed.

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    Prodigy 95 points

    Hello Nick,

    Nick Saulnier said:
    Hypothesis: the remote core is not releasing its resources before it gets shut off. Thus, it is unable to reclaim those resources when it turns back on.

    Thank you for the share. I will check the shared resource.

    Nick Saulnier said:
    Additional notes

    Ok. First I will debug about this in the current kernel's version that I am using. As I'm using freertos, I may have to implement a mechanism to listen for a stop from the remote proc on my application. But I have to read more about this.

    Nick Saulnier said:
    Setting expectations for my next response 

    No problem. This topic is not urgent at the moment. Enjoy the vacations.

    Best regards,
    Joao Lima

  • 0 in reply to Nick Saulnier
    Prodigy 95 points

    Hello Nick,

    Nick Saulnier said:
    Additional notes

    Meanwhile, we switched to SDK 8.5, since this SDK has the supported examples for what we want to analyze and implement.

    With this in mind, can you please indicate what they are and what is the location of the code related to stopping the remote proc in kernel 6.1?

    Given that we are using kernel 6.6, we want to evaluate the cost of porting.

    Thank you.
    Joao Lima

  • 0 in reply to Joao Lima
    TI__Guru* 83305 points

    Hello Joao,

    I am back and slowly catching up on everything.

    The Linux remoteproc driver in AM64x SDK 8.5 will not support graceful shutdown of remote cores. I think we added that feature for AM64x in SDK 9.1, and finished adding the documentation for it in SDK 9.2. In SDK 8.5, if you want to load new firmware into a remote core, the suggested method is to reboot the processor.

    For documentation about the R5F side of the graceful shutdown code, refer to the AM64x academy, multicore module, page "Graceful shutdown", section "MCU+ code to enable graceful shutdown": https://dev.ti.com/tirex/explore/node?a=7qm9DIS__LATEST&node=A__AVt2qZLTY3BgKr3D4YIv8w__AM64-ACADEMY__WI1KRXP__LATEST

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    TI__Guru* 83305 points

    I do not fully understand your statement "evaluate the cost of porting to kernel 6.6". TI will finish porting our code to kernel 6.6 within the next month or so, and you'll be able to access that code either

    through an official SDK release,

    through the git repo (for just the Linux kernel)
    https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/?h=ti-linux-6.6.y-cicd 

    or through CICD (for the latest filesystem & version of the SDK)
    https://software-dl.ti.com/cicd-report/linux/index.html?section=platform&platform=am64xx 

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    Prodigy 95 points

    Hello Nick,

    Thank you for your answer.

    Nick Saulnier said:
    The Linux remoteproc driver in AM64x SDK 8.5 will not support graceful shutdown of remote cores. I think we added that feature for AM64x in SDK 9.1, and finished adding the documentation for it in SDK 9.2. In SDK 8.5, if you want to load new firmware into a remote core, the suggested method is to reboot the processor.

    Thank you for the clarification.

    Nick Saulnier said:
    I do not fully understand your statement "evaluate the cost of porting to kernel 6.6". TI will finish porting our code to kernel 6.6 within the next month or so, and you'll be able to access that code either

    Good news. It's almost being released then. otherwise we would have to implement an auxiliary mechanism to reboot the remote core.

    Regards,
    Joao Lima

  • 0 in reply to Joao Lima
    TI__Guru* 83305 points

    Hello Joao,

    Glad to hear that the timeframe for SDK 10.0 should work for your design cycle. If you run into any challenges with the SDK 10.0 release, please let us know. In the meantime, feel free to reply here if you have followup questions, or create a new thread if you have a question about a new subject!

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    Prodigy 95 points

    Hello Nick,

    Deal. Thank you for your availability.

    Regards,
    Joao Lima