AM3356: Cannot enter sleep mode with kernel 6.1.49

Hi Bin-san

Thank you for your quick response.

This is a fatal problem for our products.
So, I hope it will be fixed in a few weeks.

Best regards
Katsuyama

Hi Katsuyama-san,

It is planned to be fixed in AM335x SDK9.3, which will be released by mid/end of next month.

Hi Katsuyama-san,

What wakeup source will you be using?

Hi Bin-san

We will use GPIO0 and UARTs as wakeup source.

Best regards
Katsuyama

Hi Katsuyama-san,

Noted. Thanks.

Hi Bin-san,

Thank you.

By the way, I just realized I got the version number wrong in this title. It must be "6.1.46".

I understand new fixed version will be "6.1.49".

Hi Katsuyama-san,

The AM335x SDK9.3 will be based on mainline kernel v6.1.105 or newer.

Hi Bin-san

What is the status of countermeasures?

If the kernel version is 6.1.113 or higher, that would be great.
Because vulnerabilities such as CVE-2024-49884 have been addressed.

Best regards
Katsuyama

Hi Katsuyama-san,

Currently the SDK9.3 release candidate is based on v6.1.112. The branch will be sync'd with the upstream stable branch again today which is v6.1.119 as of now and run through the test cycles. If there is no any issue in this update, the SDK9.3 will be at least based on v6.1.119.

Hi Bin-san,

Thank you for your reply.
I'm very happy to hear that the next kernel base version will be 6.1.119.

I previously heard that the release would be in the middle or end of this month.
When do you think it will be?

Best regards,
Katsuyama

Hi Katsuyama-san,

The SDK release candidate is in the last few rounds of testing. As far as I know the release is still on schedule.

Hi Katsuyama-san,

$ echo standby > /sys/power/state
bash: echo: write error: Invalid argument

Sorry I just noticed you were reporting issue in standby usecase. Do you only require standby mode but not deepsleep mode?

There was a miscommunication in the issue report to our sw dev team. Then now resume/wakeup only work in deepsleep suspend, but not in standby mode. The miscommunication has been corrected now, and our sw dev team is investigating the standby resume issue. It is likely too late to include the solution in SDK9.3 release, but the fix patch will be available on the kernel 6.1.y branch on git.ti.com once the issue is fixed.

Hi Bin-san,

Thank you for the information.

> Do you only require standby mode but not deepsleep mode?
Yes, I only require standby mode. I don't use deepsleep mode.

We hope that the fix will be released as soon as possible, and ideally included in SDK9.3.

Best regards,
Katsuyama

Hi Katsuyama-san,

Sorry it is unlikely to be included in SDK9.3. The SDK9.3 will be released in a couple weeks and the code is already frozen. But I will let you know as soon as the patch is available on the kernel repo on git.ti.com.

Hi Bin-san,

Can you show me the URL for SDK 9.3 when it is released?
I will Check periodically to see if it has been uploaded.

Best regards,
Katsuyama

Hi Katsuyama-san,

The SDK9.3 release is at https://www.ti.com/tool/PROCESSOR-SDK-AM335X

But the release itself won't be updated. The future kernel update will be on the ti-linux-6.1.y-cicd branch linked below.

https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/log/?h=ti-linux-6.1.y-cicd

Hi Katsuyama-san,

Sorry I think I misunderstood your question, and my previous response didn't answer it.

AM335x SDK9.3 has been released last week. Please refer to the link below.

https://www.ti.com/tool/PROCESSOR-SDK-AM335X

Hi Bin-san,

Thank you for your reply.

In case of SDK 9.1, I download it from the following URL.

software-dl.ti.com/.../Overview_Building_the_SDK.html

I want to know this style URL for SDK 9.3.

Best regards,
Katsuyama

Hi Katsuyama-san,

Takeharu Katsuyama said:

In case of SDK 9.1, I download it from the following URL.

software-dl.ti.com/.../Overview_Building_the_SDK.html

This links to the SDK9.1 documentation. The corresponding link for SDK9.3 is

https://software-dl.ti.com/processor-sdk-linux/esd/AM335X/09_03_05_02/exports/docs/linux/Overview_Building_the_SDK.html

Hi Bin-san,

I saw software-dl.ti.com/.../Overview_Building_the_SDK.html.
I'm a little confused because the contents were very different from the one
for SDK 9.1.

Is the following SDK 9.1 method still possible in SDK 9.3?

  $ git clone https://git.ti.com/git/arago-project/oe-layersetup.git tisdk
  $ cd tisdk
  $ ./oe-layertool-setup.sh -f configs/processor-sdk/processor-sdk-09.03.05.02-legacy-config.txt

Best regards,
Katsuyama

Hi Katsuyama-san,

The documentation scheme is changed in SDK9.3, but the build instruction is the same as that in SDK9.1 doc:

git clone https://git.ti.com/git/arago-project/oe-layersetup.git tisdk
cd tisdk
./oe-layertool-setup.sh -f configs/processor-sdk/<oeconfig-file>
cd build
. conf/setenv
MACHINE=<machine> bitbake -k tisdk-default-image

Link https://software-dl.ti.com/processor-sdk-linux/esd/AM335X/09_03_05_02/exports/docs/linux/Release_Specific_Yocto_layer_Configuration.html#yocto-layer-configuration shows the oeconfig-file is "processor-sdk-09.03.05.02-legacy-config.txt", so yes, your instruction is correct.

I also wanted to let you know that the SDK provide prebuilt Linux images, which you can download and install the SDK package linked below, just in case you can use it without re-build the SDK in Yocto.

https://dr-download.ti.com/software-development/software-development-kit-sdk/MD-1BUptXj3op/09.03.05.02/ti-processor-sdk-linux-am335x-evm-09.03.05.02-Linux-x86-Install.bin

Hi Bin-san,

I was able to build the 'tisdk-base-image' of SDK 9.3 successfully.
Thank you very much!

By the way, I'll be checking the kernel repositories regularly, when do you think the fix for the sleep issue will be available?

Best regards,
Katsuyama

Hi Katsuyama-san,

The developer is actively debugging the issue, there is some understanding of the problem, but the root cause has not been identified yet, so at this moment it is not clear when the fix will be available,

Hi Bin-san,

A happy new year!

How is debugging the sleep issue going?
I checked the repository and couldn't find any commits about it.
I am looking forward to the patch being released.

Best regards,
Katsuyama

Hi Katsuyama-san,

Most of us were out of office in the past two weeks for the holidays, I don't believe it has much debugging work done recently. I will keep you posted.

Hi Bin-san,

Has there been any progress?

Hi Bin-san,

Thank you for the patches.
I applied them, but unfortunately nothing has changed.
Following kernel messages are still shown.

[   64.489759] remoteproc remoteproc0: request_firmware failed: -110
[   64.496060] wkup_m3_ipc 44e11324.wkup_m3_ipc: rproc_boot failed

And rtcwake shows followings.

rtcwake -s 10 -m standby 
rtcwake: unrecognized suspend state 'standby' 

[   64.489759] remoteproc remoteproc0: request_firmware failed: -110
[   64.496060] wkup_m3_ipc 44e11324.wkup_m3_ipc: rproc_boot failed

This error should be already fixed in the SDK9.3.

How did you generate the kernel .config file? Please use command 'make linux' in the SDK top level directory to build the kernel.

Hi Bin-san,

> Please use command 'make linux' in the SDK top level directory to build the kernel.

I'm afraid I can't understand this method.
I made .config as follows:
---
1. cp -a 'kernel-source' directory in 'tisdk/build/arago-tmp-default-glibc/arago-tmp-default-glibc/work-shared/am335x-evm ' ./linux
2. cd linux
3. apply patches
4. make mrproper
5. make am335x_xxx_defconfig
---
I'll upload am335x_xxx_defconfig after.

Then build kernel and modules. And deploy them as follows.
---
5. make
6. make INSTALL_MOD_PATH=xxxx modules_install
7. deploy zImage and modules on target board.
---

Could this be caused by the cortex m3 firmware?

Hi Katsuyama-san,

Takeharu Katsuyama said:

1. cp -a 'kernel-source' directory in 'tisdk/build/arago-tmp-default-glibc/arago-tmp-default-glibc/work-shared/am335x-evm ' ./linux

Ensure the "kernel-source" is take from the AM335x SDK9.3 release.

Takeharu Katsuyama said:

5. make am335x_xxx_defconfig

The SDK9.3 kernel doesn't have this "am335x_xxx_defconfig" file. Please use the following command to generate .config.

make multi_v7_defconfig ti_multi_v7_prune.config no_smp.config

Hi Bin-san,

am335x_xxx_defconfig is what I customized after migrating from an old kernel.

By the way, I tried as you said.
I got the error "mpc.h: No such file or directory".
Here's the log.

$ make mrproper
$ make multi_v7_defconfig ti_multi_v7_prune.config no_smp.config
  ...
$ make
  SYNC    include/config/auto.conf.cmd
  SYSHDR  arch/arm/include/generated/uapi/asm/unistd-oabi.h
  SYSHDR  arch/arm/include/generated/uapi/asm/unistd-eabi.h
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  scripts/dtc/libfdt/fdt.o
  HOSTCC  scripts/dtc/libfdt/fdt_ro.o
  HOSTCC  scripts/dtc/libfdt/fdt_wip.o
  HOSTCC  scripts/dtc/libfdt/fdt_sw.o
  HOSTCC  scripts/dtc/libfdt/fdt_rw.o
  HOSTCC  scripts/dtc/libfdt/fdt_strerror.o
  HOSTCC  scripts/dtc/libfdt/fdt_empty_tree.o
  HOSTCC  scripts/dtc/libfdt/fdt_addresses.o
  HOSTCC  scripts/dtc/libfdt/fdt_overlay.o
  HOSTCC  scripts/dtc/fdtoverlay.o
  HOSTLD  scripts/dtc/fdtoverlay
  HOSTCXX scripts/gcc-plugins/arm_ssp_per_task_plugin.so
In file included from scripts/gcc-plugins/gcc-common.h:95,
                 from scripts/gcc-plugins/arm_ssp_per_task_plugin.c:3:
/opt/arago-2023.10/sysroots/x86_64-arago-linux/usr/lib/arm-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/11.5.0/plugin/include/builtins.h:23:10: fatal error: mpc.h: No such file or directory
   23 | #include <mpc.h>
      |          ^~~~~~~
compilation terminated.
make[2]: *** [scripts/gcc-plugins/Makefile:54: scripts/gcc-plugins/arm_ssp_per_task_plugin.so] Error 1
make[1]: *** [scripts/Makefile.build:505: scripts/gcc-plugins] Error 2
make: *** [Makefile:1268: scripts] Error 2

Hi Bin-san,

Once I disabled CONFIG_GCC_PLUGINS the error about mcp.h disappeared and the build completed successfully.

The following kernel message no longer appears:

[   64.489759] remoteproc remoteproc0: request_firmware failed: -110
[   64.496060] wkup_m3_ipc 44e11324.wkup_m3_ipc: rproc_boot failed

But I still can't enter sleep mode.

Below is the error message.

$ echo standby > /sys/power/state
bash: echo: write error: Invalid argument
$ rtcwake -s 10 -m standby
rtcwake: unrecognized suspend state 'standby'

Hi Bin-san,

I can't see 'standby' in /sys/power/state.

$ cat /sys/power/state
freeze mem

Hi Katsuyama-san,

Takeharu Katsuyama said:

By the way, I tried as you said.
I got the error "mpc.h: No such file or directory".

I think you need to install some packages on PC Ubuntu to solve this problem:

$ sudo apt install libgmp3-dev libmpc-dev

Takeharu Katsuyama said:

Once I disabled CONFIG_GCC_PLUGINS the error about mcp.h disappeared and the build completed successfully.

I don't recommend to disable it, as the SDK always has it enabled. I am not sure about the behavior if it is disabled.

Takeharu Katsuyama said:

I can't see 'standby' in /sys/power/state.

Let's first ensure we are on the same page - instead of using your kernel build process, can you please do the following to ensure you can get deepsleep or standby resume working?

- download and install the AM335x SDK9.3 release from ti.com;
- go to the SDK installation directory;
- run command "make linux", which compiles the kernel zImage, modules, and dtbs;
- use your prefer methods to populate the kernel zImage, modules, and dtb to your board root filesystem.

Hi Bin-san,

I installed libgmp3-dev libmpc-dev and enabled CONFIG_GCC_PLUGINS, following error occurs.

cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: undefined symbol: _ZN8opt_pass14set_pass_paramEjb
cc1: error: cannot load plugin ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: ./scripts/gcc-plugins/arm_ssp_per_task_plugin.so: undefined symbol: _ZN8opt_pass14set_pass_paramEjb
make[1]: *** [scripts/Makefile.build:250: scripts/mod/empty.o] error 1

 - download and install the AM335x SDK9.3 release from ti.com;
 - go to the SDK installation directory;
 - run command "make linux", which compiles the kernel zImage, modules, and dtbs;
 - use your prefer methods to populate the kernel zImage, modules, and dtb to your board root filesystem.

I don't understand "make linux".
In which directory should I run it?

I built with the following steps according to software-dl.ti.com/.../Overview_Building_the_SDK.html .

  $ sudo apt install libgmp3-dev libmpc-dev
  $ git clone https://git.ti.com/git/arago-project/oe-layersetup.git tisdk
  $ cd tisdk/
  $ ./oe-layertool-setup.sh -f configs/processor-sdk/processor-sdk-09.03.05.02-legacy-config.txt
  $ cd build/
  $ . conf/setenv
  $ MACHINE=am335x-evm bitbake -k tisdk-base-image
  $ cd ../..
  $ cp -a tisdk/build/arago-tmp-default-glibc/arago-tmp-default-glibc/work-shared/am335x-evm/kernel-source ./linux
  $ cd linux
  $ git apply 7776.0001-soc-ti-omap-prm-Fix-STANDBY-handling-of-PER-power-do.patch
  $ git apply 7776.0002-ARM-OMAP2-Fix-l4ls-clk-domain-handling-in-STANDBY.patch
  $ source /opt/arago-2023.10/environment-setup-armv7at2hf-neon-oe-linux-gnueabi
  $ export MACHINE=am335x-evm
  $ export ARCH=arm
  $ make mrproper
  $ make multi_v7_defconfig ti_multi_v7_prune.config no_smp.config
  $ make 
    I got the above error!

I built toolchain as below.

$ MACHINE=am335x-evm bitbake -k meta-toolchain-arago-tisdk

My Ubunutu version is 22.04LTS.

It might be the same problem as below.

 lists.openembedded.org/.../89633477

Hi Katsuyama-san,

Takeharu Katsuyama said:

I don't understand "make linux".
In which directory should I run it?

I mentioned it in the instruction, "go to the SDK installation directory". The directory has the following content.

dev@ula:am335x-plsdk-9352$ ls
bin            example-applications    filesystem  linux-devkit  manifest  Rules.make
board-support  external-toolchain-dir  licenses    makerules     Makefile  setup.sh

Takeharu Katsuyama said:

I built with the following steps according to software-dl.ti.com/.../Overview_Building_the_SDK.html .

The procedure compiles the entire SDK, including the root filesystem. it would take hours to finish.

But the "make linux" instruction I mentioned only compile the kernel, which is much faster.

Hi Katsuyama-san,

Glad to hear to it working on your side now. Thanks for the update.

The two patches are under review now, they should be committed to the ti-linux-6.1.y branch on git.ti.com very soon.