Other Parts Discussed in Thread: TMDX654IDKEVM
Dear TI team,
We are using the PROCESSOR-SDK-LINUX-RT-AM65X "Version: 08.06.00.47" to generate the Linux image for our custom board.
We have found that processes using the AM65x hardware cryptographic accelerator can get stuck in an Uninterruptible Sleep (D) State.
We were able to reproduce the issue using the "speed" command from the cryptodev-tests package.
If two instances of the speed command are running concurrently, the two processes can get stuck in the state:D (see the next traces generated using /proc/sysrq-trigger):
[ 4596.599966] task:speed state:D stack: 0 pid: 1512 ppid: 1511 flags:0x00000001
[ 4596.599991] Call trace:
[ 4596.599994] __switch_to+0xc0/0x118
[ 4596.600019] __schedule+0x298/0x818
[ 4596.600032] schedule+0xb0/0x148
[ 4596.600041] schedule_timeout+0x250/0x2b8
[ 4596.600052] wait_for_completion+0x84/0x110
[ 4596.600062] cryptodev_cipher_encrypt+0x9c/0xf0 [cryptodev]
[ 4596.600091] hash_n_crypt.isra.0+0x11c/0x128 [cryptodev]
[ 4596.600112] crypto_run+0xcc/0x490 [cryptodev]
[ 4596.600132] cryptodev_ioctl+0x3c8/0xb20 [cryptodev]
[ 4596.600152] __arm64_sys_ioctl+0xa8/0xf0
[ 4596.600164] el0_svc_common.constprop.0+0x78/0x1c8
[ 4596.600179] do_el0_svc+0x24/0xa0
[ 4596.600190] el0_svc+0x14/0x20
[ 4596.600198] el0_sync_handler+0xb0/0xb8
[ 4596.600206] el0_sync+0x180/0x1c0
[ 4596.600216] task:speed state:D stack: 0 pid: 1513 ppid: 1451 flags:0x00000001
[ 4596.600229] Call trace:
[ 4596.600232] __switch_to+0xc0/0x118
[ 4596.600243] __schedule+0x298/0x818
[ 4596.600252] schedule+0xb0/0x148
[ 4596.600260] schedule_timeout+0x250/0x2b8
[ 4596.600270] wait_for_completion+0x84/0x110
[ 4596.600279] cryptodev_cipher_encrypt+0x9c/0xf0 [cryptodev]
[ 4596.600300] hash_n_crypt.isra.0+0x11c/0x128 [cryptodev]
[ 4596.600321] crypto_run+0xcc/0x490 [cryptodev]
[ 4596.600341] cryptodev_ioctl+0x3c8/0xb20 [cryptodev]
[ 4596.600361] __arm64_sys_ioctl+0xa8/0xf0
[ 4596.600370] el0_svc_common.constprop.0+0x78/0x1c8
[ 4596.600382] do_el0_svc+0x24/0xa0
[ 4596.600392] el0_svc+0x14/0x20
[ 4596.600400] el0_sync_handler+0xb0/0xb8
[ 4596.600408] el0_sync+0x180/0x1c0
The issue is not reproduced if only an instance of the speed command is running.
In a TMDX654IDKEVM board were it is installed the latest PROCESSOR-SDK-LINUX-RT-AM65X (Version: 09.01.00.01) TI Linux image (tisdk-default-image-am65xx-evm)
we have repeated the test explained above, with the same result:
root@am65xx-evm:~# uname -a
Linux am65xx-evm 6.1.46-rt13-gb4f1767f9e #1 SMP PREEMPT_RT Thu Nov 30 22:21:00 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
root@am65xx-evm:~# echo w > /proc/sysrq-trigger
[ 668.499478] sysrq: Show Blocked State
[ 668.503397] task:speed state:D stack:0 pid:1019 ppid:1018 flags:0x00000205
[ 668.511763] Call trace:
[ 668.514235] __switch_to+0xc8/0x120
[ 668.517745] __schedule+0x3d4/0x680
[ 668.521245] schedule+0x58/0xd0
[ 668.524398] schedule_timeout+0xc0/0xf0
[ 668.528242] wait_for_completion+0x74/0x160
[ 668.532435] cryptodev_cipher_encrypt+0x94/0xf0 [cryptodev]
[ 668.538035] hash_n_crypt.isra.0+0xec/0x120 [cryptodev]
[ 668.543279] crypto_run+0xc8/0x470 [cryptodev]
[ 668.547745] cryptodev_ioctl+0x3f4/0xa00 [cryptodev]
[ 668.552730] __arm64_sys_ioctl+0xa0/0xe0
[ 668.556665] el0_svc_common.constprop.0+0x5c/0x134
[ 668.561466] do_el0_svc+0x1c/0x2c
[ 668.564789] el0_svc+0x24/0xf0
[ 668.567850] el0t_64_sync_handler+0xb4/0x130
[ 668.572126] el0t_64_sync+0x148/0x14c
[ 668.575813] task:speed state:D stack:0 pid:1021 ppid:1020 flags:0x00000205
[ 668.584168] Call trace:
[ 668.586616] __switch_to+0xc8/0x120
[ 668.590125] __schedule+0x3d4/0x680
[ 668.593623] schedule+0x58/0xd0
[ 668.596774] schedule_timeout+0xc0/0xf0
[ 668.600616] wait_for_completion+0x74/0x160
[ 668.604808] cryptodev_cipher_encrypt+0x94/0xf0 [cryptodev]
[ 668.610400] hash_n_crypt.isra.0+0xec/0x120 [cryptodev]
[ 668.615645] crypto_run+0xc8/0x470 [cryptodev]
[ 668.620160] cryptodev_ioctl+0x3f4/0xa00 [cryptodev]
[ 668.625146] __arm64_sys_ioctl+0xa0/0xe0
[ 668.629077] el0_svc_common.constprop.0+0x5c/0x134
[ 668.633876] do_el0_svc+0x1c/0x2c
[ 668.637199] el0_svc+0x24/0xf0
[ 668.640261] el0t_64_sync_handler+0xb4/0x130
[ 668.644538] el0t_64_sync+0x148/0x14c
