AM6442: rngd.service "No HW SUPPORT"

Hi Jakub,

Do you see the same issue if you use the prebuilt WIC image in the Processor SDK package?

Hi Bin Liu

No, I there is no such issue.

Hi Jakub,

I am routing your query to our Yocto expert for comments.

Hi Jakub,

yes that could be a Kernel configuration issue. To figure out what's going on I'd suggest comparing the active config running our SDK image that you confirmed earlier doesn't exhibit this issue against the active config of running your own image. One of the best ways to do this avoiding any confusion of if you are really looking at the right data is to harvest the config from a live/running system like this:

root@am64xx-evm:~# zcat /proc/config.gz
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 5.10.140 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="aarch64-none-linux-gnu-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=90201
CONFIG_LD_VERSION=233010000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0
<snip>

Look for anything that could be related to that. Doing a 'diff' would be a good idea here.

If this doesn't yield any insight I'd then compare the bootlogs in detail for any clues.

Regards, Andreas

Thank you for response. 

Unfortunately the configuration for Random Number Generator is the same in my image and SDK image. Message "No HW SUPPORT" comes from rng-tools/6.15-r0/git/rngd_rndr.c.

In rngd_rndr.c below function return value 1. 

int init_rndr_entropy_source(struct rng *ent_src)
{
        if (!(getauxval(AT_HWCAP2) & HWCAP2_RNG)) {
                message_entsrc(ent_src, LOG_DAEMON|LOG_INFO, "No HW SUPPORT\n");
                return 1;
        }
        message_entsrc(ent_src,LOG_DAEMON|LOG_INFO, "Enabling aarch64 RNDR rng support\n");
        if (ent_src->rng_options[DARN_OPT_AES].int_val && init_openssl(ent_src))
                return 1;
        return 0;
}

I read ( here: https://man7.org/linux/man-pages/man3/getauxval.3.html) that AT_HWCAP stands for "An architecture and ABI dependent bit-mask". On the same hardware but with SDK linux image this AT_HWCAP is different and init_rmdr_entropy_source return sucessfully.  How could that be?

I also changed status of rng device tree node:

crypto: crypto@40900000 {

		compatible = "ti,am64-sa2ul";

		reg = <0x00 0x40900000 0x00 0x1200>;

		power-domains = <&k3_pds 133 TI_SCI_PD_SHARED>;

		#address-cells = <2>;

		#size-cells = <2>;

		ranges = <0x00 0x40900000 0x00 0x40900000 0x00 0x30000>;

		dmas = <&main_pktdma 0xc001 0>, <&main_pktdma 0x4002 0>,

		       <&main_pktdma 0x4003 0>;

		dma-names = "tx", "rx1", "rx2";



		rng: rng@40910000 {

			compatible = "inside-secure,safexcel-eip76";

			reg = <0x00 0x40910000 0x00 0x7d>;

			interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;

			clocks = <&k3_clks 133 1>;

			status = "okey";

		};

	};

But nothing changed...

Jakub,

what Kernel tree are you using? For devices like AM64x/AM62x you should be using the current/latest ti-linux-5.10.y from https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/log/?h=ti-linux-5.10.y at this time for best-possible coverage (as we build out the upstream story over time).

Regards, Andreas

I'm using the same kernel ti-linux-5.10.y from TI git repository.

What on the AM6442-SK board provides hardware support for the random number generator? Is this functionality built into the CPU? I wanted to use an external TPM but it turned out that the there is a lack of tpm-rng driver module (file tpm-rng.c not exists in kernel sources)  so using this option is not possible.

You can find out how the HW support works behind the scenes by grepping the Kernel source tree for the DTS compatible string....

$ git grep inside-secure,safexcel-eip76

This will lead you to the driver file at drivers/char/hw_random/omap-rng.c. To debug this further one method (one of many) would be to instrument that code such as the probe function with printk() statements to see if it actually gets run, and if not to figure out why not.

Your specific issue could actually be because your device tree seems mis-configured. Your code snippet posted earlier says status = "okey" but the DTS parsing logic expects either "ok" or "okay" to activate a device, see drivers/of/fdt.c. Can you please double-check this aspect.

Regards, Andreas

Jakub,

a colleague just pointed out the below documentation page specific to the HW RNG, please review. There's a lot of good discussion/detail there that might be of interest to you.

https://software-dl.ti.com/processor-sdk-linux/esd/AM64X/08_04_01_04/exports/docs/linux/Foundational_Components/Kernel/Kernel_Drivers/Crypto.html#:%7E:text=Using%20the%20TRNG%20Hardware%20Accelerator

Regards, Andreas

Thank you for response

I fixed typo in rng node, as you pointed. Also I enabled CONFIG_CRYPTO_DEV_SA2UL=y. Booting process hangs on function of_get_omap_rng_device_details from omap-rng.c. To be more precise function hangs after getting irq number (this number is 346). After that there is function call devm_request_irq(dev, irq, omap4_rng_irq,
IRQF_TRIGGER_NONE, dev_name(dev), priv); and this function never return. Do you have idea what is wrong?

Also in this link there is a statement:

"The Crypto API Driver is a set of Linux drivers that provide access to the hardware cryptographic accelerators available on AM335x/AM437x/AM57x/DRA7/AM65/J721E/J721S2/AM62 devices."

but I have AM6442 board.

On another hand few lines later there is array:

* AM335X       : MD5, SHA1, SHA224, SHA256, AES, DES
* AM437X       : MD5, SHA1, SAH224, SHA256, SHA384, SHA512, AES, DES, DES3DES
* AM57x/DRA7   : AES, DES, DES3DES
* AM65x/J721e  : AES, DES3DES, SHA1, SHA256, SHA512
* J721S2       : AES, DES3DES, SHA1, SHA256, SHA512
* AM64X/AM62X  : AES, SHA256, SHA512

So how to interprate this?

Jakub Grad said:

 To be more precise function hangs after getting irq number (this number is 346). After that there is function call devm_request_irq(dev, irq, omap4_rng_irq,
IRQF_TRIGGER_NONE, dev_name(dev), priv); and this function never return. Do you have idea what is wrong?

I took a step back and looked into this a bit more and and things become a little clearer now. Enabling the RNG device in the DTS is NOT the correct way to solve your issues. It is disabled intentionally for Linux use in our DTS files, as the device is used from OP-TEE. Your issues getting the interrupts assigned are likely related to some conflicts with OP-TEE using that peripheral. So basically you should undo your Kernel config / DTS changes in this regard. And then start looking if there is anything OP-TEE related that is different in your setup vs. the TI SDK v8.4 build that doesn't show the "No HW SUPPORT" error during boot.

Regards, Andreas

Jakub Grad said:

"The Crypto API Driver is a set of Linux drivers that provide access to the hardware cryptographic accelerators available on AM335x/AM437x/AM57x/DRA7/AM65/J721E/J721S2/AM62 devices."

but I have AM6442 board.

Good catch but this is a documentation inaccuracy, there should be no difference between all the devices including AM64. I took a note so we can get this corrected.

Jakub Grad said:

On another hand few lines later there is array:

From discussions I vaguely recall that support for some older standards that are no longer recommended to be used has been removed from the HW IP, most likely to save a little cost/die size area. Those differences are not relevant for the issue you are seeing.

Regards, Andreas

Andreas Dannenberg said:

It is disabled intentionally for Linux use in our DTS files, as the device is used from OP-TEE

See this commit below for example. OP-TEE is definitely the "owner" of the RNG HW peripheral. This may give you some clues to debug your own custom tree setup.

https://github.com/OP-TEE/optee_os/commit/d6c5d0037b46f46caf71d67d7825d4b722cbcddf

Regards, Andreas

Thank you for response.

I reverted all config/device tree changes as you suggest. In kernel menuconfig I see 3 options related to OP-TEE:
CONFIG_OPTEE=y
CONFIG_HW_RANDOM_OPTEE=y
CONFIG_SHM_NUM_PRIV_PAGES=1

Comparing this to SDK image .config only CONFIG_HW_RANDOM_OPTEE differs and is set to "m". All other options are the same. Setting CONFIG_HW_RANDOM_OPTEE=y or CONFIG_HW_RANDOM_OPTEE =m does not resolve problem...

dmesg shows thath optee driver is set up successfully. So I don't really know what is wrong...

systemctl status rngd.service still raport "No HW SUPPORT" and htop shows 100% load on CPU executing /usr/sbin/rngd -f -r /dev/hwrng

Andreas Dannenberg said:

See this commit below for example. OP-TEE is definitely the "owner" of the RNG HW peripheral. This may give you some clues to debug your own custom tree setup.

https://github.com/OP-TEE/optee_os/commit/d6c5d0037b46f46caf71d67d7825d4b722cbcddf

Regards, Andreas

There is no such directory like core/arch/arm/plat-k3 in kernel source tree. I found only drivers/char/hw_random/optee-rng.c so I cannot check if optee have the same config.

Jakub Grad said:

There is no such directory like core/arch/arm/plat-k3 in kernel source tree. I found only drivers/char/hw_random/optee-rng.c so I cannot check if optee have the same config.

The link I sent was a reference to a commit in OP-TEE, not the Kernel. My point was that you should double check you are using a suitable version of OP-TEE and that everything related is setup correctly. Since we are using upstream OP-TEE for our devices (from that very Github link referenced earlier) I wouldn't really expect you are using something different but still wanted to double check.

But just to test, can you use your own filesystem image (rootfs/Kernel/DTB) together with the unmodified U-Boot from the AM64 SDK v8.4? Meaning tiboot3.bin, tispl.bin, u-boot.img from the SDK installer. OP-TEE gets baked into those boot files (specifically, into tispl.bin), so using this set of known-good boot files may help narrow down where your issue is.

Regards, Andreas

Thank you for response.

Unfortunately nothing helps. I also installed SDK 8.4 image to check how board behaves. In htop also I see 100 % load of CPU by hwrng process but for much shorter amount of time. In our project we have TPM connected to board. I wondering if task of generating random numbers cannot be done by TPM? I see that in kernel config there is an option CONFIG_HW_RANDOM_TPM. It is possible to some how replace OP-TEE with TPM? What steps should I do to enable usage TPM as hardware random generator? Can you provide some example of how should looks like device tree node for this purpose?

Let me add that in our Yocto Linux TPM is visible as the /dev/tpm device and can be used in user space.

Is it possible to completely disable this OPTEE? In kernel config I found option: CONFIG_OPTEE=y but should I do something more? I want to completely abandon OPTEE and set the TPM chip as TRNG.

I set up these kconfig options:

CONFIG_OPTEE=n
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TPM=y

and I can see that in  cat /sys/class/misc/hw_random/rng_available there is: tpm-rng-0

also in: cat /sys/class/misc/hw_random/rng_current there is: tpm-rng-0

So it seems that tmp as /dev/hwrng is enabled but still after boot up CPU load is 100%...

Why?

Hi Jakub,

OPTEE does more than just RNG. An external TPM can enhance OPTEE. And you can have multiple RNG providers at the same time. I would not recommend removing/disabling OPTEE at this time but instead keep the SDK-provided design intact (also removing OPTEE completely would involve re-building ATF if I remember correctly, it's been a while I worked on this part).

Seems like your key concern really is around the 100% CPU usage for longer-than-expected time. You already started bi-secting the issue which is good, and concluded that even replacing the 3 U-Boot files with the ones coming from the current AM64x SDK v8.4 did not improve things. How about continuing this part? Can you use the TI Kernel from the SDK as-is?

Regards, Andreas

Hi Andreas,

Isn't it strange that even in the SDK image, the /usr/sbin/rngd -f -r /dev/hwrng task take 100% CPU load? Since there is hardware support for the random number generator on AM6442, shouldn't this overhead be much lower?

Hi Jakub,
I'd like to understand this in a bit more in detail:

1. How exactly do you use htop to determine the 100% CPU load for /usr/sbin/rngd? Do you quickly log into the console right after boot and run htop?
2. How long is the 100% CPU load caused by the SDK image as per your observation?
3. How long is the 100% CPU load caused by your own image as per your observation?

Regards, Andreas

Hi Andreas,

Bellow are answers for your questions:

1. Yes, after I log in immediately open htop.

2. On SDK image (version 8.4) it takes about 25 second long.

3. On my Yocto image it takes 2 minutes and 3 seconds. Yocto image has isolated one CPU so on two cores it could be faster, but 25 seconds is still unacceptable. 

BR

Jakub

Hi Jakub,

thanks for the inputs, that's very helpful.

Jakub Grad said:

2. On SDK image (version 8.4) it takes about 25 second long

Not knowing the exact mechanics behind it even 25s sounds way too much to me. Let me reach out to one of our internal experts to see if this is in line with what is expected and get back to you.

Regards, Andreas

Hello Andreas

Any news regarding to this issue with RNG?

Hello Andreas

Can you tell me what is the status of HW RNG development?

Any update regarding this topic?

Hi Jakub,

I noticed this case has been open for a while already, please let me know if this still needs attention.

Regards, Andreas