AM625: SDK 08: AM6252 remoteproc crash on boot

Hello Jonathan,

Remoteproc is unrelated to your error. Remoteproc is the Linux driver that controls non-Linux cores on the processor. So we would expect it to initialize the M4F core, and then attach to the device manager (DM) R5F core, which should have been booted earlier in the boot flow. That is exactly what is happening.

Did you disable the additional A53 cores that are not on the AM6252 device? Please reference
https://dev.ti.com/tirex/explore/node?node=A__AGMop5fjikXYw819LJNi-w__linux_academy_am62x__XaWts8R__LATEST

I have not tried it on my end, but I think there could be a watchdog error if you do not use the Linux devicetree to properly disable the A53 cores that do not exist.

Regards,

Nick

Nick Saulnier said:

Remoteproc is unrelated to your error. Remoteproc is the Linux driver that controls non-Linux cores on the processor. So we would expect it to initialize the M4F core, and then attach to the device manager (DM) R5F core, which should have been booted earlier in the boot flow. That is exactly what is happening.

Did you disable the additional A53 cores that are not on the AM6252 device? Please reference

Removing the cores does not remove this kernel dump.

Thanks for the pointer to the watchdog driver.

[   17.633071]  rti_wdt_probe+0x1e8/0x4f0 [rti_wdt]

The device tree enables 5 different nodes for this driver. main_rti0, main_rti1, main_rti2, main_rti3, main_rti15.  I assume 0-3 represent the different arm cores. Any idea what rti15 represents?

Disabling main_rti2, and main_rti3 was required to remove the kernel dump. Note removing the cores from the cluster0 node was also required to prevent dtb build issues.  This is mentioned in other e2e posts but not in the linked linux academy.

This should be added to https://dev.ti.com/tirex/explore/node?node=A__AGMop5fjikXYw819LJNi-w__linux_academy_am62x__XaWts8R__LATEST

Great, did you forget to delete the cpu@2 and cpu@3 nodes or are those optional to remove?

Hello Jonathan,

I am not sure I understand your question.

At this point in time, the boot flow does not check for AM6254 vs AM6252 vs AM6231 vs etc. So we just write a single Linux devicetree for the superset device AM6254, and then customers have to manually update their Linux devicetree for "subset" devices that do not have all the features of AM6254. 

I believe it is theoretically possible for uboot to check for the part number and then modify the Linux devicetree settings, but we do not currently have a requirement for that feature. So it might be added in the future, but I cannot guarantee it right now.

Your testing was helpful for me here because I only have superset devices on my desk, so I won't see this error, even if I disable the A53 cores on my superset device.

Regards,

Nick

Sorry Nick, I think i confused you. I meant on your FAQ.  It mentions deleting the cluster0/cores and disabling the rti's but doesn't delete the cpu nodes. 

Nick Saulnier said:

I believe it is theoretically possible for uboot to check for the part number and then modify the Linux devicetree settings, but we do not currently have a requirement for that feature. So it might be added in the future, but I cannot guarantee it right now.

I am working on this for our SOM.  I'm currently going to use our factory config to determine number of cores and if a GPU is present.  Do you know if there are any registers in the processor that could be used instead to determine this info?

This code works as is, just needs to detect the number of cores and if a gpu is present.

#include <common.h>
#include <dm/ofnode.h>
#include <fdt_support.h>
#include <linux/libfdt.h>

/*
 * Remove cpu nodes from the device tree based on the number of CPUs
 */
int ft_fixup_cpu(void *blob, struct bd_info *bd, int num_present_cpus)
{
	int ret;
	int nodeoffset;
	char buf[64];

	// Ex: For 2 present cpus, we want to disable cpu@2 and cpu@3
	for (int cpu_num = 3; cpu_num >= num_present_cpus; cpu_num--) {
		printf("%s: Disabling CPU %d\n", __func__, cpu_num);

		/* Find the CPU node */
		snprintf(buf, sizeof(buf), "/cpus/cpu@%d", cpu_num);
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		/* Delete the CPU node */
		ret = fdt_del_node(blob, nodeoffset);
		if (ret < 0) {
			printf("%s: Could not delete %s node\n", __func__, buf);
			return ret;
		}

		/* Find the cpu cluster node */
		snprintf(buf, sizeof(buf), "/cpus/cpu-map/cluster0/core%d", cpu_num);
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		/* Delete the cpu cluster node */
		ret = fdt_del_node(blob, nodeoffset);
		if (ret < 0) {
			printf("%s: Could not delete %s node\n", __func__, buf);
			return ret;
		}

		/* Disable the cpu watchdog */
		snprintf(buf, sizeof(buf), "main_rti%d", cpu_num);
		ret = fdt_status_disabled_by_alias(blob, buf);
		if (ret < 0) {
			printf("%s: Could not disable %s node\n", __func__, buf);
			return ret;
		}
	}

	return ret;
}

/*
 * Remove gpu nodes from the device tree if gpu is not present
 */
int ft_fixup_gpu(void *blob, struct bd_info *bd, bool gpu_present)
{
	int ret;

	if (!gpu_present) {
		printf("%s: Disabling GPU\n", __func__);

		/* Disable the gpu node */
		ret = fdt_status_disabled_by_alias(blob, "gpu");
		if (ret < 0) {
			printf("%s: Could not disable gpu node\n", __func__);
			return ret;
		}

		/* Disable the gpu watchdog */
		ret = fdt_status_disabled_by_alias(blob, "main_rti15");
		if (ret < 0) {
			printf("%s: Could not disable main_rti15 node\n", __func__);
			return ret;
		}
	}

	return ret;
}

/*
 * This function is called right before the kernel is booted. "blob" is the
 * device tree that will be passed to the kernel.
 */
int ft_system_setup(void *blob, struct bd_info *bd)
{
	int ret = 0;

	// Don't abort the boot if the device tree nodes can't be fixed up
	(void)ft_fixup_cpu(blob, bd, 1);

	(void)ft_fixup_gpu(blob, bd, false);

	return ret;
}

Hello Jonathan,

using /delete-node/ is sufficient. No need to make additional modifications to disable the unused A53 cores.

For the question about detecting in uboot, please reference this thread:
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1259573/processor-sdk-am62x-how-to-read-the-number-of-cpu-cores-under-uboot

If you need additional support around uboot, let me know and I'll pass your thread over to a team member with more uboot experience.

Regards,

Nick

Thanks for the link.  Any idea if its possible to determine the speed grade from a register?  Currently I'm expecting only the AM6231 SOMs to not support 1.4Ghz, but basing it off the speed grade would be better.

Nevermind that is also apart of the USER_ID register.

Glad you found it! Feel free to ask followup questions, I just may need to send more detailed uboot questions to another team member to comment.

If you wrote any cool code that you think we should add to uboot to auto-detect the part and adjust the devicetree file, feel free to attach it here and I'll create a new requirement with a link to your sample code.

Regards,

Nick

This code is working, though as noted in the comments I added some new aliases to the kernel device tree to make it easier.

// SPDX-License-Identifier: GPL-2.0+
/*
 * MitySOM-AM62x
 *
 * (C) Copyright 2023 Critical Link <www.criticallink.com>
 * 	Jonathan Cormier <jcormier@criticallink.com>
 */

#include <common.h>
#include <fdt_support.h>

#include <asm/io.h> /* readl */

// AM625 Datasheet, Table 5-1. Device Comparison
#define MMR0_JTAG_USER_ID	0x43000018

// DEVICE_ID [31:13]
#define MMR0_JTAG_USER_ID_DEVICE_ID_SHIFT	13
// C Feature (With PRUSS)
#define DEVICE_ID_C_AM6254	0x1D123
#define DEVICE_ID_C_AM6252	0x1D0A3
#define DEVICE_ID_C_AM6234	0x1D103
#define DEVICE_ID_C_AM6232	0x1D083
// G Feature (Without PRUSS)
#define DEVICE_ID_G_AM6254	0x1D127
#define DEVICE_ID_G_AM6252	0x1D0A7
#define DEVICE_ID_G_AM6251	0x1D067
#define DEVICE_ID_G_AM6234	0x1D107
#define DEVICE_ID_G_AM6232	0x1D087
#define DEVICE_ID_G_AM6231	0x1D047
// 6204/6202/6201 have no DSS
#define DEVICE_ID_G_AM6204	0x1D307
#define DEVICE_ID_G_AM6202	0x1D287
#define DEVICE_ID_G_AM6201	0x1D247

// SPEED [10:6] - 1 = A Speed, ..., 26 = Z Speed
#define MMR0_JTAG_USER_ID_SPEED_SHIFT	6
#define MMR0_JTAG_USER_ID_SPEED_MASK	0x1F
#define SPEED_G_300MHZ	7
#define SPEED_K_800Mhz	11
#define SPEED_S_1000MHZ	19
#define SPEED_T_1400MHZ	20

// TEMP [5:3]
#define MMR0_JTAG_USER_ID_TEMP_SHIFT	3
#define MMR0_JTAG_USER_ID_TEMP_MASK	0x7
#define TEMP_0_90_C	3
#define TEMP_N40_105_C	4
#define TEMP_N40_125_C	5

int am62_get_number_of_cpus(u32 device_id)
{
	switch (device_id) {
	case DEVICE_ID_C_AM6254:
	case DEVICE_ID_G_AM6254:
	case DEVICE_ID_C_AM6234:
	case DEVICE_ID_G_AM6234:
	case DEVICE_ID_G_AM6204:
		return 4;
	case DEVICE_ID_C_AM6252:
	case DEVICE_ID_G_AM6252:
	case DEVICE_ID_C_AM6232:
	case DEVICE_ID_G_AM6232:
	case DEVICE_ID_G_AM6202:
		return 2;
	case DEVICE_ID_G_AM6251:
	case DEVICE_ID_G_AM6231:
	case DEVICE_ID_G_AM6201:
		return 1;
	default:
		printf("Unable to detect cpu count, unknown MMR0_JTAG_USER_ID: 0x%08x\n", device_id);
		return 1; // Limit to 1 CPU
	}
}

bool am62_gpu_present(u32 device_id)
{
	switch(device_id) {
	case DEVICE_ID_G_AM6254:
	case DEVICE_ID_C_AM6254:
	case DEVICE_ID_G_AM6252:
	case DEVICE_ID_C_AM6252:
	case DEVICE_ID_G_AM6251:
		return true;
	case DEVICE_ID_G_AM6234:
	case DEVICE_ID_C_AM6234:
	case DEVICE_ID_G_AM6232:
	case DEVICE_ID_C_AM6232:
	case DEVICE_ID_G_AM6231:
	case DEVICE_ID_G_AM6204:
	case DEVICE_ID_G_AM6202:
	case DEVICE_ID_G_AM6201:
		return false;
	default:
		printf("Unable to detect gpu presence, unknown MMR0_JTAG_USER_ID: 0x%08x\n", device_id);
		return 0; // Disable GPU
	}
}

bool am62_pruss_present(u32 device_id)
{
	switch(device_id) {
	case DEVICE_ID_C_AM6254:
	case DEVICE_ID_C_AM6252:
	case DEVICE_ID_C_AM6234:
	case DEVICE_ID_C_AM6232:
		return true;
	case DEVICE_ID_G_AM6254:
	case DEVICE_ID_G_AM6252:
	case DEVICE_ID_G_AM6251:
	case DEVICE_ID_G_AM6234:
	case DEVICE_ID_G_AM6232:
	case DEVICE_ID_G_AM6231:
	case DEVICE_ID_G_AM6204:
	case DEVICE_ID_G_AM6202:
	case DEVICE_ID_G_AM6201:
		return false;
	default:
		printf("Unable to detect pruss presence, unknown MMR0_JTAG_USER_ID: 0x%08x\n", device_id);
		return false; // Disable PRUSS
	}
}

bool am62_dss_present(u32 device_id)
{
	switch(device_id) {
	case DEVICE_ID_C_AM6254:
	case DEVICE_ID_C_AM6252:
	case DEVICE_ID_C_AM6234:
	case DEVICE_ID_C_AM6232:
	case DEVICE_ID_G_AM6254:
	case DEVICE_ID_G_AM6252:
	case DEVICE_ID_G_AM6251:
	case DEVICE_ID_G_AM6234:
	case DEVICE_ID_G_AM6232:
	case DEVICE_ID_G_AM6231:
		return true;
	case DEVICE_ID_G_AM6204:
	case DEVICE_ID_G_AM6202:
	case DEVICE_ID_G_AM6201:
		return false;
	default:
		printf("Unable to detect dss presence, unknown MMR0_JTAG_USER_ID: 0x%08x\n", device_id);
		return false; // Disable DSS
	}
}

/*
 * Remove cpu nodes from the device tree based on the number of CPUs
 * Note: Requires alias nodes to be setup in the device tree for main_rti1/2/3/4
 */
int ft_fixup_cpu(void *blob, struct bd_info *bd, int num_present_cpus)
{
	int ret;
	int nodeoffset;
	char buf[64];

	printf("%s: Configuring device tree for %d cpus\n", __func__, num_present_cpus);

	// Ex: For 2 present cpus, we want to disable cpu@2 and cpu@3
	for (int cpu_num = 3; cpu_num >= num_present_cpus; cpu_num--) {
		/* Find the CPU node */
		snprintf(buf, sizeof(buf), "/cpus/cpu@%d", cpu_num);
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		/* Delete the CPU node */
		ret = fdt_del_node(blob, nodeoffset);
		if (ret < 0) {
			printf("%s: Could not delete %s node\n", __func__, buf);
			return ret;
		}

		/* Find the cpu cluster node */
		snprintf(buf, sizeof(buf), "/cpus/cpu-map/cluster0/core%d", cpu_num);
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		/* Delete the cpu cluster node */
		ret = fdt_del_node(blob, nodeoffset);
		if (ret < 0) {
			printf("%s: Could not delete %s node\n", __func__, buf);
			return ret;
		}

		/* Disable the cpu watchdog */
		snprintf(buf, sizeof(buf), "main_rti%d", cpu_num);
		ret = fdt_status_disabled_by_alias(blob, buf);
		if (ret < 0) {
			printf("%s: Could not disable %s node\n", __func__, buf);
			return ret;
		}
	}

	return ret;
}

/*
 * Remove 1.4 GHz speed grade from the device tree since the kernel won't auto detect it
 * Note: Requires alias nodes to be setup in the device tree for vdd_core
 */
int ft_fixup_speed_grade(void *blob, struct bd_info *bd, int cpu_speed_grade)
{
	int ret;
	int nodeoffset;
	char buf[32];

	if (cpu_speed_grade < SPEED_T_1400MHZ) {
		// The current TI kernel doesn't support changing the CORE voltage based on the speed grade
		// the mitysom device tree assumes the CPU can run at 1.4 GHz and sets the CORE voltage to 0.85V
		// so remove opp 1.4Ghz from the device tree and set the CORE voltage to 0.75V
		// Ideally the kernel would handle this for us by dynamically setting the CORE voltage based on the OPP
		printf("%s: Disabling 1.4 GHz speed grade and setting vdd_core to 0.75V\n", __func__);

		/* Find the opp node */
		snprintf(buf, sizeof(buf), "/opp-table/opp-1400000000");
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		/* Delete the CPU node */
		ret = fdt_del_node(blob, nodeoffset);
		if (ret < 0) {
			printf("%s: Could not delete %s node\n", __func__, buf);
			return ret;
		}

		/* 0.85V is only valid when running at 1.4Ghz, need to reduce core voltage to 0.75V */
		snprintf(buf, sizeof(buf), "vdd_core");
		nodeoffset = fdt_path_offset(blob, buf);
		if (nodeoffset < 0) {
			printf("%s: Could not find %s node\n", __func__, buf);
			return nodeoffset;
		}

		ret = fdt_setprop_u32(blob, nodeoffset, "regulator-min-microvolt", 750000);
		if (ret < 0) {
			printf("%s: Could not set regulator-min-microvolt for %s node\n", __func__, buf);
			return ret;
		}

		ret = fdt_setprop_u32(blob, nodeoffset, "regulator-max-microvolt", 750000);
		if (ret < 0) {
			printf("%s: Could not set regulator-min-microvolt for %s node\n", __func__, buf);
			return ret;
		}
	}

	return ret;
}

/*
 * Disable gpu nodes in the device tree if gpu is not present
 * Note: Requires alias nodes to be setup in the device tree for main_rti15 and gpu
 */
int ft_fixup_gpu(void *blob, struct bd_info *bd, bool gpu_present)
{
	int ret;

	printf("%s: Configuring device tree for GPU: %s\n", __func__, gpu_present ? "present" : "absent");

	if (!gpu_present) {
		/* Disable the gpu node */
		ret = fdt_status_disabled_by_alias(blob, "gpu");
		if (ret < 0) {
			printf("%s: Could not disable gpu node\n", __func__);
			return ret;
		}

		/* Disable the gpu watchdog */
		ret = fdt_status_disabled_by_alias(blob, "main_rti15");
		if (ret < 0) {
			printf("%s: Could not disable main_rti15 node\n", __func__);
			return ret;
		}
	}

	return ret;
}

/*
 * Disable pruss nodes in the device tree if pruss is not present
 * Note: Requires alias nodes to be setup in the device tree for pruss
 */
int ft_fixup_pruss(void *blob, struct bd_info *bd, bool pruss_present)
{
	int ret;

	printf("%s: Configuring device tree for PRUSS: %s\n", __func__, pruss_present ? "present" : "absent");

	if (!pruss_present) {
		/* Disable the pruss node */
		ret = fdt_status_disabled_by_alias(blob, "pruss");
		if (ret < 0) {
			printf("%s: Could not disable pruss node\n", __func__);
			return ret;
		}
	}

	return ret;
}

/*
 * Disable dss nodes in the device tree if dss is not present
 * Note: Requires alias nodes to be setup in the device tree for dss
 */
int ft_fixup_dss(void *blob, struct bd_info *bd, bool dss_present)
{
	int ret;

	printf("%s: Configuring device tree for DSS: %s\n", __func__, dss_present ? "present" : "absent");

	if (!dss_present) {
		/* Disable the dss node */
		ret = fdt_status_disabled_by_alias(blob, "dss");
		if (ret < 0) {
			printf("%s: Could not disable dss node\n", __func__);
			return ret;
		}
	}

	return ret;
}

/*
 * This function is called right before the kernel is booted. "blob" is the
 * device tree that will be passed to the kernel.
 */
int ft_system_setup(void *blob, struct bd_info *bd)
{
	int ret = 0;

	u32 user_id = readl(MMR0_JTAG_USER_ID);
	u32 device_id = user_id >> MMR0_JTAG_USER_ID_DEVICE_ID_SHIFT;
	int num_present_cpus = am62_get_number_of_cpus(device_id);
	int cpu_speed_grade = (user_id >> MMR0_JTAG_USER_ID_SPEED_SHIFT) & MMR0_JTAG_USER_ID_SPEED_MASK;
	bool gpu_present = am62_gpu_present(device_id);
	bool pruss_present = am62_pruss_present(device_id);
	bool dss_present = am62_dss_present(device_id);

	// Don't abort the boot if the device tree nodes can't be fixed up

	(void)ft_fixup_cpu(blob, bd, num_present_cpus);

	(void)ft_fixup_speed_grade(blob, bd, cpu_speed_grade);

	(void)ft_fixup_gpu(blob, bd, gpu_present);

	(void)ft_fixup_pruss(blob, bd, pruss_present);

	(void)ft_fixup_dss(blob, bd, dss_present);

	return ret;
}

Thanks for sharing your code with us Jonathan! I have gone ahead and filed a software requirement for TI to look at adding an autodetect functionality into uboot and linked to your code as an example.

Future readers, if you want to get an update on whether that requirement got accepted, implemented, etc, reference SITREQ-3302 (or this thread) when you ask.

Regards,

Nick