PROCESSOR-SDK-AM62X: Booting Arago Kirkstone

Hi Michael,

our upcoming SDK v9.x series (next year) will be based on Kirkstone, and there is active development on that happening at the moment. I'd suspect basic things to be working already so I'm surprised by your "zero terminal output" comment but then I've not played with that myself yet. Let me go check with the development team tomorrow to see what the status is and give you some additional insight. In any case since this is really targeted for next year release it is not ready yet for official consumption / officially supported, so it would only be something we'd recommend you look at for prototyping/experimentation purposes, NOT for a product.

Regards, Andreas

Thank you for the reply Andreas. Our product launch is planned for next year, so depending on when SDK v9.x is available in 2022 that could work out for us.

In the interim, if we cannot boot and use Kirkstone we'll have to attempt to upgrade some packages to newer upstream versions using Dunfell. Specifically, we are looking at needing docker-compose 1.29.x, which on the surface looks to require a good amount of effort due to changes in meta-virtualization between Dunfell and Kirkstone. Similarly, we are looking to use Python3.9 or 3.10 instead of Dunfell's Python3.8.

Is there a way I could get debug output from the BootROM to help diagnose this error?

Hi Michael,

thanks for the background on your end, makes sense. I got a chance to talk to one of our developers about the state of 'kirkstone' and he basically said as long as you build the HEAD commits for the different Yocto trees (the arago-kirkstone-config.txt seems to setup just that) it should result in images that boot. The biggest items at the moment being worked out are around GPU support which is not done/working yet, so you would want to build minimal images or turn off DISTRO_FEATURE=gpu. Given this context I would not expect you see a "blank screen" boot issue however, and I'd like to help to dig into this a bit further.

1. I assume when you build 'dunfell' images your U-Boot console comes up as expected, using the same hardware / board / UART setup, correct?

2. Assuming your basic HW setup is known-good (1), no console output usually points to some type of issue with the initial boot binary, also known as tiboot3.bin. There's several build steps involved with that including but not limited to certificate-wrapping done post U-Boot SPL R5 build using the "k3-image-gen" tool, that will add in a "TIFS" firmware image and the certificate and create the final boot image. So either U-Boot SPL R5 is mis-configured in some way, or there is some issue with the post-processing/build steps.

I'll see I can kick off a build here locally to see if I can get Kirkstone to build and boot and then report back here but it may take me a day or two.

Regards, Andreas

Hello Andreas,

Thank you for the follow up. I did encounter a related GPU issue. Specifically that Weston is looking for libgbm 21.1.1, but ti-img-rogue-umlibs.git provides 21.0.x. I ended up working around this the naive / hard way by removing the gpu and matric features from tisdk-default-image.

diff --git a/meta-arago-distro/recipes-core/images/tisdk-default-image.bb b/meta-arago-distro/recipes-core/images/tisdk-default-image.bb
index 8cd104299bd9a2e6f7411d3433f21bd49dfa0990..eff3ea81025f2281f29102091e44c69e952597e0 100644
--- a/meta-arago-distro/recipes-core/images/tisdk-default-image.bb
+++ b/meta-arago-distro/recipes-core/images/tisdk-default-image.bb
@@ -12,19 +12,12 @@ IMAGE_INSTALL += "\
packagegroup-arago-console \
packagegroup-arago-base-tisdk \
ti-test \
- ${@bb.utils.contains('MACHINE_FEATURES','gpu','packagegroup-arago-tisdk-graphics','',d)} \
- ${@bb.utils.contains('MACHINE_FEATURES','gpu','packagegroup-arago-tisdk-gtk','',d)} \
- ${@bb.utils.contains('MACHINE_FEATURES','gpu','packagegroup-arago-tisdk-qte','',d)} \
${@['','packagegroup-arago-tisdk-opencl'][oe.utils.all_distro_features(d, 'opencl', True, False) and bb.utils.contains('MACHINE_FEATURES', 'dsp', True, False, d)]} \
packagegroup-arago-tisdk-connectivity \
packagegroup-arago-tisdk-crypto \
- packagegroup-arago-tisdk-matrix \
- packagegroup-arago-tisdk-matrix-extra \
- packagegroup-arago-tisdk-multimedia \
packagegroup-arago-tisdk-amsdk \
packagegroup-arago-tisdk-addons \
packagegroup-arago-tisdk-addons-extra \
- ${@bb.utils.contains('MACHINE_FEATURES','gpu','packagegroup-arago-tisdk-hmi','packagegroup-arago-base-tisdk-server-extra',d)} \
ti-analytics \
ti-demos \
"

Building tisdk-base-image instead of the 'patched' tisdk-default-image resulted in the same behaviour. I will rerun the build to have logs to grep. I am afraid I had to free up some space for a Dunfell build.

Answers to questions:

1. Yes. The same hardware setup boots the Dunfell images successfully

2. The EVM I have appears to be Silicon 1.0 ( mark number XAM6254A ). If I understand this document for the AM64x correctly, and if it applied equivalently to the AM62x, https://software-dl.ti.com/processor-sdk-linux/esd/AM64X/08_04_01_04/exports/docs/linux/Foundational_Components_Migration_Guide.html, could we be looking at a situation where the Kirkstone builds are assuming the SoC is HS-FS capable, and producing incompatible images for Silicon 1.0? That is, building assuming Silicon 2.0, but what is on my EVM is 1.0? I will look into this area in my build logs to try to understand tiboot3.bin and k3-image-gen work flow.

Thank you,

Michael

Hi Michael,

I was able to successfully build 'tisdk-base-image' based off configs/arago-kirkstone-config.txt without any modifications/issues (the base image has no graphics so it's a good way for quick tests with less dependencies). I had a quick look at the hexdump of the resulting arago-tmp-default-glibc/deploy/images/am62xx-evm/tiboot3.bin initial boot image and it seems to have all the needed pieces baked in. I'll try in a bit here to see if it is bootable and report back.

Michael Magyar said:

could we be looking at a situation where the Kirkstone builds are assuming the SoC is HS-FS capable, and producing incompatible images for Silicon 1.0? That is, building assuming Silicon 2.0, but what is on my EVM is 1.0

This is a very good thought and if you were using AM64x this would definitely an area to scrutinize but for AM62x all devices available in the public at this time are GP devices (no HS variants) and also there's only one silicon revision (SR) so things should _still_ be very straightforward

Anyways let me play with the image and report back here.

Regards, Andreas

Michael Magyar said:

I will look into this area in my build logs to try to understand tiboot3.bin and k3-image-gen work flow.

Yocto will do all those things for you, but if you want a spelled-out description of that U-Boot build flow that would enable you to do it all manually, here are the steps:

https://dev.ti.com/tirex/explore/node?node=A__AFjV37M4u-vbrb9WTUWeZw__linux_academy_am62x__XaWts8R__LATEST

Regards, Andreas

Michael,

so my kirkstone image also doesn't boot, which means I can re-create your problem. The issue seems to be around the image generation/wrapping of tiboot3.bin. The image generator used on 'kirkstone' an the way it is setup/used is way behind where it is on current 'dunfell' (our production branches), and there are some pending patches that haven't gotten merged yet into 'kirkstone'. Unless you have a ton of time to spare I would NOT recommend you trying to fix this. This particular piece of our Yocto setup is pretty tricky. But let me work internally to provide a set of missing patches or something to get you going. You running into this has definitely gotten some attention with us to get this working again, it'll be in our all's best interest. Will get back on this soon.

Regards, Andreas

Hello Andreas,

Thank you for confirming ( even before my local rebuild was able to complete ). I will be able to focus on our m4f co-processor firmware requirements, and a few other components while the internal TI team guides those patches onto the Kirkstone branchs.

Thank you for the link to the manual uBoot instructions. While I don't have the spare time to investigate deeply, I am curious about this area. In previous projects with different vendors I have had to modify uBoot, and uBoot's environment uEnv.txt to meet the product's needs. For the AM62x platform is the correct area in Yocto:

./meta-arago/meta-arago-distro/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bb and
./meta-arago/meta-arago-distro/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile/Makefile_sysfw-image
./meta-ti/recipes-bsp/ti-sci-fw/ti-sci-fw_git.bb
./meta-ti/recipes-bsp/ti-linux-fw/ti-linux-fw.inc
?

My best guess at the moment would be out of date target branches for git://git.ti.com/git/k3-image-gen/k3-image-gen.git and git://git.ti.com/git/processor-firmware/ti-linux-firmware.git? I can also learn from the diff when the patches are applied . Very likely I'm way off base here as this is a new platform for me.

Thank you for your help with this issue.

Michael

Hi Michael,

I figured out why the current Kirkstone HEAD/HEAD/HEAD builds don't boot. Actually the created initial binary boot artifact tiboot3.bin is perfectly bootable (so correctly signed, containing the correct image artifact), but there's something going wrong with the SD card creation process itself. I traced it down to a change that was made to support EFI. If you undo this change as shown below (this is in meta-ti), it should create a bootable image:

$ git diff
diff --git a/meta-ti-bsp/conf/machine/include/k3.inc b/meta-ti-bsp/conf/machine/include/k3.inc
index 8a32e0fe..46b13257 100644
--- a/meta-ti-bsp/conf/machine/include/k3.inc
+++ b/meta-ti-bsp/conf/machine/include/k3.inc
@@ -45,7 +45,7 @@ IMAGE_EFI_BOOT_FILES ?= "${IMAGE_BOOT_FILES}"
 EFI_PROVIDER ?= "grub-efi"
 MACHINE_FEATURES += "efi"

-WKS_FILE ?= "sdimage-2part-efi.wks"
+WKS_FILE ?= "sdimage-2part.wks"
 do_image_wic[depends] += "virtual/bootloader:do_deploy"
 do_image_wic[mcdepends] += "mc::k3r5:virtual/bootloader:do_deploy mc::k3r5:ti-sci-fw:do_deploy"
 do_image_tar[mcdepends] += "mc::k3r5:virtual/bootloader:do_deploy mc::k3r5:ti-sci-fw:do_deploy"

Can you please try this on your end.

Note that this is just a quick hack to get you going. A proper fix would involve trying to figure out why exactly the boot image creation fails. I compared partition maps they do look the same. Perhaps a part of the image gets overwritten by the "grub" bootloader, leaving the ROM unable to perform the boot. And a "better HACK" may involve removing "efi" from MACHINE_FEATURES, and perhaps other things. But this is definitely the right area to poke.

Regards, Andreas

Thank you Andreas! I rebuilt over night with that commit reverted locally. I am able to boot Kirkstone with the tisdk-base-image.

Can you please point me to some documentation about EFI support on the AM62x devices? I had not noticed that supported, and feel I should do some reading on the implications and how I might use it.