AM625: RPMessage character device files do not appear in /dev

Leon Rigoni

Part Number: AM625

Tool/software:

Dear TI team

I am trying to get RPMessage to work on a Yocto-built image for an AM625, but the character device files don't show up in /dev, even though the moudles are loaded and the MCU announced its service. I already had a working example with an image from the company Phytec, but since I am not using their board anymore, I want to go back to a TI-only build. Unfortunately I cannot find any significant difference between the two in order to find the error.

Do you have an idea what I could be missing? Thank you very much!

Please check the text file for information about my system and what I'm doing.

Fullscreen RPMsg_Debug_Info.txt Download

uname -a
Linux am62xx-evm 6.1.119-ti-ge4e8b16e66f5 #1 SMP PREEMPT Sat Dec  7 07:14:10 UTC 2024 aarch64 GNU/Linux


dmesg | grep rpmsg
[    3.738902] virtio_rpmsg_bus virtio0: rpmsg host is online
[    3.773832] virtio_rpmsg_bus virtio0: creating channel rpmsg_chrdev addr 0x1
[    3.829193] virtio_rpmsg_bus virtio1: rpmsg host is online


dmesg | grep remoteproc
[    3.566622] k3-m4-rproc 5000000.m4fss: configured M4 for remoteproc mode
[    3.580849] remoteproc remoteproc0: 5000000.m4fss is available
[    3.709686] remoteproc remoteproc0: powering up 5000000.m4fss
[    3.715889] remoteproc remoteproc0: Booting fw image am62-mcu-m4f0_0-fw, size 2400160
[    3.751410] remoteproc remoteproc0: remote processor 5000000.m4fss is now up
[    3.803573] remoteproc remoteproc1: 78000000.r5f is available
[    3.809608] remoteproc remoteproc1: attaching to 78000000.r5f
[    3.843133] remoteproc remoteproc1: remote processor 78000000.r5f is now attached
[    3.947905] remoteproc remoteproc2: 30074000.pru is available
[    3.954315] remoteproc remoteproc3: 30078000.pru is available


lsmod | grep rpmsg
rpmsg_ctrl 16384 0 - Live 0xffff800001281000
rpmsg_char 16384 1 rpmsg_ctrl, Live 0xffff80000122a000
virtio_rpmsg_bus 20480 0 - Live 0xffff800001171000
rpmsg_ns 16384 1 virtio_rpmsg_bus, Live 0xffff800001119000


cat /sys/class/remoteproc/remoteproc0/name
5000000.m4fss


cat /sys/class/remoteproc/remoteproc0/state
running


ls /sys/class/rpmsg/
rpmsg0       rpmsg_ctrl0  rpmsg_ctrl1


cat /sys/class/rpmsg/rpmsg0/name
rpmsg_chrdev


ls -l /dev | grep rpmsg
<no output, should be rpmsg0, rpmsg1, rpmsg_ctrl0, rpmsg_ctrl1>


Searched menuconfig for "rpmsg":
# CONFIG_RPMSG_TTY is not set
# CONFIG_SND_SOC_FSL_RPMSG is not set
# CONFIG_SND_SOC_IMX_RPMSG is not set
# CONFIG_CROS_EC_RPMSG is not set
# Rpmsg drivers
CONFIG_RPMSG=y
CONFIG_RPMSG_CHAR=m
CONFIG_RPMSG_CTRL=m
CONFIG_RPMSG_NS=m
CONFIG_RPMSG_MTK_SCP=m
CONFIG_RPMSG_QCOM_GLINK=y
CONFIG_RPMSG_QCOM_GLINK_RPM=y
CONFIG_RPMSG_QCOM_GLINK_SMEM=m
CONFIG_RPMSG_QCOM_SMD=y
CONFIG_RPMSG_VIRTIO=m
CONFIG_RPMSG_PRU=m
# end of Rpmsg drivers
# Rpmsg virtual device drivers
CONFIG_RPMSG_KDRV=y
# CONFIG_RPMSG_KDRV_DEMO is not set
# CONFIG_RPMSG_KDRV_DISPLAY is not set
CONFIG_RPMSG_KDRV_ETH_SWITCH=y
# end of Rpmsg virtual device drivers
CONFIG_SAMPLE_RPMSG_CLIENT=m


Searched menuconfig for "mailbox":
CONFIG_ARM_SCMI_TRANSPORT_MAILBOX=y
CONFIG_MAILBOX=y
CONFIG_APPLE_MAILBOX=y
# CONFIG_MAILBOX_TEST is not set


Device tree M4F node:
			m4fss@5000000 {
				compatible = "ti,am64-m4fss";
				reg = <0x00 0x5000000 0x00 0x30000 0x00 0x5040000 0x00 0x10000>;
				reg-names = "iram\0dram";
				ti,sci = <0x05>;
				ti,sci-dev-id = <0x09>;
				ti,sci-proc-ids = <0x18 0xff>;
				resets = <0x07 0x09 0x01>;
				firmware-name = "am62-mcu-m4f0_0-fw";
				wakeup-source;
				mboxes = <0x08 0x09>;
				memory-region = <0x0a 0x0b>;
				phandle = <0x66>;
			};


Device tree mailbox nodes and more:
			mailbox@4d000000 {
				compatible = "ti,am654-secure-proxy";
				#mbox-cells = <0x01>;
				reg-names = "target_data\0rt\0scfg";
				reg = <0x00 0x4d000000 0x00 0x80000 0x00 0x4a600000 0x00 0x80000 0x00 0x4a400000 0x00 0x80000>;
				interrupt-names = "rx_012";
				interrupts = <0x00 0x22 0x04>;
				phandle = <0x12>;
			};
...
		mailbox@43600000 {
			compatible = "ti,am654-secure-proxy";
			#mbox-cells = <0x01>;
			reg-names = "target_data\0rt\0scfg";
			reg = <0x00 0x43600000 0x00 0x10000 0x00 0x44880000 0x00 0x20000 0x00 0x44860000 0x00 0x20000>;
			status = "disabled";
			phandle = <0x76>;
		};
...
		mailbox@29000000 {
			compatible = "ti,am64-mailbox";
			reg = <0x00 0x29000000 0x00 0x200>;
			interrupts = <0x00 0x4c 0x04 0x00 0x4d 0x04>;
			#mbox-cells = <0x01>;
			ti,mbox-num-users = <0x04>;
			ti,mbox-num-fifos = <0x10>;
			phandle = <0x08>;

			mbox-m4-0 {
				ti,mbox-rx = <0x00 0x00 0x00>;
				ti,mbox-tx = <0x01 0x00 0x00>;
				phandle = <0x09>;
			};

			mbox-r5-0 {
				ti,mbox-rx = <0x02 0x00 0x00>;
				ti,mbox-tx = <0x03 0x00 0x00>;
				phandle = <0x0c>;
			};
		};
...
		mailbox0_cluster0 = "/bus@f0000/mailbox@29000000";
		mbox_m4_0 = "/bus@f0000/mailbox@29000000/mbox-m4-0";
		mbox_r5_0 = "/bus@f0000/mailbox@29000000/mbox-r5-0";


Code on the MCU M4F: I am using the following function calls:
RPMessage_waitForLinuxReady(SystemP_WAIT_FOREVER);
RPMessage_announce(ID_A53_0, localEndpt, "rpmsg_chrdev");

The debugger shows that both functions are called and return with success. The exact same MCU firmware also worked on Phytec's Linux. The SDK version is 09.01.00.39.

4 months ago

0 Nick Saulnier 4 months ago

TI__Guru** 101250 points

Hello Leon,

So far things look the way I would expect... can you show me your code that is failing to run?

Based on the fact that you are defining an RPMsg endpoint of 0x1, I assume that you are no longer using the default IPC RPMsg echo example (which defines endpoints at 13 & 14 by default). https://github.com/TexasInstruments/mcupsdk-core-k3/blob/k3_main/examples/drivers/ipc/ipc_rpmsg_echo_linux/ipc_rpmsg_echo.c

Please also show me your service and endpoint definitions in your MCU+ code.

I want to make sure you are defining a service name that Linux will recognize, since the service names have been changing around a bit recently. See https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1497459/faq-am625-how-to-get-the-linux-kernel-rpmsg-example-working-on-linux-sdk-10-0-10-1-11-0 for the discussion on ti.ipc4.ping-pong. There is a similar discussion going on around rpmsg_chrdev vs rpmsg-raw in the rpmsg-char driver.

Regards,

Nick

0 Leon Rigoni 4 months ago in reply to Nick Saulnier

Intellectual 315 points

Hello Nick,

Thanks a lot for the quick response.

Here are the two code fragments from Linux and MCU (stripped down a lot, of course):

Fullscreen Failing_code_Linux.txt Download

char ept_dev[RPMSG_DEV_NAME_MAXLEN] = {0};
rpmsg_char_init(nullptr);
rpmsg_char_dev_t* rcdev = rpmsg_char_open(M4F_MCU0_0, nullptr, RPMSG_ADDR_ANY, 1, ept_dev, 0); // returns nullptr

Fullscreen Code_MCU.txt Download

static void IPC_ServerThread(void* pArgs) {
    RPMessage_waitForLinuxReady(SystemP_WAIT_FOREVER);

    IPC_Socket serverSocket; // my internal structure, shouldn't be relevant
    while (!IPC_CreateSocket(&serverSocket, 1, 123, ID_A53_0))
        vTaskDelay(10);
    ...
}

bool IPC_CreateSocket(IPC_Socket* socket, uint16_t localEndpt, uint16_t remoteEndpt, uint16_t remoteCoreId) {
    RPMessage_CreateParams createParams;
    RPMessage_CreateParams_init(&createParams);
    createParams.localEndPt = localEndpt;
    RPMessage_construct(&socket->rpm_obj, &createParams);
    socket->packetCounter = 0xffff;
    socket->localEndpt = localEndpt;
    socket->remoteEndpt = remoteEndpt;
    socket->remoteCoreId = remoteCoreId;
    if (remoteCoreId == ID_A53_0) { // announcement only necessary for Linux cores
        char* serviceName = "rpmsg_chrdev";
        if (RPMessage_announce(remoteCoreId, localEndpt, serviceName) != SystemP_SUCCESS) {
            return false; // debugger shows that this does NOT happen
        }
    }
    ... // we actually arrive here

    return true;
}

Kind regards,

Leon

0 Leon Rigoni 3 months ago in reply to Leon Rigoni

Intellectual 315 points

Hello Nick,

I've just experienced something weird: I changed the kernel from normal to RT (6.1.119) and added device mapper support to the kernel config - seemingly unrelated. But now the rpmsg device files rpmsg0/1 and rpmsg0/1_ctrl suddenly appear and work!

What could be causing this? I have no clue what is going on there...

Thanks

0 Nick Saulnier 3 months ago in reply to Leon Rigoni

TI__Guru** 101250 points

Hello Leon,

Glad to hear that things are working now! Now to reverse engineer what actually happened...

Did you determine which change caused things to start working? (i.e., switching from normal to RT kernel, or adding device mapper support)

I have not looked at device mapper before today. I am not aware of userspace RPMsg communication depending on that driver, but I will do a little more digging.

If changing kernel versions causes things to start working, it would be good if you can point to exactly which kernel repo and version you are using.

For example, if the "normal" kernel was the upstream kernel, but the RT kernel was ti-rt-linux-6.1.y, I can tell you that we were absolutely carrying some patches in kernel 6.1 ti-linux-kernel related to remoteproc and rpmsg that had not been upstreamed: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/log/?h=ti-rt-linux-6.1.y

Regards,

Nick

0 Leon Rigoni 3 months ago in reply to Nick Saulnier

Intellectual 315 points

Hello Nick,

I did not find out whether it was the device mapper or not, but I can hardly belive it.

The kernel versions are 6.1.119 in both cases.

There were also other weird side effects: For example, in the normal kernel I had SSH without SCP, in the RT build I had SCP. Why this has anything to do with the kernel is totally unclear to me.

Kind regards,

Leon

Processors

Processors forum

AM625: RPMessage character device files do not appear in /dev