AM6548: UART0 can not receive

Hi user5772774,

I followed the link you provided and also explicitly looked at chapter 5.3.17.2 but I'm not clear what your issue is. For example, I don't see MCU_UART0 being mentioned in that section.

Can you please provide more detailed steps, logs, screenshots, etc. to describe and re-create the issue you are seeing.

Thanks, Andreas

Hi TI team,

SDK Version: ti-processor-sdk-linux-am65xx-evm-06.00.00.07

Board: EVM

Step 1: in the kernel shell, run the command: ln -sf /lib/firmware/rti_dwwdtest/am65xx_evm/csl_rti_dwwd_test_app_mcu1_0_release.xer5f /lib/firmware/am65x-mcu-r5f0_0-fw

Step 2: reboot the EVM board

Step 3: in the kernel shell, run the command: cat  /dev/ttyS1, then PC send something to the ttyS1 of  EVM board  via serial , the ttyS1 can not receive anything

if don't do Step1 to load the mcu firmware, the Step3 is ok

Hi user5772774,

thanks for the steps on how to re-create the issue. I'll need to run this on my AM65x board locally to see what is going on, unfortunately we had some terrible weather here last night at Dallas and part of our campus is currently closed. It will probably be at least a day or two before I can get to my hardware and look into this. Will keep you posted in case it takes longer.

Regards, Andreas

Ok, thanks

user5772774,

it looks like our campus may re-open on Thursday, but that's not 100% clear yet. Anyways, just letting you know we are all still waiting to be able to get back to our workstations / boards, etc.

Regards, Andreas

user5772774,

while I still don't have access to my office and my hardware and with that can't do any board-level testing/validation I think I might have found an explanation from looking at the csl_rti_dwwd_test_app_mcu1_0_release.xer5f source code.

First, that source code is part of the TI RTOS SDK available here: http://software-dl.ti.com/processor-sdk-rtos/esd/AM65X/latest/index_FDS.html in folder "\ti\pdk_am65xx_1_0_5\packages\ti\csl\example\rti\rti_dwwdtest_app" after installation. Inspecting the source code it becomes evident that the code performs pinmux setup for selected peripherals. The related pinmux helper function and initialization functions are located here: "\ti\pdk_am65xx_1_0_5\packages\ti\board\src\am65xx_evm".

Now, to the interesting part. You say you lose the ability to receive anything on the ttyS1 UART (which I suppose is the MCU_UART0 on the device) after that csl_rti_dwwd_test_app_mcu1_0_release.xer5f firmware is running. Now looking at that example's pinmux source code, we see that the MCU_UART0 is indeed affected by the code, and the settings performed are as follows:

static pinmuxPerCfg_t gMcu_uart0PinCfg[] =
{
    /* MCU_UART0 -> MCU_UART0_RXD -> P4 */
    {
        PIN_MCU_OSPI1_D1, PIN_MODE(4) | \
        ((PIN_PULL_DISABLE | PIN_INPUT_ENABLE) & (~PIN_PULL_DIRECTION))
    },
    /* MCU_UART0 -> MCU_UART0_TXD -> P5 */
    {
        PIN_MCU_OSPI1_D2, PIN_MODE(4) | \
        ((PIN_PULL_DISABLE) & (~PIN_PULL_DIRECTION & ~PIN_INPUT_ENABLE))
    },
    /* MCU_UART0 -> MCU_UART0_CTS -> P1 */
    {
        PIN_MCU_OSPI1_D3, PIN_MODE(4) | \
        ((PIN_PULL_DISABLE | PIN_INPUT_ENABLE) & (~PIN_PULL_DIRECTION))
    },
    /* MCU_UART0 -> MCU_UART0_RTS -> P4 */
    {
        PIN_MCU_OSPI1_CSN1, PIN_MODE(4) | \
        ((PIN_PULL_DISABLE | PIN_INPUT_ENABLE) & (~PIN_PULL_DIRECTION))
    },
    {PINMUX_END}
};

Note how the MCU_UART0_TXD pin is configured as output, and the MCU_UART0_RXD pin is configured as an input - both as they should be. But there seems to be an error in the handshake signal setup. See how both MCU_UART0_CTS and MCU_UART0_RTS are signals are setup the same way, even though one is supposed to be an input and one an output? This looks wrong. So the reason for your issues is probably because RTS is setup as an input instead as an output it prohibits your PC from sending data to the AM654x. I think if you change the configuration of RTS to match that of TXD and re-build that code that issue should go away.

It is probably better to remove all pinmux setup from the MCU code not directly needed for your function. I don't think it is a good idea to have several pieces of code running on several cores stomping over each others configuration code. I'm going to file an issue against the SDK internally to get this fixed.

Let me know should you decide to re-build and re-try the code and if that fixes it, or if we need to dig into this some more.

Regards, Andreas

Hello,

Thank you for your support, I will try to rebuild on your way

Hello,

I have tried it, but it still is not ok!

Hi user5772774,

which board do you have? The "AM65x EVM" or the "AM65x Industrial Development Kit"? While both kits share a fair amount of hardware (processor board) they are being represented by different folders in the RTOS SDK:

• C:\ti\pdk_am65xx_1_0_5\packages\ti\board\src\am65xx_evm
• C:\ti\pdk_am65xx_1_0_5\packages\ti\board\src\am65xx_idk

So first, I'd like to make sure we are modifying the right folder.

Then, the main application file is this here: C:\ti\pdk_am65xx_1_0_5\packages\ti\csl\example\rti\rti_dwwdtest_app\rti_app_dwwd_v1.c

Can you please try the following:

1. Make a modification to the file that would make it obvious and can be used to confirm that you are indeed building the right file, and finally running that actual file on the device. For example, add an infinite while(1) {} loop near the top, to trap program execution, which you should be able to witness running on the AM654 device. Again, this is to confirm your build/update process works as you expect.
2. Experiment with making changes to that file's main() function to see if you can narrow down the issue. For example, remove the call to Board_init(). Now the code won't work properly for sure. But does the UART now work? etc.  Basically, try a few things to narrow down the issue and report back.

Above I was basically outlining what I'd try myself to pin this down, hoping it would enable you to make some progress. If nothing helps I may get access to an AM654x board next week (fingers crossed) to debug this directly.

Regards,
Andreas

Hello,

1. I only connected the RXD and TXD pins of PC uart and EVM uart, so the RTS pin cannot affect the communication.

2. After running the MCU firmware in Linux, I have reconfigured the pinmux of the RXD, TXD, RTS and CTS pins to the correct mode, but the receiving function of the MCU uart0 still does not work.

3. I guess it should be that other parts of the MCU firmware affect the receiving function of uart0.

Hi user5772774,

I appreciate your patience on this. I was finally able to get hold of and setup an AM654x EVM board and was able to re-create the concern you described. Running the RTOS RTI DWWD test firmware on the R5 indeed impacts how MCU_UART0 operates. I can transmit data to the PC still with this firmware running, but I can no longer receive data. I also tried force-configuring serial port handshake operation and just as you indicated that has no impact so the reason needs to be found somewhere else.

I did notice that when starting up the MCU firmware a diagnostic message is transmitted out of MCU_UART0, so this confirms that this firmware is definitely touching the UART. While I cannot yet point to what exactly makes the MCU_UART0 usage break from a Linux point of view (the AM65xx RTOS SDK takes forever to build, and I wanted to go through at least one full build before experimenting with the code some more) I suspect it might be something interrupt or DMA related as the CSL UART driver is touching these components.

CSL RTI APP: RTI App WDT timeout interrupt configured.
CSL RTI APP: RTI App Resource Table at 41c09000.

At any rate, different cores/programs/drivers should not really share a single UART, which is I suppose why you want the RTOS demo firmware to no longer access the MCU_UART0, that you can continue using it from Linux, correct?

user5772774 said:

3. I guess it should be that other parts of the MCU firmware affect the receiving function of uart0.

I suppose you have probably tried removing the calls to Board_init() in rti_app_dwwd_v1.c? I'll experiment with that on my end as soon as my initial RTOS build is done and report back tomorrow.

Regards, Andreas

Hi user5772774,

quick update here. After rebuilding the RTOS SDK which took about 7 hours am RTI WDT test application was created with the same name like the one that comes with the Processor SDK Linux. The build output was located here:

adannenb@beast:/opt/ti-processor-sdk-rtos-am65xx-evm-06.01.00.08
$ find . -name csl_rti_dwwd_test_app_mcu1_0_release.xer5f
./pdk_am65xx_1_0_6/packages/ti/binary/csl_rti_dwwd_test_app/bin/am65xx_evm/csl_rti_dwwd_test_app_mcu1_0_release.xer5f

Plugging this file into the AM654 EVM filesystem as image am65x-mcu-r5f0_0-fw the MCU_UART0 keeps working as expected, both for RX and TX using simple commands:

 _____                    _____           _         _   
|  _  |___ ___ ___ ___   |  _  |___ ___  |_|___ ___| |_ 
|     |  _| .'| . | . |  |   __|  _| . | | | -_|  _|  _|
|__|__|_| |__,|_  |___|  |__|  |_| |___|_| |___|___|_|  
              |___|                    |___|            

Arago Project http://arago-project.org am65xx-evm ttyS2

Arago 2019.05 am65xx-evm ttyS2

am65xx-evm login: root
root@am65xx-evm:~# stty -F /dev/ttyS1 -ixon -crtscts 115200
root@am65xx-evm:~# cat /dev/ttyS1
hhahaha
heehehhe
^C
root@am65xx-evm:~# echo test22222 > /dev/ttyS1
root@am65xx-evm:~# echo test11111 > /dev/ttyS1
root@am65xx-evm:~# echo test33333 > /dev/ttyS1
root@am65xx-evm:~# dmesg | grep remoteproc0
[    6.832656] remoteproc remoteproc0: 41000000.r5f is available
[    6.935169] remoteproc remoteproc0: powering up 41000000.r5f
[    6.941121] remoteproc remoteproc0: Booting fw image am65x-mcu-r5f0_0-fw, size 510784

The filesize matches the file I built from the RTOS SDK. However no debug prints are generated like with the original firmware such as "CSL RTI APP: RTI App WDT timeout interrupt configured". From looking at the source code something should have gotten printed which it did not which made me think something is broken with this new firmware. Sure enough, double-checking with the JTAG debugger shows that the firmware has crashed (no JTAG access at all). Whereas with the original FW image located at /lib/firmware/rti_dwwdtest/am65xx_evm/csl_rti_dwwd_test_app_mcu1_0_release.xer5f those debug prints are emitted as reported earlier, AND there is no issue accessing this firmware.

Also running the 'watchdog' demo now results in a CPU exception. So clearly something is fundamentally broken here with the firmware I built.

 _____                    _____           _         _   
|  _  |___ ___ ___ ___   |  _  |___ ___  |_|___ ___| |_ 
|     |  _| .'| . | . |  |   __|  _| . | | | -_|  _|  _|
|__|__|_| |__,|_  |___|  |__|  |_| |___|_| |___|___|_|  
              |___|                    |___|            

Arago Project http://arago-project.org am65xx-evm ttyS2

Arago 2019.05 am65xx-evm ttyS2

am65xx-evm login: root
root@am65xx-evm:~# watchdog
[   40.890631] SError Interrupt on CPU3, code 0xbf000000 -- SError
[   40.890637] CPU: 3 PID: 1034 Comm: watchdog Tainted: G           O      4.19.38-g4dae378bbe #1
[   40.890639] Hardware name: Texas Instruments AM654 Base Board (DT)
[   40.890642] pstate: 80000005 (Nzcv daif -PAN -UAO)
[   40.890644] pc : vprintk_func+0x20/0x1c0
[   40.890646] lr : printk+0x60/0x80
[   40.890648] sp : ffff000011e4f940
[   40.890650] x29: ffff000011e4f940 x28: ffff000011e4fd20 
[   40.890656] x27: 0000000000000002 x26: 0000000000000000 
[   40.890661] x25: 0000000000000000 x24: ffff800841339310 
[   40.890666] x23: ffff800845371600 x22: ffff000008ab4700 
[   40.890671] x21: ffff800840533578 x20: ffff000011e4f9d0 
[   40.890676] x19: ffff000008c13648 x18: 0000000000000000 
[   40.890681] x17: 0000000000000000 x16: ffff8008459d0000 
[   40.890686] x15: 0000000000000000 x14: 0000000000000000 
[   40.890690] x13: 0000000000000000 x12: 0000000000000000 
[   40.890695] x11: fefefefefefefeff x10: ffff000011e4fa60 
[   40.890701] x9 : 00000000ffffffc8 x8 : ffff000011e4fa20 
[   40.890707] x7 : 0000000000000000 x6 : 0000002c3a674b3f 
[   40.890712] x5 : 00ffffffffffffff x4 : 00000000ffff02ce 
[   40.890716] x3 : 0000000000000000 x2 : ffff8008459d0000 
[   40.890721] x1 : ffff000011e4f9d0 x0 : ffff000008ab4700 
[   40.890727] Kernel panic - not syncing: Asynchronous SError Interrupt
[   40.890731] CPU: 3 PID: 1034 Comm: watchdog Tainted: G           O      4.19.38-g4dae378bbe #1
[   40.890735] Hardware name: Texas Instruments AM654 Base Board (DT)
[   40.890737] Call trace:
[   40.890739]  dump_backtrace+0x0/0x158
[   40.890740]  show_stack+0x14/0x20
[   40.890742]  dump_stack+0x90/0xb4
[   40.890744]  panic+0x120/0x27c
[   40.890746]  nmi_panic+0x6c/0x70
[   40.890748]  arm64_serror_panic+0x74/0x80
[   40.890750]  is_valid_bugaddr+0x0/0x8
[   40.890751]  el1_error+0x7c/0xdc
[   40.890753]  vprintk_func+0x20/0x1c0
[   40.890756]  printk+0x60/0x80
[   40.890758]  keystone_wdt_start+0x60/0xc0
[   40.890760]  watchdog_start+0xec/0x150
[   40.890762]  watchdog_open+0xdc/0x138
[   40.890764]  misc_open+0x11c/0x1b0
[   40.890766]  chrdev_open+0xa4/0x1a0
[   40.890768]  do_dentry_open+0x110/0x340
[   40.890770]  vfs_open+0x28/0x30
[   40.890772]  path_openat+0x268/0x1000
[   40.890774]  do_filp_open+0x78/0xf0
[   40.890776]  do_sys_open+0x170/0x210
[   40.890778]  __arm64_sys_openat+0x20/0x28
[   40.890780]  el0_svc_common+0x84/0xf0
[   40.890782]  el0_svc_handler+0x2c/0x80
[   40.890784]  el0_svc+0x8/0xc
[   40.890859] SMP: stopping secondary CPUs
[   40.890861] Kernel Offset: disabled
[   40.890863] CPU features: 0x0,2080200c
[   40.890865] Memory Limit: none
[   41.127990] ---[ end Kernel panic - not syncing: Asynchronous SError Interrupt ]---

I need to go back to the team that developed this firmware to see what is missing here. We might not have the right file, or may not be using it correctly. Or there might be an issue with the file itself. Also in case you were able to successfully build and use that firmware (as evident from debug prints on MCU_UART0) please do let me know.

Regards, Andreas

One more thing. The steps I provided previously as well as the contents of pdk_am65xx_1_0_6_rti_dwwdtest_app_updated.tar.gz is for compatibility with the Linux SDK v06.00.00.07 as per your initial post. If you use newer versions of the Linux SDK (like v06.01.00.08) you must not make the change to the linker command file I did. The reason is that most recent Linux SDK allocates DDR for the R5FSS0_CORE1 in the 0xaxxxxxxxx address range (see Linux's k3-am654-base-board.dts), just like the most recent RTOS example code needs. Previous versions of the Linux SDK allocated the remote core DDR in the 0x9xxxxxxx range which requires the linker modification I made to re-establish compatibility.

So back to my point earlier, the best will be if you set yourself up to build that code so that you have full control over it.

Regards, Andreas

Hello,

Ok, Thanks, I will try to build a bin file that meets my needs.

Hello,

I think you'd better fix this MCU firmware issue in your released SDK, because our software needs to synchronize catchup to the latest version you released every time. For example, the MCU firmware compiled by us on SDK6.0 can't be Used on SDK6.1, so we have to re-edit the MCU firmware every time, then compile.

Hi user5772774,

understood about your concern. Actually the expectation is that Linux SDK and RTOS SDK need to be on the same major/minor version to be compatible from a shared resource point of view. I do realize that this specific fact is not obvious from our current public documentation but I already submitted feedback internally to have this clarified. Personally I would prefer us to just release a single combined SDK for both Linux and RTOS (if someone doesn't want to use a part of it then they don't have to) as multi-core and multi-OS development is becoming increasingly more common especially with the new platforms we are introducing. I'm advocating for this internally but this not something solely under my control, and there are possible other avenues to ensure better consistency between the two worlds. But please do know this general topic is on our radar screen.

This being said I'd think we are now past compatibility-breaking memory map changes so I would not expect the hiccup we encountered when mix-matching 6.0 and 6.1 SDKs to repeat itself anytime soon.

Regards, Andreas