AM625: RS485 issue

Hi Ethan,

Which Processor SDK version is used?

Which main_uart is used?

Any log showing what the test is? and what the errors are?

Hi Bin

They use SDK8.6,  main_urat1, no error is reported, but data is lost or garbled when sending and receiving

Hi Ethan,

Which pins are used for the RS485 DE for both main_uart1 and mcu_uart?

Please share the customer's device tree config for both UART ports, and describe the detail of the test procedure, I will try to reproduce it on my side and look into it.

Ethan,

I just checked the SK-AM62 schemtics, main_uart1 is on the FTDI header, in which the RXD and TXD are not exposed, how can the customer attach RS485 transceiver to it?

serial1 = &mcu_uart0;
serial2 = &main_uart0;
serial3 = &main_uart1;

main_uart1_pins_default: main-uart1-pins-default {
pinctrl-single,pins = <
AM62X_IOPAD(0x01ac, PIN_INPUT, 2) /* (E19) MCASP0_AFSR.UART1_RXD */
AM62X_IOPAD(0x01b0, PIN_OUTPUT, 2) /* (A20) MCASP0_ACLKR.UART1_TXD */
AM62X_IOPAD(0x0194, PIN_INPUT, 2) /* (B19) MCASP0_AXR3.UART1_CTSn */
AM62X_IOPAD(0x0198, PIN_OUTPUT, 2) /* (A19) MCASP0_AXR2.UART1_RTSn */
>;
};

mcu_uart0_pins_default: mcu-uart0-pins-default {
pinctrl-single,pins = <
AM62X_MCU_IOPAD(0x001c, PIN_INPUT, 0) /* (A6) MCU_UART0_CTSn */
AM62X_MCU_IOPAD(0x0020, PIN_OUTPUT, 0) /* (B6) MCU_UART0_RTSn */
AM62X_MCU_IOPAD(0x0014, PIN_INPUT, 0) /* (B5) MCU_UART0_RXD */
AM62X_MCU_IOPAD(0x0018, PIN_OUTPUT, 0) /* (A5) MCU_UART0_TXD */
>;
};

&mcu_uart0 {
pinctrl-names = "default";
pinctrl-0 = <&mcu_uart0_pins_default>;
rs485-rts-active-low;
linux,rs485-enabled-at-boot-time;
};

&main_uart1 {
pinctrl-names = "default";
pinctrl-0 = <&main_uart1_pins_default>;
rs485-rts-active-low;
linux,rs485-enabled-at-boot-time;
};

Hello Michael

Could you help us understand how can you attach RS485 transceiver to main_uart1?

And please help describe the detail of the test procedure.

BR

Ethan

We developed our own board card based on SK-AM62. The circuit diagram is shown in the attachment. The test method is that mcu_uart-RXD is connected to main_uart1-TXD, and mcu_uart-TXD is connected to main_uart1-RXD, and loop back test is conducted without flow control. About 30 to 50 characters are transmitted at a time. After hundreds of tests, incomplete end characters or garbled characters occur.

Hi Michael,

michael cao said:

We developed our own board card based on SK-AM62. The circuit diagram is shown in the attachment. The test method is that mcu_uart-RXD is connected to main_uart1-TXD, and mcu_uart-TXD is connected to main_uart1-RXD, and loop back test is conducted without flow control.

You are not using RS485 transceivers in the test? Then please remove the two RS485 related device tree properties and test again.

We have used 485 transceivers, normal rs232 testing is the same problem

Hi Michael,

The kernel drivers handle RS485 and RS232 differently. Should we just focus on your RS232 test issue first?

What SDK/kernel version do you use?

the console log you showed above is for RS485 or RS232 test? what is the baud rate?

Is the received chars in the log wrong? what is the expected string then?

Do you use the root filesystem provided in the Processor SDK? If so, please ensure the MCU application firmware is not loaded by Linux, the MCU application uses the MCU UART too, I believe.

Let's focus on rs232 first.
Version: ti-processor-sdk-linux-am62xx-evm-08.06.00.42, kernel:5.10.168 uboot:2021.01.
We conducted a test of rs232.
We set some data to be sent, accepted and verified by ourselves. Currently, the error is that the comparison between sent and obtained data is incomplete or garbled.
The filesystem is obtained from the sdk. How do you know if mcu's firmware is loaded?

Hi Michael,

The SDK provides multiple filesystem. If you use tisdk-default-image*, likely the MCU firmware is running. You can run the following command on your board Linux prompt to confirm;

# dmesg | grep m4f

If you see the following message, it means the MCU firmware is loaded and running.

[ 4.202209] remoteproc remoteproc0: powering up 5000000.m4fss
[ 4.208144] remoteproc remoteproc0: Booting fw image am62-mcu-m4f0_0-fw, size 54860

One way to prevent the firmware to run is to rename the following file to something else:

/lib/firmware/am62-mcu-m4f0_0-fw

then after reboot, you would see the following kernel message:

# dmesg | grep m4f

[ 4.250475] remoteproc remoteproc0: Direct firmware load for am62-mcu-m4f0_0-fw failed with error -2

Hi  Bin

m4 is not working. The log is as follows

# dmesg | grep m4f
[ 0.000000] OF: reserved mem: initialized node m4f-dma-memory@9cb00000, compatible id shared-dma-pool
[ 0.000000] OF: reserved mem: initialized node m4f-memory@9cc00000, compatible id shared-dma-pool
[ 4.639009] k3-m4-rproc 5000000.m4fss: device does not reserved memory regions, ret = -22
[ 4.657543] k3-m4-rproc 5000000.m4fss: reserved memory init failed, ret = -22
[ 4.670330] remoteproc remoteproc0: releasing 5000000.m4fss
[ 4.678264] k3-m4-rproc: probe of 5000000.m4fss failed with error -22

Dear Bin

I would like to know if there are any restrictions on the intercommunication between the main domain and the MCU domain on the same board, because as far as I know, it seems that not only UART intercommunication will cause data loss problems

Hi Bin

Customer has tested four UART transmissions between two boards, and there is no problem

Hi Ethan,

I tested UART5 external lookback on SK-AM62 at baud rate 921600 and don't see UART char drop or corruption.

I use a jumper wire to connect UART5 RXD to TXD (SK-AM62 J3.8 & J3.10 pins) to form an external loopback. Then in Linux using /usr/bin/serialcheck program to send and receive UART characters, and I don't see any data corruption or character drop in baud rate 921600.

# serialcheck -b 921600 -d /dev/ttyS5 -f <a-big-text-file> -l 1000000 -m d

What baud rate does the customer use when testing two UART ports on the same board? Can you please ask the customer to test using serialcheck program provided in the SDK root filesystem?

Hello Bin 

The baud rate customer use is 115200, there has always been no problem with one UART port to send and receive data spontaneously. The only problem is that two UART ports are used for data transmission.

Hi Ethan,

I tested TX/RX between main uart5 and mcu uart0 on SK-AM62 (E2) using SDK8.6 but don't see any data loss problem. Below is the details of my test. Please let me know if you or the customer can replicate the test.

On SK-AM62 board, mcu uart0 routes to the MCU HEADER (J9), TXD is to J9.19, while RXD is to J9.13. You would need to add a jumper to J8, otherwise, mcu uart0 routes to the FTDI usb connector.

main uart5 routes to the USER EXPANSION CONNECTOR J3, TXD is to J3.8, while RXD is to J3.10.

Please use two jump wires to connect J9.19 to J3.10, and J9.13 to J3.8 respectively so that main uart5 and mcu uart0 can communicate to each other.

Following is dts changes in k3-am62x-sk-common.dtsi:

aliases {
        ...
        serial1 = &mcu_uart0;
        serial5 = &main_uart5;
        ...
};
...        
&main_pmx0 {
        ...
        main_uart5_pins_default: main_uart5-pins-default {                      
                pinctrl-single,pins = <                                         
                        AM62X_IOPAD(0x01dc, PIN_OUTPUT, 1) /* (E15) MCAN0_RX.uart5.tx */
                        AM62X_IOPAD(0x01d8, PIN_INPUT, 1) /* (C15) MCAN0_TX.uart5.rx */
                >;                                                              
        };
};
...
&mcu_pmx0 {                                                                     
        mcu_uart0_pins_default: mcu_uart0-pins-default {                        
                pinctrl-single,pins = <                                         
                        AM62X_MCU_IOPAD(0x0014, PIN_INPUT, 0) /* (B5) MCU_UART0_RXD */
                        AM62X_MCU_IOPAD(0x0018, PIN_OUTPUT, 0) /* (A5) MCU_UART0_TXD */
                >;                                                              
        };                                                                      
};
...
&main_uart5 {                                                                   
        pinctrl-names = "default";                                              
        pinctrl-0 = <&main_uart5_pins_default>;                                 
        status = "okay";                                                        
}; 
...
&mcu_uart0 {                                                                    
        pinctrl-names = "default";                                              
        pinctrl-0 = <&mcu_uart0_pins_default>;                                  
        status = "okay";                                                        
};

Once boot into Linux, ensure the M4 firmware is not running. We need two terminals to run the test, one for transmit the other for receive. We could use the Linux console and a telnet terminal. Connect the SK-AM62 board ethernet port to the PC, and manually assign an IP address if the PC doesn't run DHCP server. Start a telnet terminal to SK-AM62 board.

In the Linux console, run the following command to receive on mcu uart0:

root@am62xx-evm:~# serialcheck -b 115200 -d /dev/ttyS1 -f /etc/init.d/README -l 100000 -m r

In the telnet terminal, run the following command to transmit on main uart5:

root@am62xx-evm:~# serialcheck -b 115200 -d /dev/ttyS5 -f /etc/init.d/README -l 100000 -m t

When mcu uart0 received incorrect data, the serialcheck program would print the error and details.

You can also swap ttyS5 and ttyS1 in the two commands above to transmit from mcu uart0 to main uart5.

Hi Bin

I have sent you an email, could you please have a look at customer's test code? 

Hi Ethan,

Thanks for providing the customer test program offline. The test passed when I initially ran the test program on my SK-AM62 setup between mcu_uart0 and main_uart5. However, I did see the failure after running the test multiple times. The program serialstats (you can compile it from https://github.com/nsekhar/serialcheck) reports no error in the UART port. So I suspect the data loss error in the user space or kernel tty layer due to lack of flow control.

Your offline email also mentions the customer tested with serialcheck I mentioned above, but the RX failed in the every beginning of the test, RX doesn't receive any data at all. This seems like a setup error. We can look at it in the coming meeting.

I changed my test as following to conduct the duplex mode transfer between main_uart5 and mcu_uart0. I ran the command in one line to avoid the RX timeout when another thread terminates earlier. I still use the same text file /etc/init.d/README (1151 bytes), since the test program serialcheck doesn't close the UART port between loops.

This test transmits and receives 3.45MB data. I used the program serialstats, from the same serialcheck project linked in my previous post, to confirm both UARTs indeed transmitting and receiving data.

I ran this test multiple times on SK-AM62 evm, it didn't report any error.

serialcheck -b 115200 -d /dev/ttyS1 -f /etc/init.d/README -l 3000 -m d& serialcheck -b 115200 -d /dev/ttyS5 -f /etc/init.d/README -l 3000 -m d

Hi Bin

mcu_uart0 and main_uart1 use serial_check test,
1. The file size is about 1k, rt mode (one transmission and one reception), -d mode (simultaneous transmission and reception), 3000 loops are no problem
2. The file size is 2.4M, -d mode, 1 loop, and there is a problem

root@am62xx-rs10:/etc/init.d# serialcheck -b 115200 -d /dev/ttyS1 -f /etc/init.d/README -l 1 -m d & serialcheck -b 115200 -d /dev/ttyS3 -f /etc/init.d/README -l 1 -m d
[1] 1376

timeout, RX/TX: 1318056/2445400
timeout, RX/TX: 1318024/2445400
Needed 158 reads 1 writes Oh oh, inconsistency at pos 604047 (0x9378f).

Original sample:
00093750: 73 74 65 6d 64 2d 62 61  73 65 64 20 4f 53 20 77   stemd-based OS w
00093760: 68 65 72 65 20 74 72 61  64 69 74 69 6f 6e 61 6c   here traditional
00093770: 20 69 6e 69 74 20 73 63  72 69 74 73 20 68 61 76    init scrits hav
00093780: 65 0a 62 65 65 6e 20 72  65 70 6c 61 63 65 64 20   e.been replaced
00093790: 62 79 20 6e 61 74 69 76  65 20 73 79 73 74 65 6d   by native system
000937a0: 64 20 73 65 72 76 69 63  65 73 20 66 69 6c 65 73   d services files

Received sample:
00093750: 73 74 65 6d 64 2d 62 61  73 65 64 20 4f 53 20 77   stemd-based OS w
00093760: 68 65 72 65 20 74 72 61  64 69 74 69 6f 6e 61 6c   here traditional
00093770: 20 69 6e 69 74 20 73 63  72 69 74 73 20 68 61 76    init scrits hav
00093780: 65 0a 62 65 65 6e 20 72  65 70 6c 61 63 65 64 62   e.been replacedb
00093790: 79 20 6e 61 74 69 76 65  20 73 79 73 74 65 6d 64   y native systemd
000937a0: 20 73 65 72 76 69 63 65  73 20 66 69 6c 65 73 2e    services files.
loops 1 / 1

cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 2445398 tx: 2445400 frame 0 ovr 0 par: 0 brk: 0 buf_ovrr: 1127374
Needed 121 reads 1 writes Oh oh, inconsistency at pos 1314816 (0x141000).

Original sample:
00140fd0: 2f 62 6c 6f 67 2f 70 72  6f 6a 65 63 74 73 2f 73   /blog/projects/s
00140fe0: 79 73 74 65 6d 64 2d 66  6f 72 2d 61 64 6d 69 6e   ystemd-for-admin
00140ff0: 73 2d 33 2e 68 74 6d 6c  0a 20 20 20 20 20 20 20   s-3.html.
00141000: 20 74 74 70 73 3a 2f 2f  77 77 77 2e 66 72 65 65    ttps://www.free
00141010: 64 65 73 6b 74 6f 70 2e  6f 72 67 2f 77 69 6b 69   desktop.org/wiki
00141020: 2f 53 6f 66 74 77 61 72  65 2f 73 79 73 74 65 6d   /Software/system

Received sample:
00140fd0: 2f 62 6c 6f 67 2f 70 72  6f 6a 65 63 74 73 2f 73   /blog/projects/s
00140fe0: 79 73 74 65 6d 64 2d 66  6f 72 2d 61 64 6d 69 6e   ystemd-for-admin
00140ff0: 73 2d 33 2e 68 74 6d 6c  0a 20 20 20 20 20 20 20   s-3.html.
00141000: 61 72 2e 73 65 72 76 69  63 65 20 64 75 72 69 6e   ar.service durin
00141010: 67 20 73 79 73 74 65 6d  20 69 6e 69 74 69 61 6c   g system initial
00141020: 69 7a 61 74 69 6f 6e 2e  0a 0a 54 68 61 6e 6b 20   ization...Thank
loops 1 / 1

cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 2445400 tx: 2445400 frame 0 ovr 0 par: 0 brk: 0 buf_ovrr: 1127344

Can you work with RD to see what is wrong with the customer's test code?

Hi Ethan,

I can reproduce the error with a 2.4MB file, then I found the following note that '-m d' is not supported by serialcheck:

https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/08_06_00_42/exports/docs/linux/Foundational_Components/Kernel/Kernel_Drivers/UART.html#basic-external-loopback-testing

Then follow the note, I started multiple processes to test RX/TX, then I don't see any error in tranmitting/receiving 2.4MB file:

serialcheck -b 115200 -d /dev/ttyS1 -f <bigger-file> -l 1 -m r&
serialcheck -b 115200 -d /dev/ttyS5 -f <bigger-file> -l 1 -m r&
serialcheck -b 115200 -d /dev/ttyS1 -f <bigger-file> -l 1 -m t&
serialcheck -b 115200 -d /dev/ttyS5 -f <bigger-file> -l 1 -m t&

I also used another uart test program (https://github.com/cbrake/linux-serial-test.git, the binary I compiled 4 years ago is attached below), using the following command and don't see any error after transmitting/receiving 50MB data:

./serial-test -b 115200 -p /dev/ttyS1 -s -S -A &; ./serial-test -b 115200 -p /dev/ttyS5 -s -S -A

serial-test.gz

Hi Bin

The customer wants to reproduce the test you did on the SK-AM62. From the schematic diagram, seems that i2c configuration is required. Please tell me how to adjust this part so that the test can be carried out correctly.

Hi Ethan,

I didn't use main uart1 for testing. I used main uart5. I posted the setup details of using main uart5 and mcu uart0 on May 5 above on this thread.

If you indeed want to use main uart1, you don't need i2c to control the UART1_FET_SEL signal. UART1_FET_SEL is actually a GPIO pin from the IO expander - U70 in the board schematics page 38.

Please follow the instruction below to control the UART1_FET_SEL GPIO signal:
https://dev.ti.com/tirex/explore/node?node=A__ANR3bSoQUtte3JSAV7ex0g__linux_academy_am62x__XaWts8R__LATEST

You can get the UART1_FET_SEL GPIO pin number from 'cat /sys/kernel/debug/gpio' too.

Hi Bin

I think it doesn’t make much sense to always change the test tool to test, and it doesn’t solve the customer’s problem. I sent you the source code of the customer’s test code, and this problem can also be reproduced on our board. Can you please see where the customer's test problem is?

Hi Ethan,

Ethan Tian said:

Can you please see where the customer's test problem is?

We don't support customer's applications. seriakcheck program was used in SDK UART testing (except the duplex mode), and linux-serial-test was often used in UART testing. So please check if linux-serial-test still reports issue on custom board or your SK-AM62.

root@am62xx-rs10:~# ./serial-test -b 115200 -p /dev/ttyS1 -s -S -A &
[1] 1092
Linux serial test app
[  208.598986] omap8250 4a00000.serial: ttyS1 (1): invalid RTS setting, using RTS_ON_SEND instead
root@am62xx-rs10:~# ./serial-test -b 115200 -p /dev/ttyS3 -s -S -A
Linux serial test app[  210.134564] omap8250 2810000.serial: ttyS3 (3): invalid RTS setting, using RTS_ON_SEND instead

/dev/ttyS3: count for this session: rx=0, tx=0, rx err=1
/dev/ttyS3: TIOCGICOUNT: ret=0, rx=87, tx=2, frame = 5, overrun = 0, parity = 0, brk = 0, buf_overrun = 0
root@am62xx-rs10:~# No data received for 2.0s.
No data received for 3.3s.
No data received for 4.7s.
/dev/ttyS1: count for this session: rx=0, tx=69439, rx err=0
/dev/ttyS1: TIOCGICOUNT: ret=0, rx=2, tx=161663, frame = 0, overrun = 0, parity = 0, brk = 0, buf_overrun = 0
No data received for 6.0s.
No data received for 7.3s.
No data received for 8.7s.
No data received for 10.0s.
No data received for 11.3s.
/dev/ttyS1: count for this session: rx=0, tx=138559, rx err=0
/dev/ttyS1: TIOCGICOUNT: ret=0, rx=2, tx=230783, frame = 0, overrun = 0, parity = 0, brk = 0, buf_overrun = 0
No data received for 12.7s.
No data received for 14.0s.
No data received for 15.3s.
No data received for 16.7s.
/dev/ttyS1: count for this session: rx=0, tx=207679, rx err=0
/dev/ttyS1: TIOCGICOUNT: ret=0, rx=2, tx=299903, frame = 0, overrun = 0, parity = 0, brk = 0, buf_overrun = 0
No data received for 18.0s.
^C
root@am62xx-rs10:~# No data received for 19.3s.
No data received for 20.7s.

We failed to verify using the linux-serial-test tool. Therefore, in order to avoid divergence of problems, the following suggestions are put forward:

Based on our test tool, the exception can be reproduced on am62x-evm. Please also solve the problem based on this.

1. Solve our problem from TI-SDK driver layer or point out the problem of our tool if you think it is abnormal.

2. Verify that our tool is free of this problem on AM62x-EVM using the sdk of a lower version of am62x, such as sdk08.04 or sdk08.05.

Hi Wei,

The log seems indicating ttyS3 has rx frame error. Is this test on your board or AM62x SK EVM?

Does this error happen in every time when you run this test?

Could you please run the two commands in one line?

Wei Zeng said:

2. Verify that our tool is free of this problem on AM62x-EVM using the sdk of a lower version of am62x, such as sdk08.04 or sdk08.05.

Do you mean the original data loss issue only happens on SDK8.6 but not on SDK8.4 or 8.5?

Thank you very much. We have understood the difference of this part. May I ask whether it can be normal after DMA is enabled

Hi Wei,

What do you mean by "normal"? DMA only offload CPU cycles, but likely won't change any other characters. DMA might add latency though.