AM3352: UART stopping receiving data after long time running

Hi Tony,

Tony Tang said:

#1. Is there known issue of UART stopping receiving like this on old SDK with Kernel 3.12. 

I don't recall any issue report similar to this, especially in RS-485 use cases.

Tony Tang said:

#2. What is your suggestion to investigate this kind of issue? 

 I can think out to dump out UART registers to see if any abnormalty. 

 Check GPIO status whether the transceiver in right direction.

Yes, first we need to check if the GPIO DE pin is in the correct level in the error condition.

Since echo a char to the uart can recover the issue, it is possible the UART driver (the customer uses omap-serial.c, right?) has a bug related to timing, which puts the GPIO pin in the wrong level to disable the RX of the RS-485 transceiver.

Hi Bin,

Yes customer is using omap-serial.c

#1: Do you see any related updates/fixes in the subsequent SDK release?

As this is quite hard to reproduce (customer own lab could at most run 1 week continuously due to the environment conditions and never meet this problem), only their end customers real applications will run several weeks and about 1 out of 100 devices will have this problem randomly after several weeks.

So for their next time reproduce at the end customer side, we need prepare a list of steps in advance to capture as much information as we want to know within limited time (their end customer may not keep the environment for a long time).

#2: Hence, if there is anything else you require please let us know in advance.

Thanks,

Kevin

Hi Kevin,

Without the knowing the root cause of the issue, It is very difficult to tell if any update/fix in the new SDK would be related.

Whenever the issue is reproduced, we need to check

- if the RS485 DE pin is in the correct level, controlling the transceiver to receive data;

- if so, probe the AM3352 UART RXD pin to see if the RX data go through the transceiver.

Hi Bin,

Understood, to better prepare we could efficiently capture the information we need when next problem occurred, this afternoon we work with customers to make clear the exact commands to obtain the RS485 DE pin status, and trying on their own lab's normal products. Please have a look and see if there is anything needed to add.

Step 1: We let customer to provide their schematic diagram below to make clear which GPIO pin is controlling the RS485.

From the picture we could see that GPIO2_4 is actually controlling the RS485 DE pin.

Step 2: We checked the GPIO2 base address is 0x481AC000.

Step 3: We think GPIO_OE (0x134 offset), GPIO_DATAIN (0x138 offset), GPIO_DATAOUT (0x13C offset) are important to get the results

Step 4: We ask customers using below commands to read the GPIO_OE, GPIO_DATAIN, GPIO_DATAOUT for GPIO2_4

devmem 0x481AC134

devmem 0x481AC138

devmem 0x481AC13C

Step 5: Customer followed the above commands on their normal boards in their lab and got the below results:

root@Emerson:~# devmem 0x481AC134
0xFFFFFFEF
root@Emerson:~# devmem 0x481AC138
0x0000002E
root@Emerson:~# devmem 0x481AC13C
0x00000000

Step 6: We also let customer be familiar to get the dmesg log, so that they are able to get the correct dmesg log when problem happens. The following is the dmesg log obtained by customer in their lab.

root@Emerson:~# dmesg
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 3.12.10-BSPV2.01T013.12-am335xpsp (kcarney@NPUSDLWDGHMIBLD) (gcc version 4.9.1 20140505 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.05 - Linaro GCC 2014.05) ) #1 Thu Apr 2 08:24:33 EDT 2020
[    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine: Vertiv Global HMI (Flattened Device Tree), model: ACP AM335x Board
[    0.000000] cma: CMA: reserved 24 MiB at 9e000000
[    0.000000] Memory policy: ECC disabled, Data cache writeback
[    0.000000] On node 0 totalpages: 130304
[    0.000000] free_area_init_node: node 0, pgdat c08e8cfc, node_mem_map c0927000
[    0.000000]   Normal zone: 1024 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 130304 pages, LIFO batch:31
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] AM335X ES2.1 (sgx neon )
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 129280
[    0.000000] Kernel command line: console=ttyO0,115200n8 consoleblank=0 mtdparts=spi1.0:128k(SPL)ro,64k(HWID)ro,64k(Reserved)ro,128k(u-boot-env1),128k(u-boot-env2),1536k(u-boot)ro,512k(persistent),-(unused) root=/dev/mmcblk0p3 rw rootfstype=ext4 rootwait
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 482572K/521216K available (6134K kernel code, 585K rwdata, 2028K rodata, 346K init, 242K bss, 38644K reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0800fdc   (8164 kB)
      .init : 0xc0801000 - 0xc0857860   ( 347 kB)
      .data : 0xc0858000 - 0xc08ea610   ( 586 kB)
       .bss : 0xc08ea610 - 0xc0926f20   ( 243 kB)
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
[    0.000000] Total of 128 interrupts on 1 active controller
[    0.000000] OMAP clockevent source: timer2 at 24000000 Hz
[    0.000000] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
[    0.000000] OMAP clocksource: timer1 at 24000000 Hz
[    0.000000] Console: colour dummy device 80x30
[    0.000336] Calibrating delay loop... 398.13 BogoMIPS (lpj=1990656)
[    0.049657] pid_max: default: 32768 minimum: 301
[    0.049776] Security Framework initialized
[    0.049832] Mount-cache hash table entries: 512
[    0.059839] CPU: Testing write buffer coherency: ok
[    0.060242] Setting up static identity map for 0xc0603518 - 0xc0603588
[    0.061203] devtmpfs: initialized
[    0.063220] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
[    0.124824] pinctrl core: initialized pinctrl subsystem
[    0.125769] regulator-dummy: no parameters
[    0.128357] NET: Registered protocol family 16
[    0.131113] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.134121] cpuidle: using governor ladder
[    0.134140] cpuidle: using governor menu
[    0.144612] platform mpu.1: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.145896] platform 49000000.edma: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.146947] gpiochip_add: registered GPIOs 0 to 31 on device: gpio
[    0.147284] OMAP GPIO hardware version 0.1
[    0.148266] gpiochip_add: registered GPIOs 32 to 63 on device: gpio
[    0.149391] gpiochip_add: registered GPIOs 64 to 95 on device: gpio
[    0.150616] gpiochip_add: registered GPIOs 96 to 127 on device: gpio
[    0.159442] platform 56000000.sgx: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.163369] DSS not supported on this SoC
[    0.163391] No ATAGs?
[    0.163404] hw-breakpoint: debug architecture 0x4 unsupported.
[    0.197012] bio: create slab <bio-0> at 0
[    0.214156] edma-dma-engine edma-dma-engine.0: TI EDMA DMA engine driver
[    0.214823] of_get_named_gpio_flags: can't parse gpios property of node '/fixedregulator@0[0]'
[    0.215147] vbat: 5000 mV 
[    0.215294] of_get_named_gpio_flags: can't parse gpios property of node '/fixedregulator@1[0]'
[    0.215582] lis3_reg: no parameters
[    0.215715] of_get_named_gpio_flags exited with status 48
[    0.216139] wlan-en-regulator: 1800 mV 
[    0.219287] drivers/tty/serial/e2_msdrv.c(1068): Creating netlink socket family 17.
[    0.219973] vgaarb: loaded
[    0.221389] SCSI subsystem initialized
[    0.221994] libata version 3.00 loaded.
[    0.222938] usbcore: registered new interface driver usbfs
[    0.223151] usbcore: registered new interface driver hub
[    0.223389] usbcore: registered new device driver usb
[    0.224475] omap_i2c 44e0b000.i2c: could not find pctldev for node /pinmux@44e10800/pinmux_i2c0_pins, deferring probe
[    0.224506] platform 44e0b000.i2c: Driver omap_i2c requests probe deferral
[    0.224778] media: Linux media interface: v0.10
[    0.224978] Linux video capture interface: v2.00
[    0.225303] pps_core: LinuxPPS API ver. 1 registered
[    0.225315] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.225470] PTP clock support registered
[    0.226896] Advanced Linux Sound Architecture Driver Initialized.
[    0.228703] Switched to clocksource timer1
[    0.250763] NET: Registered protocol family 2
[    0.251451] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[    0.251562] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
[    0.251626] TCP: Hash tables configured (established 4096 bind 4096)
[    0.251705] TCP: reno registered
[    0.251722] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    0.251747] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    0.251970] NET: Registered protocol family 1
[    0.252398] RPC: Registered named UNIX socket transport module.
[    0.252413] RPC: Registered udp transport module.
[    0.252422] RPC: Registered tcp transport module.
[    0.252430] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.252445] PCI: CLS 0 bytes, default 64
[    0.254193] PM: Loading am335x-pm-firmware.bin
[    0.448280] VFS: Disk quotas dquot_6.5.2
[    0.448365] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.449125] NFS: Registering the id_resolver key type
[    0.449218] Key type id_resolver registered
[    0.449229] Key type id_legacy registered
[    0.449274] jffs2: version 2.2. (NAND) (SUMMARY)  \xffffffc2\xffffffa9 2001-2006 Red Hat, Inc.
[    0.449487] msgmni has been set to 990
[    0.451333] NET: Registered protocol family 38
[    0.451383] io scheduler noop registered
[    0.451393] io scheduler deadline registered
[    0.451420] io scheduler cfq registered (default)
[    0.453804] pinctrl-single 44e10800.pinmux: 142 pins at pa f9e10800 size 568
[    0.608273] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.611213] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/serial@44e09000[0]'
[    0.611541] 44e09000.serial: ttyO0 at MMIO 0x44e09000 (irq = 88, base_baud = 3000000) is a OMAP UART0
[    1.230463] console [ttyO0] enabled
[    1.234795] of_get_named_gpio_flags exited with status 36
[    1.235114] 48022000.serial: ttyO1 at MMIO 0x48022000 (irq = 89, base_baud = 3000000) is a OMAP UART1
[    1.245465] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/serial@48024000[0]'
[    1.245739] 48024000.serial: ttyO2 at MMIO 0x48024000 (irq = 90, base_baud = 3000000) is a OMAP UART2
[    1.256072] of_get_named_gpio_flags exited with status 30
[    1.256387] 481a6000.serial: ttyO3 at MMIO 0x481a6000 (irq = 60, base_baud = 3000000) is a OMAP UART3
[    1.266726] of_get_named_gpio_flags exited with status 68
[    1.267040] 481a8000.serial: ttyO4 at MMIO 0x481a8000 (irq = 61, base_baud = 3000000) is a OMAP UART4
[    1.278363] omap_rng 48310000.rng: OMAP Random Number Generator ver. 20
[    1.297025] brd: module loaded
[    1.306465] loop: module loaded
[    1.315135] mtdoops: mtd device (mtddev=name/number) must be supplied
[    1.324160] edma-dma-engine edma-dma-engine.0: allocated channel for 0:17
[    1.331423] edma-dma-engine edma-dma-engine.0: allocated channel for 0:16
[    1.339049] m25p80 spi1.0: s25fl256s1 (32768 Kbytes)
[    1.344322] 8 cmdlinepart partitions found on MTD device spi1.0
[    1.350601] Creating 8 MTD partitions on "spi1.0":
[    1.355745] 0x000000000000-0x000000020000 : "SPL"
[    1.362412] 0x000000020000-0x000000030000 : "HWID"
[    1.369032] 0x000000030000-0x000000040000 : "Reserved"
[    1.375963] 0x000000040000-0x000000060000 : "u-boot-env1"
[    1.383190] 0x000000060000-0x000000080000 : "u-boot-env2"
[    1.390473] 0x000000080000-0x000000200000 : "u-boot"
[    1.397266] 0x000000200000-0x000000280000 : "persistent"
[    1.404459] 0x000000280000-0x000002000000 : "unused"
[    1.413211] vcan: Virtual CAN interface driver
[    1.417915] CAN device driver interface
[    1.423377] c_can_platform 481cc000.d_can: c_can_platform device registered (regs=fa1cc000, irq=68)
[    1.434184] c_can_platform 481d0000.d_can: c_can_platform device registered (regs=fa1d0000, irq=71)
[    1.445868] usbcore: registered new interface driver asix
[    1.451867] usbcore: registered new interface driver ax88179_178a
[    1.458527] usbcore: registered new interface driver cdc_ether
[    1.464882] usbcore: registered new interface driver r815x
[    1.470924] usbcore: registered new interface driver smsc95xx
[    1.477213] usbcore: registered new interface driver net1080
[    1.483366] usbcore: registered new interface driver cdc_subset
[    1.489823] usbcore: registered new interface driver zaurus
[    1.495991] usbcore: registered new interface driver cdc_ncm
[    1.502550] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.509476] ehci-pci: EHCI PCI platform driver
[    1.514402] ehci-omap: OMAP-EHCI Host Controller driver
[    1.521010] usbcore: registered new interface driver cdc_wdm
[    1.527230] usbcore: registered new interface driver usb-storage
[    1.534798] mousedev: PS/2 mouse device common for all mice
[    1.542947] input: ti-tsc as /devices/ocp.2/44e0d000.tscadc/TI-am335x-tsc/input/input0
[    1.553121] i2c /dev entries driver
[    1.557464] Driver for 1-wire Dallas network protocol.
[    1.564398] tps3823_wdt: configured with heartbeat=30 sec (nowayout=0)
[    1.571881] tps3823_wdt watchdog.4: initialized watchdog.
[    1.579179] of_get_named_gpio_flags exited with status 115
[    1.579197] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/mmc@48060000[0]'
[    1.579499] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
[    1.586702] edma-dma-engine edma-dma-engine.0: allocated channel for 0:24
[    1.594031] omap_hsmmc 48060000.mmc: unable to get vmmc regulator -517
[    1.600980] edma-dma-engine edma-dma-engine.0: freeing channel for 24
[    1.607781] edma-dma-engine edma-dma-engine.0: freeing channel for 25
[    1.614708] platform 48060000.mmc: Driver omap_hsmmc requests probe deferral
[    1.622900] ledtrig-cpu: registered to indicate activity on CPUs
[    1.629671] edma-dma-engine edma-dma-engine.0: allocated channel for 0:36
[    1.636898] omap-sham 53100000.sham: hw accel on OMAP rev 4.3
[    1.644392] omap-aes 53500000.aes: OMAP AES hw accel rev: 3.2
[    1.650624] edma-dma-engine edma-dma-engine.0: allocated channel for 0:5
[    1.657712] edma-dma-engine edma-dma-engine.0: allocated channel for 0:6
[    1.666355] usbcore: registered new interface driver usbhid
[    1.672291] usbhid: USB HID core driver
[    1.679248] usbcore: registered new interface driver snd-usb-audio
[    1.688090] davinci_evm sound.3: ASoC: CODEC (null) not registered
[    1.694787] davinci_evm sound.3: snd_soc_register_card failed (-517)
[    1.701545] platform sound.3: Driver davinci_evm requests probe deferral
[    1.709537] oprofile: no performance counters
[    1.714565] oprofile: using timer interrupt.
[    1.719474] TCP: cubic registered
[    1.722974] Initializing XFRM netlink socket
[    1.727531] NET: Registered protocol family 10
[    1.733055] sit: IPv6 over IPv4 tunneling driver
[    1.739201] NET: Registered protocol family 17
[    1.743907] NET: Registered protocol family 15
[    1.748566] can: controller area network core (rev 20120528 abi 9)
[    1.755241] NET: Registered protocol family 29
[    1.759997] can: raw protocol (rev 20120528)
[    1.764479] can: broadcast manager protocol (rev 20120528 t)
[    1.770444] can: netlink gateway (rev 20130117) max_hops=1
[    1.776285] 8021q: 802.1Q VLAN Support v1.8
[    1.780781] Key type dns_resolver registered
[    1.786090] cpu cpu0: cpu0 regulator not ready, retry
[    1.791582] platform cpufreq-cpu0.0: Driver cpufreq-cpu0 requests probe deferral
[    1.799862] PM: CM3 Firmware Version 18 not supported
[    1.805575] ThumbEE CPU extension supported.
[    1.814133] tps65910 0-002d: No interrupt support, no core IRQ
[    1.822635] vrtc: 1800 mV 
[    1.825789] vrtc: supplied by vbat
[    1.830759] vio: at 1500 mV 
[    1.833879] vio: supplied by vbat
[    1.838879] vdd_mpu: 912 <--> 1375 mV at 1100 mV 
[    1.843927] vdd_mpu: supplied by vbat
[    1.849264] vdd_core: 912 <--> 1150 mV at 1100 mV 
[    1.854390] vdd_core: supplied by vbat
[    1.859593] vdd3: 5000 mV 
[    1.863664] vdig1: at 1800 mV 
[    1.866957] vdig1: supplied by vbat
[    1.871926] vdig2: at 1800 mV 
[    1.875228] vdig2: supplied by vbat
[    1.880194] vpll: at 1800 mV 
[    1.883393] vpll: supplied by vbat
[    1.888231] vdac: at 1800 mV 
[    1.891457] vdac: supplied by vbat
[    1.896307] vaux1: at 1800 mV 
[    1.899624] vaux1: supplied by vbat
[    1.904555] vaux2: at 3300 mV 
[    1.907844] vaux2: supplied by vbat
[    1.912803] vaux33: at 3300 mV 
[    1.916187] vaux33: supplied by vbat
[    1.921246] vmmc: 1800 <--> 3300 mV at 3300 mV 
[    1.926106] vmmc: supplied by vbat
[    1.931810] rtc-pcf2129 0-0051: chip found, driver version 0.1
[    1.939295] rtc-pcf2129 0-0051: low voltage detected, date/time is not reliable.
[    1.947538] rtc-pcf2129 0-0051: rtc core: registered rtc-pcf2129 as rtc0
[    1.955253] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz
[    1.961821] of_get_named_gpio_flags exited with status 115
[    1.961841] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/mmc@48060000[0]'
[    1.962301] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
[    1.969551] edma-dma-engine edma-dma-engine.0: allocated0] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin MONO_LOUT
[    2.020728] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin HPLCOM
[    2.027342] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin HPRCOM
[    2.033975] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Line Out
[    2.040785] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Mic Jack
[    2.047584] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Line In
[    2.055601] davinci_evm sound.3:  tlv320aic23-hifi <-> 4803c000.mcasp mapping ok
[    2.063528] davinci_evm sound.3: ASoC: no source widget found for HPLOUT
[    2.070633] davinci_evm sound.3: ASoC: Failed to add route HPLOUT -> direct -> Headphone Jack
[    2.079643] davinci_evm sound.3: ASoC: no source widget found for HPROUT
[    2.086717] davinci_evm sound.3: ASoC: Failed to add route HPROUT -> direct -> Headphone Jack
[    2.127160] mmc0: host does not support reading read-only switch. assuming write-enable.
[    2.137733] mmc0: new high speed SDHC card at address 59b4
[    2.144207] mmcblk0: mmc0:59b4 USD   7.51 GiB 
[    2.150316]  mmcblk0: p1 p2 p3 p4
[    2.168776] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6
[    2.175217] davinci_mdio 4a101000.mdio: detected phy mask fffffffc
[    2.183234] libphy: 4a101000.mdio: probed
[    2.187497] davinci_mdio 4a101000.mdio: phy[0]: device 4a101000.mdio:00, driver SMSC LAN8710/LAN8720
[    2.197167] davinci_mdio 4a101000.mdio: phy[1]: device 4a101000.mdio:01, driver SMSC LAN8710/LAN8720
[    2.207739] Missing dual_emac_res_vlan in DT.
[    2.212399] Using 1 as Reserved VLAN for 0 slave
[    2.217296] Missing dual_emac_res_vlan in DT.
[    2.221914] Using 2 as Reserved VLAN for 1 slave
[    2.226784] Detected MACID = 58:7a:62:ac:de:d8
[    2.233009] cpsw: Detected MACID = 58:7a:62:ac:de:da
[    2.240695] rtc-pcf2129 0-0051: low voltage detected, date/time is not reliable.
[    2.248544] rtc-pcf2129 0-0051: setting system clock to 2026-01-01 19:39:17 UTC (1767296357)
[    2.260743] ALSA device list:
[    2.263897]   #0: AM33xx_EVM
[    2.336507] EXT4-fs (mmcblk0p3): recovery complete
[    2.346270] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[    2.354896] VFS: Mounted root (ext4 filesystem) on device 179:3.
[    2.369728] devtmpfs: mounted
[    2.373576] Freeing unused kernel memory: 344K (c0801000 - c0857000)
[    2.831122] EXT4-fs (mmcblk0p3): re-mounted. Opts: (null)
[    2.990418] udevd[1444]: starting version 182
[    3.387723] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/usb@47400000/usb-phy@47401300[0]'
[    3.405839] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/usb@47400000/usb-phy@47401b00[0]'
[    6.832954] musb-hdrc musb-hdrc.0.auto: Enabled SW babble control
[    6.942675] musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
[    6.942706] musb-hdrc: MHDRC RTL version 2.0 
[    6.942717] musb-hdrc: setup fifo_mode 4
[    6.942739] musb-hdrc: 28/31 max ep, 16384/16384 memory
[    6.942875] musb-hdrc musb-hdrc.0.auto: MUSB HDRC host driver
[    7.032049] musb-hdrc musb-hdrc.0.auto: new USB bus registered, assigned bus number 1
[    7.111510] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    7.118776] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    7.126410] usb usb1: Product: MUSB HDRC host driver
[    7.131671] usb usb1: Manufacturer: Linux 3.12.10-BSPV2.01T013.12-am335xpsp musb-hcd
[    7.139855] usb usb1: SerialNumber: musb-hdrc.0.auto
[    7.277046] hub 1-0:1.0: USB hub found
[    7.314324] hub 1-0:1.0: 1 port detected
[    7.384160] musb-hdrc musb-hdrc.1.auto: Enabled SW babble control
[    7.450873] musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
[    7.450903] musb-hdrc: MHDRC RTL version 2.0 
[    7.450914] musb-hdrc: setup fifo_mode 4
[    7.450936] musb-hdrc: 28/31 max ep, 16384/16384 memory
[    7.451072] musb-hdrc musb-hdrc.1.auto: MUSB HDRC host driver
[    7.521215] musb-hdrc musb-hdrc.1.auto: new USB bus registered, assigned bus number 2
[    7.629079] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
[    7.636281] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    7.643932] usb usb2: Product: MUSB HDRC host driver
[    7.649188] usb usb2: Manufacturer: Linux 3.12.10-BSPV2.01T013.12-am335xpsp musb-hcd
[    7.657361] usb usb2: SerialNumber: musb-hdrc.1.auto
[    7.762768] EXT3-fs (mmcblk0p2): error: can't find ext3 filesystem on dev mmcblk0p2.
[    7.820385] EXT2-fs (mmcblk0p2): error: can't find an ext2 filesystem on dev mmcblk0p2.
[    7.913473] EXT4-fs (mmcblk0p2): VFS: Can't find ext4 filesystem
[    7.931091] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    7.945305] hub 2-0:1.0: USB hub found
[    7.958452] EXT4-fs (mmcblk0p4): recovery complete
[    7.974624] cramfs: wrong magic
[    7.983031] hub 2-0:1.0: 1 port detected
[    7.992056] EXT4-fs (mmcblk0p4): mounted filesystem with ordered data mode. Opts: (null)
[    8.003398] FAT-fs (mmcblk0p2): bogus logical sector size 65535
[    8.009726] FAT-fs (mmcblk0p2): Can't find a valid FAT filesystem
[    8.161333] FAT-fs (mmcblk0p2): bogus logical sector size 65535
[    8.167598] FAT-fs (mmcblk0p2): Can't find a valid FAT filesystem
[   10.500907] jffs2: notice: (1906) jffs2_build_xattr_subsystem: complete building xattr subsystem, 0 of xdatum (0 unchecked, 0 orphan) and 0 of xref (0 dead, 0 orphan) found.
[   11.069477] net eth0: initializing cpsw version 1.12 (0)
[   11.078066] net eth0: phy found : id is : 0x7c0f1
[   11.094019] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   11.100232] 8021q: adding VLAN 0 to HW filter on device eth0
[   23.174702] net eth1: initializing cpsw version 1.12 (0)
[   23.190916] net eth1: phy found : id is : 0x7c0f1
[   23.215041] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[   23.221332] 8021q: adding VLAN 0 to HW filter on device eth1
[   23.919925] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[   23.932613] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  143.958206] omap_hwmod: d_can0: _wait_target_disable failed
[  143.968021] omap_hwmod: d_can0: _wait_target_disable failed
[  143.975144] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  143.987263] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  265.029749] omap_hwmod: d_can0: _wait_target_disable failed
[  265.045716] omap_hwmod: d_can0: _wait_target_disable failed
[  265.063414] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  265.083969] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  386.131288] omap_hwmod: d_can0: _wait_target_disable failed
[  386.141144] omap_hwmod: d_can0: _wait_target_disable failed
[  386.148083] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  386.157197] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  424.549280] libphy: 4a101000.mdio:00 - Link is Up - 100/Full
[  424.555528] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  507.196899] omap_hwmod: d_can0: _wait_target_disable failed
[  507.206702] omap_hwmod: d_can0: _wait_target_disable failed
[  507.217879] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  507.238312] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  628.278634] omap_hwmod: d_can0: _wait_target_disable failed
[  628.288488] omap_hwmod: d_can0: _wait_target_disable failed
[  628.295373] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  628.304609] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  749.345249] omap_hwmod: d_can0: _wait_target_disable failed
[  749.362065] omap_hwmod: d_can0: _wait_target_disable failed
[  749.373613] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  749.384987] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  870.437336] omap_hwmod: d_can0: _wait_target_disable failed
[  870.456066] omap_hwmod: d_can0: _wait_target_disable failed
[  870.486240] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  870.499617] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[  991.549552] omap_hwmod: d_can0: _wait_target_disable failed
[  991.559314] omap_hwmod: d_can0: _wait_target_disable failed
[  991.566593] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[  991.575907] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1112.616229] omap_hwmod: d_can0: _wait_target_disable failed
[ 1112.626055] omap_hwmod: d_can0: _wait_target_disable failed
[ 1112.633299] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1112.646038] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1233.687558] omap_hwmod: d_can0: _wait_target_disable failed
[ 1233.697444] omap_hwmod: d_can0: _wait_target_disable failed
[ 1233.711347] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1233.721592] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1355.761583] omap_hwmod: d_can0: _wait_target_disable failed
[ 1355.771412] omap_hwmod: d_can0: _wait_target_disable failed
[ 1355.778564] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1355.787971] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1476.826199] omap_hwmod: d_can0: _wait_target_disable failed
[ 1476.839585] omap_hwmod: d_can0: _wait_target_disable failed
[ 1476.856385] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1476.866386] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1597.905184] omap_hwmod: d_can0: _wait_target_disable failed
[ 1597.914979] omap_hwmod: d_can0: _wait_target_disable failed
[ 1597.922123] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1597.931255] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1718.969683] omap_hwmod: d_can0: _wait_target_disable failed
[ 1718.979559] omap_hwmod: d_can0: _wait_target_disable failed
[ 1718.986584] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1718.995729] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1840.040372] omap_hwmod: d_can0: _wait_target_disable failed
[ 1840.058252] omap_hwmod: d_can0: _wait_target_disable failed
[ 1840.086873] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1840.097087] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 1961.135676] omap_hwmod: d_can0: _wait_target_disable failed
[ 1961.145593] omap_hwmod: d_can0: _wait_target_disable failed
[ 1961.152541] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 1961.161700] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2082.201619] omap_hwmod: d_can0: _wait_target_disable failed
[ 2082.211597] omap_hwmod: d_can0: _wait_target_disable failed
[ 2082.218829] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2082.228079] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2203.266306] omap_hwmod: d_can0: _wait_target_disable failed
[ 2203.276107] omap_hwmod: d_can0: _wait_target_disable failed
[ 2203.283430] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2203.292711] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2324.331366] omap_hwmod: d_can0: _wait_target_disable failed
[ 2324.341150] omap_hwmod: d_can0: _wait_target_disable failed
[ 2324.348059] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2324.357367] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2445.395959] omap_hwmod: d_can0: _wait_target_disable failed
[ 2445.405820] omap_hwmod: d_can0: _wait_target_disable failed
[ 2445.415446] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2445.424956] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2566.464736] omap_hwmod: d_can0: _wait_target_disable failed
[ 2566.474530] omap_hwmod: d_can0: _wait_target_disable failed
[ 2566.481454] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2566.490799] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2687.530067] omap_hwmod: d_can0: _wait_target_disable failed
[ 2687.539968] omap_hwmod: d_can0: _wait_target_disable failed
[ 2687.546961] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2687.556379] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2808.596205] omap_hwmod: d_can0: _wait_target_disable failed
[ 2808.606006] omap_hwmod: d_can0: _wait_target_disable failed
[ 2808.612924] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2808.622316] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 2929.660803] omap_hwmod: d_can0: _wait_target_disable failed
[ 2929.670614] omap_hwmod: d_can0: _wait_target_disable failed
[ 2929.678489] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 2929.688169] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3050.726446] omap_hwmod: d_can0: _wait_target_disable failed
[ 3050.746981] omap_hwmod: d_can0: _wait_target_disable failed
[ 3050.764015] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3050.774012] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3172.813969] omap_hwmod: d_can0: _wait_target_disable failed
[ 3172.823845] omap_hwmod: d_can0: _wait_target_disable failed
[ 3172.830822] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3172.840106] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3293.878652] omap_hwmod: d_can0: _wait_target_disable failed
[ 3293.888425] omap_hwmod: d_can0: _wait_target_disable failed
[ 3293.895566] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3293.905210] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3414.944007] omap_hwmod: d_can0: _wait_target_disable failed
[ 3414.953768] omap_hwmod: d_can0: _wait_target_disable failed
[ 3414.960700] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3414.970300] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3536.009222] omap_hwmod: d_can0: _wait_target_disable failed
[ 3536.029780] omap_hwmod: d_can0: _wait_target_disable failed
[ 3536.053572] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3536.075313] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3657.123881] omap_hwmod: d_can0: _wait_target_disable failed
[ 3657.133765] omap_hwmod: d_can0: _wait_target_disable failed
[ 3657.140823] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3657.150106] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3778.189905] omap_hwmod: d_can0: _wait_target_disable failed
[ 3778.200524] omap_hwmod: d_can0: _wait_target_disable failed
[ 3778.207453] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3778.216795] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 3899.255701] omap_hwmod: d_can0: _wait_target_disable failed
[ 3899.265594] omap_hwmod: d_can0: _wait_target_disable failed
[ 3899.276463] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 3899.286834] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 4020.328080] omap_hwmod: d_can0: _wait_target_disable failed
[ 4020.347918] omap_hwmod: d_can0: _wait_target_disable failed
[ 4020.365458] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 4020.375699] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 4141.415179] omap_hwmod: d_can0: _wait_target_disable failed
[ 4141.428212] omap_hwmod: d_can0: _wait_target_disable failed
[ 4141.445750] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 4141.455799] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 4262.493861] omap_hwmod: d_can0: _wait_target_disable failed
[ 4262.505645] omap_hwmod: d_can0: _wait_target_disable failed
[ 4262.513078] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 4262.522501] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000
[ 4383.561650] omap_hwmod: d_can0: _wait_target_disable failed
[ 4383.571445] omap_hwmod: d_can0: _wait_target_disable failed
[ 4383.578448] c_can_platform 481cc000.d_can can0: setting BTR=1c02 BRPE=0000
[ 4383.587819] c_can_platform 481d0000.d_can can1: setting BTR=2701 BRPE=0000

Could you please help check the following points:

#1: If the step 1~5 are correct to get the RS485 DE status? (Especially based on step 5 result, is it reasonable to indicate the product currently is running as expected?)

#2: Please check if the step 6 dmesg log is as expected. We want to make sure if it already covered expected information we need. For example, if there is any overrun issue or any other problems happened, shall it will be present in the dmesg log by obtaining in this way?

#3: We also think maybe dumping some UART registers will also be beneficial when problem happens. As shown in their schematic, customer is using the UART4 in this case, could you also suggest some UART registers with the address for customer to dump?

Thanks,

Kevin

Hi Kevin,

I will check the register details. But at first, are the customer able to probe the UART signals on the board when the issue happens?

I think we first need to probe the DE pin (GPIO2_4) to see if it is low.

If it is high, we need to probe the UART4 TXD pin to see if the AM335x UART4 is constantly transmitting data.

If DE pin is low, we need to probe the UART4 RXD pin to see if the UART data bits are on the pin, so that we know the RX485 transceiver is working fine.

Hi Bin,

Understand the concern, if it is happening on customer's lab (never happened before), they are able to probe UART RX/TXD pins, but if happening on their end customer's product, it is quite difficult to do that. I will continue highlight this point to customer side in parallel.

Please help to check:

#1: If the above steps to read GPIO2_4 register is appropriate or not?

#2: If the above dmesg log is sufficient, or any other log required to ensure there is no overrun happened?

#3: If there are any suggested UART4 registers for customer to read?

Thanks,

Kevin

Hi Kevin,

The register addresses are correct, but I am not sure if the DATAIN/DATAOUT registers can be directly read to get the status of the pin. I need to check this on the EVM.

What is the pinmux setting for GPIO2_4 on the custom board?

I don't think kernel log prints any uart overrun information. You probably need the program serialstat provided in project https://github.com/TexasInstruments/serialcheck

You also might want to run command "cat /proc/tty/driver/serial" on the failure board to see if it provides any information.

Hi Bin,

Customer is using the ZCZ package, so the pinmux should be U6.

Thanks,

Kevin

Kevin,

What is the value written to this pin (U6) padconfig register in Linux kernel? 

Bin Liu said:

You probably need the program serialstat provided in project https://github.com/TexasInstruments/serialcheck

I built the serialstats on target board with command: root@am62xx-evm:/home/serialcheck-master# gcc serialstats.c -o serialstats

Want to confirm on how to use the serialstats, is it continually monitoring status and there isn't output if no errors if without -o(once) option?

root@am62xx-evm:/home/serialcheck-master# ./serialstats --help
Usage: serialstats [OPTION...]
uart stats tool

-d, --device=FILE serial node device
-i, --interval=NUM interval in seconds
-o, --once print stats once and exit
-?, --help Give this help list
--usage Give a short usage message

Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
root@am62xx-evm:/home/serialcheck-master# ./serialstats --device=/dev/ttyS0

^C
root@am62xx-evm:/home/serialcheck-master# ./serialstats --device=/dev/ttyS1
^C
root@am62xx-evm:/home/serialcheck-master# ./serialstats --device=/dev/ttyS1 -o
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 0 tx: 0 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0
root@am62xx-evm:/home/serialcheck-master#

Hi Bin,

The following are the steps we read out the value of pin (U6) padconfig register:

Step 1: The U6 is referring to GPMC_Wen control_module register

Step 2: The offset of GPMC_Wen control_module register is 0x898

Step 3: The base address for control module is 0x44E10000

Step 4: The address for U6 padconfig register will be 0x44E10000 + 0x898 = 0x44E10898

Step 5: Customer uses following command to read out the value in their lab after running their board for some times:

root@Emerson:~# devmem 0x44E10898
0x0000000F

The 0x0000000F is the value writing to the pin (U6) padconfig register.

Thanks,

Kevin

Hi Bin,

We have let customer tested cat /proc/tty/driver/serial, and got the below results:

root@Emerson:~# cat /proc/tty/driver/serial
serinfo:1.0 driver revision:
0: uart:unknown port:00000000 irq:0
1: uart:unknown port:00000000 irq:0
2: uart:unknown port:00000000 irq:0
3: uart:unknown port:00000000 irq:0

We think the reason is that, in the SDK3.x customer used, the DTS is not supported.

As a workaround, we think the most accurate & reliable way is to read out all error related registers to see if any errors occured. We have created below steps for customers to try when the problem happens, please have a check, thanks.

Step 1: The base address for UART4 (customer used) is 0x481A8000

Step 2: LSR_UART (offset 0x14) records OE (overrun error), PE (parity error), and FE (frame error)

Step 3: Apart from LSR_UART, we think IER_UART (offset 0x4), IIR_UART (offset 0x8), LCR (offset 0xC), MCR (offset 0x10), and TLR (offset 0x1C) could also be helpful for the analysis, we will let customer read out these registers when problem happens to support our further analysis. Do you think this is okay to replace the usage of  "cat /proc/tty/driver/serial" ?

LSR_UART (offset 0x14):  devmem 0x481A8014

IER_UART (offset 0x4):  devmem 0x481A8004

IIR_UART (offset 0x8): devmem 0x481A8008

LCR (offset 0xC): devmem 0x481A800C

MCR (offset 0x10): devmem 0x481A8010

TLR (offset 0x1C): devmem 0x481A801C

Thanks,

Kevin

Hi Tony,

Tony Tang said:

is it continually monitoring status and there isn't output if no errors if without -o(once) option?

Without '-o' parameter, the program prints the message in every 10 seconds if '-i' is not used either. You can use '-i' to specify the print interval.

Hi Kevin,

Kevin Peng said:

We have let customer tested cat /proc/tty/driver/serial, and got the below results:

This result doesn't have any helpful information.

Kevin Peng said:

The 0x0000000F is the value writing to the pin (U6) padconfig register.

This looks good to me.

Kevin Peng said:

Step 2: LSR_UART (offset 0x14) records OE (overrun error), PE (parity error), and FE (frame error)

The OE, PE, FE reflect the error for the particular byte received. The register value will be reset once the byte has been read out of the UART RX FIFO. So I am not sure if the information would be helpful. However the error status has been recorded in the kernel internal data structure which can be queried by the 'serialstat' program mentioned above.

By the way, when reading the DE pin (GPIO2_4) state, please use its DATAOUT register (0x481AC13C). The DATAIN register won't tell the GPIO pin output level.

Bin Liu said:

This result doesn't have any helpful information.

Should the output have irq count at least? As there is not any count output from this command, Seems like the command cat /proc/tty/driver/serial" doesn't take effect.

Kevin Peng 

serialstats.c is simply reading TIOCGICOUNT count and print. it should work independent of Linux version if can be built successfully on target system. 

Tony Tang said:

Should the output have irq count at least? As there is not any count output from this command, Seems like the command cat /proc/tty/driver/serial" doesn't take effect.

We can ignore this procfs entry. I haven't touched omap_serial driver for many years, and now I am not sure if it is covered by this procfs entry. We can get irq count from /proc/interrupts.

Tony Tang said:

it should work independent of Linux version if can be built successfully on target system. 

Just FYI, you can also cross-compile it on a host PC, using a 32-bit (ARMv7) toolchain.

Hi Bin,

Customer just reproduced this issue on their end customer side today.

Please see the log below for register dump:

devmem 0x481A8014;devmem 0x481A8004;devmem 0x481A8008;devmem 0x481A800C;devmem 0x481A8010;devmem 0x481A801C;devmem 0x481AC134;devmem 0x481AC138;devmem 0x481AC13C
Bus error
Bus error
Bus error
Bus error
Bus error
Bus error
0xFFFFFFEF
0x0000002E
0x00000000

Thanks,

Kevin

Bus error seems telling the UART module goes into suspend state. Please keep this board in this failure state, I will experiment on the EVM tomorrow then tell you the command to the Linux sysfs to check the UART runtime PM state.

devmem 0x481AC134;devmem 0x481AC138;devmem 0x481AC13C

What are these 3 registers - 0x481AC13x?

Bin Liu said:

devmem 0x481AC134;devmem 0x481AC138;devmem 0x481AC13C

What are these 3 registers - 0x481AC13x?

Never mind, they are for the gpio DE pin.

I let them to check UART4 status this afternoon:  devmem 0x44E00078

Tony,

Yeah you know this better than me

It seems the UART is in runtime suspend state, and its clock is gated.

Hi Bin,

Since UART4 status register dump has Bus Error, we suspect its clocking is properly enabled or not. Hence, we provide a complete register dump below to confirm:

CM_WKUP_UART0_CLKCTRL:  devmem 0x44E004B4
0x00000002

CM_PER_UART1_CLKCTRL:   devmem 0x44E0006C
0x00000002

CM_PER_UART2_CLKCTRL:   devmem 0x44E00070
0x00000002

CM_PER_UART3_CLKCTRL:   devmem 0x44E00074
0x00000002

CM_PER_UART4_CLKCTRL:   devmem 0x44E00078
0x00030000

CM_PER_UART5_CLKCTRL:   devmem 0x44E00038
0x00030000

CM_PER_GPIO2_CLKCTRL：devmem 0x44E000B0
0x00000002

Referring to the below image, it indicates that UART4 clocking is disabled by some reasons. (UART5 is also disabled because they did not enable in the SW, customer only uses UART4)

We have double confirmed letting customer to read UART5 status registers & UART2 status registers wishing to see UART5 has Bus Error & UART2 can successfully read the status to double confirm the UART4 status bus error is because the clock is disabled by some reason:

UART 5:
IER UART (Offset 4):   devmem 0x481AA004
Bus error
IIR UART (Offset 8):   devmem 0x481AA008
Bus error
SPR (Offset 1C):       devmem 0x481AA01C
Bus error

UART 2:
IER UART (Offset 4):   devmem 0x48024004
0x00000005
IIR UART (Offset 8):   devmem 0x48024008
0x000000C1
SPR (Offset 1C):       devmem 0x4802401C
0x00000000

This result proves our assumption that it is because UART4 clock is disabled by some reason that cause bus error in register dump.

Applied serialstats tool , output as below:

root@am33x:~ ./serialstats --device=/dev/ttyO4 -o   
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 396082106 tx: -2106099644 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0
root@am33x:~ ^C
root@am33x:~ ./serialstats --device=/dev/ttyO4 -o   
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 396082106 tx: -2106099644 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0
root@am33x:~ ./serialstats --device=/dev/ttyO4
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 0 tx: 0 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 0 tx: 0 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0
cts: 0 dsr: 0 rng: 0 dcd: 0 rx: 0 tx: 0 frame error 0 overuns 0 parity: 0 break: 0 buffer overrun: 0

//The output without -o option are 0 is correct, as it is (new_counters.x-old_counter.x) in while(1), there is not increament. so 0.

It seems that there is no obvious errors recorded in the tool. Only strange thing is the Tx is a negative number. Does it have any potential impacts?

cat /proc/tty/driver/OMAP-SERIAL to confirm the status: the output is same as that of serialstats.

root@am33x:~ cat /proc/tty/driver/OMAP-SERIAL
serinfo:1.0 driver revision:
0: uart:OMAP UART0 mmio:0x44E09000 irq:88 tx:39039 rx:0 RTS|CTS|DTR|DSR
1: uart:OMAP UART1 mmio:0x48022000 irq:89 tx:0 rx:0 CTS|DSR|CD|RI
2: uart:OMAP UART2 mmio:0x48024000 irq:90 tx:0 rx:0 RTS|CTS|DTR|DSR
3: uart:OMAP UART3 mmio:0x481A6000 irq:60 tx:0 rx:0 CTS|DSR
4: uart:OMAP UART4 mmio:0x481A8000 irq:61 tx:-2106099644 rx:396082106 RTS|CTS|DTR|DSR

Full DMESG log as below:

4718.session3.log

There are 3 lines showing UART_Open is because at that time customer used the serialstats tool, so there is no problem.

Do you have any suggestions for the further debugging please?

Thanks,

Kevin

Hi Bin,

We have tried, but the end customer cannot accept give additional time remaining the environment. Therefore before customer recover the system, we let customer read all the peripheral clocking status to record:

root@am33x:~ devmem 0x44E00000;devmem 0x44E00004;devmem 0x44E0000C;devmem 0x44E00014;devmem 0x44E00018;devmem 0x44E0001C;devmem 0x44E00024;devmem 0x44E00028;devmem 0x44E0002C;devmem 0x44E00030;devmem 0x44E00034;devmem 0x44E00038;devmem 0x44E0003C;devmem 0x44E00040;devmem 0x44E00044;devmem 0x44E00048;devmem 0x44E0004C;devmem 0x44E00050;devmem 0x44E00060;devmem 0x44E00068;devmem 0x44E0006C;devmem 0x44E00070;devmem 0x44E00074;devmem 0x44E00078;devmem 0x44E0007C;devmem 0x44E00080;devmem 0x44E00084;devmem 0x44E00088;devmem 0x44E000AC;devmem 0x44E000B0;devmem 0x44E000B4;devmem 0x44E000BC;devmem 0x44E000C0;devmem 0x44E000C4;devmem 0x44E000CC;devmem 0x44E000D4;devmem 0x44E000D8;devmem 0x44E000DC;devmem 0x44E000E0;devmem 0x44E000E4;devmem 0x44E000E8;devmem 0x44E000EC;devmem 0x44E000F0;devmem 0x44E000F4;devmem 0x44E000F8;devmem 0x44E000FC;devmem 0x44E00100;devmem 0x44E0010C;devmem 0x44E00110;devmem 0x44E0011C;devmem 0x44E00120;devmem 0x44E0012C;devmem 0x44E00130;devmem 0x44E00140;devmem 0x44E00144;devmem 0x44E00148;devmem 0x44E0014C;devmem 0x44E00150
0x00224D02
0x0000000A
0x000000D6
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00030000
0x00030000
0x00000002
0x00030000
0x00030000
0x00030000
0x00030000
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00030000
0x00000002
0x00030000
0x00030000
0x00000002
0x00000002
0x00040002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00000002
0x00000002
0x00070000
0x00070000
0x00030000
0x00030000
0x00030000
0x00030000
0x00000002
0x00000002
0x00030000
0x00000002
0x0000007A
0x00000002
0x00000001
0x00060002
0x00000001
0x00000012
0x00000012
0x00000002
0x00000012

After that, customer uses their previous way to recover the system by: echo 1 >/dev/tty04

Just after this, the system recovers properly, we have let customer re-read the CM_PER_UART4_CLKCTRL to confirm:

root@am33x:~ devmem 0x44E00078
0x00000002

It shows the expected status. And we have let re-read all the peripheral clocking status again (only last one not read well, because command not put properly, so you could just compare the rest of them except the last 0x44E00150)

root@am33x:~ devmem 0x44E00000;devmem 0x44E00004;devmem 0x44E0000C;devmem 0x44E00014;devmem 0x44E00018;devmem 0x44E0001C;devmem 0x44E00024;devmem 0x44E00028;devmem 0x44E0002C;devmem 0x44E00030;devmem 0x44E00034;devmem 0x44E00038;devmem 0x44E0003C;devmem 0x44E00040;devmem 0x44E00044;devmem 0x44E00048;devmem 0x44E0004C;devmem 0x44E00050;devmem 0x44E00060;devmem 0x44E00068;devmem 0x44E0006C;devmem 0x44E00070;devmem 0x44E00074;devmem 0x44E00078;devmem 0x44E0007C;devmem 0x44E00080;devmem 0x44E00084;devmem 0x44E00088;devmem 0x44E000AC;devmem 0x44E000B0;devmem 0x44E000B4;devmem 0x44E000BC;devmem 0x44E000C0;devmem 0x44E000C4;devmem 0x44E000CC;devmem 0x44E000D4;devmem 0x44E000D8;devmem 0x44E000DC;devmem 0x44E000E0;devmem 0x44E000E4;devmem 0x44E000E8;devmem 0x44E000EC;devmem 0x44E000F0;devmem 0x44E000F4;devmem 0x44E000F8;devmem 0x44E000FC;devmem 0x44E00100;devmem 0x44E0010C;devmem 0x44E00110;devmem 0x44E0011C;devmem 0x44E00120;devmem 0x44E0012C;devmem 0x44E00130;devmem 0x44E00140;devmem 0x44E00144;devmem 0x44E00148;devmem 0x44E0014C;
0x00224D02
0x0000000A
0x000000D6
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00030000
0x00030000
0x00000002
0x00030000
0x00030000
0x00030000
0x00030000
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00030000
0x00000002
0x00030000
0x00030000
0x00000002
0x00000002
0x00040002
0x00000002
0x00000002
0x00000002
0x00000002
0x00000002
0x00030000
0x00000002
0x00000002
0x00070000
0x00070000
0x00030000
0x00030000
0x00030000
0x00030000
0x00000002
0x00000002
0x00030000
0x00000002
0x0000007A
0x00000002
0x00000001
0x00060002
0x00000001
0x00000012
0x00000012
0x00000002

The dmesg log after system recover becomes:

root@am33x:~ dmesg
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 3.12.10-BSPV2.01T013.12-am335xpsp (kcarney@NPUSDLWDGHMIBLD) (gcc version 4.9.1 20140505 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.05 - Linaro GCC 2014.05) ) #1 Thu Apr 2 08:24:33 EDT 2020
[    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine: Vertiv Global HMI (Flattened Device Tree), model: ACP AM335x Board
[    0.000000] cma: CMA: reserved 24 MiB at 9e000000
[    0.000000] Memory policy: ECC disabled, Data cache writeback
[    0.000000] On node 0 totalpages: 130304
[    0.000000] free_area_init_node: node 0, pgdat c08e8cfc, node_mem_map c0927000
[    0.000000]   Normal zone: 1024 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 130304 pages, LIFO batch:31
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] AM335X ES2.1 (sgx neon )
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 129280
[    0.000000] Kernel command line: console=ttyO0,115200n8 consoleblank=0 mtdparts=spi1.0:128k(SPL)ro,64k(HWID)ro,64k(Reserved)ro,128k(u-boot-env1),128k(u-boot-env2),1536k(u-boot)ro,512k(persistent),-(unused) root=/dev/mmcblk0p3 rw rootfstype=ext4 rootwait
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 482572K/521216K available (6134K kernel code, 585K rwdata, 2028K rodata, 346K init, 242K bss, 38644K reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0800fdc   (8164 kB)
      .init : 0xc0801000 - 0xc0857860   ( 347 kB)
      .data : 0xc0858000 - 0xc08ea610   ( 586 kB)
       .bss : 0xc08ea610 - 0xc0926f20   ( 243 kB)
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
[    0.000000] Total of 128 interrupts on 1 active controller
[    0.000000] OMAP clockevent source: timer2 at 24000000 Hz
[    0.000000] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
[    0.000000] OMAP clocksource: timer1 at 24000000 Hz
[    0.000000] Console: colour dummy device 80x30
[    0.000260] Calibrating delay loop... 663.55 BogoMIPS (lpj=3317760)
[    0.049759] pid_max: default: 32768 minimum: 301
[    0.049841] Security Framework initialized
[    0.049881] Mount-cache hash table entries: 512
[    0.056384] CPU: Testing write buffer coherency: ok
[    0.056719] Setting up static identity map for 0xc0603518 - 0xc0603588
[    0.057413] devtmpfs: initialized
[    0.058740] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
[    0.116204] pinctrl core: initialized pinctrl subsystem
[    0.116884] regulator-dummy: no parameters
[    0.118846] NET: Registered protocol family 16
[    0.120660] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.122651] cpuidle: using governor ladder
[    0.122664] cpuidle: using governor menu
[    0.129029] platform mpu.1: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.129886] platform 49000000.edma: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.130649] gpiochip_add: registered GPIOs 0 to 31 on device: gpio
[    0.130867] OMAP GPIO hardware version 0.1
[    0.131571] gpiochip_add: registered GPIOs 32 to 63 on device: gpio
[    0.132340] gpiochip_add: registered GPIOs 64 to 95 on device: gpio
[    0.133112] gpiochip_add: registered GPIOs 96 to 127 on device: gpio
[    0.138748] platform 56000000.sgx: FIXME: clock-name 'fck' DOES NOT exist in dt!
[    0.141311] DSS not supported on this SoC
[    0.141325] No ATAGs?
[    0.141334] hw-breakpoint: debug architecture 0x4 unsupported.
[    0.162221] bio: create slab <bio-0> at 0
[    0.173039] edma-dma-engine edma-dma-engine.0: TI EDMA DMA engine driver
[    0.173503] of_get_named_gpio_flags: can't parse gpios property of node '/fixedregulator@0[0]'
[    0.173736] vbat: 5000 mV 
[    0.173839] of_get_named_gpio_flags: can't parse gpios property of node '/fixedregulator@1[0]'
[    0.174030] lis3_reg: no parameters
[    0.174126] of_get_named_gpio_flags exited with status 48
[    0.174435] wlan-en-regulator: 1800 mV 
[    0.176445] drivers/tty/serial/e2_msdrv.c(1068): Creating netlink socket family 17.
[    0.176885] vgaarb: loaded
[    0.177838] SCSI subsystem initialized
[    0.178244] libata version 3.00 loaded.
[    0.178854] usbcore: registered new interface driver usbfs
[    0.178992] usbcore: registered new interface driver hub
[    0.179147] usbcore: registered new device driver usb
[    0.179950] omap_i2c 44e0b000.i2c: could not find pctldev for node /pinmux@44e10800/pinmux_i2c0_pins, deferring probe
[    0.179972] platform 44e0b000.i2c: Driver omap_i2c requests probe deferral
[    0.180157] media: Linux media interface: v0.10
[    0.180285] Linux video capture interface: v2.00
[    0.180496] pps_core: LinuxPPS API ver. 1 registered
[    0.180503] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.180599] PTP clock support registered
[    0.181495] Advanced Linux Sound Architecture Driver Initialized.
[    0.182734] Switched to clocksource timer1
[    0.196731] NET: Registered protocol family 2
[    0.197224] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[    0.197292] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
[    0.197332] TCP: Hash tables configured (established 4096 bind 4096)
[    0.197390] TCP: reno registered
[    0.197402] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    0.197417] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    0.197572] NET: Registered protocol family 1
[    0.197894] RPC: Registered named UNIX socket transport module.
[    0.197904] RPC: Registered udp transport module.
[    0.197910] RPC: Registered tcp transport module.
[    0.197915] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.197926] PCI: CLS 0 bytes, default 64
[    0.199178] PM: Loading am335x-pm-firmware.bin
[    0.319572] VFS: Disk quotas dquot_6.5.2
[    0.319635] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.320114] NFS: Registering the id_resolver key type
[    0.320201] Key type id_resolver registered
[    0.320210] Key type id_legacy registered
[    0.320240] jffs2: version 2.2. (NAND) (SUMMARY)  \xffffffc2\xffffffa9 2001-2006 Red Hat, Inc.
[    0.320383] msgmni has been set to 990
[    0.321585] NET: Registered protocol family 38
[    0.321620] io scheduler noop registered
[    0.321628] io scheduler deadline registered
[    0.321646] io scheduler cfq registered (default)
[    0.323399] pinctrl-single 44e10800.pinmux: 142 pins at pa f9e10800 size 568
[    0.423349] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.425359] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/serial@44e09000[0]'
[    0.425606] 44e09000.serial: ttyO0 at MMIO 0x44e09000 (irq = 88, base_baud = 3000000) is a OMAP UART0
[    1.040969] console [ttyO0] enabled
[    1.045110] of_get_named_gpio_flags exited with status 36
[    1.045351] 48022000.serial: ttyO1 at MMIO 0x48022000 (irq = 89, base_baud = 3000000) is a OMAP UART1
[    1.055487] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/serial@48024000[0]'
[    1.055681] 48024000.serial: ttyO2 at MMIO 0x48024000 (irq = 90, base_baud = 3000000) is a OMAP UART2
[    1.065777] of_get_named_gpio_flags exited with status 30
[    1.066008] 481a6000.serial: ttyO3 at MMIO 0x481a6000 (irq = 60, base_baud = 3000000) is a OMAP UART3
[    1.076113] of_get_named_gpio_flags exited with status 68
[    1.076346] 481a8000.serial: ttyO4 at MMIO 0x481a8000 (irq = 61, base_baud = 3000000) is a OMAP UART4
[    1.087172] omap_rng 48310000.rng: OMAP Random Number Generator ver. 20
[    1.101601] brd: module loaded
[    1.108805] loop: module loaded
[    1.115840] mtdoops: mtd device (mtddev=name/number) must be supplied
[    1.124216] edma-dma-engine edma-dma-engine.0: allocated channel for 0:17
[    1.131366] edma-dma-engine edma-dma-engine.0: allocated channel for 0:16
[    1.138872] m25p80 spi1.0: s25fl256s1 (32768 Kbytes)
[    1.144183] 8 cmdlinepart partitions found on MTD device spi1.0
[    1.150393] Creating 8 MTD partitions on "spi1.0":
[    1.155447] 0x000000000000-0x000000020000 : "SPL"
[    1.161550] 0x000000020000-0x000000030000 : "HWID"
[    1.167619] 0x000000030000-0x000000040000 : "Reserved"
[    1.174046] 0x000000040000-0x000000060000 : "u-boot-env1"
[    1.180673] 0x000000060000-0x000000080000 : "u-boot-env2"
[    1.187400] 0x000000080000-0x000000200000 : "u-boot"
[    1.193668] 0x000000200000-0x000000280000 : "persistent"
[    1.200230] 0x000000280000-0x000002000000 : "unused"
[    1.207800] vcan: Virtual CAN interface driver
[    1.212475] CAN device driver interface
[    1.217552] c_can_platform 481cc000.d_can: c_can_platform device registered (regs=fa1cc000, irq=68)
[    1.227970] c_can_platform 481d0000.d_can: c_can_platform device registered (regs=fa1d0000, irq=71)
[    1.238914] usbcore: registered new interface driver asix
[    1.244784] usbcore: registered new interface driver ax88179_178a
[    1.251320] usbcore: registered new interface driver cdc_ether
[    1.257571] usbcore: registered new interface driver r815x
[    1.263492] usbcore: registered new interface driver smsc95xx
[    1.269653] usbcore: registered new interface driver net1080
[    1.275713] usbcore: registered new interface driver cdc_subset
[    1.282052] usbcore: registered new interface driver zaurus
[    1.288072] usbcore: registered new interface driver cdc_ncm
[    1.294457] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.301307] ehci-pci: EHCI PCI platform driver
[    1.306151] ehci-omap: OMAP-EHCI Host Controller driver
[    1.312362] usbcore: registered new interface driver cdc_wdm
[    1.318490] usbcore: registered new interface driver usb-storage
[    1.325659] mousedev: PS/2 mouse device common for all mice
[    1.333090] input: ti-tsc as /devices/ocp.2/44e0d000.tscadc/TI-am335x-tsc/input/input0
[    1.342590] i2c /dev entries driver
[    1.346764] Driver for 1-wire Dallas network protocol.
[    1.353164] tps3823_wdt: configured with heartbeat=30 sec (nowayout=0)
[    1.360414] tps3823_wdt watchdog.4: initialized watchdog.
[    1.367164] of_get_named_gpio_flags exited with status 115
[    1.367177] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/mmc@48060000[0]'
[    1.367415] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
[    1.374661] edma-dma-engine edma-dma-engine.0: allocated channel for 0:24
[    1.381837] omap_hsmmc 48060000.mmc: unable to get vmmc regulator -517
[    1.388723] edma-dma-engine edma-dma-engine.0: freeing channel for 24
[    1.395497] edma-dma-engine edma-dma-engine.0: freeing channel for 25
[    1.402341] platform 48060000.mmc: Driver omap_hsmmc requests probe deferral
[    1.410310] ledtrig-cpu: registered to indicate activity on CPUs
[    1.416943] edma-dma-engine edma-dma-engine.0: allocated channel for 0:36
[    1.424138] omap-sham 53100000.sham: hw accel on OMAP rev 4.3
[    1.431132] omap-aes 53500000.aes: OMAP AES hw accel rev: 3.2
[    1.437279] edma-dma-engine edma-dma-engine.0: allocated channel for 0:5
[    1.444336] edma-dma-engine edma-dma-engine.0: allocated channel for 0:6
[    1.452430] usbcore: registered new interface driver usbhid
[    1.458315] usbhid: USB HID core driver
[    1.464299] usbcore: registered new interface driver snd-usb-audio
[    1.472418] davinci_evm sound.3: ASoC: CODEC (null) not registered
[    1.479046] davinci_evm sound.3: snd_soc_register_card failed (-517)
[    1.485750] platform sound.3: Driver davinci_evm requests probe deferral
[    1.493431] oprofile: no performance counters
[    1.498314] oprofile: using timer interrupt.
[    1.503083] TCP: cubic registered
[    1.506561] Initializing XFRM netlink socket
[    1.511078] NET: Registered protocol family 10
[    1.516345] sit: IPv6 over IPv4 tunneling driver
[    1.522067] NET: Registered protocol family 17
[    1.526883] NET: Registered protocol family 15
[    1.531580] can: controller area network core (rev 20120528 abi 9)
[    1.538170] NET: Registered protocol family 29
[    1.542875] can: raw protocol (rev 20120528)
[    1.547378] can: broadcast manager protocol (rev 20120528 t)
[    1.553370] can: netlink gateway (rev 20130117) max_hops=1
[    1.559209] 8021q: 802.1Q VLAN Support v1.8
[    1.563694] Key type dns_resolver registered
[    1.568804] cpu cpu0: cpu0 regulator not ready, retry
[    1.574226] platform cpufreq-cpu0.0: Driver cpufreq-cpu0 requests probe deferral
[    1.582366] PM: CM3 Firmware Version 18 not supported
[    1.587965] ThumbEE CPU extension supported.
[    1.595439] tps65910 0-002d: No interrupt support, no core IRQ
[    1.603389] vrtc: 1800 mV 
[    1.606476] vrtc: supplied by vbat
[    1.611160] vio: at 1500 mV 
[    1.614259] vio: supplied by vbat
[    1.619000] vdd_mpu: 912 <--> 1375 mV at 1325 mV 
[    1.624008] vdd_mpu: supplied by vbat
[    1.629063] vdd_core: 912 <--> 1150 mV at 1100 mV 
[    1.634152] vdd_core: supplied by vbat
[    1.639118] vdd3: 5000 mV 
[    1.642995] vdig1: at 1800 mV 
[    1.646249] vdig1: supplied by vbat
[    1.650953] vdig2: at 1800 mV 
[    1.654230] vdig2: supplied by vbat
[    1.658940] vpll: at 1800 mV 
[    1.662099] vpll: supplied by vbat
[    1.666726] vdac: at 1800 mV 
[    1.669892] vdac: supplied by vbat
[    1.674528] vaux1: at 1800 mV 
[    1.677776] vaux1: supplied by vbat
[    1.682476] vaux2: at 3300 mV 
[    1.685745] vaux2: supplied by vbat
[    1.690456] vaux33: at 3300 mV 
[    1.693815] vaux33: supplied by vbat
[    1.698611] vmmc: 1800 <--> 3300 mV at 3300 mV 
[    1.703430] vmmc: supplied by vbat
[    1.708494] rtc-pcf2129 0-0051: chip found, driver version 0.1
[    1.716246] rtc-pcf2129 0-0051: rtc core: registered rtc-pcf2129 as rtc0
[    1.723734] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz
[    1.730086] of_get_named_gpio_flags exited with status 115
[    1.730099] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/mmc@48060000[0]'
[    1.730506] edma-dma-engine edma-dma-engine.0: allocated channel for 0:25
[    1.737700] edma-dma-engine edma-dma-engine.0: allocated channel for 0:24
[    1.785456] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin MONO_LOUT
[    1.792330] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin HPLCOM
[    1.798938] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin HPRCOM
[    1.805525] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Line Out
[    1.812284] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Mic Jack
[    1.819049] tlv320aic23-codec 0-001b: ASoC: DAPM unknown pin Line In
[    1.828586] davinci_evm sound.3:  tlv320aic23-hifi <-> 4803c000.mcasp mapping ok
[    1.836434] davinci_evm sound.3: ASoC: no source widget found for HPLOUT
[    1.843521] davinci_evm sound.3: ASoC: Failed to add route HPLOUT -> direct -> Headphone Jack
[    1.852464] davinci_evm sound.3: ASoC: no source widget found for HPROUT
[    1.859502] davinci_evm sound.3: ASoC: Failed to add route HPROUT -> direct -> Headphone Jack
[    1.899949] mmc0: host does not support reading read-only switch. assuming write-enable.
[    1.910419] mmc0: new high speed SDHC card at address 59b4
[    1.916703] mmcblk0: mmc0:59b4 USD   7.51 GiB 
[    1.922516]  mmcblk0: p1 p2 p3 p4
[    1.932782] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6
[    1.939184] davinci_mdio 4a101000.mdio: detected phy mask fffffffc
[    1.946911] libphy: 4a101000.mdio: probed
[    1.951127] davinci_mdio 4a101000.mdio: phy[0]: device 4a101000.mdio:00, driver SMSC LAN8710/LAN8720
[    1.960743] davinci_mdio 4a101000.mdio: phy[1]: device 4a101000.mdio:01, driver SMSC LAN8710/LAN8720
[    1.971049] Missing dual_emac_res_vlan in DT.
[    1.975668] Using 1 as Reserved VLAN for 0 slave
[    1.980531] Missing dual_emac_res_vlan in DT.
[    1.985116] Using 2 as Reserved VLAN for 1 slave
[    1.989958] Detected MACID = 94:a9:a8:55:4c:60
[    1.995721] cpsw: Detected MACID = 94:a9:a8:55:4c:62
[    2.002832] rtc-pcf2129 0-0051: setting system clock to 2025-12-22 19:23:54 UTC (1766431434)
[    2.014056] ALSA device list:
[    2.017205]   #0: AM33xx_EVM
[    2.661322] EXT4-fs (mmcblk0p3): recovery complete
[    2.695836] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[    2.704404] VFS: Mounted root (ext4 filesystem) on device 179:3.
[    2.711945] devtmpfs: mounted
[    2.715526] Freeing unused kernel memory: 344K (c0801000 - c0857000)
[    3.327974] EXT4-fs (mmcblk0p3): re-mounted. Opts: (null)
[    3.458593] udevd[1443]: starting version 182
[    3.768531] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/usb@47400000/usb-phy@47401300[0]'
[    3.774636] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/usb@47400000/usb-phy@47401b00[0]'
[    6.500179] musb-hdrc musb-hdrc.0.auto: Enabled SW babble control
[    6.584111] musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
[    6.584134] musb-hdrc: MHDRC RTL version 2.0 
[    6.584141] musb-hdrc: setup fifo_mode 4
[    6.584156] musb-hdrc: 28/31 max ep, 16384/16384 memory
[    6.584258] musb-hdrc musb-hdrc.0.auto: MUSB HDRC host driver
[    6.657428] musb-hdrc musb-hdrc.0.auto: new USB bus registered, assigned bus number 1
[    6.735727] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    6.742943] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    6.750520] usb usb1: Product: MUSB HDRC host driver
[    6.755742] usb usb1: Manufacturer: Linux 3.12.10-BSPV2.01T013.12-am335xpsp musb-hcd
[    6.763889] usb usb1: SerialNumber: musb-hdrc.0.auto
[    7.056173] hub 1-0:1.0: USB hub found
[    7.088884] hub 1-0:1.0: 1 port detected
[    7.146989] musb-hdrc musb-hdrc.1.auto: Enabled SW babble control
[    7.215661] musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn)
[    7.215684] musb-hdrc: MHDRC RTL version 2.0 
[    7.215691] musb-hdrc: setup fifo_mode 4
[    7.215705] musb-hdrc: 28/31 max ep, 16384/16384 memory
[    7.215808] musb-hdrc musb-hdrc.1.auto: MUSB HDRC host driver
[    7.226845] EXT4-fs (mmcblk0p4): recovery complete
[    7.231901] EXT4-fs (mmcblk0p4): mounted filesystem with ordered data mode. Opts: (null)
[    7.291888] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    7.324139] musb-hdrc musb-hdrc.1.auto: new USB bus registered, assigned bus number 2
[    7.394828] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
[    7.401971] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    7.409606] usb usb2: Product: MUSB HDRC host driver
[    7.414834] usb usb2: Manufacturer: Linux 3.12.10-BSPV2.01T013.12-am335xpsp musb-hcd
[    7.422968] usb usb2: SerialNumber: musb-hdrc.1.auto
[    7.695189] hub 2-0:1.0: USB hub found
[    7.733659] hub 2-0:1.0: 1 port detected
[    8.992768] jffs2: notice: (1921) jffs2_build_xattr_subsystem: complete building xattr subsystem, 0 of xdatum (0 unchecked, 0 orphan) and 0 of xref (0 dead, 0 orphan) found.
[    9.459087] net eth1: initializing cpsw version 1.12 (0)
[    9.468194] net eth1: phy found : id is : 0x7c0f1
[    9.484966] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[    9.491114] 8021q: adding VLAN 0 to HW filter on device eth1
[    9.613779] c_can_platform 481cc000.d_can can0: setting BTR=1c05 BRPE=0000
[    9.705731] c_can_platform 481d0000.d_can can1: setting BTR=1c05 BRPE=0000
[   10.097032] Assembly Num: enp_hwid_asy=02359456 rev A03 - Loc: C - Date: SEP 20 2022 - Seq#: 0225 YEAR: 22 MONTH: 09 DAY: 20 SEQ: 0225
[   12.644067] net eth0: initializing cpsw version 1.12 (0)
[   12.652284] net eth0: phy found : id is : 0x7c0f1
[   12.668428] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   12.674631] 8021q: adding VLAN 0 to HW filter on device eth0
[  701.299530] c_can_platform 481cc000.d_can can0: msg lost in buffer 16
[685471.613402] libphy: 4a101000.mdio:01 - Link is Up - 10/Full
[685471.625905] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[685473.693067] libphy: 4a101000.mdio:01 - Link is Down
[685474.693774] libphy: 4a101000.mdio:01 - Link is Up - 100/Full
[685508.803005] libphy: 4a101000.mdio:01 - Link is Down
[8104906.574075] libphy: 4a101000.mdio:01 - Link is Up - 100/Full
[8106168.627940] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f28014
[8106896.178624] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f49014
[8107197.642108] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f7b014
[8107197.655624] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6fef004
[8107197.670466] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f57008
[8107197.684294] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f8b00c
[8107197.697742] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6fac010
[8107197.711357] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f8201c
[8108388.652986] libphy: 4a101000.mdio:01 - Link is Down
[8116556.663233] libphy: 4a101000.mdio:01 - Link is Up - 100/Full
[8120475.684014] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f13004
[8120499.796227] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f0d008
[8120554.496531] Unhandled fault: external abort on non-linefetch (0x1018) at 0xb6f7201c
[8122957.072874] usb 2-1: new high-speed USB device number 2 using musb-hdrc
[8122957.223645] usb 2-1: New USB device found, idVendor=3535, idProduct=6387
[8122957.230891] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[8122957.238610] usb 2-1: Product: USB DISK
[8122957.242839] usb 2-1: Manufacturer: aigo    
[8122957.247422] usb 2-1: SerialNumber: A4A233E956AED54D
[8122957.260965] usb-storage 2-1:1.0: USB Mass Storage device detected
[8122957.274294] scsi0 : usb-storage 2-1:1.0
[8122958.273537] scsi 0:0:0:0: Direct-Access     aigo     U312             PMAP PQ: 0 ANSI: 6
[8122958.288451] sd 0:0:0:0: [sda] 122880000 512-byte logical blocks: (62.9 GB/58.5 GiB)
[8122958.312310] sd 0:0:0:0: [sda] Write Protect is off
[8122958.317600] sd 0:0:0:0: [sda] Mode Sense: 45 00 00 00
[8122958.318536] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[8122958.586048]  sda: sda1
[8122958.598243] sd 0:0:0:0: [sda] Attached SCSI removable disk
[8122958.772087] EXT3-fs (sda): error: can't find ext3 filesystem on dev sda.
[8122958.780612] EXT2-fs (sda): error: can't find an ext2 filesystem on dev sda.
[8122958.789383] EXT4-fs (sda): VFS: Can't find ext4 filesystem
[8122958.797719] cramfs: wrong magic
[8122958.803178] FAT-fs (sda): invalid media value (0x01)
[8122958.808592] FAT-fs (sda): Can't find a valid FAT filesystem
[8122958.816573] FAT-fs (sda): invalid media value (0x01)
[8122958.821972] FAT-fs (sda): Can't find a valid FAT filesystem
[8122959.210519] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[8123192.348042] usb 2-1: USB disconnect, device number 2
[8123256.282957] usb 2-1: new high-speed USB device number 3 using musb-hdrc
[8123256.433681] usb 2-1: New USB device found, idVendor=3535, idProduct=6387
[8123256.440920] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[8123256.448617] usb 2-1: Product: USB DISK
[8123256.452743] usb 2-1: Manufacturer: aigo    
[8123256.457319] usb 2-1: SerialNumber: A4A233E956AED54D
[8123256.473736] usb-storage 2-1:1.0: USB Mass Storage device detected
[8123256.497491] scsi1 : usb-storage 2-1:1.0
[8123257.504227] scsi 1:0:0:0: Direct-Access     aigo     U312             PMAP PQ: 0 ANSI: 6
[8123257.518477] sd 1:0:0:0: [sda] 122880000 512-byte logical blocks: (62.9 GB/58.5 GiB)
[8123257.535607] sd 1:0:0:0: [sda] Write Protect is off
[8123257.540866] sd 1:0:0:0: [sda] Mode Sense: 45 00 00 00
[8123257.543925] sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[8123257.678943]  sda: sda1
[8123257.692288] sd 1:0:0:0: [sda] Attached SCSI removable disk
[8123257.901292] EXT3-fs (sda): error: can't find ext3 filesystem on dev sda.
[8123257.917931] EXT2-fs (sda): error: can't find an ext2 filesystem on dev sda.
[8123257.933181] EXT4-fs (sda): VFS: Can't find ext4 filesystem
[8123257.941490] cramfs: wrong magic
[8123257.946410] FAT-fs (sda): invalid media value (0x01)
[8123257.951810] FAT-fs (sda): Can't find a valid FAT filesystem
[8123257.959631] FAT-fs (sda): invalid media value (0x01)
[8123257.965092] FAT-fs (sda): Can't find a valid FAT filesystem
[8123258.501883] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[8124361.635580] uart_open(1827): ttyO4: tty->count = 2
[8124458.598337] uart_open(1827): ttyO4: tty->count = 2
[8124502.623692] uart_open(1827): ttyO4: tty->count = 2
[8124960.401037] usb 2-1: USB disconnect, device number 3
[8132189.551175] uart_open(1827): ttyO4: tty->count = 2

Thanks,

Kevin

Hi Kevin,

Running kernel 3.12 on EVM, then I did command:

    # find /sys -name '*.serial'

and got:

    /sys/bus/platform/devices/44e09000.serial
    /sys/bus/platform/drivers/omap_uart/44e09000.serial
    /sys/devices/ocp.2/44e09000.serial

its first line "44e09000.serial" is for the console UART0. But if you run this command on the custom board with UART4 enabled, you should also get:

    /sys/bus/platform/devices/481a8000.serial

For UART4, now run command:

    # cd /sys/bus/platform/devices/481a8000.serial/power/

then run command:

    # cat control

I expect you get:

    auto

run command:

    # cat runtime_status

I expect you get:

    suspended

which tells UART4 is in suspend state. Finally run command:

    # cat autosuspend_delay_ms

I got '-1' on evm, but I am wondering if you see a positive value on the custom board.

Finally, please run command:

    # echo on  > control

on the custom board, then check if this bring the UART4 back in active state.

If yes, the final solution for this issue is to do "echo on > control" right after Linux is booted and before the application is started.

Hi Bin,

Thanks for your support for the online meeting with customer.

There are 3 points we need to look at:

1: Based on your suggestions in the meeting to modify the suspend functions. We have work with customer & obtain the patch below. Please have a check on it, if everything is ok, thanks.

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/6710.my_5F00_code_5F00_change.patch

2: Customer has an idea that once next time there is a device re-occur the problem, customer plans to apply echo on  > control, if UART recovered immediately after that, customer will have more confidence that this command will be more likely resolve the issue if we apply it in advance. (This is customer's idea, but my concern is if the function for this command is try to prevent UART suspend, then if it is already suspend, it may not have the function to recover the UART? How do you think about this point please?)

3: Customer also has a plan to write a script in the background that continuously reads the UART clock register to check if it is enabled, and if not, writes to enable it. (This is customer's idea, but our concern is if UART is already suspended, then only writing clock to enable might not be sufficient, we may also need driver initialization, how do you think so?)

Thanks,

Kevin

Hi Kevin,

Kevin Peng said:

1: Based on your suggestions in the meeting to modify the suspend functions. We have work with customer & obtain the patch below. Please have a check on it, if everything is ok, thanks.

The patch is functionally correct. But I recommend to use dev_warn() instead of printk(). You can refer to other places in omap-serial.c to see how dev_warn() is used. dev_warn() will print the UART port address in the message, so up->port.line is not needed in the print message.

In testing this patch on the customer side, since the "autosuspend_delay_ms" is set to -1, serial_omap_runtime_suspend() function should not be called. I am thinking if the customer can manually change autosuspend_delay_ms to a positive number, such as 100ms, and don't transmit data to the UART port, to see if the printk() message shows up on the console.

Kevin Peng said:

2: Customer has an idea that once next time there is a device re-occur the problem, customer plans to apply echo on  > control, if UART recovered immediately after that, customer will have more confidence that this command will be more likely resolve the issue if we apply it in advance.

I think this is a good idea. When everything runs normally, when a module is already in runtime suspend state, change control from auto to on will immediately resume the module.

Again, since we don't know what gated the clk, we won't be able to tell if this would prevent the issue. we would have to test this and see.

Kevin Peng said:

3: Customer also has a plan to write a script in the background that continuously reads the UART clock register to check if it is enabled, and if not, writes to enable it.

This unlikely would work. The software (kernel) has its own state machine which won't be changed/corrected by just modifying the UART clk control register.

Hi Bin,

Please see the following updates:

Customer has followed your suggestion changing the printk() to dev_warn() for both serial_omap_suspend & serial_omap_runtime_suspend functions shown below:

Then customer conducts 3 experiments below:

1: When control is set to auto and autosuspend_delay_ms is set to 10000, if the serial port does not receive data for more than 10 seconds, the following message will be printed in the code:

[  101.289096] omap_uart 481a8000.serial:  serial_omap_runtime_suspend uart port num = 4

At this point, the register of UART4 remains normal (shown below as 0x00000002), indicating that the code patch effectively prevents the serial port from entering runtime suspension (cross-validated with an unpatched kernel, where the register value becomes 0x00030000 after suspension).

root@Emerson:/sys/bus/platform/devices/481a8000.serial/power# devmem 0x44E00078
0x00000002

2: When control is set to on and autosuspend_delay_ms is set to 10000, if the serial port does not receive data for more than 10 seconds, the message in the code will not be printed.

The register of UART4 is also normal, this indicates that control setting to on has higher priority to prevent system going to suspend functions even though autosuspend_delay is not -1 and not receiving the data.

3: Inputting echo freeze > /sys/power/state to put the system into sleep (to test serial_omap_suspend function, not has runtime) results in the following log:

[   56.404467] omap_uart 481a8000.serial:  serial_omap_suspend uart port num = 4

At this point, the register of UART4 remains normal, indicating that the code patch effectively prevents the serial port from entering system suspension.

root@Emerson:/sys/bus/platform/devices/481a8000.serial/power# devmem 0x44E00078
0x00000002

The overall result seems promising, and regarding to the test 3, will other modules also go to suspend function?

Thanks,

Kevin

Hi Kevin,

The patch looks good to me. By the way, you no longer need to print up->port.line now, since dev_warn() prints uart base address as the log prefix (like 481a8000.serial in the message), so we already know which uart prints the messages.

Kevin Peng said:

and regarding to the test 3, will other modules also go to suspend function?

No, any of the modules refuse to suspend will make the entire Linux not going to suspend at all.