Peripheral PLL issue with AM335x

Santhosh Ramani

Expert 2765 points

Other Parts Discussed in Thread: AM3352

Hello,

We have a custom design with AM3352 (ZCE) package and we have run into the problem mentioned in these threads:

e2e.ti.com/.../1011560

e2e.ti.com/.../265620

In our case the symptom seems to be that the baud rate on the 2 UARTs in use changes from 9600 (expected) to 12593 (observed). We have used auto baud rate detection with a protocol analyzer to determine the exact baud rate. Additionally, we have also observed the USB interface doesn't recognize any USB devices plugged into the board. We have also observed the I2C, clock frequency also changes - 400Khz (expected) to 555Khz (observed)

Please find below the schematic for the OSC0 and OSC1 circuits (we are not using OSC1 and left it floating) and based on the errata advisory we have updated the design; the crystal ground is connected to ground very close of the crystal. We observe this once in a while when the unit is power cycled and once in this state it never recovers. We have not tried a software reboot - I will update the thread with that result shortly.

This is the layout of the crystal and the clock inputs to the processor.

Do you guys have any thoughts on how we can resolve this issue. Thank you so much for your help and replies.

Regards

Santhosh

over 8 years ago

0 Santhosh Ramani over 8 years ago

Expert 2765 points

Sorry, couldn't paste the picture of the schematic and layout.

0 peaves over 8 years ago in reply to Santhosh Ramani

TI__Guru 55265 points

You need to understand the noise source and how it is coupling into the crystal circuit.

I see you have a copper pour under the crystal circuit, which could be acting like an antenna. You mentioned this signal was connected to the PCB digital ground and it didn’t eliminate the problem. This connection needs to provide a much lower inductance path to ground than the mutual coupling of the noise to the crystal circuit.

You may also need to shield the reference clock signals from other signals as they are routed to the oscillator terminals.

Noise is normally created by currents flowing through large loop areas. Are you routing large groups of fast toggling signals off board through a connector that does not provide a good ratio of return paths back to ground through the same connector?

You may want to temporarily replace the crystal with the same frequency 1.8 volt LVCMOS clock source to be certain your issue is related to noise coupling into the crystal circuit. The fast rise/fall times of the LVCMOS clock makes it very hard for noise to couple into the reference clock signal.

Regards,
Paul

0 Brad Griffis over 8 years ago in reply to peaves

TI__Guru*** 125430 points

Please grab a Clock Tree Tool dump for the case that things are working properly and the case where things are not. Name the files clearly, zip them, and post them here.

0 Brad Griffis over 8 years ago in reply to Brad Griffis

TI__Guru*** 125430 points

Here is a link for directions on getting the clock tree dump. Sorry, I meant to post this in my earlier reply:

processors.wiki.ti.com/.../AM335x_Clock_Tree_Tool

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Guys,

Sorry about the really late reply, I was side tracked with a different issue. As of now, we have a work around where we are rebooting the unit which allows the PLL to be reset. I will definitely look into the LVCMOS to identify the cause.

The unit doesn't have JTAG access, so I don't know if I can use Clock Tree dump. I wanted to know if there is a software method to re-initialize the PLL from Linux?

Thank you

Regards

Santhosh

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

If you're on a recent Processor SDK you can also use omapconf to get a CTT dump. I think the command is "omapconf export ctt". Note however that a bunch of registers were missing for AM335x in omapconf up until a patch 5 months ago, so if you're not using Proc SDK 2.00.01 or Proc SDK 2.00.02 then you probably would need to rebuild the latest version (or just grab it from the file system of a newer version).

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Brad,

I am using SDK 7.0 and so don't have the latest SDK. Additionally, I was thinking if I can detect this issue in u-boot. My plan is as follows:

1. Use a different UART (not the console) to send x bytes of data at a given baud rate.

2. The transmission should be done in y ms (+/- some variance - which should be 5% of the baud)

3. If the time taken is outside the boundary, then I reset the board. (do_reset function)

Does this sound plausible solution? This is based on the fact that the baud rate is wrong when the PLL is working incorrectly.

Thank you

Regards

Santhosh

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

I have built the latest omapconf using the toolchain bundled with SDK 7.00. Here's the output: omapconf.zip

Can you please grab the corresponding rd1 files (e.g. "./omapconf export ctt good-dump.rd1" and "./omapconf export ctt bad-dump.rd1")?

Given that several people have bumped into this issue, I would like to understand the underlying issue.

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Brad,

Once again sorry for the delayed response, ran into another issue with the PMIC. I will get you this data in a day or two.

Just some additional inputs:

1. I implemented the scheme I discussed in my previous post - checking the baud rate in u-boot. We had used this scheme to check the baudrate once the kernel boots up, so I was thinking if I can detect it at an early stage I can recover quickly.

2. So far I have not had a single instance of PLL corruption - I'm not running the OS, just u-boot and of course MLO. The unit powers up, MLO (SPL) executes, after the PLLs are setup I check the baudrate on UART1.

3. If the baud rate is outside +/- 3% then I perform a reset. So far, I've not had a single instance of "warm reset". I will continue running the test overnight and get back to you with the results along with the CTT dump.

Thank you for your help.
Regards

Santhosh

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

I'm unable to run the omapconf on my target board - are there any specific requirements for the root file system and/or the kernel that would allow omapconf to run properly. I am not even able to get the version information with --version.

Regards

Santhosh

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

I was able to run the omapconf on a unit that has wired ethernet, but I'm unable to the run the units without wired ethernet. I'm attaching the system log - this is the file /var/log/messages - that is the only difference that I can see, there are no network interfaces on the units that are under test. The wired ethernet is for bench testing and deployment only - do you think this is the reason why I'm not able to run omapconf on the other units?

Thank you

Regards

Santhosh

-filesystemfile/__key/communityserver-discussions-components-files/791/systemlog.txt?_=635981414280000000" class="code-download ui-tip" data-tip="Download systemlog.txt">systemlog.txt data-theme="light" style="height:922px;min-width:1230px;margin:0px;" data-gutter="true">Feb 3 04:52:13 HCC syslog.info syslogd started: BusyBox v1.22.1 kernel: klogd started: BusyBox v1.22.1 (2016-04-25 10:45:37 CDT) 0.000000] Booting Linux on physical CPU 0x0 0.000000] Linux version 3.12.10-ti2013.12.01 (sramani@sramani-Linux) (gcc version 4.7.3 20130226 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) ) #1 Thu Apr 14 12:42:36 CDT 2016 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache 0.000000] Machine: Generic AM33XX (Flattened Device Tree), model: Aprilaire Home Comfort Control 0.000000] cma: CMA: reserved 24 MiB at 86000000 0.000000] Memory policy: ECC disabled, Data cache writeback 0.000000] On node 0 totalpages: 32256 0.000000] free_area_init_node: node 0, pgdat c06716e0, node_mem_map c06a6000 0.000000] Normal zone: 256 pages used for memmap 0.000000] Normal zone: 0 pages reserved 0.000000] Normal zone: 32256 pages, LIFO batch:7 0.000000] CPU: All CPU(s) started in SVC mode. 0.000000] AM335X ES2.1 (neon ) 0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768 0.000000] pcpu-alloc: [0] 0 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32000 0.000000] Kernel command line: console=ttyO0,115200n8 root=ubi0:rootfs rw ubi.mtd=NAND.rootfs,2048 rootfstype=ubifs rootwait=1 noinitrd 0.000000] PID hash table entries: 512 (order: -1, 2048 bytes) 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) 0.000000] Memory: 96320K/129024K available (4147K kernel code, 417K rwdata, 1356K rodata, 640K init, 205K bss, 32704K reserved, 0K highmem) 0.000000] Virtual kernel memory layout: 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB) 0.000000] fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB) 0.000000] vmalloc : 0xc8800000 - 0xff000000 ( 872 MB) 0.000000] lowmem : 0xc0000000 - 0xc8000000 ( 128 MB) 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB) 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB) 0.000000] .text : 0xc0008000 - 0xc0568004 (5505 kB) 0.000000] .init : 0xc0569000 - 0xc060904c ( 641 kB) 0.000000] .data : 0xc060a000 - 0xc0672590 ( 418 kB) 0.000000] .bss : 0xc0672590 - 0xc06a5c80 ( 206 kB) 0.000000] NR_IRQS:16 nr_irqs:16 16 0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts 0.000000] Total of 128 interrupts on 1 active controller 0.000000] OMAP clockevent source: timer2 at 25000000 Hz 0.000000] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 171798ms 0.000000] OMAP clocksource: timer1 at 25000000 Hz 0.000000] Console: colour dummy device 80x30 0.000638] Calibrating delay loop... 199.06 BogoMIPS (lpj=995328) 0.069351] pid_max: default: 32768 minimum: 301 0.069551] Security Framework initialized 0.069656] Mount-cache hash table entries: 512 0.088257] CPU: Testing write buffer coherency: ok 0.088983] Setting up static identity map for 0xc0413028 - 0xc0413098 0.090850] devtmpfs: initialized 0.094790] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3 0.168943] omap_hwmod: debugss: _wait_target_disable failed 0.170279] pinctrl core: initialized pinctrl subsystem 0.172061] regulator-dummy: no parameters 0.176572] NET: Registered protocol family 16 0.181654] DMA: preallocated 256 KiB pool for atomic coherent allocations 0.204002] platform mpu.1: FIXME: clock-name 'fck' DOES NOT exist in dt! 0.206376] platform 49000000.edma: FIXME: clock-name 'fck' DOES NOT exist in dt! 0.208358] gpiochip_add: registered GPIOs 0 to 31 on device: gpio 0.209072] OMAP GPIO hardware version 0.1 0.210966] gpiochip_add: registered GPIOs 32 to 63 on device: gpio 0.213215] gpiochip_add: registered GPIOs 64 to 95 on device: gpio 0.215475] gpiochip_add: registered GPIOs 96 to 127 on device: gpio 0.229498] omap-gpmc 50000000.gpmc: could not find pctldev for node /pinmux@44e10800/nandflash_pins_default, deferring probe 0.229562] platform 50000000.gpmc: Driver omap-gpmc requests probe deferral 0.230476] platform 56000000.sgx: FIXME: clock-name 'fck' DOES NOT exist in dt! 0.234853] DSS not supported on this SoC 0.234891] hw-breakpoint: debug architecture 0x4 unsupported. 0.284214] bio: create slab <bio-0> at 0 0.317291] edma-dma-engine edma-dma-engine.0: TI EDMA DMA engine driver 0.322179] SCSI subsystem initialized 0.324116] usbcore: registered new interface driver usbfs 0.324551] usbcore: registered new interface driver hub 0.325044] usbcore: registered new device driver usb 0.326743] omap_i2c 44e0b000.i2c: could not find pctldev for node /pinmux@44e10800/pinmux_i2c0_pins, deferring probe 0.326799] platform 44e0b000.i2c: Driver omap_i2c requests probe deferral 0.326860] omap_i2c 4819c000.i2c: could not find pctldev for node /pinmux@44e10800/pinmux_i2c2_pins, deferring probe 0.326897] platform 4819c000.i2c: Driver omap_i2c requests probe deferral 0.327676] pps_core: LinuxPPS API ver. 1 registered 0.327702] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> 0.328053] PTP clock support registered 0.332040] Switched to clocksource timer1 0.375639] NET: Registered protocol family 2 0.377013] TCP established hash table entries: 1024 (order: 1, 8192 bytes) 0.377098] TCP bind hash table entries: 1024 (order: 0, 4096 bytes) 0.377146] TCP: Hash tables configured (established 1024 bind 1024) 0.377320] TCP: reno registered 0.377353] UDP hash table entries: 256 (order: 0, 4096 bytes) 0.377402] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes) 0.377859] NET: Registered protocol family 1 0.378630] RPC: Registered named UNIX socket transport module. 0.378659] RPC: Registered udp transport module. 0.378676] RPC: Registered tcp transport module. 0.378693] RPC: Registered tcp NFSv4.1 backchannel transport module. 0.380433] NetWinder Floating Point Emulator V0.97 (double precision) 0.381693] PM: Loading am335x-pm-firmware.bin 0.668705] VFS: Disk quotas dquot_6.5.2 0.668875] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) 0.669848] NFS: Registering the id_resolver key type 0.669997] Key type id_resolver registered 0.670019] Key type id_legacy registered 0.670142] msgmni has been set to 236 0.673402] NET: Registered protocol family 38 0.673489] io scheduler noop registered 0.673509] io scheduler deadline registered 0.673576] io scheduler cfq registered (default) 0.677609] pinctrl-single 44e10800.pinmux: 142 pins at pa f9e10800 size 568 0.753048] Console: switching to colour frame buffer device 100x30 0.780698] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled 0.785786] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/serial@44e09000[0]' 0.786400] 44e09000.serial: ttyO0 at MMIO 0x44e09000 (irq = 88, base_baud = 3000000) is a OMAP UART0 1.400334] console [ttyO0] enabled 1.405306] of_get_named_gpio_flags exited with status 104 1.406033] 48022000.serial: ttyO1 at MMIO 0x48022000 (irq = 89, base_baud = 3000000) is a OMAP UART1 1.417149] of_get_named_gpio_flags exited with status 6 1.417754] 481a8000.serial: ttyO4 at MMIO 0x481a8000 (irq = 61, base_baud = 3000000) is a OMAP UART4 1.430635] omap_rng 48310000.rng: OMAP Random Number Generator ver. 20 1.461235] brd: module loaded 1.476838] loop: module loaded 1.487027] mtdoops: mtd device (mtddev=name/number) must be supplied 1.498183] edma-dma-engine edma-dma-engine.0: allocated channel for 0:17 1.505567] edma-dma-engine edma-dma-engine.0: allocated channel for 0:16 1.516227] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver 1.523272] ehci-omap: OMAP-EHCI Host Controller driver 1.529890] usbcore: registered new interface driver usb-storage 1.540476] of_get_named_gpio_flags: can't parse gpios property of node '/ocp/usb@47400000/usb-phy@47401300[0]' 1.543003] musb-hdrc musb-hdrc.0.auto: Enabled SW babble control 1.549692] musb-hdrc musb-hdrc.0.auto: Falied to request rx1. 1.556155] musb-hdrc musb-hdrc.0.auto: musb_init_controller failed with status -517 1.564509] platform musb-hdrc.0.auto: Driver musb-hdrc requests probe deferral 1.605332] mousedev: PS/2 mouse device common for all mice 1.612730] i2c /dev entries driver 1.620227] omap_wdt: OMAP Watchdog Timer Rev 0x01: initial timeout 60 sec 1.628888] edma-dma-engine edma-dma-engine.0: allocated channel for 0:36 1.636399] omap-sham 53100000.sham: hw accel on OMAP rev 4.3 1.645163] omap-aes 53500000.aes: OMAP AES hw accel rev: 3.2 1.651345] edma-dma-engine edma-dma-engine.0: allocated channel for 0:5 1.658703] edma-dma-engine edma-dma-engine.0: allocated channel for 0:6 1.668788] usbcore: registered new interface driver usbhid 1.674807] usbhid: USB HID core driver 1.683338] oprofile: no performance counters 1.688853] oprofile: using timer interrupt. 1.694166] TCP: cubic registered 1.697700] Initializing XFRM netlink socket 1.702342] NET: Registered protocol family 17 1.707166] NET: Registered protocol family 15 1.712284] Key type dns_resolver registered 1.718989] ThumbEE CPU extension supported. 1.727627] omap-gpmc 50000000.gpmc: GPMC revision 6.0 1.733239] gpmc_mem_init: disabling cs 0 mapped at 0x0-0x1000000 1.741643] ONFI param page 0 valid 1.745464] ONFI flash detected 1.748841] NAND device: Manufacturer ID: 0x2c, Chip ID: 0xda (Micron MT29F2G08ABAEAH4), 256MiB, page size: 2048, OOB size: 64 1.760966] omap2-nand: detected x8 NAND flash 1.765721] nand: using OMAP_ECC_BCH8_CODE_HW ECC scheme 1.771415] omap2-nand: using custom ecc layout 1.776534] 8 ofpart partitions found on MTD device omap2-nand.0 1.782972] Creating 8 MTD partitions on "omap2-nand.0": 1.788634] 0x000000000000-0x000000020000 : "NAND.SPL" 1.797610] 0x000000020000-0x000000040000 : "NAND.SPL.backup1" 1.807129] 0x000000040000-0x000000060000 : "NAND.SPL.backup2" 1.816608] 0x000000060000-0x000000080000 : "NAND.SPL.backup3" 1.826064] 0x000000080000-0x000000100000 : "NAND.u-boot" 1.835188] 0x000000100000-0x000000140000 : "NAND.device-tree" 1.844606] 0x000000140000-0x000000600000 : "NAND.kernel" 1.857344] 0x000000600000-0x000010000000 : "NAND.rootfs" 2.087356] vdds_ddr: 1800 mV 2.093618] vdd_peripheral: 3300 mV 2.100321] vdd_mpu: 1100 mV 2.105626] LDO1: 1800 mV 2.110587] LDO2: 3300 mV 2.114692] LDO3: at 1800 mV 2.118892] LDO4: at 2800 mV 2.123337] tps65217 0-0024: TPS65217 ID 0x7 version 1.2 2.130194] of_get_named_gpio_flags exited with status 60 2.131711] input: TSC2007 Touchscreen as /devices/ocp.2/44e0b000.i2c/i2c-0/0-0048/input/input0 2.142693] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz 2.152645] sht21 2-0040: initialized 2.157565] rtc-pcf8563 2-0051: chip found, driver version 0.4.3 2.165485] rtc-pcf8563 2-0051: rtc core: registered rtc-pcf8563 as rtc0 2.172839] omap_i2c 4819c000.i2c: bus 2 rev0.11 at 400 kHz 2.179434] musb-hdrc musb-hdrc.0.auto: Enabled SW babble control 2.189689] musb-hdrc: ConfigData=0xde (UTMI-8, dyn FIFOs, bulk combine, bulk split, HB-ISO Rx, HB-ISO Tx, SoftConn) 2.189721] musb-hdrc: MHDRC RTL version 2.0 2.189743] musb-hdrc: setup fifo_mode 4 2.189786] musb-hdrc: 28/31 max ep, 16384/16384 memory 2.189986] musb-hdrc musb-hdrc.0.auto: MUSB HDRC host driver 2.197060] musb-hdrc musb-hdrc.0.auto: new USB bus registered, assigned bus number 1 2.205981] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 2.213273] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 2.220950] usb usb1: Product: MUSB HDRC host driver 2.226270] usb usb1: Manufacturer: Linux 3.12.10-ti2013.12.01 musb-hcd 2.233326] usb usb1: SerialNumber: musb-hdrc.0.auto 2.240660] hub 1-0:1.0: USB hub found 2.244925] hub 1-0:1.0: 1 port detected 2.251072] UBI: attaching mtd7 to ubi0 2.742324] usb 1-1: new high-speed USB device number 2 using musb-hdrc 2.913032] usb 1-1: New USB device found, idVendor=03f0, idProduct=5307 2.920172] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 2.927804] usb 1-1: Product: v165w 2.931519] usb 1-1: Manufacturer: HP 2.935448] usb 1-1: SerialNumber: AA4B0801C0030000039 2.974536] usb-storage 1-1:1.0: USB Mass Storage device detected 2.993171] scsi0 : usb-storage 1-1:1.0 3.609984] UBI: scanning is finished 3.630285] UBI: attached mtd7 (name "NAND.rootfs", size 250 MiB) to ubi0 3.637665] UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes 3.644905] UBI: min./max. I/O unit sizes: 2048/2048, sub-page size 512 3.651941] UBI: VID header offset: 2048 (aligned 2048), data offset: 4096 3.659267] UBI: good PEBs: 2000, bad PEBs: 0, corrupted PEBs: 0 3.665680] UBI: user volume: 1, internal volumes: 1, max. volumes count: 128 3.673287] UBI: max/mean erase counter: 2/1, WL threshold: 4096, image sequence number: 620953357 3.682830] UBI: available PEBs: 0, total reserved PEBs: 2000, PEBs reserved for bad PEB handling: 40 3.692667] UBI: background thread "ubi_bgt0d" started, PID 756 3.762132] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6 3.768634] davinci_mdio 4a101000.mdio: no live phy, scanning all 3.776002] davinci_mdio: probe of 4a101000.mdio failed with error -5 3.784398] Random MACID = 0a:9b:1c:6a:19:aa 3.807572] rtc-pcf8563 2-0051: setting system clock to 2010-02-03 04:52:12 UTC (1265172732) 3.842336] UBIFS: background thread "ubifs_bgt0_0" started, PID 765 3.868355] UBIFS: recovery needed 4.064168] UBIFS: recovery completed 4.068352] UBIFS: mounted UBI device 0, volume 0, name "rootfs" 4.074804] UBIFS: LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes 4.084564] UBIFS: FS size: 239222784 bytes (228 MiB, 1884 LEBs), journal size 9023488 bytes (8 MiB, 72 LEBs) 4.095118] UBIFS: reserved for root: 0 bytes (0 KiB) 4.100510] UBIFS: media format: w4/r0 (latest is w4/r0), UUID 44EF20E6-7242-436C-9041-A7AB459D30B7, small LPT model 4.136155] VFS: Mounted root (ubifs filesystem) on device 0:14. 4.144551] devtmpfs: mounted 4.192223] usb 1-1: reset high-speed USB device number 2 using musb-hdrc 4.726041] scsi 0:0:0:0: Direct-Access hp v165w 1100 PQ: 0 ANSI: 4 4.740928] sd 0:0:0:0: [sda] 31405824 512-byte logical blocks: (16.0 GB/14.9 GiB) 4.750360] sd 0:0:0:0: [sda] Write Protect is off 4.755537] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00 4.756192] sd 0:0:0:0: [sda] No Caching mode page found 4.761851] sd 0:0:0:0: [sda] Assuming drive cache: write through 4.772250] sd 0:0:0:0: [sda] No Caching mode page found 4.777923] sd 0:0:0:0: [sda] Assuming drive cache: write through 4.786001] sda: sda1 4.792778] sd 0:0:0:0: [sda] No Caching mode page found 4.798473] sd 0:0:0:0: [sda] Assuming drive cache: write through 4.805005] sd 0:0:0:0: [sda] Attached SCSI removable disk 4.813520] Freeing unused kernel memory: 640K (c0569000 - c0609000) 5.396826] udevd[791]: starting version 1.9 6.018537] PM: request_firmware failed 9.552692] net eth0: initializing cpsw version 1.12 (0) 9.560837] libphy: PHY 4a101000.mdio:01 not found 9.566002] net eth0: phy 4a101000.mdio:01 not found on slave 0 sshd[920]: Server listening on 0.0.0.0 port 22.

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

I would not expect your networking to have any impact whatsoever on omapconf. Check the md5sum for the working and non-working version.

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Brad,

Yeah I thought so, I performed a sha1sum on the omapconf file in both locations and they are identical. Here is the sha1sum result:

58e028a08df6d7d928930de9257d56007ac27b8a

Any other thoughts?

- Santhosh

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

I was finally able to get the omapconf for good and bad condition ( I had to rename the files to .txt since the post didn't allow rd1 files to be uploaded). I'm attaching the files here, I do see that there are few differences. Please let me know if you need any additional information.

I wanted to know if the Linux Kernel re-initializes the PLLs, I thought that u-boot initialized them and they are not modified by the kernel.

- Santhosh

good_1.txt

DeviceName AM335x1.0
0x44E00000 0x031A4502
0x44E00004 0x0000000A
0x44E00008 0x00000101
0x44E0000C 0x00000056
0x44E00010 0x00070000
0x44E00014 0x00000002
0x44E00018 0x00000002
0x44E0001C 0x00000002
0x44E00020 0x00070000
0x44E00024 0x00000002
0x44E00028 0x00000002
0x44E0002C 0x00000002
0x44E00030 0x00000002
0x44E00034 0x00030000
0x44E00038 0x00030000
0x44E0003C 0x00030000
0x44E00040 0x00000002
0x44E00044 0x00000002
0x44E00048 0x00030000
0x44E0004C 0x00000002
0x44E00050 0x00030000
0x44E00054 0x00030000
0x44E00058 0x00030000
0x44E00060 0x00000002
0x44E00064 0x00020002
0x44E00068 0x00030000
0x44E0006C 0x00000002
0x44E00070 0x00030000
0x44E00074 0x00030000
0x44E00078 0x00000002
0x44E0007C 0x00030000
0x44E00080 0x00000002
0x44E00084 0x00030000
0x44E00088 0x00030000
0x44E0008C 0x00030000
0x44E00090 0x00000002
0x44E00094 0x00030000
0x44E00098 0x00030000
0x44E0009C 0x00030000
0x44E000A0 0x00030000
0x44E000A4 0x00030000
0x44E000A8 0x00030000
0x44E000AC 0x00000002
0x44E000B0 0x00000002
0x44E000B4 0x00000002
0x44E000B8 0x00030000
0x44E000BC 0x00000002
0x44E000C0 0x00030000
0x44E000C4 0x00030000
0x44E000C8 0x00030000
0x44E000CC 0x00030000
0x44E000D0 0x00020002
0x44E000D4 0x00000002
0x44E000D8 0x00030000
0x44E000DC 0x00000002
0x44E000E0 0x00000002
0x44E000E4 0x00070000
0x44E000E8 0x00070000
0x44E000EC 0x00030000
0x44E000F0 0x00030000
0x44E000F4 0x00030000
0x44E000F8 0x00030000
0x44E000FC 0x00000002
0x44E00100 0x00000002
0x44E00104 0x00030000
0x44E0010C 0x00030000
0x44E00110 0x00030000
0x44E0011C 0x0000007A
0x44E00120 0x00000002
0x44E00124 0x00070000
0x44E00128 0x00030000
0x44E0012C 0x00000001
0x44E00130 0x00060002
0x44E00134 0x00030000
0x44E00138 0x00030000
0x44E0013C 0x00030000
0x44E00140 0x00000001
0x44E00144 0x00000012
0x44E00148 0x00000012
0x44E0014C 0x00020002
0x44E00150 0x00000001
0x44E00400 0x00007E16
0x44E00404 0x00000002
0x44E00408 0x00000002
0x44E0040C 0x00000002
0x44E00410 0x00000002
0x44E00414 0x12510000
0x44E00418 0x00000009
0x44E0041C 0x00000000
0x44E00420 0x00000001
0x44E00424 0x00000000
0x44E00428 0x00000000
0x44E0042C 0x00012C18
0x44E00430 0x00000000
0x44E00434 0x00000001
0x44E00438 0x00000000
0x44E0043C 0x00000000
0x44E00440 0x0000A618
0x44E00444 0x00000000
0x44E00448 0x00000001
0x44E0044C 0x00000000
0x44E00450 0x00000000
0x44E00454 0x00004218
0x44E00458 0x00000000
0x44E0045C 0x00000001
0x44E00460 0x00000000
0x44E00464 0x00000000
0x44E00468 0x0003E818
0x44E0046C 0x00000000
0x44E00470 0x00000001
0x44E00474 0x00000000
0x44E00478 0x00000000
0x44E0047C 0x00000300
0x44E00480 0x0000022A
0x44E00484 0x00000228
0x44E00488 0x00000007
0x44E0048C 0x00000007
0x44E00490 0x00000007
0x44E00494 0x00000007
0x44E00498 0x00000007
0x44E0049C 0x0403C018
0x44E004A0 0x00000201
0x44E004A4 0x00000201
0x44E004A8 0x00000201
0x44E004AC 0x00000285
0x44E004B0 0x00040002
0x44E004B4 0x00000002
0x44E004B8 0x00000002
0x44E004BC 0x00000002
0x44E004C0 0x00030000
0x44E004C4 0x00000002
0x44E004C8 0x00030000
0x44E004CC 0x00000006
0x44E004D0 0x00000002
0x44E004D4 0x00030000
0x44E00504 0x00000001
0x44E00508 0x00000001
0x44E0050C 0x00000001
0x44E00510 0x00000001
0x44E00514 0x00000004
0x44E00518 0x00000001
0x44E0051C 0x00000001
0x44E00520 0x00000000
0x44E00528 0x00000000
0x44E0052C 0x00000000
0x44E00530 0x00000000
0x44E00534 0x00000000
0x44E00538 0x00000001
0x44E0053C 0x00000000
0x44E00600 0x00000006
0x44E00604 0x00000002
0x44E00700 0x00000080
0x44E00800 0x00020002
0x44E00804 0x00000201
0x44E00900 0x00000001
0x44E00904 0x00070000
0x44E00908 0x00070000
0x44E0090C 0x00000001
0x44E00910 0x00030000
0x44E00914 0x00030000
0x44E00A00 0x00000001
0x44E00A20 0x00030000
0x44E00B00 0x00000000
0x44E00B04 0x00000500
0x44E00B08 0x00000000
0x44E00B0C 0x00000100
0x44E00B10 0x00000000
0x44E00C00 0x00000003
0x44E00C04 0x00000000
0x44E00C08 0x01E60007
0x44E00C0C 0xEE0000EB
0x44E00D00 0x00000008
0x44E00D04 0x00000008
0x44E00D08 0x00000000
0x44E00D0C 0x00000000
0x44E00E00 0x01FF0007
0x44E00E04 0x000003F7
0x44E00E08 0x00000000
0x44E00F00 0x00000000
0x44E00F04 0x00001006
0x44E00F08 0x00000001
0x44E00F0C 0x78000017
0x44E00F10 0x00000003
0x44E00F14 0x00000000
0x44E00F18 0x00000003
0x44E00F1C 0x00000000
0x44E01000 0x00000004
0x44E01004 0x00000000
0x44E01100 0x00060047
0x44E01104 0x00000001
0x44E01110 0x00000037
0x44E01114 0x00000000
0x44E01200 0x00000000
0x44E01204 0x00000000
0x44E10040 0x00800313

3678.bad_1.txt

DeviceName AM335x1.0
0x44E00000 0x031A4502
0x44E00004 0x0000000A
0x44E00008 0x00000101
0x44E0000C 0x00000016
0x44E00010 0x00070000
0x44E00014 0x00070000
0x44E00018 0x00000002
0x44E0001C 0x00000002
0x44E00020 0x00070000
0x44E00024 0x00000002
0x44E00028 0x00000002
0x44E0002C 0x00000002
0x44E00030 0x00000002
0x44E00034 0x00030000
0x44E00038 0x00030000
0x44E0003C 0x00030000
0x44E00040 0x00000002
0x44E00044 0x00000002
0x44E00048 0x00030000
0x44E0004C 0x00000002
0x44E00050 0x00030000
0x44E00054 0x00030000
0x44E00058 0x00030000
0x44E00060 0x00000002
0x44E00064 0x00020002
0x44E00068 0x00030000
0x44E0006C 0x00000002
0x44E00070 0x00030000
0x44E00074 0x00030000
0x44E00078 0x00000002
0x44E0007C 0x00030000
0x44E00080 0x00000002
0x44E00084 0x00030000
0x44E00088 0x00030000
0x44E0008C 0x00030000
0x44E00090 0x00000002
0x44E00094 0x00030000
0x44E00098 0x00030000
0x44E0009C 0x00030000
0x44E000A0 0x00030000
0x44E000A4 0x00030000
0x44E000A8 0x00030000
0x44E000AC 0x00000002
0x44E000B0 0x00000002
0x44E000B4 0x00000002
0x44E000B8 0x00030000
0x44E000BC 0x00000002
0x44E000C0 0x00030000
0x44E000C4 0x00030000
0x44E000C8 0x00030000
0x44E000CC 0x00030000
0x44E000D0 0x00020002
0x44E000D4 0x00000002
0x44E000D8 0x00030000
0x44E000DC 0x00000002
0x44E000E0 0x00000002
0x44E000E4 0x00070000
0x44E000E8 0x00070000
0x44E000EC 0x00030000
0x44E000F0 0x00030000
0x44E000F4 0x00030000
0x44E000F8 0x00030000
0x44E000FC 0x00000002
0x44E00100 0x00000002
0x44E00104 0x00030000
0x44E0010C 0x00030000
0x44E00110 0x00030000
0x44E0011C 0x0000000A
0x44E00120 0x00000002
0x44E00124 0x00070000
0x44E00128 0x00030000
0x44E0012C 0x00000001
0x44E00130 0x00060002
0x44E00134 0x00030000
0x44E00138 0x00030000
0x44E0013C 0x00030000
0x44E00140 0x00000001
0x44E00144 0x00000001
0x44E00148 0x00000012
0x44E0014C 0x00020002
0x44E00150 0x00000001
0x44E00400 0x00007E16
0x44E00404 0x00000002
0x44E00408 0x00000002
0x44E0040C 0x00000002
0x44E00410 0x00000002
0x44E00414 0x12510000
0x44E00418 0x00000009
0x44E0041C 0x00000000
0x44E00420 0x00000001
0x44E00424 0x00000000
0x44E00428 0x00000000
0x44E0042C 0x00012C18
0x44E00430 0x00000000
0x44E00434 0x00000001
0x44E00438 0x00000000
0x44E0043C 0x00000000
0x44E00440 0x0000A018
0x44E00444 0x00000000
0x44E00448 0x00000001
0x44E0044C 0x00000000
0x44E00450 0x00000000
0x44E00454 0x00004218
0x44E00458 0x00000000
0x44E0045C 0x00000001
0x44E00460 0x00000000
0x44E00464 0x00000000
0x44E00468 0x0003E818
0x44E0046C 0x00000000
0x44E00470 0x00000001
0x44E00474 0x00000000
0x44E00478 0x00000000
0x44E0047C 0x00000300
0x44E00480 0x0000022A
0x44E00484 0x00000028
0x44E00488 0x00000007
0x44E0048C 0x00000007
0x44E00490 0x00000007
0x44E00494 0x00000007
0x44E00498 0x00000007
0x44E0049C 0x0403C018
0x44E004A0 0x00000201
0x44E004A4 0x00000201
0x44E004A8 0x00000201
0x44E004AC 0x00000285
0x44E004B0 0x00040002
0x44E004B4 0x00000002
0x44E004B8 0x00000002
0x44E004BC 0x00000002
0x44E004C0 0x00030000
0x44E004C4 0x00000002
0x44E004C8 0x00030000
0x44E004CC 0x00000006
0x44E004D0 0x00000002
0x44E004D4 0x00030000
0x44E00504 0x00000001
0x44E00508 0x00000001
0x44E0050C 0x00000001
0x44E00510 0x00000001
0x44E00514 0x00000004
0x44E00518 0x00000001
0x44E0051C 0x00000001
0x44E00520 0x00000000
0x44E00528 0x00000000
0x44E0052C 0x00000000
0x44E00530 0x00000000
0x44E00534 0x00000000
0x44E00538 0x00000001
0x44E0053C 0x00000000
0x44E00600 0x00000006
0x44E00604 0x00000002
0x44E00700 0x00000080
0x44E00800 0x00020002
0x44E00804 0x00000201
0x44E00900 0x00000001
0x44E00904 0x00070000
0x44E00908 0x00070000
0x44E0090C 0x00000001
0x44E00910 0x00030000
0x44E00914 0x00030000
0x44E00A00 0x00000001
0x44E00A20 0x00030000
0x44E00B00 0x00000000
0x44E00B04 0x00008500
0x44E00B08 0x00000000
0x44E00B0C 0x00008100
0x44E00B10 0x00000000
0x44E00C00 0x00000003
0x44E00C04 0x00000000
0x44E00C08 0x01E60007
0x44E00C0C 0xEE0000EB
0x44E00D00 0x00000008
0x44E00D04 0x00000008
0x44E00D08 0x00000000
0x44E00D0C 0x00000000
0x44E00E00 0x01FF0007
0x44E00E04 0x000003F7
0x44E00E08 0x00000000
0x44E00F00 0x00000000
0x44E00F04 0x00001006
0x44E00F08 0x00000001
0x44E00F0C 0x78000017
0x44E00F10 0x00000003
0x44E00F14 0x00000000
0x44E00F18 0x00000003
0x44E00F1C 0x00000000
0x44E01000 0x00000004
0x44E01004 0x00000000
0x44E01100 0x00060047
0x44E01104 0x00000001
0x44E01110 0x00000037
0x44E01114 0x00000000
0x44E01200 0x00000000
0x44E01204 0x00000000
0x44E10040 0x00800313

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

Here are some addition files of both good and bad cases, I'm hoping these shed some light into the situation. I have 4 units that are running the same software, same RFS, same kernel and uboot. They have been power cycled and on each boot up I run the omapconf so that I can get the good and bad files. The bad ones are created when the baud rate is wrong (indicated by the PLL issue) and the good ones are when there are no problems.

- Santhosh

omapconf-output.zip

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

I did some quick diffs of various combinations of good-good, bad-bad, bad-good and the failures look to be extremely consistent.

PRM_IRQSTATUS_MPU (address 0x44E00B04)
• DPLL_PER_RECAL_ST (Bit 15) set on bad unit

I've done some quick checking of your DPLL_PER configuration and nothing jumped out at me as having an issue. This could potentially indicate a hardware issue on your board. I'm going to consult some of my colleagues to see what further suggestions they might have.

FYI, I imagine you can adjust your workaround to simply check the bit I mentioned instead of having to do anything with the UART. That is slightly better than what you're doing now, but better yet I'd like to see the issue avoid altogether.

0 -DK- over 7 years ago in reply to Brad Griffis

TI__Mastermind 27720 points

Please post the PER_PLL M/N/M2/SD settings currently in use.

0 Brad Griffis over 7 years ago in reply to Brad Griffis

TI__Guru*** 125430 points

What PMIC are you using, which output is powering VDDA1P8V_USB0, and what else is on that same rail?

0 Brad Griffis over 7 years ago in reply to -DK-

TI__Guru*** 125430 points

-DK- said:
Please post the PER_PLL M/N/M2/SD settings currently in use.

I've extracted this info from his clock dumps using CTT:

M=960

N=24 (/25), with 25 MHz external clock input

SD_DIV=4

M2=5

This results in CLKOUT = 960 MHz and DPLL_PER_M2 = 192 MHz.

0 Brad Griffis over 7 years ago in reply to Brad Griffis

TI__Guru*** 125430 points

Did you ever try Paul's suggestion of removing your 25 MHz crystal and feeding a 1.8V 25 MHz square wave? That would help us understand if the issue pertains to your clocking.

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Brad,

Thanks for your replies. Here are answers to your questions:

Brad Griffis said:
Did you ever try Paul's suggestion of removing your 25 MHz crystal and feeding a 1.8V 25 MHz square wave? That would help us understand if the issue pertains to your clocking.

No not yet, I was busy with the software work-around as it will allow me to continue without a hardware modification. Additionally, I would like to add that I have tried to detect this problem in u-boot with the same method that I mentioned and I have not been able to detect even an single occurrence of this problem. Which leads me to believe that the problem occurs when

a. The current consumption of the unit increases and there is an increase of switching signals (DDR, LCD, etc.) when the OS boots.

b. The OS must re-initialize the PLLs. My understanding is that once a PLL is locked it shouldn't change but if it's reinitilaized while the OS boots and there is increase in noise due to increased power draw there is a possibility of this problem happening during boot (OS).

Brad Griffis said:
I've done some quick checking of your DPLL_PER configuration and nothing jumped out at me as having an issue. This could potentially indicate a hardware issue on your board. I'm going to consult some of my colleagues to see what further suggestions they might have.

I think this is almost guaranteed - I think there is a hardware problem as well, but I don't know where to start looking. I will perform the test that Paul is suggesting and get back to you with the results.

Brad Griffis said:
FYI, I imagine you can adjust your workaround to simply check the bit I mentioned instead of having to do anything with the UART. That is slightly better than what you're doing now, but better yet I'd like to see the issue avoid altogether.

Agreed. But this is still not the best work-around since I lose a lot of time booting but it's much better than what I have currently in place.

Thank you once again for your replies and patience. I will perform the test Paul is suggesting - do you have a part in mind that I can use so that I don't spend a lot of time searching and use the part that you are familiar with so that the results are consistent?

Regards

Santhosh

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

Can you update the u-boot DPLL PER configuration to use these values:

M=384
N=9
M2=5
SD = 4

There are actually two things I want to figure out based on this one test:

1. This configuration is optimized for faster lock time and tracking (i.e. since the reference clock going into the DPLL is faster due to a smaller pre-divider). If there's an issue with the DPLL losing lock because it can't track fast enough, this will help.

2. You can check the DPLL registers in the kernel. If they've maintained these values (which I *think* they will) that will prove the kernel is not reconfiguring the DPLL.

Please send an updated rd1 file after you've made the test. That will allow me to validate that you've correctly made the change. You can then do some additional testing to see if it improves the behavior.

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Hello Brad,

I was able to find the file where I can modify the values you have suggested. However, I don't know what is SD? Based on the structure defined in clock.h I was only able to make modifications based on the following:

struct dpll_params {
	u32 m;
	u32 n;
	s8 m2;
	s8 m3;
	s8 m4;
	s8 m5;
	s8 m6;
};

Can you please let me know when I can find the parameter SD?

- Santhosh

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

Okay, I was able to figure it out. In the file clock.h, there are definitions for CLKSEL DPLL and I was able to add the mask and shifter for SD. I added the following:

#define CM_CLKSEL_DPLL_SD_SHIFT			24
#define CM_CLKSEL_DPLL_SD_MASK			(0x7F << 24)

These definitions were already there:

#define CM_CLKSEL_DPLL_M_SHIFT			8
#define CM_CLKSEL_DPLL_M_MASK			(0x7FF << 8)
#define CM_CLKSEL_DPLL_N_SHIFT			0
#define CM_CLKSEL_DPLL_N_MASK			0x7F

Based on this I made some modifications to clock.c in the function do_setup_dpll

/* Set SD */
temp &= ~CM_CLKSEL_DPLL_SD_MASK;
temp |= (4 << CM_CLKSEL_DPLL_SD_SHIFT) & CM_CLKSEL_DPLL_SD_MASK;

Please do let me know if these changes are correct.

- Santhosh

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

Careful with your changes. DPLL_PER is the only PLL that has this SD field. If you're adding that into code that runs for all DPLL's that may be an issue. Your original register dump already showed SD_DIV = 4, so I don't know you need to do anything along those lines. Just change the multiplier and dividers and then grab a new ctt capture from Linux.

0 Santhosh Ramani over 7 years ago in reply to Brad Griffis

Expert 2765 points

Okay, understood. I will leave the SD section alone and just update the M, N, and M2. I will post the results in 2-3 hours.

0 Santhosh Ramani over 7 years ago in reply to Santhosh Ramani

Expert 2765 points

Hello Brad,

Here are the results from the changes - it seems the PLL issue now happens lot more often.

- Santhosh

conf-output.zip

0 Brad Griffis over 7 years ago in reply to Santhosh Ramani

TI__Guru*** 125430 points

I see the values "stuck" so that confirms the DPLL is not being reconfigured by Linux. That said, it seems like you have some kind of noise issue on your board. It could be at the main clock, or it might be noise being coupled into VDDA1P8V_USB0. Do you have a ferrite bead on VDDA1P8V_USB0? That would be helpful in isolating from some of the board noise.

One other thought that comes to mind is that there might be something on your board that when you enable it you get a big in-rush of current (e.g. maybe a display, a connectivity device, a power rail, etc.). You may want to do some instrumentation of the u-boot code to print out PRM_IRQSTATUS_MPU at various points during the boot process. You might discover that the bit always gets set at the same time, which may correlate to something else in your system, i.e. that may help you understand the root of the issue.

Processors

Processors forum

Peripheral PLL issue with AM335x