PROCESSOR-SDK-AM335X: Kernel boot stops with "Waiting for root device PARTUUID=..."

In the pinmux section I also have 

I have no PMIC on the board would it be related to this fact?

Since there is no PMIC could it really be something different stopping the kernel..... 

Here a full console output:

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 5.10.100-g7a7a3af903 (tm@WSTM6) (arm-none-linux-gnueabihf-gcc (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 9.2.1 20191025, GNU ld (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209) #12 PREEMPT Fri Sep 9 11:04:19 CEST 2022
[ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Prolon PID6000
[ 0.000000] Memory policy: Data cache writeback
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 48 MiB at 0x9c800000
[ 0.000000] Zone ranges:
[ 0.000000] Normal [mem 0x0000000080000000-0x000000009fefffff]
[ 0.000000] HighMem empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000080000000-0x000000009fefffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000009fefffff]
[ 0.000000] CPU: All CPU(s) started in SVC mode.
[ 0.000000] AM335X ES1.0 (sgx neon)
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 129666
[ 0.000000] Kernel command line: console=ttyS0,115200n8 root=PARTUUID=f1c7fdfc-02 rw rootfstype=ext4 rootwait
[ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] Memory: 452392K/523264K available (10240K kernel code, 695K rwdata, 3400K rodata, 1024K init, 280K bss, 21720K reserved, 49152K cma-reserved, 0K highmem)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] rcu: RCU event tracing is enabled.
[ 0.000000] Trampoline variant of Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[ 0.000000] IRQ: Found an INTC at 0x(ptrval) (revision 5.0) with 128 interrupts
[ 0.000000] random: get_random_bytes called from start_kernel+0x30c/0x4c8 with crng_init=0
[ 0.000000] TI gptimer clocksource: always-on /ocp/interconnect@44c00000/segment@200000/target-module@31000
[ 0.000012] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
[ 0.000034] clocksource: dmtimer: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
[ 0.000427] TI gptimer clockevent: 24000000 Hz at /ocp/interconnect@48000000/segment@0/target-module@40000
[ 0.001575] Console: colour dummy device 80x30
[ 0.001671] Calibrating delay loop... 718.02 BogoMIPS (lpj=3590144)
[ 0.120503] pid_max: default: 32768 minimum: 301
[ 0.120697] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[ 0.120717] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[ 0.121718] CPU: Testing write buffer coherency: ok
[ 0.121805] CPU0: Spectre v2: using BPIALL workaround
[ 0.122774] Setting up static identity map for 0x80100000 - 0x80100060
[ 0.122949] rcu: Hierarchical SRCU implementation.
[ 0.123061] EFI services will not be available.
[ 0.123490] devtmpfs: initialized
[ 0.135481] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
[ 0.135787] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.135817] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[ 0.139598] pinctrl core: initialized pinctrl subsystem
[ 0.140828] DMI not present or invalid.
[ 0.141373] NET: Registered protocol family 16
[ 0.143972] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.162897] l3-aon-clkctrl:0000:0: failed to disable
[ 0.164379] thermal_sys: Registered thermal governor 'fair_share'
[ 0.164393] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.164412] thermal_sys: Registered thermal governor 'step_wise'
[ 0.164421] thermal_sys: Registered thermal governor 'user_space'
[ 0.164782] cpuidle: using governor ladder
[ 0.164825] cpuidle: using governor menu
[ 1.345485] No ATAGs?
[ 1.345519] hw-breakpoint: debug architecture 0x4 unsupported.
[ 1.373425] iommu: Default domain type: Translated
[ 1.375832] SCSI subsystem initialized
[ 1.376354] mc: Linux media interface: v0.10
[ 1.376412] videodev: Linux video capture interface: v2.00
[ 1.376536] pps_core: LinuxPPS API ver. 1 registered
[ 1.376550] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 1.376578] PTP clock support registered
[ 1.376653] EDAC MC: Ver: 3.0.0
[ 1.378111] Advanced Linux Sound Architecture Driver Initialized.
[ 1.379676] clocksource: Switched to clocksource dmtimer
[ 1.389009] NET: Registered protocol family 2
[ 1.389251] IP idents hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 1.390811] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
[ 1.390860] TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
[ 1.390917] TCP bind hash table entries: 4096 (order: 2, 16384 bytes, linear)
[ 1.390970] TCP: Hash tables configured (established 4096 bind 4096)
[ 1.391100] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 1.391127] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 1.391277] NET: Registered protocol family 1
[ 1.391933] RPC: Registered named UNIX socket transport module.
[ 1.391953] RPC: Registered udp transport module.
[ 1.391963] RPC: Registered tcp transport module.
[ 1.391972] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 1.391993] PCI: CLS 0 bytes, default 64
[ 1.393079] hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
[ 1.394498] Initialise system trusted keyrings
[ 1.394873] workingset: timestamp_bits=14 max_order=17 bucket_order=3
[ 1.399956] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 1.400853] NFS: Registering the id_resolver key type
[ 1.400910] Key type id_resolver registered
[ 1.400922] Key type id_legacy registered
[ 1.401040] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 1.401055] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[ 1.401103] ntfs: driver 2.1.32 [Flags: R/O].
[ 1.401798] Key type asymmetric registered
[ 1.401820] Asymmetric key parser 'x509' registered
[ 1.401879] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[ 1.401893] io scheduler mq-deadline registered
[ 1.401904] io scheduler kyber registered
[ 1.805493] omap-mailbox 480c8000.mailbox: omap mailbox rev 0x400
[ 1.890031] debugfs: Directory '49000000.dma' with parent 'dmaengine' already present!
[ 1.890085] edma 49000000.dma: TI EDMA DMA engine driver
[ 1.904994] pinctrl-single 44e10800.pinmux: 142 pins, size 568
[ 1.969853] Serial: 8250/16550 driver, 10 ports, IRQ sharing enabled
[ 1.975175] printk: console [ttyS0] disabled
[ 1.975410] 44e09000.serial: ttyS0 at MMIO 0x44e09000 (irq = 20, base_baud = 3000000) is a 8250
[ 2.503383] random: fast init done
[ 2.637187] printk: console [ttyS0] enabled
[ 2.643656] 48022000.serial: ttyS1 at MMIO 0x48022000 (irq = 24, base_baud = 3000000) is a 8250
[ 2.654576] 48024000.serial: ttyS2 at MMIO 0x48024000 (irq = 25, base_baud = 3000000) is a 8250
[ 2.665583] 481a8000.serial: ttyS4 at MMIO 0x481a8000 (irq = 36, base_baud = 3000000) is a 8250
[ 2.676370] 481aa000.serial: ttyS5 at MMIO 0x481aa000 (irq = 37, base_baud = 3000000) is a 8250
[ 2.688613] omap_rng 48310000.rng: Random Number Generator ver. 20
[ 2.695228] random: crng init done
[ 2.721371] brd: module loaded
[ 2.732647] loop: module loaded
[ 2.746317] i2c /dev entries driver
[ 2.753727] cpuidle: enable-method property 'ti,am3352' found operations
[ 2.761356] sdhci: Secure Digital Host Controller Interface driver
[ 2.767575] sdhci: Copyright(c) Pierre Ossman
[ 2.773400] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.782091] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.793876] NET: Registered protocol family 10
[ 2.799609] Segment Routing with IPv6
[ 2.803567] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 2.810440] NET: Registered protocol family 17
[ 2.815336] Key type dns_resolver registered
[ 2.820386] omap_voltage_late_init: Voltage driver support not added
[ 2.827503] Loading compiled-in X.509 certificates
[ 2.847068] OMAP GPIO hardware version 0.1
[ 2.853087] omap_i2c 44e0b000.i2c: bus 0 rev0.11 at 400 kHz
[ 2.860450] omap_i2c 4802a000.i2c: bus 1 rev0.11 at 100 kHz
[ 2.872248] omap_gpio 44e07000.gpio: Could not set line 6 debounce to 200000 microseconds (-22)
[ 2.882853] ALSA device list:
[ 2.885861] No soundcards found.
[ 2.890709] sdhci-omap 48060000.mmc: Got CD GPIO
[ 2.895707] sdhci-omap 48060000.mmc: supply vqmmc not found, using dummy regulator
[ 2.956666] mmc0: SDHCI controller on 48060000.mmc [48060000.mmc] using External DMA
[ 2.966928] Waiting for root device PARTUUID=f1c7fdfc-02...

Hi Thomas,

Our Linux MMC expert is out of office, but I have seen many similar boot issues which were likely related to SD card detect. I see the device tree mmc1 note uses gpio0_6 for SD card detect, but is the gpio0[5] pinmux defined in your board DTS?

Thomas Maltesen said:

  AM33XX_IOPAD(0x95c, PIN_INPUT | MUX_MODE7) /* (A16) spi0_cs0.gpio0[5] */

Is this suppose to be gpio0_6?

This pinmux was irrelevant. Sorry for that, it is from an ordinary gpio declaration and is not related to MMC.

     AM33XX_IOPAD(0x95c, PIN_INPUT | MUX_MODE7) /* (A16) spi0_cs0.gpio0[5] */

The card detect declaration is here as part of the mmc pin declaration:

    AM33XX_IOPAD(0x960, PIN_INPUT | MUX_MODE5) /* (C15) spi0_cs1.mmc0_sdcd */

Hi Thomas,

Thomas Maltesen said:

The card detect declaration is here as part of the mmc pin declaration:

    AM33XX_IOPAD(0x960, PIN_INPUT | MUX_MODE5) /* (C15) spi0_cs1.mmc0_sdcd */

I am not sure this will work in card detect - as you defined the pinmux to use the mmc0_sdcd pin, but the &mmc1 DT node uses "cd-gpios" property to tell to use GPIO0_6 for card detect. Can you please add "broken-cd;" property in &mmc1 node to see if it solves the issue? I think "broken-cd;" to tell the driver to bypass card detect.

Also your DT shows "vmmcsd_fixed" regulator only provides 3.3v, no 1.8v. If so, I think you need to add "no-1-8-v;" property in &mmc1 node to tell the driver not to switch to 1.8v mode.

Thanks Bin Liu - Any help is greatly appreciated as I am getting a little desperate ;-)

I think I am misunderstanding or confusing this about GPIO0_6 and the Card Detect. I am now not sure what is the correct way do to it.

Card detect is attached to the C15 pin as an active low signal.

What is the correct way to declare this in the devicetree.

This pin can be declared as GPIO6_0 in pinmux, but could also be set as above in mode 5 as mmc sdcd pin.

And then how should this correctly be set as part of &mmc1 declaration....

I have tried boken-cd but without any change, I will retry. I am unsure about this as I have a card detect signal attached on pin C15....

Also I will check about 1.8 V.

Can you provide a little info on the 1.8v issue you are mentioning.

SD Card has am 3V3 supply and nothing else. Where does 1.8V come into the picture?

Am I misuinderstanding the meaning of the fixed regulator...?

Hi Thomas,

MMC is not my expertise, so I only know a little bit of it. So bare with me...

I know the MMC controller has a dedicated sdcd pin, but I see the AM335x GPEVM uses gpio0_6 pin to connect to the card detect of the sdcard slot and the MMC driver uses this gpio pin for card detect. So I am trying to understand how software can use the dedicated sdcd pin, I am trying to contact our MMC HW expert but haven't got response yet.

Meanwhile, what is your board design related to card detect? which AM335x pin is connected to the card detect of the sdcard slot?

Based on my limited knowledge of MMC Spec, MMC can work in either 3.3v, or 1.8v (for high speed class), so the MMC power supply can provides 3.3v and 1.8v, but I never looked to understand how the power supply is implemented (I am a SW guy...). But I know 'no-1-8-v;' DT property can prevent MMC to switch to 1.8v, which in turn doesn't switch to high speed class mode.

Thomas Maltesen said:

Card detect is attached to the C15 pin as an active low signal.

I missed this message. Looks like your board uses the mmc sdcd pin. Let me look to understand how SW works with this pin. But since you don't use gpio for card detect, I think you should remove the "cd-gpios" from your mmc1 DT node.

Hi Thomas,

Bin Liu said:

But since you don't use gpio for card detect, I think you should remove the "cd-gpios" from your mmc1 DT node.

Please definitely test this, removing the 'cd-gpios' from the DT node. Please see the following e2e thread:

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/579502/linux-am3358-understanding-mmc0-card-detect/2135086#2135086

Still no success.

I have tried the following two devicetree versions:

I have now also tried the following:

I was not 100% sure on naming 0 vs 1 counting in the ti,hwmods. So I also tried:

Same result!

So I removed the "ti,hwmods" and now I'm back to normal problem stopping with "Waiting for root device ..."

But I spotted another line, not sure if this is related:

    2.525156] omap_voltage_late_init: Voltage driver support not added

This seems unrelated to the mmc devicetreee as event the most simple declaration also has this output.

&mmc1 {

Hi Bin Liu

Any chance getting some information from the MMC expert you are mentioning. I am really stuck here?

I have added kernel debug to see if this could help:

In the following I have removed other unrelated traces, but something strange is happing that I do not understand why. After initialization seems to be disabled...

Why should it be disabled....

Thomas Maltesen said:

            AM33XX_IOPAD(0x960, PIN_INPUT | MUX_MODE5) /* (C15) spi0_cs1.mmc0_sdcd */

Can you please try to change this sdcd pinmux from MODE5 to MODE7, which is GPIO0_6. Then keep the following line in your &mmc1 DT node:

cd-gpios = <&gpio0 6 GPIO_ACTIVE_LOW>;

Now this "can't parse 'cd-gpio' property" message should go away. And let me know this solves the issue.

I have tried this method, as I think I found it was the way it was declared in Beagle Bone Black. But it does not help, same issue. The "can't parse..." dissapears and another trace about gpio0_6 unable to set (debounce as I remember). This trace is also there on our older board with PMIC, so I ignore it.

It think it has to do with the fact that there is no PMIC on the board and that it is removed from the devicetree.

I have tried different declaration explaining fixed power in the devicetree but not found anything that helps. Maybe there is a bug somewhere if there is no PMIC declared.....

I have been searching for the code that disables the PMIC after it is initialized, It must be some upper layer. I am afraid it looks for some power information and if not found it disables the mmc. Anyway just guessing and searching.

Hi Thomas,

Can you please rerun the test and capture the debounce log? The kernel driver needs this GPIO interrupt to know the SD card is present, but I don't this Card Detect pin has anything to do with PMIC.

Also when the kernel boot failed, have you probed the MMC interface to know if the 3.3v VDD does present and the pin C15 (mmc0_sdcd) is low?

For the following devicetree:

... pinmux section

I get the log (I have added the two lines to the omap_hsmmc driver):

[ 2.470692] sdhci: Secure Digital Host Controller Interface driver
[ 2.476998] sdhci: Copyright(c) Pierre Ossman
[ 2.482932] omap_hsmmc: omap_hsmmc_probe matching device...
[ 2.490276] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.497957] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.509773] NET: Registered protocol family 10
[ 2.515666] Segment Routing with IPv6
[ 2.519483] omap_hsmmc: omap_hsmmc_probe returns success (0)
[ 2.520486] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 2.533428] NET: Registered protocol family 17
[ 2.538393] Key type dns_resolver registered
[ 2.543947] omap_voltage_late_init: Voltage driver support not added
[ 2.551186] Loading compiled-in X.509 certificates
[ 2.570457] OMAP GPIO hardware version 0.1
[ 2.580946] ALSA device list:
[ 2.584321] No soundcards found.
[ 2.588697] Waiting for root device PARTUUID=ae4aef5c-02...

For the following devicetree:

... pinmux section

I get the log (I have added the four lines to the omap_hsmmc driver):

[ 2.475074] sdhci: Secure Digital Host Controller Interface driver
[ 2.481294] sdhci: Copyright(c) Pierre Ossman
[ 2.487211] omap_hsmmc: omap_hsmmc_probe matching device...
[ 2.505102] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.512632] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.518911] omap_hsmmc: omap_hsmmc_probe returns error (-517)
[ 2.524467] NET: Registered protocol family 10
[ 2.536194] Segment Routing with IPv6
[ 2.539986] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 2.546846] NET: Registered protocol family 17
[ 2.551708] Key type dns_resolver registered
[ 2.556685] omap_voltage_late_init: Voltage driver support not added
[ 2.563789] Loading compiled-in X.509 certificates
[ 2.583083] OMAP GPIO hardware version 0.1
[ 2.592772] omap_hsmmc: omap_hsmmc_probe matching device...
[ 2.600542] ALSA device list:
[ 2.603550] No soundcards found.
[ 2.607504] omap_gpio 44e07000.gpio: Could not set line 6 debounce to 200000 microseconds (-22)
[ 2.617149] omap_hsmmc 48060000.mmc: Got CD GPIO
[ 2.648953] omap_hsmmc: omap_hsmmc_probe returns success (0)
[ 2.650796] Waiting for root device PARTUUID=ae4aef5c-02...

Another observation that confuses me.

If i remove the devicetree node:

compatible = "ti,am33xx-hsmmc";

Something happens, it seems it does not call the omap_hsmmc module probe functions...It does still not work, but is there something here?

[ 2.470511] sdhci: Secure Digital Host Controller Interface driver
[ 2.476818] sdhci: Copyright(c) Pierre Ossman
[ 2.482592] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.502740] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.524292] NET: Registered protocol family 10
[ 2.529990] Segment Routing with IPv6
[ 2.539806] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 2.547295] NET: Registered protocol family 17
[ 2.552333] Key type dns_resolver registered
[ 2.557247] omap_voltage_late_init: Voltage driver support not added
[ 2.564702] Loading compiled-in X.509 certificates
[ 2.576625] mmc0: SDHCI controller on 48060000.mmc [48060000.mmc] using External DMA
[ 2.594999] OMAP GPIO hardware version 0.1
[ 2.605673] ALSA device list:
[ 2.608685] No soundcards found.
[ 2.613444] Waiting for root device PARTUUID=ae4aef5c-02...