Hello,
We designed a board with an AM5728 (Linux). Connected to its PCIe interface we have two Marvell 9170 (PCIe to 2xSATA converter), so that 4 drives can be connected to our board.
We are quite happy with the performance of the board. However we have a serious issue that prevents our product to go to the market.
The problem we have is related to the identification of the drives at start time:
o If the board was previously shut down by software (from O.S: reboot, poweroff ….) the next power on is successful (all drives are detected and reads/writes to disks get no errors. Always)
o If the board was unexpectedly shut down (unplugging the power wire) there is a high possibility that in the next power on some drives (or even all of them) are not detected or generate errors at start time so that O.S. disables them.
We want to know what could cause this behavior. It seems a state is stored somewhere and depending on how it was previously closed, PICe powers on one way or another.
Do you have any hint about what could be causing this issue?
"lspci -vv" and "dmesg | grep pci" when all disks (4 out of 4) are correctly identified at start time:
root@nepARIAv4:~# lspci -vv
0000:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 177
Region 0: Memory at 20100000 (64-bit, non-prefetchable) [size=1M]
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: 00001000-00001fff
Memory behind bridge: 20200000-202fffff
Prefetchable memory behind bridge: 20300000-203fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000ee103000 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: pcieport
0000:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) (prog-if 01 [AHCI 1.0])
Subsystem: Marvell Technology Group Ltd. Device 9170
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 186
Region 0: I/O ports at 1010 [size=8]
Region 1: I/O ports at 1020 [size=4]
Region 2: I/O ports at 1018 [size=8]
Region 3: I/O ports at 1024 [size=4]
Region 4: I/O ports at 1000 [size=16]
Region 5: Memory at 20200000 (32-bit, non-prefetchable) [size=512]
Expansion ROM at 20300000 [disabled] [size=64K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: ee103000 Data: 0001
Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Kernel driver in use: ahci
Kernel modules: ahci
0001:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 179
Region 0: Memory at 30100000 (64-bit, non-prefetchable) [size=1M]
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: 00000000-00000fff
Memory behind bridge: 30200000-302fffff
Prefetchable memory behind bridge: 30300000-303fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000ee098000 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: pcieport
0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) (prog-if 01 [AHCI 1.0])
Subsystem: Marvell Technology Group Ltd. Device 9170
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 188
Region 0: I/O ports at 10010 [size=8]
Region 1: I/O ports at 10020 [size=4]
Region 2: I/O ports at 10018 [size=8]
Region 3: I/O ports at 10024 [size=4]
Region 4: I/O ports at 10000 [size=16]
Region 5: Memory at 30200000 (32-bit, non-prefetchable) [size=512]
Expansion ROM at 30300000 [disabled] [size=64K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: ee098000 Data: 0001
Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Kernel driver in use: ahci
Kernel modules: ahci
root@nepARIAv4:~# dmesg | pci
-bash: pci: command not found
root@nepARIAv4:~# dmesg | grep pci
[ 0.737189] dra7-pcie 51000000.pcie: Linked as a consumer to phy-4a094000.pciephy.1
[ 0.737347] dra7-pcie 51000000.pcie: GPIO lookup for consumer (null)
[ 0.737355] dra7-pcie 51000000.pcie: using device tree for GPIO lookup
[ 0.737390] of_get_named_gpiod_flags: parsed 'gpios' property of node '/ocp/axi@0/pcie@51000000[0]' - status (0)
[ 0.737426] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_probe devid=0 reset_gpio= 313=0x139
[ 0.737480] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_unaligned_memaccess devid=0 args[0]= 20=0x14
[ 0.737589] OF: PCI: host bridge /ocp/axi@0/pcie@51000000 ranges:
[ 0.837736] dra7-pcie 51000000.pcie: link up
[ 0.837865] dra7-pcie 51000000.pcie: PCI host bridge to bus 0000:00
[ 0.837879] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.837891] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 0.837902] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[ 0.837935] pci 0000:00:00.0: [104c:8888] type 01 class 0x060400
[ 0.837966] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[ 0.838030] pci 0000:00:00.0: supports D1
[ 0.838037] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 0.838383] pci 0000:01:00.0: [1b4b:9170] type 00 class 0x010601
[ 0.838493] pci 0000:01:00.0: reg 0x10: [io 0x8000-0x8007]
[ 0.838540] pci 0000:01:00.0: reg 0x14: [io 0x8040-0x8043]
[ 0.838587] pci 0000:01:00.0: reg 0x18: [io 0x8100-0x8107]
[ 0.838633] pci 0000:01:00.0: reg 0x1c: [io 0x8140-0x8143]
[ 0.838680] pci 0000:01:00.0: reg 0x20: [io 0x800000-0x80000f]
[ 0.838726] pci 0000:01:00.0: reg 0x24: [mem 0x00900000-0x009001ff]
[ 0.838773] pci 0000:01:00.0: reg 0x30: [mem 0xd0000000-0xd000ffff pref]
[ 0.839017] pci 0000:01:00.0: PME# supported from D3hot
[ 0.862837] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff 64bit]
[ 0.862857] pci 0000:00:00.0: BAR 8: assigned [mem 0x20200000-0x202fffff]
[ 0.862869] pci 0000:00:00.0: BAR 9: assigned [mem 0x20300000-0x203fffff pref]
[ 0.862881] pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 0.862897] pci 0000:01:00.0: BAR 6: assigned [mem 0x20300000-0x2030ffff pref]
[ 0.862908] pci 0000:01:00.0: BAR 5: assigned [mem 0x20200000-0x202001ff]
[ 0.862932] pci 0000:01:00.0: BAR 4: assigned [io 0x1000-0x100f]
[ 0.862954] pci 0000:01:00.0: BAR 0: assigned [io 0x1010-0x1017]
[ 0.862977] pci 0000:01:00.0: BAR 2: assigned [io 0x1018-0x101f]
[ 0.863000] pci 0000:01:00.0: BAR 1: assigned [io 0x1020-0x1023]
[ 0.863022] pci 0000:01:00.0: BAR 3: assigned [io 0x1024-0x1027]
[ 0.863045] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 0.863056] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]
[ 0.863068] pci 0000:00:00.0: bridge window [mem 0x20200000-0x202fffff]
[ 0.863079] pci 0000:00:00.0: bridge window [mem 0x20300000-0x203fffff pref]
[ 0.863393] pcieport 0000:00:00.0: Signaling PME with IRQ 177
[ 0.863524] pcieport 0000:00:00.0: AER enabled with IRQ 177
[ 0.863667] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_probe devid=0 gen= 1=0x1
[ 0.863862] dra7-pcie 51800000.pcie: Linked as a consumer to phy-4a095000.pciephy.2
[ 0.863968] dra7-pcie 51800000.pcie: GPIO lookup for consumer (null)
[ 0.863975] dra7-pcie 51800000.pcie: using device tree for GPIO lookup
[ 0.864006] of_get_named_gpiod_flags: parsed 'gpios' property of node '/ocp/axi@1/pcie@51800000[0]' - status (0)
[ 0.864038] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_probe devid=0 reset_gpio= 220=0xdc
[ 0.864064] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_unaligned_memaccess devid=0 args[0]= 20=0x14
[ 0.864162] OF: PCI: host bridge /ocp/axi@1/pcie@51800000 ranges:
[ 0.964293] dra7-pcie 51800000.pcie: link up
[ 0.964416] dra7-pcie 51800000.pcie: PCI host bridge to bus 0001:00
[ 0.964428] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 0.964441] pci_bus 0001:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff])
[ 0.964452] pci_bus 0001:00: root bus resource [mem 0x30013000-0x3fffffff]
[ 0.964481] pci 0001:00:00.0: [104c:8888] type 01 class 0x060400
[ 0.964511] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[ 0.964565] pci 0001:00:00.0: supports D1
[ 0.964572] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 0.964905] pci 0001:01:00.0: [1b4b:9170] type 00 class 0x010601
[ 0.965015] pci 0001:01:00.0: reg 0x10: [io 0x18000-0x18007]
[ 0.965062] pci 0001:01:00.0: reg 0x14: [io 0x18040-0x18043]
[ 0.965109] pci 0001:01:00.0: reg 0x18: [io 0x18100-0x18107]
[ 0.965155] pci 0001:01:00.0: reg 0x1c: [io 0x18140-0x18143]
[ 0.965202] pci 0001:01:00.0: reg 0x20: [io 0x800000-0x80000f]
[ 0.965249] pci 0001:01:00.0: reg 0x24: [mem 0x00900000-0x009001ff]
[ 0.965296] pci 0001:01:00.0: reg 0x30: [mem 0xd0000000-0xd000ffff pref]
[ 0.965541] pci 0001:01:00.0: PME# supported from D3hot
[ 0.992827] pci 0001:00:00.0: BAR 0: assigned [mem 0x30100000-0x301fffff 64bit]
[ 0.992846] pci 0001:00:00.0: BAR 8: assigned [mem 0x30200000-0x302fffff]
[ 0.992858] pci 0001:00:00.0: BAR 9: assigned [mem 0x30300000-0x303fffff pref]
[ 0.992869] pci 0001:00:00.0: BAR 7: assigned [io 0x10000-0x10fff]
[ 0.992885] pci 0001:01:00.0: BAR 6: assigned [mem 0x30300000-0x3030ffff pref]
[ 0.992897] pci 0001:01:00.0: BAR 5: assigned [mem 0x30200000-0x302001ff]
[ 0.992921] pci 0001:01:00.0: BAR 4: assigned [io 0x10000-0x1000f]
[ 0.992944] pci 0001:01:00.0: BAR 0: assigned [io 0x10010-0x10017]
[ 0.992966] pci 0001:01:00.0: BAR 2: assigned [io 0x10018-0x1001f]
[ 0.992989] pci 0001:01:00.0: BAR 1: assigned [io 0x10020-0x10023]
[ 0.993011] pci 0001:01:00.0: BAR 3: assigned [io 0x10024-0x10027]
[ 0.993034] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[ 0.993045] pci 0001:00:00.0: bridge window [io 0x10000-0x10fff]
[ 0.993056] pci 0001:00:00.0: bridge window [mem 0x30200000-0x302fffff]
[ 0.993067] pci 0001:00:00.0: bridge window [mem 0x30300000-0x303fffff pref]
[ 0.993368] pcieport 0001:00:00.0: Signaling PME with IRQ 179
[ 0.993497] pcieport 0001:00:00.0: AER enabled with IRQ 179
[ 0.993637] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_probe devid=0 gen= 1=0x1
"lspci -vv" and "dmesg | grep pci" when only some disks (2 out of 4) are correctly identified at start time:
root@nepARIAv4:~# lspci -vv
0000:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 177
Region 0: Memory at 20100000 (64-bit, non-prefetchable) [size=1M]
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000ee0fe000 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: pcieport
0001:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 179
Region 0: Memory at 30100000 (64-bit, non-prefetchable) [size=1M]
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: 00000000-00000fff
Memory behind bridge: 30200000-302fffff
Prefetchable memory behind bridge: 30300000-303fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000ee08c000 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: pcieport
0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) (prog-if 01 [AHCI 1.0])
Subsystem: Marvell Technology Group Ltd. Device 9170
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 186
Region 0: I/O ports at 10010 [size=8]
Region 1: I/O ports at 10020 [size=4]
Region 2: I/O ports at 10018 [size=8]
Region 3: I/O ports at 10024 [size=4]
Region 4: I/O ports at 10000 [size=16]
Region 5: Memory at 30200000 (32-bit, non-prefetchable) [size=512]
Expansion ROM at 30300000 [disabled] [size=64K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: ee08c000 Data: 0001
Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Kernel driver in use: ahci
Kernel modules: ahci
root@nepARIAv4:~# dmesg | grep "pci"
[ 0.736773] dra7-pcie 51000000.pcie: Linked as a consumer to phy-4a094000.pciephy.1
[ 0.736931] dra7-pcie 51000000.pcie: GPIO lookup for consumer (null)
[ 0.736938] dra7-pcie 51000000.pcie: using device tree for GPIO lookup
[ 0.736972] of_get_named_gpiod_flags: parsed 'gpios' property of node '/ocp/axi@0/pcie@51000000[0]' - status (0)
[ 0.737009] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_probe devid=0 reset_gpio= 313=0x139
[ 0.737064] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_unaligned_memaccess devid=0 args[0]= 20=0x14
[ 0.737174] OF: PCI: host bridge /ocp/axi@0/pcie@51000000 ranges:
[ 0.837318] dra7-pcie 51000000.pcie: link up
[ 0.837447] dra7-pcie 51000000.pcie: PCI host bridge to bus 0000:00
[ 0.837460] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.837472] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 0.837482] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[ 0.837516] pci 0000:00:00.0: [104c:8888] type 01 class 0x060400
[ 0.837548] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[ 0.837607] pci 0000:00:00.0: supports D1
[ 0.837614] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 0.881351] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff 64bit]
[ 0.881370] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 0.881688] pcieport 0000:00:00.0: Signaling PME with IRQ 177
[ 0.881823] pcieport 0000:00:00.0: AER enabled with IRQ 177
[ 0.881946] dra7-pcie 51000000.pcie: indra => dra7xx_pcie_probe devid=0 gen= 1=0x1
[ 0.882131] dra7-pcie 51800000.pcie: Linked as a consumer to phy-4a095000.pciephy.2
[ 0.882236] dra7-pcie 51800000.pcie: GPIO lookup for consumer (null)
[ 0.882243] dra7-pcie 51800000.pcie: using device tree for GPIO lookup
[ 0.882274] of_get_named_gpiod_flags: parsed 'gpios' property of node '/ocp/axi@1/pcie@51800000[0]' - status (0)
[ 0.882307] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_probe devid=0 reset_gpio= 220=0xdc
[ 0.882332] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_unaligned_memaccess devid=0 args[0]= 20=0x14
[ 0.882431] OF: PCI: host bridge /ocp/axi@1/pcie@51800000 ranges:
[ 0.982562] dra7-pcie 51800000.pcie: link up
[ 0.982683] dra7-pcie 51800000.pcie: PCI host bridge to bus 0001:00
[ 0.982695] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 0.982735] pci_bus 0001:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff])
[ 0.982745] pci_bus 0001:00: root bus resource [mem 0x30013000-0x3fffffff]
[ 0.982775] pci 0001:00:00.0: [104c:8888] type 01 class 0x060400
[ 0.982804] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[ 0.982858] pci 0001:00:00.0: supports D1
[ 0.982864] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 0.983202] pci 0001:01:00.0: [1b4b:9170] type 00 class 0x010601
[ 0.983310] pci 0001:01:00.0: reg 0x10: [io 0x18000-0x18007]
[ 0.983357] pci 0001:01:00.0: reg 0x14: [io 0x18040-0x18043]
[ 0.983404] pci 0001:01:00.0: reg 0x18: [io 0x18100-0x18107]
[ 0.983450] pci 0001:01:00.0: reg 0x1c: [io 0x18140-0x18143]
[ 0.983496] pci 0001:01:00.0: reg 0x20: [io 0x800000-0x80000f]
[ 0.983543] pci 0001:01:00.0: reg 0x24: [mem 0x00900000-0x009001ff]
[ 0.983590] pci 0001:01:00.0: reg 0x30: [mem 0xd0000000-0xd000ffff pref]
[ 0.983834] pci 0001:01:00.0: PME# supported from D3hot
[ 1.012831] pci 0001:00:00.0: BAR 0: assigned [mem 0x30100000-0x301fffff 64bit]
[ 1.012851] pci 0001:00:00.0: BAR 8: assigned [mem 0x30200000-0x302fffff]
[ 1.012863] pci 0001:00:00.0: BAR 9: assigned [mem 0x30300000-0x303fffff pref]
[ 1.012876] pci 0001:00:00.0: BAR 7: assigned [io 0x10000-0x10fff]
[ 1.012896] pci 0001:01:00.0: BAR 6: assigned [mem 0x30300000-0x3030ffff pref]
[ 1.012908] pci 0001:01:00.0: BAR 5: assigned [mem 0x30200000-0x302001ff]
[ 1.012931] pci 0001:01:00.0: BAR 4: assigned [io 0x10000-0x1000f]
[ 1.012955] pci 0001:01:00.0: BAR 0: assigned [io 0x10010-0x10017]
[ 1.012977] pci 0001:01:00.0: BAR 2: assigned [io 0x10018-0x1001f]
[ 1.013000] pci 0001:01:00.0: BAR 1: assigned [io 0x10020-0x10023]
[ 1.013022] pci 0001:01:00.0: BAR 3: assigned [io 0x10024-0x10027]
[ 1.013047] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[ 1.013058] pci 0001:00:00.0: bridge window [io 0x10000-0x10fff]
[ 1.013069] pci 0001:00:00.0: bridge window [mem 0x30200000-0x302fffff]
[ 1.013080] pci 0001:00:00.0: bridge window [mem 0x30300000-0x303fffff pref]
[ 1.013378] pcieport 0001:00:00.0: Signaling PME with IRQ 179
[ 1.013506] pcieport 0001:00:00.0: AER enabled with IRQ 179
[ 1.013647] dra7-pcie 51800000.pcie: indra => dra7xx_pcie_probe devid=0 gen= 1=0x1
