This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PCIe link issue between TMS320C6655 and Spartan 6 FPGA

Other Parts Discussed in Thread: TMS320C6655, TMS320C6657

We are currently using PCIe link between TMS320C6655 and Spartan 6 FPGA.   We get past detect state and go into link training but basically we cannot get past link training.  The Spartan 6 only supports 1 PCIe1.0 lane so the max link speed is going to be 2.5Gb/s even though the DSP supports PCIe2.0 and up to 2 lanes.  We have been able to do some preliminary eye diagram analysis of DSP TX diff pair to FPGA and FPGA TX diff pair to DSP.  The DSP TX diff pair to FPGA eye diagram is partially closed and the suspected problem whereas the FPGA TX diff pair to DSP look pretty good and appears to be within compliance.   We have looked over DSP and FPGA device configuration and not found anything which has improved eye between DSP and FPGA.  We have also looked at the common 100MHz clock we are supplying to both FPGA nd DSP and clocks look to meet PCIe1.0 jitter requirement.  FYI...We currently have the Spartan 6 EVM and the TMS320C6657 EVM connected via a daughter card we previously developed for proof of concept and PCIe link is working okay in that application.  We are open to suggestions to resolving this problem and currently working to get PCIe IBIS-AMI model to do some board level simulations.  

James

  • Hi James,
    Please check wheather below PCIe FAQ wiki has pointers to link issue,
    processors.wiki.ti.com/.../PCI_Express_(PCIe)_Resource_Wiki_for_Keystone_Devices

    Also I would recommend you to search through e2e forum for similar issue with Spartan 6 FPGA,
    e2e.ti.com/.../245625
    e2e.ti.com/.../193398
    e2e.ti.com/.../155435
    e2e.ti.com/.../306544

    Thank you.
  • Raja,

    Above FAQ wiki and forum links did not really help.  We previously developed a daughter card to connect Spartan 6 FPGA and 6657 EVM to validate proof of concept for PCIe for our design.  We were able to get that interface working with the various interconnections between boards.  Our custom board which is not currently working is a point to point connection between 6655 DSP and Spatan 6 FPGA with only about 1.5-1.75" inches of matched trace length and no board to board interconnects.  

    James

  • Hi,

    Thanks for you information.

    I am not sure the PCIe point to point connection trace length is issue or not. I will check with my hardware team to get the maximum trace length.

    Provide more information about your setup, DSP is configured as EP or RC. Have you track the LTSSM_STATE debug0 register in DSP side? It should be useful to track the issue, for more information refer Appendix A on PCIe user guide(SPRUGS6D). Insure link training completion and success by observing LTSSM_STATE field in DEBUG0 register change to 0x11.

    Thanks,
  • DSP is configured as Root Complex and FPGA as endpoint. We have tracked the LTSSM_STATE some and we can see it going between modes 0 and 2 primarily and occasionally up to 04 but never reached 0x11, L0, State...Basically link training really never appears to complete in our custom application...
  • Hi,

    Try to re-Initiate link training can be initiated by asserting LTSSM_EN bit in the CMD_STATUS register (CMD_STATUS[LTSSM_EN]=1). IF you using debugger means open memory browser and write PCIE CMD_STATUS to 0x00000007 manually.

    Thanks,
  • This was tried by setting LTSSM_EN bit numerous times without success ... We do not feel that we have a configuration issue based on our successful development on 6657 evaluation platform. We are planning to dump the debug registers on our custom platform and can include in another post. Any other suggestions to resolving this issue!!!
  • We captured some debug registers on our platform. Data dumped below.

    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 1
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 2b
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 6
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 8f
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 0
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : b4
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 2a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : ff
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : b
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 9d
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : fe
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
  • Hi,

    Have you using the same code(DSP and FPGA EVM test code) on custom board for both chips(DSP and FPGA)? Have you using TI provided PCIe test code for your testing?

    In your custom board the PCIe reference clock is common or seperate for DSP and FPGA.

    Thanks,

  • Yes...We have used same baseline code from EVM test platforms on our custom board for both devices with some register changes required for our custom application which we resolved previously when we worked to get PCIe working on Evaluation Platforms. As you have seen above, we have dumped the debug registers for your analysis and feedback. In our custom board, we are using a common PCIe reference clock for both DSP and FPGA. Any feedback regarding minimum length of PCIe traces. I know the max length could be upwards of 10-12" but as I noted above in previous post our point to point trace lengths between DSP and FPGA are around 1.5-1.75"...
  • The PCIe point to point trace length is fine for your design. For my understanding, the same software is running on both platforms means it should be fine for software side.

    Please take a look at below PCIe FAQ wiki and check the power sequence on your custom board.
    processors.wiki.ti.com/.../PCI_Express_(PCIe)_Resource_Wiki_for_Keystone_Devices

    Thanks,
  • Ganaphathi,

    Could you elaborate. Basically we are powering DSP core then I/O per 6655 Datasheet. In regards to PCIe Wiki power sequence, the procedure listed is below. We do not have a PCIe edge connector/slot in our design. We have a point to point connection between 2 DSP and FPGA on same board. We are following this basic flow with exception that our link does not come up...

    1. PCIe slot on PC host provides power and reference clock to PCIe module on EVM.
    2. PCIe boot code on EVM initializes C66x PCIe module and ready for link up.
    3. PCIe root complex in PC host is powered up and link up is established between PCIe root complex (RC) in host and PCIe end point (EP) in EVM.
    4. PC host enumeration (BIOS) starts to scan the PCIe bus.
    5. PCIe end point in EVM is enumerated and registered in PC host OS.
  • Below is our PCIe C Code taken originally from your evaluation platform for your review.

    //* FILE: PCIe.c *
    //* AUTHOR: *
    //* UNIT NAME: PCI Express Bus Setup Utilities *
    //* FUNCTION: To initialize the PCI Express bus so it will talk to the *
    //* Master FPGA. *
    //* INPUTS: *
    //* OUTPUTS: *
    //**************************************************************************
    #include "computation_engine.h"
    #ifdef C66_PLATFORMS
    #include "csl_cpsw_3gfAux.h"
    #endif

    #include <ti/csl/csl_gpio.h>
    #include <ti/csl/cslr_gpio.h>
    #include <ti/csl/csl_gpioAux.h>
    #include <ti/csl/csl_bootcfgAux.h>
    #include <ti/csl/csl_edma3.h>
    #include <ti/csl/csl_edma3Aux.h>
    #include <ti/csl/csl_cacheAux.h>
    #include <ti/csl/csl_chip.h>
    #include <ti/csl/csl_xmcAux.h>
    #include <ti/csl/cslr_uart.h>
    #include <ti/csl/src/intc/csl_intcAux.h>
    #include <math.h>
    #include "metrics.h"

    /**************************************************************************
    ** Configuration
    ****************************************************************************/

    //**************************************************************************************
    //************************************* PCIe Structures ********************************
    //**************************************************************************************
    /* last element in the buffer is a marker that indicates the buffer status: full/empty */
    #define PCIE_EXAMPLE_MAX_CACHE_LINE_SIZE 128
    #define PCIE_EXAMPLE_UINT32_SIZE 4 /* preprocessor #if requires a real constant, not a sizeof() */
    #define PCIE_EXAMPLE_DSTBUF_BYTES ((PCIE_BUFSIZE_APP + 1) * PCIE_EXAMPLE_UINT32_SIZE)
    #define PCIE_EXAMPLE_DSTBUF_REM (PCIE_EXAMPLE_DSTBUF_BYTES % PCIE_EXAMPLE_MAX_CACHE_LINE_SIZE)
    #define PCIE_EXAMPLE_DSTBUF_PAD (PCIE_EXAMPLE_DSTBUF_REM ? (PCIE_EXAMPLE_MAX_CACHE_LINE_SIZE - PCIE_EXAMPLE_DSTBUF_REM) : 0)
    #define PCIE_EXAMPLE_BUF_EMPTY 0
    #define PCIE_EXAMPLE_BUF_FULL 1


    /* Status of the call to initialize the platform */
    extern void *pcieBase;

    /* Does not need to be aligned (even for cache) since it is only accessed locally */
    extern uint32_t srcBuf[PCIE_BUFSIZE_APP+1];

    extern volatile unsigned int cregister TSCL;

    /* Global config variable that controls
    the PCIe mode. It is global so it can be poked
    from CCS. It should be set either to EP or RC. */
    extern pcieMode_e PcieModeGbl;


    /*****************************************************************************
    * Function: Power domain configuration
    ****************************************************************************/
    pcieRet_e pciePowerCfg(void)
    {
    /* Turn on the PCIe power domain */
    if (CSL_PSC_getPowerDomainState (CSL_PSC_PD_PCIEX) != PSC_PDSTATE_ON)
    {
    /* Enable the domain */
    CSL_PSC_enablePowerDomain (CSL_PSC_PD_PCIEX);
    /* Enable MDCTL */
    CSL_PSC_setModuleNextState (CSL_PSC_LPSC_PCIEX, PSC_MODSTATE_ENABLE);
    /* Apply the domain */
    CSL_PSC_startStateTransition (CSL_PSC_PD_PCIEX);
    /* Wait for it to finish */
    while (! CSL_PSC_isStateTransitionDone (CSL_PSC_PD_PCIEX));
    }
    else
    {
    platform_write ("Power domain is already enabled. You probably re-ran without device reset (which is OK)\n");
    }

    return pcie_RET_OK;
    }

    /*****************************************************************************
    * Function: Utility function to introduce delay
    ****************************************************************************/
    void cycleDelay (uint32_t count)
    {
    uint32_t sat;

    if (count <= 0)
    return;

    sat = TSCL + count;
    while (TSCL < sat);
    }

    /*****************************************************************************
    * Function: Serdes configuration
    ****************************************************************************/
    pcieRet_e pcieSerdesCfg(void)
    {
    uint16_t cfg;

    CSL_BootCfgUnlockKicker();

    /* Provide PLL reference clock to SERDES inside PCIESS
    Program PLL settings and enable PLL from PCIe SERDES.*/
    cfg = 0x01C9; /* value based on PCIe userguide */
    CSL_BootCfgSetPCIEConfigPLL(cfg);

    CSL_BootCfgLockKicker();

    /*Wait for PLL to lock (3000 CLKIN1 cycles) */
    cycleDelay(10000);

    return pcie_RET_OK;
    }

    /*****************************************************************************
    * Function: Enable/Disable LTSSM (Link Training)
    ****************************************************************************/
    pcieRet_e pcieLtssmCtrl(Pcie_Handle handle, uint8_t enable)
    {
    pcieCmdStatusReg_t cmdStatus;
    pcieRegisters_t setRegs;
    pcieRegisters_t getRegs;
    pcieRet_e retVal;

    memset (&cmdStatus, 0, sizeof(cmdStatus));
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.cmdStatus = &cmdStatus;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read CMD STATUS register failed!\n");
    return retVal;
    }

    if(enable)
    cmdStatus.ltssmEn = 1;
    else
    cmdStatus.ltssmEn = 0;

    setRegs.cmdStatus = &cmdStatus;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET CMD STATUS register failed!\n");
    return retVal;
    }

    return pcie_RET_OK;
    }

    /*****************************************************************************
    * Function: Configure PCIe in Root Complex Mode
    ****************************************************************************/
    pcieRet_e pcieCfgRC(Pcie_Handle handle)
    {
    pcieRet_e retVal;

    pcieCmdStatusReg_t cmdStatus;
    pcieObSizeReg_t obSize;
    pcieGen2Reg_t gen2;
    pcieType1Bar32bitIdx_t type1Bar32bitIdx;
    pcieStatusCmdReg_t statusCmd;
    pcieDevStatCtrlReg_t devStatCtrl;
    pcieAccrReg_t accr;
    pcieLinkStatCtrlReg_t LinkStatCtrl;
    pcieEndianReg_t endian; /**< @brief Endian Register*/


    pcieRegisters_t setRegs;
    pcieRegisters_t getRegs;


    memset (&cmdStatus, 0, sizeof(cmdStatus));
    memset (&obSize, 0, sizeof(obSize));
    memset (&gen2, 0, sizeof(gen2));
    memset (&type1Bar32bitIdx, 0, sizeof(type1Bar32bitIdx));
    memset (&statusCmd, 0, sizeof(statusCmd));
    memset (&devStatCtrl, 0, sizeof(devStatCtrl));
    memset (&accr, 0, sizeof(accr));
    memset (&LinkStatCtrl, 0, sizeof(LinkStatCtrl));

    /*Disable link training*/
    if ((retVal = pcieLtssmCtrl(handle, FALSE)) != pcie_RET_OK)
    {
    platform_write ("Failed to disable Link Training!\n");
    return retVal;
    }

    /* Configure the size of the translation regions */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    obSize.size = pcie_OB_SIZE_1MB;
    setRegs.obSize = &obSize;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET OB_SIZE register failed!\n");
    return retVal;
    }

    /* Setting PL_GEN2 */
    memset (&setRegs, 0, sizeof(setRegs));
    gen2.numFts = 0xF;
    gen2.dirSpd = 0x0;
    gen2.lnEn = 1;
    setRegs.gen2 = &gen2;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET GEN2 register failed!\n");
    return retVal;
    }

    // David Added
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&LinkStatCtrl, 0, sizeof(LinkStatCtrl));
    memset (&endian, 0, sizeof(endian));

    getRegs.linkStatCtrl = &LinkStatCtrl;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read LINK STATUS register failed!\n");
    return retVal;
    }
    LinkStatCtrl.commonClkCfg = 1;
    setRegs.linkStatCtrl = &LinkStatCtrl;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET LINK STATUS register failed!\n");
    return retVal;
    }


    endian.mode = 0;
    setRegs.endian = &endian;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET endian register failed!\n");
    return retVal;
    }
    // End David Added


    /* Configure BAR Masks */
    /* First need to enable writing on BAR mask registers */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    getRegs.cmdStatus = &cmdStatus;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read CMD STATUS register failed!\n");
    return retVal;
    }
    cmdStatus.dbi = 1;
    setRegs.cmdStatus = &cmdStatus;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET CMD STATUS register failed!\n");
    return retVal;
    }

    /* Configure Masks*/
    memset (&setRegs, 0, sizeof(setRegs));
    type1Bar32bitIdx.reg.reg32 = PCIE_BAR_MASK0;
    setRegs.type1Bar32bitIdx = &type1Bar32bitIdx;

    /* BAR 0 */
    type1Bar32bitIdx.idx = 0; /* configure BAR 0*/
    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET BAR MASK register failed!\n");
    return retVal;
    }

    /* BAR 1 */
    type1Bar32bitIdx.reg.reg32 = PCIE_BAR_MASK1;
    type1Bar32bitIdx.idx = 1; /* configure BAR 1*/
    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET BAR MASK register failed!\n");
    return retVal;
    }

    /* Disable writing on BAR Masks */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    getRegs.cmdStatus = &cmdStatus;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read CMD STATUS register failed!\n");
    return retVal;
    }
    cmdStatus.dbi = 0;
    setRegs.cmdStatus = &cmdStatus;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET CMD STATUS register failed!\n");
    return retVal;
    }


    /* Enable memory access and mastership of the bus */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.statusCmd = &statusCmd;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read Status Comand register failed!\n");
    return retVal;
    }
    statusCmd.memSp = 1;
    statusCmd.busMs = 1;
    statusCmd.resp = 1;
    statusCmd.serrEn = 1;
    setRegs.statusCmd = &statusCmd;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET Status Command register failed!\n");
    return retVal;
    }

    /* Enable Error Reporting */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.devStatCtrl = &devStatCtrl;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Regad Device Status Control register failed!\n");
    return retVal;
    }

    //devStatCtrl.maxSz = 1;
    //devStatCtrl.maxPayld = 0;
    devStatCtrl.reqRp = 1;
    devStatCtrl.fatalErRp = 1;
    devStatCtrl.nFatalErRp = 1;
    devStatCtrl.corErRp = 1;
    setRegs.devStatCtrl = &devStatCtrl;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET Device Status Control register failed!\n");
    return retVal;
    }

    /* Enable ECRC */
    memset (&setRegs, 0, sizeof(setRegs));

    accr.chkEn=1;
    accr.chkCap=1;
    accr.genEn=1;
    accr.genCap=1;
    setRegs.accr = &accr;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET ACCR register failed!\n");
    return retVal;
    }

    return pcie_RET_OK;
    }

    /*****************************************************************************
    * Function: Configure and enable Outbound Address Translation
    ****************************************************************************/
    pcieRet_e pcieObTransCfg(Pcie_Handle handle, uint32_t obAddrLo, uint32_t obAddrHi, uint8_t region)
    {
    pcieRet_e retVal;

    pcieRegisters_t setRegs;
    pcieRegisters_t getRegs;

    pcieCmdStatusReg_t cmdStatus;

    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    /* Set outbound offset registers */
    if ((retVal = Pcie_cfgObOffset(handle, obAddrLo, obAddrHi, region)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure ObOffset registers!\n");
    return retVal;
    }

    /*enable Outbound address translation*/
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.cmdStatus = &cmdStatus;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read CMD STATUS register failed!\n");
    return retVal;
    }
    cmdStatus.obXltEn = 1;
    setRegs.cmdStatus = &cmdStatus;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET CMD STATUS register failed!\n");
    return retVal;
    }

    return pcie_RET_OK;
    }


    /*****************************************************************************
    * Function: Configure and enable Inbound Address Translation
    ****************************************************************************/
    pcieRet_e pcieIbTransCfg(Pcie_Handle handle, pcieIbTransCfg_t *ibCfg)
    {
    pcieRet_e retVal;

    pcieRegisters_t setRegs;
    pcieRegisters_t getRegs;

    pcieCmdStatusReg_t cmdStatus;

    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    /* Set outbound offset registers */
    if ((retVal = Pcie_cfgIbTrans(handle, ibCfg)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure Inbound Translation registers!\n");
    return retVal;
    }

    /*enable Inbound address translation*/
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.cmdStatus = &cmdStatus;
    if ((retVal = Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read CMD STATUS register failed!\n");
    return retVal;
    }
    cmdStatus.ibXltEn = 1;
    setRegs.cmdStatus = &cmdStatus;

    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET CMD STATUS register failed!\n");
    return retVal;
    }

    return pcie_RET_OK;
    }


    /*****************************************************************************
    * Function: Initialize application buffers
    ****************************************************************************/
    void pcieInitAppBuf(void)
    {
    uint32_t i;

    for (i=0; i<PCIE_BUFSIZE_APP; i++)
    {
    dstBuf.buf[i] = 0;
    srcBuf[i] = i + 1;
    }
    srcBuf[PCIE_BUFSIZE_APP] = PCIE_EXAMPLE_BUF_FULL;
    dstBuf.buf[PCIE_BUFSIZE_APP] = PCIE_EXAMPLE_BUF_EMPTY;
    }

    void pcie_register_debug(Pcie_Handle handle)
    {
    pcieRegisters_t getRegs;
    pcieRegisters_t setRegs;

    pcieRootErrStReg_t RootErrStatus;
    pcieDevStatCtrlReg_t DevStatCtrl;
    pcieLinkCapReg_t LinkCap;
    pcieLinkStatCtrlReg_t LinkStatCtrl;
    pcieLinkCtrl2Reg_t LinkCtrl2;

    pcieVndDevIdReg_t VndDev;
    pcieStatusCmdReg_t statusCmd;
    pcieRevIdReg_t revId;
    pcieSerdesCfg0Reg_t serdesCfg0; /**< @brief SERDES config 0 Register*/
    pcieSerdesCfg1Reg_t serdesCfg1; /**< @brief SERDES config 1 Register*/

    pcieDebug0Reg_t debug0; /**< @brief Debug 0*/
    pcieDebug1Reg_t debug1; /**< @brief Debug 1 Register*/

    pcieType1BusNumReg_t type1BusNum; /**< @brief Latency Timer and Bus Number */
    pcieType1SecStatReg_t type1SecStat; /**< @brief Secondary Status and IO space */
    pcieType1MemspaceReg_t type1Memspace; /**< @brief Memory Limit*/
    pciePrefMemReg_t prefMem; /**< @brief Prefetch Memory Limit and Base*/
    pciePrefBaseUpperReg_t prefBaseUpper; /**< @brief Prefetch Memory Base Upper*/
    pciePrefLimitUpperReg_t prefLimitUpper; /**< @brief Prefetch Memory Limit Upper*/
    pcieType1IOSpaceReg_t type1IOSpace; /**< @brief IO Base and Limit Upper 16 bits */
    pciePciesCapReg_t pciesCap; /**< @brief PCI Express Capabilities Register*/
    pcieDeviceCapReg_t deviceCap; /**< @brief Device Capabilities Register*/
    pcieSubIdReg_t subId; // Subsystem ID
    pcieType0Bar32bitIdx_t type0Bar32bitIdx;

    memset (&type0Bar32bitIdx, 0, sizeof(type0Bar32bitIdx));
    memset (&debug0, 0, sizeof(debug0));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&setRegs, 0, sizeof(setRegs));

    //Debug Code
    platform_write ("Read Root Complex Regs!\n");
    //#1 Vendor & Device
    cycleDelay(100);
    getRegs.vndDevId = &VndDev;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("RC - Read Vendor and device ID Reg failed!\n");
    }
    else
    {
    platform_write ("RC - Read Vendor : %x Device ID: %x\n",VndDev.vndId,VndDev.devId);
    }


    //#2 Status Command Reg
    cycleDelay(100);
    getRegs.statusCmd = &statusCmd;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Status Command Reg failed!\n");
    }
    else
    {
    platform_write ("Status Command Reg raw : %x\n",statusCmd.raw);
    platform_write ("Status Command Reg parity : %x\n",statusCmd.parity);
    platform_write ("Status Command Reg sysError : %x\n",statusCmd.sysError);
    platform_write ("Status Command Reg mstAbort : %x\n",statusCmd.mstAbort);
    platform_write ("Status Command Reg tgtAbort : %x\n",statusCmd.tgtAbort);
    platform_write ("Status Command Reg sigTgtAbort: %x\n",statusCmd.sigTgtAbort);
    platform_write ("Status Command Reg parError : %x\n",statusCmd.parError);
    platform_write ("Status Command Reg capList : %x\n",statusCmd.capList);
    platform_write ("Status Command Reg stat : %x\n",statusCmd.stat);
    platform_write ("Status Command Reg dis : %x\n",statusCmd.dis);
    platform_write ("Status Command Reg serrEn : %x\n",statusCmd.serrEn);
    platform_write ("Status Command Reg resp : %x\n",statusCmd.resp);
    platform_write ("Status Command Reg busMs : %x\n",statusCmd.busMs);
    platform_write ("Status Command Reg memSp : %x\n",statusCmd.memSp);
    platform_write ("Status Command Reg ioSp : %x\n",statusCmd.ioSp);
    }

    //#3 Root Error Status
    cycleDelay(100);
    getRegs.rootErrSt = &RootErrStatus;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Root Error Status Reg failed!\n");
    }
    else
    {
    platform_write ("Root Error Status aerIntMsg: %x\n",RootErrStatus.aerIntMsg);
    platform_write ("Root Error Status ferrRcv : %x\n",RootErrStatus.ferrRcv);
    platform_write ("Root Error Status nfErr : %x\n",RootErrStatus.nfErr);
    platform_write ("Root Error Status uncorFata: %x\n",RootErrStatus.uncorFatal);
    platform_write ("Root Error Status multFnf : %x\n",RootErrStatus.multFnf);
    platform_write ("Root Error Status errFnf : %x\n",RootErrStatus.errFnf);
    platform_write ("Root Error Status multCor : %x\n",RootErrStatus.multCor);
    platform_write ("Root Error Status corrErr : %x\n",RootErrStatus.corrErr);
    }

    //#4 Device Status and Control
    cycleDelay(100);
    getRegs.devStatCtrl = &DevStatCtrl;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Device Status and Control Reg failed!\n");
    }
    else
    {
    platform_write ("Device Status and Control raw : %x\n",DevStatCtrl.raw);
    platform_write ("Device Status and Control tpend : %x\n",DevStatCtrl.tpend);
    platform_write ("Device Status and Control auxPwr : %x\n",DevStatCtrl.auxPwr);
    platform_write ("Device Status and Control rqDet : %x\n",DevStatCtrl.rqDet);
    platform_write ("Device Status and Control fatalEr : %x\n",DevStatCtrl.fatalEr);
    platform_write ("Device Status and Control nFatalEr : %x\n",DevStatCtrl.nFatalEr);
    platform_write ("Device Status and Control corrEr : %x\n",DevStatCtrl.corrEr);
    platform_write ("Device Status and Control maxSz : %x\n",DevStatCtrl.maxSz);
    platform_write ("Device Status and Control noSnoop : %x\n",DevStatCtrl.noSnoop);
    platform_write ("Device Status and Control auxPwrEn : %x\n",DevStatCtrl.auxPwrEn);
    platform_write ("Device Status and Control phantomEn : %x\n",DevStatCtrl.phantomEn);
    platform_write ("Device Status and Control xtagEn : %x\n",DevStatCtrl.xtagEn);
    platform_write ("Device Status and Control maxPayld : %x\n",DevStatCtrl.maxPayld);
    platform_write ("Device Status and Control relaxed : %x\n",DevStatCtrl.relaxed);
    platform_write ("Device Status and Control reqRp : %x\n",DevStatCtrl.reqRp);
    platform_write ("Device Status and Control fatalErRp : %x\n",DevStatCtrl.fatalErRp);
    platform_write ("Device Status and Control nFatalErRp: %x\n",DevStatCtrl.nFatalErRp);
    platform_write ("Device Status and Control corErRp : %x\n",DevStatCtrl.corErRp);
    }

    //#5 Link Capabilities
    cycleDelay(100);
    getRegs.linkCap = &LinkCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Link Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("Link Capabilities portNum : %x\n",LinkCap.portNum);
    platform_write ("Link Capabilities bwNotifyCap : %x\n",LinkCap.bwNotifyCap);
    platform_write ("Link Capabilities dllRepCap : %x\n",LinkCap.dllRepCap);
    platform_write ("Link Capabilities downErrRepCap: %x\n",LinkCap.downErrRepCap);
    platform_write ("Link Capabilities clkPwrMgmt : %x\n",LinkCap.clkPwrMgmt);
    platform_write ("Link Capabilities l1ExitLat : %x\n",LinkCap.l1ExitLat);
    platform_write ("Link Capabilities losExitLat : %x\n",LinkCap.losExitLat);
    platform_write ("Link Capabilities asLinkPm : %x\n",LinkCap.asLinkPm);
    platform_write ("Link Capabilities maxLinkWidth : %x\n",LinkCap.maxLinkWidth);
    platform_write ("Link Capabilities maxLinkSpeed : %x\n",LinkCap.maxLinkSpeed);
    }

    //#6 Link Status and Control
    cycleDelay(100);
    getRegs.linkStatCtrl = &LinkStatCtrl;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Link Status and Control Reg failed!\n");
    }
    else
    {
    platform_write ("Link Status and Control linkBwStatus : %x\n",LinkStatCtrl.linkBwStatus);
    platform_write ("Link Status and Control linkBwMgmtStatus: %x\n",LinkStatCtrl.linkBwMgmtStatus);
    platform_write ("Link Status and Control dllActive : %x\n",LinkStatCtrl.dllActive);
    platform_write ("Link Status and Control slotClkCfg : %x\n",LinkStatCtrl.slotClkCfg);
    platform_write ("Link Status and Control linkTraining : %x\n",LinkStatCtrl.linkTraining);
    platform_write ("Link Status and Control undef : %x\n",LinkStatCtrl.undef);
    platform_write ("Link Status and Control negotiatedLinkWd: %x\n",LinkStatCtrl.negotiatedLinkWd);
    platform_write ("Link Status and Control linkSpeed : %x\n",LinkStatCtrl.linkSpeed);
    platform_write ("Link Status and Control linkBwIntEn : %x\n",LinkStatCtrl.linkBwIntEn);
    platform_write ("Link Status and Control linkBwMgmtIntEn : %x\n",LinkStatCtrl.linkBwMgmtIntEn);
    platform_write ("Link Status and Control hwAutoWidthDis : %x\n",LinkStatCtrl.hwAutoWidthDis);
    platform_write ("Link Status and Control clkPwrMgmtEn : %x\n",LinkStatCtrl.clkPwrMgmtEn);
    platform_write ("Link Status and Control extSync : %x\n",LinkStatCtrl.extSync);
    platform_write ("Link Status and Control commonClkCfg : %x\n",LinkStatCtrl.commonClkCfg);
    platform_write ("Link Status and Control retrainLink : %x\n",LinkStatCtrl.retrainLink);
    platform_write ("Link Status and Control linkDisable : %x\n",LinkStatCtrl.linkDisable);
    platform_write ("Link Status and Control rcb : %x\n",LinkStatCtrl.rcb);
    platform_write ("Link Status and Control activeLinkPm : %x\n",LinkStatCtrl.activeLinkPm);
    }

    //#7 Link Status and Control
    cycleDelay(100);
    getRegs.linkCtrl2 = &LinkCtrl2;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Link Control Reg2 failed!\n");
    }
    else
    {
    platform_write ("Link Control Reg2 deEmp : %x\n",LinkCtrl2.deEmph);
    platform_write ("Link Control Reg2 pollDeemph : %x\n",LinkCtrl2.pollDeemph);
    platform_write ("Link Control Reg2 cmplSos : %x\n",LinkCtrl2.cmplSos);
    platform_write ("Link Control Reg2 entrModCompl : %x\n",LinkCtrl2.entrModCompl);
    platform_write ("Link Control Reg2 txMargin : %x\n",LinkCtrl2.txMargin);
    platform_write ("Link Control Reg2 selDeemph : %x\n",LinkCtrl2.selDeemph);
    platform_write ("Link Control Reg2 hwAutoSpeedDis: %x\n",LinkCtrl2.hwAutoSpeedDis);
    platform_write ("Link Control Reg2 entrCompl : %x\n",LinkCtrl2.entrCompl);
    platform_write ("Link Control Reg2 tgtSpeed : %x\n",LinkCtrl2.tgtSpeed);
    }

    //#8 SerDes Cfg Reg0
    cycleDelay(100);
    getRegs.serdesCfg0 = &serdesCfg0;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Read SerDes Cfg Reg0 failed!\n");
    }
    else
    {
    platform_write ("Read SerDes Cfg Reg0 txLoopback: %x\n",serdesCfg0.txLoopback);
    platform_write ("Read SerDes Cfg Reg0 txMsync : %x\n",serdesCfg0.txMsync);
    platform_write ("Read SerDes Cfg Reg0 txCm : %x\n",serdesCfg0.txCm);
    platform_write ("Read SerDes Cfg Reg0 txInvpair : %x\n",serdesCfg0.txInvpair);
    platform_write ("Read SerDes Cfg Reg0 rxLoopback: %x\n",serdesCfg0.rxLoopback);
    platform_write ("Read SerDes Cfg Reg0 rxEnoc : %x\n",serdesCfg0.rxEnoc);
    platform_write ("Read SerDes Cfg Reg0 rxEq : %x\n",serdesCfg0.rxEq);
    platform_write ("Read SerDes Cfg Reg0 rxCdr : %x\n",serdesCfg0.rxCdr);
    platform_write ("Read SerDes Cfg Reg0 rxLos : %x\n",serdesCfg0.rxLos);
    platform_write ("Read SerDes Cfg Reg0 rxAlign : %x\n",serdesCfg0.rxAlign);
    platform_write ("Read SerDes Cfg Reg0 rxInvpair : %x\n",serdesCfg0.rxInvpair);
    }

    //#9 SerDes Cfg Reg1
    cycleDelay(100);
    getRegs.serdesCfg1 = &serdesCfg1;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Read SerDes Cfg Reg1 failed!\n");
    }
    else
    {
    platform_write ("Read SerDes Cfg Reg1 txLoopback: %x\n",serdesCfg1.txLoopback);
    platform_write ("Read SerDes Cfg Reg1 txMsync : %x\n",serdesCfg1.txMsync);
    platform_write ("Read SerDes Cfg Reg1 txCm : %x\n",serdesCfg1.txCm);
    platform_write ("Read SerDes Cfg Reg1 txInvpair : %x\n",serdesCfg1.txInvpair);
    platform_write ("Read SerDes Cfg Reg1 rxLoopback: %x\n",serdesCfg1.rxLoopback);
    platform_write ("Read SerDes Cfg Reg1 rxEnoc : %x\n",serdesCfg1.rxEnoc);
    platform_write ("Read SerDes Cfg Reg1 rxEq : %x\n",serdesCfg1.rxEq);
    platform_write ("Read SerDes Cfg Reg1 rxCdr : %x\n",serdesCfg1.rxCdr);
    platform_write ("Read SerDes Cfg Reg1 rxLos : %x\n",serdesCfg1.rxLos);
    platform_write ("Read SerDes Cfg Reg1 rxAlign : %x\n",serdesCfg1.rxAlign);
    platform_write ("Read SerDes Cfg Reg1 rxInvpair : %x\n",serdesCfg1.rxInvpair);
    }

    //#10 Debug Reg0
    cycleDelay(100);
    getRegs.debug0 = &debug0;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Read Debug Reg0 failed!\n");
    }
    else
    {
    platform_write ("Read Debug Reg0 tsLnkCtrl : %x\n",debug0.tsLnkCtrl);
    platform_write ("Read Debug Reg0 tsLaneK237 : %x\n",debug0.tsLaneK237);
    platform_write ("Read Debug Reg0 tsLinkK237 : %x\n",debug0.tsLinkK237);
    platform_write ("Read Debug Reg0 rcvdIdle0 : %x\n",debug0.rcvdIdle0);
    platform_write ("Read Debug Reg0 rcvdIdle1 : %x\n",debug0.rcvdIdle1);
    platform_write ("Read Debug Reg0 pipeTxData : %x\n",debug0.pipeTxData);
    platform_write ("Read Debug Reg0 pipeTxDataK: %x\n",debug0.pipeTxDataK);
    platform_write ("Read Debug Reg0 skipTx : %x\n",debug0.skipTx);
    platform_write ("Read Debug Reg0 ltssmState : %x\n",debug0.ltssmState);
    }

    //#11 Debug Reg1
    cycleDelay(100);
    getRegs.debug1 = &debug1;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Read Debug Reg1 failed!\n");
    }
    else
    {
    platform_write ("Read Debug Reg1 scramblerDisable: %x\n",debug1.scramblerDisable);
    platform_write ("Read Debug Reg1 linkDisable : %x\n",debug1.linkDisable);
    platform_write ("Read Debug Reg1 linkInTraining : %x\n",debug1.linkInTraining);
    platform_write ("Read Debug Reg1 rcvrRevrsPolEn : %x\n",debug1.rcvrRevrsPolEn);
    platform_write ("Read Debug Reg1 trainingRstN : %x\n",debug1.trainingRstN);
    platform_write ("Read Debug Reg1 pipeTxdetectrxLb: %x\n",debug1.pipeTxdetectrxLb);
    platform_write ("Read Debug Reg1 pipeTxelecidle : %x\n",debug1.pipeTxelecidle);
    platform_write ("Read Debug Reg1 pipeTxcompliance: %x\n",debug1.pipeTxcompliance);
    platform_write ("Read Debug Reg1 appInitRst : %x\n",debug1.appInitRst);
    platform_write ("Read Debug Reg1 rmlhTsLinkNum : %x\n",debug1.rmlhTsLinkNum);
    platform_write ("Read Debug Reg1 xmlhLinkUp : %x\n",debug1.xmlhLinkUp);
    platform_write ("Read Debug Reg1 rmlhInskipRcv : %x\n",debug1.rmlhInskipRcv);
    platform_write ("Read Debug Reg1 rmlhTs1Rcvd : %x\n",debug1.rmlhTs1Rcvd);
    platform_write ("Read Debug Reg1 rmlhTs2Rcvd : %x\n",debug1.rmlhTs2Rcvd);
    platform_write ("Read Debug Reg1 rmlhRcvdLaneRev : %x\n",debug1.rmlhRcvdLaneRev);
    }

    //#12 Capabilities Reg
    cycleDelay(100);
    getRegs.pciesCap = &pciesCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("Capabilities Reg intMsg : %x\n",pciesCap.intMsg);
    platform_write ("Capabilities Reg sltImplN : %x\n",pciesCap.sltImplN);
    platform_write ("Capabilities Reg dportType: %x\n",pciesCap.dportType);
    platform_write ("Capabilities Reg pcieCap : %x\n",pciesCap.pcieCap);
    platform_write ("Capabilities Reg nextCap : %x\n",pciesCap.nextCap);
    platform_write ("Capabilities Reg capId : %x\n",pciesCap.capId);
    }

    //#13 Device Capabilities Reg
    cycleDelay(100);
    getRegs.deviceCap = &deviceCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Device Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("Device Capabilities Reg raw : %x\n",deviceCap.raw);
    platform_write ("Device Capabilities Reg pwrLimitScale: %x\n",deviceCap.pwrLimitScale);
    platform_write ("Device Capabilities Reg pwrLimitValue: %x\n",deviceCap.pwrLimitValue);
    platform_write ("Device Capabilities Reg errRpt : %x\n",deviceCap.errRpt);
    platform_write ("Device Capabilities Reg l1Latency : %x\n",deviceCap.l1Latency);
    platform_write ("Device Capabilities Reg l0Latency : %x\n",deviceCap.l0Latency);
    platform_write ("Device Capabilities Reg extTagFld : %x\n",deviceCap.extTagFld);
    platform_write ("Device Capabilities Reg phantomFld : %x\n",deviceCap.phantomFld);
    platform_write ("Device Capabilities Reg maxPayldSz : %x\n",deviceCap.maxPayldSz);
    }

    //#14 Latency Timer and Bus Number
    cycleDelay(100);
    getRegs.type1BusNum = &type1BusNum;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Latency Timer and Bus Number Reg failed!\n");
    }
    else
    {
    platform_write ("Latency Timer and Bus Number Reg secLatTmr: %x\n",type1BusNum.secLatTmr);
    platform_write ("Latency Timer and Bus Number Reg subBusNum: %x\n",type1BusNum.subBusNum);
    platform_write ("Latency Timer and Bus Number Reg secBusNum: %x\n",type1BusNum.secBusNum);
    platform_write ("Latency Timer and Bus Number Reg priBusNum: %x\n",type1BusNum.priBusNum);
    }

    //#15 Secondary Status and IO space
    cycleDelay(100);
    getRegs.type1SecStat = &type1SecStat;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Secondary Status and IO space Reg failed!\n");
    }
    else
    {
    platform_write ("Secondary Status and IO space Reg dtctPError : %x\n",type1SecStat.dtctPError);
    platform_write ("Secondary Status and IO space Reg rxSysError : %x\n",type1SecStat.rxSysError);
    platform_write ("Secondary Status and IO space Reg rxMstAbort : %x\n",type1SecStat.rxMstAbort);
    platform_write ("Secondary Status and IO space Reg rxTgtAbort : %x\n",type1SecStat.rxTgtAbort);
    platform_write ("Secondary Status and IO space Reg txTgtAbort : %x\n",type1SecStat.txTgtAbort);
    platform_write ("Secondary Status and IO space Reg mstDPErr : %x\n",type1SecStat.mstDPErr);
    platform_write ("Secondary Status and IO space Reg IOLimit : %x\n",type1SecStat.IOLimit);
    platform_write ("Secondary Status and IO space Reg IOLimitAddr: %x\n",type1SecStat.IOLimitAddr);
    platform_write ("Secondary Status and IO space Reg IOBase : %x\n",type1SecStat.IOBase);
    platform_write ("Secondary Status and IO space Reg IOBaseAddr : %x\n",type1SecStat.IOBaseAddr);
    }

    //#16 Memory Limit
    cycleDelay(100);
    getRegs.type1Memspace = &type1Memspace;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Memory Limit Reg failed!\n");
    }
    else
    {
    platform_write ("Memory Limit Reg limit : %x\n",type1Memspace.limit);
    platform_write ("Memory Limit Reg base : %x\n",type1Memspace.base);
    }

    //#17 Prefetch Memory Limit and Base
    cycleDelay(100);
    getRegs.prefMem = &prefMem;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Prefetch Memory Limit and Base Reg failed!\n");
    }
    else
    {
    platform_write ("Prefetch Memory Limit and Base Reg limit : %x\n",prefMem.limit);
    platform_write ("Prefetch Memory Limit and Base Reg limitAddr: %x\n",prefMem.limitAddr);
    platform_write ("Prefetch Memory Limit and Base Reg base : %x\n",prefMem.base);
    platform_write ("Prefetch Memory Limit and Base Reg baseAddr : %x\n",prefMem.baseAddr);
    }

    //#18 Prefetch Memory Base Upper
    cycleDelay(100);
    getRegs.prefBaseUpper = &prefBaseUpper;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Prefetch Memory Base Upper Reg failed!\n");
    }
    else
    {
    platform_write ("Prefetch Memory Base Upper Reg base : %x\n",prefBaseUpper.base);

    }

    //#19 Prefetch Memory Limit Upper
    cycleDelay(100);
    getRegs.prefLimitUpper = &prefLimitUpper;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Prefetch Memory Limit Upper Reg failed!\n");
    }
    else
    {
    platform_write ("Prefetch Memory Limit Upper limit : %x\n",prefLimitUpper.limit);

    }
    //#20 IO Base and Limit Upper 16 bits
    cycleDelay(100);
    getRegs.type1IOSpace = &type1IOSpace;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("IO Base and Limit Upper 16 bits Reg failed!\n");
    }
    else
    {
    platform_write ("IO Base and Limit Upper 16 bits Reg IOBase : %x\n",type1IOSpace.IOBase);
    platform_write ("IO Base and Limit Upper 16 bits Reg IOLimit: %x\n",type1IOSpace.IOLimit);
    }


    platform_write ("--------------------------\n");
    platform_write ("Read EP Regs!\n");
    //#1 Vendor & Device
    cycleDelay(100);
    getRegs.vndDevId = &VndDev;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Read Vendor and device ID Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Read Vendor : %x Device ID: %x\n",VndDev.vndId,VndDev.devId);
    }

    //#2 sub ID
    cycleDelay(100);
    getRegs.subId = &subId;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - sub ID Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Read subId raw : %x\n",subId.raw);
    }

    //#3 Rev ID Reg
    cycleDelay(100);
    getRegs.revId = &revId;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Rev ID Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Rev ID Reg classCode: %x\n",revId.classCode);
    platform_write ("EP - Rev ID Reg revId : %x\n",revId.revId);
    }

    //#4 Link Capabilities
    cycleDelay(100);
    getRegs.linkCap = &LinkCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Link Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Link Capabilities portNum : %x\n",LinkCap.portNum);
    platform_write ("EP - Link Capabilities bwNotifyCap : %x\n",LinkCap.bwNotifyCap);
    platform_write ("EP - Link Capabilities dllRepCap : %x\n",LinkCap.dllRepCap);
    platform_write ("EP - Link Capabilities downErrRepCap: %x\n",LinkCap.downErrRepCap);
    platform_write ("EP - Link Capabilities clkPwrMgmt : %x\n",LinkCap.clkPwrMgmt);
    platform_write ("EP - Link Capabilities l1ExitLat : %x\n",LinkCap.l1ExitLat);
    platform_write ("EP - Link Capabilities losExitLat : %x\n",LinkCap.losExitLat);
    platform_write ("EP - Link Capabilities asLinkPm : %x\n",LinkCap.asLinkPm);
    platform_write ("EP - Link Capabilities maxLinkWidth : %x\n",LinkCap.maxLinkWidth);
    platform_write ("EP - Link Capabilities maxLinkSpeed : %x\n",LinkCap.maxLinkSpeed);
    }

    //#5 Capabilities Reg
    cycleDelay(100);
    getRegs.pciesCap = &pciesCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Capabilities Reg raw : %x\n",pciesCap.raw);
    platform_write ("EP - Capabilities Reg intMsg : %x\n",pciesCap.intMsg);
    platform_write ("EP - Capabilities Reg sltImplN : %x\n",pciesCap.sltImplN);
    platform_write ("EP - Capabilities Reg dportType: %x\n",pciesCap.dportType);
    platform_write ("EP - Capabilities Reg pcieCap : %x\n",pciesCap.pcieCap);
    platform_write ("EP - Capabilities Reg nextCap : %x\n",pciesCap.nextCap);
    platform_write ("EP - Capabilities Reg capId : %x\n",pciesCap.capId);
    }

    //#6 Device Capabilities Reg
    cycleDelay(100);
    getRegs.deviceCap = &deviceCap;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Device Capabilities Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Device Capabilities Reg raw : %x\n",deviceCap.raw);
    platform_write ("EP - Device Capabilities Reg pwrLimitScale: %x\n",deviceCap.pwrLimitScale);
    platform_write ("EP - Device Capabilities Reg pwrLimitValue: %x\n",deviceCap.pwrLimitValue);
    platform_write ("EP - Device Capabilities Reg errRpt : %x\n",deviceCap.errRpt);
    platform_write ("EP - Device Capabilities Reg l1Latency : %x\n",deviceCap.l1Latency);
    platform_write ("EP - Device Capabilities Reg l0Latency : %x\n",deviceCap.l0Latency);
    platform_write ("EP - Device Capabilities Reg extTagFld : %x\n",deviceCap.extTagFld);
    platform_write ("EP - Device Capabilities Reg phantomFld : %x\n",deviceCap.phantomFld);
    platform_write ("EP - Device Capabilities Reg maxPayldSz : %x\n",deviceCap.maxPayldSz);
    }

    //#7 Latency Timer and Bus Number
    cycleDelay(100);
    getRegs.type1BusNum = &type1BusNum;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Latency Timer and Bus Number Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Latency Timer and Bus Number Reg secLatTmr: %x\n",type1BusNum.secLatTmr);
    platform_write ("EP - Latency Timer and Bus Number Reg subBusNum: %x\n",type1BusNum.subBusNum);
    platform_write ("EP - Latency Timer and Bus Number Reg secBusNum: %x\n",type1BusNum.secBusNum);
    platform_write ("EP - Latency Timer and Bus Number Reg priBusNum: %x\n",type1BusNum.priBusNum);
    }

    //#8 Secondary Status and IO space
    cycleDelay(100);
    getRegs.type1SecStat = &type1SecStat;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Secondary Status and IO space Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Secondary Status and IO space Reg dtctPError : %x\n",type1SecStat.dtctPError);
    platform_write ("EP - Secondary Status and IO space Reg rxSysError : %x\n",type1SecStat.rxSysError);
    platform_write ("EP - Secondary Status and IO space Reg rxMstAbort : %x\n",type1SecStat.rxMstAbort);
    platform_write ("EP - Secondary Status and IO space Reg rxTgtAbort : %x\n",type1SecStat.rxTgtAbort);
    platform_write ("EP - Secondary Status and IO space Reg txTgtAbort : %x\n",type1SecStat.txTgtAbort);
    platform_write ("EP - Secondary Status and IO space Reg mstDPErr : %x\n",type1SecStat.mstDPErr);
    platform_write ("EP - Secondary Status and IO space Reg IOLimit : %x\n",type1SecStat.IOLimit);
    platform_write ("EP - Secondary Status and IO space Reg IOLimitAddr: %x\n",type1SecStat.IOLimitAddr);
    platform_write ("EP - Secondary Status and IO space Reg IOBase : %x\n",type1SecStat.IOBase);
    platform_write ("EP - Secondary Status and IO space Reg IOBaseAddr : %x\n",type1SecStat.IOBaseAddr);
    }

    //#9 Memory Limit
    cycleDelay(100);
    getRegs.type1Memspace = &type1Memspace;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Memory Limit Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Memory Limit Reg limit : %x\n",type1Memspace.limit);
    platform_write ("EP - Memory Limit Reg base : %x\n",type1Memspace.base);
    }

    //#10 Prefetch Memory Limit and Base
    cycleDelay(100);
    getRegs.prefMem = &prefMem;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Prefetch Memory Limit and Base Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Prefetch Memory Limit and Base Reg limit : %x\n",prefMem.limit);
    platform_write ("EP - Prefetch Memory Limit and Base Reg limitAddr: %x\n",prefMem.limitAddr);
    platform_write ("EP - Prefetch Memory Limit and Base Reg base : %x\n",prefMem.base);
    platform_write ("EP - Prefetch Memory Limit and Base Reg baseAddr : %x\n",prefMem.baseAddr);
    }

    //#11 Prefetch Memory Base Upper
    cycleDelay(100);
    getRegs.prefBaseUpper = &prefBaseUpper;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Prefetch Memory Base Upper Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Prefetch Memory Base Upper Reg base : %x\n",prefBaseUpper.base);

    }

    //#12 Prefetch Memory Limit Upper
    cycleDelay(100);
    getRegs.prefLimitUpper = &prefLimitUpper;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Prefetch Memory Limit Upper Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Prefetch Memory Limit Upper limit : %x\n",prefLimitUpper.limit);

    }
    //#13 IO Base and Limit Upper 16 bits
    cycleDelay(100);
    getRegs.type1IOSpace = &type1IOSpace;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - IO Base and Limit Upper 16 bits Reg failed!\n");
    }
    else
    {
    platform_write ("EP - IO Base and Limit Upper 16 bits Reg IOBase : %x\n",type1IOSpace.IOBase);
    platform_write ("EP - IO Base and Limit Upper 16 bits Reg IOLimit: %x\n",type1IOSpace.IOLimit);
    }
    //#14 Status Command Reg
    cycleDelay(100);
    getRegs.statusCmd = &statusCmd;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Status Command Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Status Command Reg raw : %x\n",statusCmd.raw);
    platform_write ("EP - Status Command Reg parity : %x\n",statusCmd.parity);
    platform_write ("EP - Status Command Reg sysError : %x\n",statusCmd.sysError);
    platform_write ("EP - Status Command Reg mstAbort : %x\n",statusCmd.mstAbort);
    platform_write ("EP - Status Command Reg tgtAbort : %x\n",statusCmd.tgtAbort);
    platform_write ("EP - Status Command Reg sigTgtAbort: %x\n",statusCmd.sigTgtAbort);
    platform_write ("EP - Status Command Reg parError : %x\n",statusCmd.parError);
    platform_write ("EP - Status Command Reg capList : %x\n",statusCmd.capList);
    platform_write ("EP - Status Command Reg stat : %x\n",statusCmd.stat);
    platform_write ("EP - Status Command Reg dis : %x\n",statusCmd.dis);
    platform_write ("EP - Status Command Reg serrEn : %x\n",statusCmd.serrEn);
    platform_write ("EP - Status Command Reg resp : %x\n",statusCmd.resp);
    platform_write ("EP - Status Command Reg busMs : %x\n",statusCmd.busMs);
    platform_write ("EP - Status Command Reg memSp : %x\n",statusCmd.memSp);
    platform_write ("EP - Status Command Reg ioSp : %x\n",statusCmd.ioSp);
    }

    //#15 Device Status and Control
    cycleDelay(100);
    getRegs.devStatCtrl = &DevStatCtrl;
    if (Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs) != pcie_RET_OK)
    {
    platform_write ("EP - Device Status and Control Reg failed!\n");
    }
    else
    {
    platform_write ("EP - Device Status and Control raw : %x\n",DevStatCtrl.raw);
    platform_write ("EP - Device Status and Control tpend : %x\n",DevStatCtrl.tpend);
    platform_write ("EP - Device Status and Control auxPwr : %x\n",DevStatCtrl.auxPwr);
    platform_write ("EP - Device Status and Control rqDet : %x\n",DevStatCtrl.rqDet);
    platform_write ("EP - Device Status and Control fatalEr : %x\n",DevStatCtrl.fatalEr);
    platform_write ("EP - Device Status and Control nFatalEr : %x\n",DevStatCtrl.nFatalEr);
    platform_write ("EP - Device Status and Control corrEr : %x\n",DevStatCtrl.corrEr);
    platform_write ("EP - Device Status and Control maxSz : %x\n",DevStatCtrl.maxSz);
    platform_write ("EP - Device Status and Control noSnoop : %x\n",DevStatCtrl.noSnoop);
    platform_write ("EP - Device Status and Control auxPwrEn : %x\n",DevStatCtrl.auxPwrEn);
    platform_write ("EP - Device Status and Control phantomEn : %x\n",DevStatCtrl.phantomEn);
    platform_write ("EP - Device Status and Control xtagEn : %x\n",DevStatCtrl.xtagEn);
    platform_write ("EP - Device Status and Control maxPayld : %x\n",DevStatCtrl.maxPayld);
    platform_write ("EP - Device Status and Control relaxed : %x\n",DevStatCtrl.relaxed);
    platform_write ("EP - Device Status and Control reqRp : %x\n",DevStatCtrl.reqRp);
    platform_write ("EP - Device Status and Control fatalErRp : %x\n",DevStatCtrl.fatalErRp);
    platform_write ("EP - Device Status and Control nFatalErRp: %x\n",DevStatCtrl.nFatalErRp);
    platform_write ("EP - Device Status and Control corErRp : %x\n",DevStatCtrl.corErRp);
    }

    //#16 Root Error Status
    cycleDelay(100);
    getRegs.rootErrSt = &RootErrStatus;
    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Root Error Status Reg failed!\n");
    }
    else
    {
    platform_write ("Root Error Status aerIntMsg: %x\n",RootErrStatus.aerIntMsg);
    platform_write ("Root Error Status ferrRcv : %x\n",RootErrStatus.ferrRcv);
    platform_write ("Root Error Status nfErr : %x\n",RootErrStatus.nfErr);
    platform_write ("Root Error Status uncorFata: %x\n",RootErrStatus.uncorFatal);
    platform_write ("Root Error Status multFnf : %x\n",RootErrStatus.multFnf);
    platform_write ("Root Error Status errFnf : %x\n",RootErrStatus.errFnf);
    platform_write ("Root Error Status multCor : %x\n",RootErrStatus.multCor);
    platform_write ("Root Error Status corrErr : %x\n",RootErrStatus.corrErr);
    }
    }
    /*****************************************************************************
    * Function: Check LTSSM status and wait for the link to be up
    ****************************************************************************/
    void pcieWaitLinkUp(Pcie_Handle handle)
    {
    pcieRootErrStReg_t RootErrStatus;
    pcieLnkCtrlReg_t lnkCtrl; /**< @brief Port Link Control*/
    pcieRstCmdReg_t rstCmd;
    pcieRegisters_t getRegs;
    pcieRegisters_t setRegs;

    pcieStatusCmdReg_t statusCmd;

    pcieDebug0Reg_t debug0; /**< @brief Debug 0*/
    pcieDebug1Reg_t debug1; /**< @brief Debug 1 Register*/


    pcieType0Bar32bitIdx_t type0Bar32bitIdx;
    pcieCmdStatusReg_t cmdStatus;

    memset (&type0Bar32bitIdx, 0, sizeof(type0Bar32bitIdx));
    memset (&debug0, 0, sizeof(debug0));
    memset (&debug1, 0, sizeof(debug1));
    memset (&rstCmd, 0, sizeof(rstCmd));
    memset (&lnkCtrl, 0, sizeof(lnkCtrl));
    memset (&cmdStatus, 0, sizeof(cmdStatus));
    memset (&RootErrStatus, 0, sizeof( RootErrStatus));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&setRegs, 0, sizeof(setRegs));

    uint8_t ltssmState = 0;

    while(ltssmState != pcie_LTSSM_L0)
    {
    cycleDelay(100);
    // Enable link training

    getRegs.debug0 = &debug0;
    getRegs.debug1 = &debug1;
    getRegs.rstCmd = &rstCmd;
    getRegs.lnkCtrl = &lnkCtrl;
    getRegs.cmdStatus = &cmdStatus;
    getRegs.rootErrSt = &RootErrStatus;


    if (Pcie_readRegs (handle, pcie_LOCATION_LOCAL, &getRegs) != pcie_RET_OK)
    {
    platform_write ("Read LTSSM state failed!\n");
    return;
    }
    ltssmState = debug0.ltssmState;
    //if (ltssmState != pcie_LTSSM_L0)
    //{
    // rstCmd.initRst = 1;
    // lnkCtrl.rstAsrt = 1;
    // setRegs.rstCmd = &rstCmd;
    // setRegs.lnkCtrl = &lnkCtrl;
    // if (Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs) != pcie_RET_OK)
    // {
    // platform_write ("SET OB_SIZE register failed!\n");
    // return;
    // }
    // cycleDelay(200);
    // if (pcieLtssmCtrl(handle, TRUE) != pcie_RET_OK)
    // {
    // platform_write ("Failed to Enable Link Training!\n");
    // return;
    // }
    //}


    platform_write ("Read Command Stat ltssmEn : %x\n",cmdStatus.ltssmEn);
    platform_write ("Read Debug Reg0 tsLnkCtrl : %x\n",debug0.tsLnkCtrl);
    platform_write ("Read Debug Reg0 tsLaneK237 : %x\n",debug0.tsLaneK237);
    platform_write ("Read Debug Reg0 tsLinkK237 : %x\n",debug0.tsLinkK237);
    platform_write ("Read Debug Reg0 rcvdIdle0 : %x\n",debug0.rcvdIdle0);
    platform_write ("Read Debug Reg0 rcvdIdle1 : %x\n",debug0.rcvdIdle1);
    platform_write ("Read Debug Reg0 pipeTxData : %x\n",debug0.pipeTxData);
    platform_write ("Read Debug Reg0 pipeTxDataK: %x\n",debug0.pipeTxDataK);
    platform_write ("Read Debug Reg0 skipTx : %x\n",debug0.skipTx);
    platform_write ("Read Debug Reg0 ltssmState : %x\n",debug0.ltssmState);

    platform_write ("Read Debug Reg1 scramblerDisable: %x\n",debug1.scramblerDisable);
    platform_write ("Read Debug Reg1 linkDisable : %x\n",debug1.linkDisable);
    platform_write ("Read Debug Reg1 linkInTraining : %x\n",debug1.linkInTraining);
    platform_write ("Read Debug Reg1 rcvrRevrsPolEn : %x\n",debug1.rcvrRevrsPolEn);
    platform_write ("Read Debug Reg1 trainingRstN : %x\n",debug1.trainingRstN);
    platform_write ("Read Debug Reg1 pipeTxdetectrxLb: %x\n",debug1.pipeTxdetectrxLb);
    platform_write ("Read Debug Reg1 pipeTxelecidle : %x\n",debug1.pipeTxelecidle);
    platform_write ("Read Debug Reg1 pipeTxcompliance: %x\n",debug1.pipeTxcompliance);
    platform_write ("Read Debug Reg1 appInitRst : %x\n",debug1.appInitRst);
    platform_write ("Read Debug Reg1 rmlhTsLinkNum : %x\n",debug1.rmlhTsLinkNum);
    platform_write ("Read Debug Reg1 xmlhLinkUp : %x\n",debug1.xmlhLinkUp);
    platform_write ("Read Debug Reg1 rmlhInskipRcv : %x\n",debug1.rmlhInskipRcv);
    platform_write ("Read Debug Reg1 rmlhTs1Rcvd : %x\n",debug1.rmlhTs1Rcvd);
    platform_write ("Read Debug Reg1 rmlhTs2Rcvd : %x\n",debug1.rmlhTs2Rcvd);
    platform_write ("Read Debug Reg1 rmlhRcvdLaneRev : %x\n",debug1.rmlhRcvdLaneRev); \
    pcie_serdes_status = PCIE_SERDES_STS;
    platform_write ("pcie_serdes_status : %x\n",pcie_serdes_status);
    platform_write ("Read Root Error Stat aerIntMsg : %x\n",RootErrStatus.aerIntMsg);
    platform_write ("Read Root Error Stat ferrRcv : %x\n",RootErrStatus.ferrRcv);
    platform_write ("Read Root Error Stat nfErr : %x\n",RootErrStatus.nfErr);
    platform_write ("Read Root Error Stat uncorFatal : %x\n",RootErrStatus.uncorFatal);
    platform_write ("Read Root Error Stat multFnf : %x\n",RootErrStatus.multFnf);
    platform_write ("Read Root Error Stat errFnf : %x\n",RootErrStatus.errFnf);
    platform_write ("Read Root Error Stat multCor : %x\n",RootErrStatus.multCor);
    platform_write ("Read Root Error Stat corrErr : %x\n",RootErrStatus.corrErr);
    }

    //************** Set EP Mem Access Enable ***********
    /* Enable memory access and mastership of the bus */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));

    getRegs.statusCmd = &statusCmd;
    if ((Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("Read Status Comand register failed!\n");
    }
    statusCmd.memSp = 1;
    statusCmd.busMs = 0;
    statusCmd.resp = 1;
    statusCmd.serrEn = 1;
    setRegs.statusCmd = &statusCmd;

    if ((Pcie_writeRegs (handle, pcie_LOCATION_REMOTE, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("SET Status Command register failed!\n");
    }
    //************** End - EP Mem Access Enable *********

    //****************************** Configure BAR0 on EP **********************************
    /* Configure BAR Masks */
    /* First need to enable writing on BAR mask registers */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    getRegs.cmdStatus = &cmdStatus;
    if ((Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("EP - Read CMD STATUS register failed!\n");
    }
    cmdStatus.dbi = 1;
    setRegs.cmdStatus = &cmdStatus;

    if ((Pcie_writeRegs (handle, pcie_LOCATION_REMOTE, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("EP - SET CMD STATUS register failed!\n");
    }

    /* BAR 0 */
    memset (&setRegs, 0, sizeof(setRegs));
    type0Bar32bitIdx.idx = 0; /* configure BAR 0*/
    type0Bar32bitIdx.reg.reg32 = PCIE_BAR_MASK1;
    setRegs.type0Bar32bitIdx = &type0Bar32bitIdx;

    if ((Pcie_writeRegs (handle, pcie_LOCATION_REMOTE, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("EP - SET BAR MASK register failed!\n");
    }

    /* Disable writing on BAR Masks */
    memset (&setRegs, 0, sizeof(setRegs));
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&cmdStatus, 0, sizeof(cmdStatus));

    getRegs.cmdStatus = &cmdStatus;
    if ((Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &getRegs)) != pcie_RET_OK)
    {
    platform_write ("EP - Read CMD STATUS register failed!\n");
    }
    cmdStatus.dbi = 0;
    setRegs.cmdStatus = &cmdStatus;

    if ((Pcie_writeRegs (handle, pcie_LOCATION_REMOTE, &setRegs)) != pcie_RET_OK)
    {
    platform_write ("EP - SET CMD STATUS register failed!\n");
    }
    //****************************** End- Configure BAR0 on EP *****************************

    pcie_register_debug(handle);

    }

    /*****************************************************************************
    * Function: Converts a core local L2 address to a global L2 address
    * Input addr: L2 address to be converted to global.
    * return: uint32_t Global L2 address
    *****************************************************************************/
    uint32_t pcieConvert_CoreLocal2GlobalAddr (uint32_t addr)
    {
    uint32_t coreNum;

    /* Get the core number. */
    coreNum = CSL_chipReadReg(CSL_CHIP_DNUM);

    /* Compute the global address. */
    return ((1 << 28) | (coreNum << 24) | (addr & 0x00ffffff));
    }

    void PCIE_Setup(void)
    {
    TSCL = 1;
    uint16_t lock=0;
    pcieRet_e retVal;
    pcieIbTransCfg_t ibCfg;
    pcieBarCfg_t barCfg;
    Pcie_Handle handle = NULL;



    platform_write ("**********************************************\n");
    platform_write ("* PCIE RC Mode Setup *\n");
    platform_write ("**********************************************\n\n");

    platform_write ("Version #: 0x%08x; string %s\n\n", Pcie_getVersion(), Pcie_getVersionStr());

    // Initialize application buffers
    pcieInitAppBuf();

    // Configure SERDES
    if ((retVal = pcieSerdesCfg()) != pcie_RET_OK)
    {
    platform_write ("PCIe Serdes config failed (%d)\n", (int)retVal);
    exit(1);
    }

    // Set the PCIe mode
    if ((retVal = Pcie_setMode(PcieModeGbl)) != pcie_RET_OK)
    {
    platform_write ("Set PCIe Mode failed (%d)\n", (int)retVal);
    exit(1);
    }

    // Power up PCIe Module
    if ((retVal = pciePowerCfg()) != pcie_RET_OK)
    {
    platform_write ("PCIe Power Up failed (%d)\n", (int)retVal);
    exit(1);
    }

    platform_write ("PCIe Power Up.\n");

    // Wait until the PCIe SERDES PLL locks
    while (!lock)
    {
    CSL_BootCfgGetPCIEPLLLock(&lock);
    }

    if ((retVal = Pcie_open(0, &handle)) != pcie_RET_OK)
    {
    platform_write ("Open failed (%d)\n", (int)retVal);
    exit(1);
    }

    platform_write ("PLL configured.\n");

    // Configure application registers for Root Complex
    if ((retVal = pcieCfgRC(handle)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure PCIe in RC mode (%d)\n", (int)retVal);
    exit(1);
    }

    // Configure Address Translation

    barCfg.location = pcie_LOCATION_LOCAL;
    barCfg.mode = pcie_RC_MODE;
    barCfg.base = PCIE_IB_LO_ADDR_M;
    barCfg.prefetch = pcie_BAR_NON_PREF;
    barCfg.type = pcie_BAR_TYPE32;
    barCfg.memSpace = pcie_BAR_MEM_MEM;
    barCfg.idx = PCIE_BAR_IDX_M;

    if ((retVal = Pcie_cfgBar(handle, &barCfg)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure BAR (%d)\n", (int)retVal);
    exit(1);
    }

    ibCfg.ibBar = PCIE_BAR_IDX_M; // Match BAR that was configured above
    ibCfg.ibStartAddrLo = PCIE_IB_LO_ADDR_M | 1;
    ibCfg.ibStartAddrHi = PCIE_IB_HI_ADDR_M;
    ibCfg.ibOffsetAddr = (uint32_t)pcieConvert_CoreLocal2GlobalAddr ((uint32_t)dstBuf.buf);
    ibCfg.region = PCIE_IB_REGION_M;

    if ((retVal = pcieIbTransCfg(handle, &ibCfg)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure Inbound Translation (%d)\n", (int)retVal);
    exit(1);
    }
    else
    {
    platform_write ("Successfully configured Inbound Translation!\n");
    }

    if ((retVal = pcieObTransCfg (handle, PCIE_OB_LO_ADDR_M, PCIE_OB_HI_ADDR_M, PCIE_OB_REGION_M)) != pcie_RET_OK)
    {
    platform_write ("Failed to configure Outbound Address Translation (%d)\n", (int)retVal);
    exit(1);
    }
    else
    {
    platform_write ("Successfully configured Outbound Translation!\n");
    }


    platform_write ("Starting link training...\n");

    // Enable link training
    if ((retVal = pcieLtssmCtrl(handle, TRUE)) != pcie_RET_OK)
    {
    platform_write ("Failed to Enable Link Training!\n", (int)retVal);
    exit(1);
    }

    // Wait for link to be up
    dev_status = DEVSTAT;
    pcieWaitLinkUp(handle);

    platform_write ("Link is up.\n");


    // **********************************************************************
    // * Push a single message to the EP then verify that it is echoed back *
    // **********************************************************************

    // Write from RC to EP
    if ((retVal = Pcie_getMemSpaceRange (handle, &pcieBase, NULL)) != pcie_RET_OK)
    {
    platform_write ("getMemSpaceRange failed\n", (int)retVal);
    exit(1);
    }

    //*******************************************************************************************
    //*********** PCI Express Memory allocation for sample buffers and commands ****************
    //*******************************************************************************************
    pcieBase0_V1 = (uint32_t *)pcieBase + 0*PCIE_N;
    pcieBase1_V1 = ((uint32_t *)pcieBase) + 1*PCIE_N;
    pcieBase0_I1 = ((uint32_t *)pcieBase) + 2*PCIE_N;
    pcieBase1_I1 = ((uint32_t *)pcieBase) + 3*PCIE_N;

    pcieBase0_V2 = ((uint32_t *)pcieBase) + 4*PCIE_N;
    pcieBase1_V2 = ((uint32_t *)pcieBase) + 5*PCIE_N;
    pcieBase0_I2 = ((uint32_t *)pcieBase) + 6*PCIE_N;
    pcieBase1_I2 = ((uint32_t *)pcieBase) + 7*PCIE_N;

    pcieBase0_V3 = ((uint32_t *)pcieBase) + 8*PCIE_N;
    pcieBase1_V3 = ((uint32_t *)pcieBase) + 9*PCIE_N;
    pcieBase0_I3 = ((uint32_t *)pcieBase) + 10*PCIE_N;
    pcieBase1_I3 = ((uint32_t *)pcieBase) + 11*PCIE_N;

    pcieTest0 = (((uint32_t *)pcieBase) + PCIE_N -1);
    pcieTest1 = (((uint32_t *)pcieBase) + 2*PCIE_N - 1);
    pcieBase2 = ((uint32_t *)pcieBase) + 1024;
    DELAY3;
    }
    //*****************************************************************************
    //******************************* End PCIe Structures *************************
    //*****************************************************************************
  • Hi,

    I have reviewed your code, it should be fine for PCIe initialization. You using TI provide MCSDK code(RC part) for your testing.

    Have you run this example via CCS? Please share the debug log message.

    Thanks,
  • Ganapahti,

    See debug captures below. They were continually read out over a period of time looking for change...

    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : f7
    Read Debug Reg0 pipeTxDataK: 1
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 84
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : bc
    Read Debug Reg0 pipeTxDataK: 1
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 73
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : f7
    Read Debug Reg0 pipeTxDataK: 1
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 51
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : bc
    Read Debug Reg0 pipeTxDataK: 1
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : e0
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 6a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 209
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : f9
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : c1
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : f7
    Read Debug Reg0 pipeTxDataK: 1
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 0
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 7e
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 6
    Read Debug Reg1 scramblerDisable: 0
    Read Debug Reg1 linkDisable : 0
    Read Debug Reg1 linkInTraining : 0
    Read Debug Reg1 rcvrRevrsPolEn : 0
    Read Debug Reg1 trainingRstN : 1
    Read Debug Reg1 pipeTxdetectrxLb: 0
    Read Debug Reg1 pipeTxelecidle : 1
    Read Debug Reg1 pipeTxcompliance: 0
    Read Debug Reg1 appInitRst : 0
    Read Debug Reg1 rmlhTsLinkNum : 0
    Read Debug Reg1 xmlhLinkUp : 0
    Read Debug Reg1 rmlhInskipRcv : 0
    Read Debug Reg1 rmlhTs1Rcvd : 0
    Read Debug Reg1 rmlhTs2Rcvd : 0
    Read Debug Reg1 rmlhRcvdLaneRev : 0
    pcie_serdes_status : 201
    Read Root Error Stat aerIntMsg : 0
    Read Root Error Stat ferrRcv : 0
    Read Root Error Stat nfErr : 0
    Read Root Error Stat uncorFatal : 0
    Read Root Error Stat multFnf : 0
    Read Root Error Stat errFnf : 0
    Read Root Error Stat multCor : 0
    Read Root Error Stat corrErr : 0
    Read Command Stat ltssmEn : 1
    Read Debug Reg0 tsLnkCtrl : 0
    Read Debug Reg0 tsLaneK237 : 0
    Read Debug Reg0 tsLinkK237 : 0
    Read Debug Reg0 rcvdIdle0 : 0
    Read Debug Reg0 rcvdIdle1 : 0
    Read Debug Reg0 pipeTxData : 4a
    Read Debug Reg0 pipeTxDataK: 0
    Read Debug Reg0 skipTx : 0
    Read Debug Reg0 ltssmState : 2
  • Hi,

    On your custom board debug log, LTSSM_STATE bits in DEBUG0 register keep switching between POLL_ACTIVE (0x2) and DETECT_WAIT (0x6) state but never attains the desired value of L0 (0x11).

    It means the PCIe link is not detected, please check your custom board hardware connection.

    The same issue find in alpha release of Edison EVM (it is labelled as v1.0.2.1), After the hardware changes the issue is resolved. For more information refer below thread.
    e2e.ti.com/.../1290181

    Thanks,
  • Ganapathi,

    As you had noted above, our software/configuration looks good for our PCIe implementation. Also, it does look like Edison EVM had similiar problem to ours but I am not familiar with this hardware platform and exactly what they have changed. It looks like they have connected RESETFULLZ which we already have connected in our design. Also, not sure what R511 and R512 are for in their design. Could you elaborate and/or point me to this schematic? so that i can see exactly what they have changed and if their is any relevance to what we are doing in our custom design...

    Remove "R510" resistor and connect "SOC_RESETFULLZ" signal from U43 IC pin# 6 to U75 IC (SATA IC) Pin# 45>
    Need to Replace R511 and R512 with 100nF capacitor?
  • Hi,

    Yes, your software/configuration looks good for PCIe implementation. Software not a issue on your setup.

    I have provide the K2E EVM thread for just reference purpose only, I am not confirm the same hardware issue on your custom board.

    For my understanding, The PCIe link is not detected properly on your custom board, so please check your custom board hardware connection with the help of your hardware team.

    Thanks,

  • Ganaphathi,

    I am the sole hardware team on this design. I had TI do a schematic check before releasing our 1st Proto Brd for FAB and any minor issues pointed out were addressed. I would like to have DSP BU re-review PCIe portion of our custom design to double check. It is crucial that we get this interface working on our board as we are getting ready for a quick re-spin. Could you get a conference call setup with DSP BU to discuss our design.
  • Hi,

    Your custom board schematic reviews are not supported on the E2E forum. Please contact your local Local Field Applications Engineers (FAEs.) for support of these check. If you are not sure who your local FAE is, then please contact your local technical sales representative and they will be able to put you in contact with your local FAE.

    Thanks,
  • Hi,

    Any update. Could you please help us to close this thread.

    Thanks,