C6678 PCIe GEN2 problem

user4489731

Hello，

I‘m using C6678 dsp and PCIe interface to communicating with DM8168,C6678 work as EP and DM8168 as RC.The data I want to transfer in DM8168 DDR,and outbound to C6678 PCIe data space（the address is 0x60000000）,then use EDMA to move these data to C6678 DDR（the address is 0x90000000）.

I have tested the transfer rate，only 352MB/S,less than expect（about 650MB/s）.I have checke LINK_STAT_CTRL register,the value is 0x10120080,in other words,the value of NEGOTIATED_LINK_WD field is 0x1,only enable one lane.But I think I have set up GEN2 mode,the value of another relevant register:

LINK_CTRL:0x30120

Link_CAP:0x35422

PL_GEN2:0x20F(before link up,I set the DIR_SPD=1,but it is not change in memory view)

can you help me to analyze why only enable one lane?

Regards,

simon,

over 9 years ago

0 Raja over 9 years ago

TI__Guru* 81335 points

Welcome to the TI E2E forum. I hope you will find many good answers here and in the TI.com documents and in the TI Wiki Pages (for processor issues). Be sure to search those for helpful information and to browse for the questions others may have asked on similar topics (e2e.ti.com). Please read all the links below my signature.

We will get back to you on the above query shortly. Thank you for your patience.

Thank you.

PS: We strongly recommend you to create new e2e thread for your queries instead of following up on an old/closed e2e thread, new threads gets more attention than old threads and can provide link of old threads or information on the new post for clarity and faster response.

0 Titusrathinaraj Stalin over 9 years ago

TI__Guru** 116100 points

Dear Simon,
Which example are you working ?
If its C6678 PCIe example, then please change the following line to run with 2 lanes.

C:\ti\pdk_C6678_1_1_2_6\packages\ti\drv\exampleProjects\PCIE_exampleProject
C:\ti\pdk_C6678_1_1_2_6\packages\ti\drv\pcie\example\sample\pcie_sample.c

/* Setting PL_GEN2 */
memset (&setRegs, 0, sizeof(setRegs));
gen2.numFts = 0xF;
gen2.dirSpd = 0x0;
gen2.lnEn = 2; //Titus : 2 lanes enabled
setRegs.gen2 = &gen2;

0 user4489731 over 9 years ago in reply to Titusrathinaraj Stalin

Intellectual 890 points

Hi,Stalin,

thanks for your answer.

I have tried the project you suggested,and checked the value of register.

when set gen2.InEn=1,the value of PL_GEN2 is 0x10F,LINK_STAT_CTRL is 0x10120080;

when set gen2.InEn=2,the value of PL_GEN2 is 0x20F,LINK_STAT_CTRL is 0x10120080;

so you can see,when change the value of gen2.InEn,the value of PL_GEN2 is chenged,but the value of LINK_STAT_CTRL not change.

Do you think the problem is related to DM8168？DM8168 also have PCIe register.

Regards,

Simon

0 lding over 9 years ago in reply to user4489731

TI__Guru* 95265 points

Simon,

DM8168 uses the same PCIE IP as the C6678. You need to check the same registers setting on DM8168 side as well:

PL_LINK_CONTROL
LINK_CAP
PL_GEN2

http://www.ti.com/lit/ug/sprugx8c/sprugx8c.pdf

Regards, Eric

0 user4489731 over 9 years ago in reply to lding

Intellectual 890 points

Hi,Eric,

I have checked the value of registers setting on DM8168，the value is：

PL_LINK_CTRL:0x30120(only one lane)

PL_GEN2:0x20F（enable GEN2）

LINK_CAP:0x135422(because it's work as RC mode)

I'm very confused,Why I enabled two lane,but the value of PL_LINK_CTRL show the PCIe mode is GEN1?

Regards,Simon

0 lding over 9 years ago in reply to user4489731

TI__Guru* 95265 points

Simon,

The discussion mixed up the # of lanes and GEN1/2. GEN1/2 is related to speed of either 2.5G or 5.0Gbps. The # of lanes can be 1 or 2. The problem you asked is why you only get 1 lane instead of 2 lanes. This has nothing to do with GEN1 or 2.

From the register settings, all the required to enable x2 lane are set. How the DM8168 is connected to C6678? With above setting, do you have DM8168 or C6678 worked with another remote successfully for x2?

Regards, Eric

0 user4489731 over 9 years ago in reply to lding

Intellectual 890 points

Hi，Eric

Today，I checked the value of LINK_STAT_CTRL register in DM8168，it is 0x30120008；and the value of LINK_STAT_CTRL register in C6678 is 0x10120080；can we get some other useful information about this problem from these two value?

I checked above all registers after link training complete,link training can be successfully.I don't have test DM8168 or C6678 worked with another remote.

Regards,

Simon

0 lding over 9 years ago in reply to user4489731

TI__Guru* 95265 points

Simon,

How the PCIE reference clock setup? Does each chip have it is own reference clock? Or the clock is passed from one to the other? Who is RC and who is EP? Is there any PCIE switch in between? Is this test bewteen TI EVM or some customized board?

We tested C6678 EVM ----PCIE----C6678 EVM in the past, we have all 4 combinations (x1/x2 lanes, GEN1/2) work as expected. DM8168 has the same PCIE IP as C6678, the registers controling x1/x2 lanes looks good.

Regards, Eric

0 user4489731 over 9 years ago in reply to lding

Intellectual 890 points

Hi,Eric,

the clock is passed from DM8168 to C6678,C6678 is EP and DM8168 is RC,There isn't PCIE switch in between.It is DM8168 EVM——PCIE——C6678 EVM.

Maybe,This problem is caused by the referenc clock.Where can I check the PCIE clock set up?in DM8168 u-boot or kernel？what‘s value should be set about PCIE reference clock for lane x2？100MHZ?

Thanks for your help.

Regards,

Simon

0 user4489731 over 9 years ago in reply to lding

Intellectual 890 points

Hi,

I get some information after input "lspci- vvv",As shown below:

00:00.0 Class 0604: Device 104c:b800 (rev 01)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Region 0: Memory at <ignored> (32-bit, non-prefetchable)
        Region 1: Memory at <ignored> (32-bit, prefetchable)
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: 21800000-218fffff
        Prefetchable memory behind bridge: 20000000-217fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000 Data: 0000
        Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s <2us, L1 <64us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

01:00.0 Class 0480: Device 104c:b005 (rev 01)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 48
        Region 0: Memory at 21800000 (32-bit, non-prefetchable) [size=1M]
        Region 1: Memory at 21400000 (32-bit, prefetchable) [size=512K]
        Region 2: Memory at 21000000 (32-bit, prefetchable) [size=4M]
        Region 3: Memory at 20000000 (32-bit, prefetchable) [size=16M]
        Region 4: Memory at 21490000 (32-bit, prefetchable) [size=4K]
        Region 5: Memory at 21480000 (32-bit, prefetchable) [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000 Data: 0000
        Capabilities: [70] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s <2us, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch+ ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn+ ChkCap+ ChkEn+

Can we get some useful information from this？

Best wishes,

Simon

0 lding over 9 years ago in reply to user4489731

TI__Guru* 95265 points

Simon,

PCIE reference clock is 100MHz regardless of x1 or x2 lanes. I am not familiar with DM8168 and not sure how the reference clock is configured.

I tested a Keystone II EVM ---- PCIE breakout card ----- Keystone II EVM connection via breakout card yesterday to verify the PCIE test code. There is no issue to get x2 lane. The breakout card doesn't passing/distributing PCIE clock, so the each EVM uses it is own on-board PCIE reference clock. The Keystone II, Keystone I and DM816x use the same PCIE IP. The PCIE registers configured should be applicable to all three devices.

The register dump here with my setup:

RC:

PL_LINK_CTRL (offset 0x1710): 0x00030120
LINK_CAP (offset 0x107C): 0x00135422
LINK_STAT_CTRL (offset 0x1080): 0x30220008
PL_GEN2 (offset 0x180C): 0x10F

EP:

PL_LINK_CTRL (offset 0x1710): 0x00030120
LINK_CAP (offset 0x107C): 0x00035422
LINK_STAT_CTRL (offset 0x1080): 0x10220000
PL_GEN2 (offset 0x180C): 0x10F

So by defailt LINK_CAP is x2 lanes alreay, in PL_LINK_CTRL you need to enable x2. The configuration in PL_GEN2 seems doesn't matter. In the LINK_STAT_CTRL bit 6 is cleared for seperate reference clock.

How the DM8168 is connected to 6678 EVM, what is this breakout card? How does it pass through reference clock? As 6678 also generates on board clock to PCIE, how do you select which clock is used? Is possible for you to test each EVM uses its own clock?

Regards, Eric

0 user4489731 over 9 years ago in reply to lding

Intellectual 890 points

Hi,Eric,

I‘m not sure because I just participate this project soon.I think after set LINK_STAT_CTRL bit[28] to 1,the description is “This bit indicates that the component uses the same physical reference clock that the platform provides on the connector.”it means C6678 use DM8168 reference clock pass through,right？

And I find some information about DM8168 PCIE reference clock,it's 250MHZ,and C6678 need 100MHZ.If C6678 use PCIE reference clock 250MHZ can impact GEN2?

Thank you very much for your help.

Best wishes,

Simon

Processors

Processors forum

C6678 PCIe GEN2 problem