This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
Other Parts Discussed in Thread: TMS320C6678
Hello,
I am trying to connect two eval6678 through a MICROTCA backplane, loading the PCIe example program and configuring one board as root complex and the other one as endpoint. The Endpoint succeeds to execute the link training phase but the root complex stops during the link training looping on this. The clock is feed through the switch of the backplane (NAT-MCH). Coul you have any suggestions please?
Regards,
Massimiliano
Massimiliano,
I assume you already configure the SW5[3] to 1 (OFF) on both of the 6678 EVMs to provide the common clock from backplane to the RC and EP PCIe, is it correct?
May I ask which PCIe example program you are testing please? Is it the PCIe LLD project in the PDK package (C:\ti\pdk_C6678_1_1_2_5\packages\ti\drv\exampleProjects\PCIE_exampleProject)?
If so, you should configure the two EVMs in no boot mode when plugging into the backplane, since the LLD example will setup the PCIe module.
And you built two test cases with PCIe mode to be RC (PcieModeGbl = pcie_RC_MODE) and EP (PcieModeGbl = pcie_EP_MODE) and loaded to the individual EVM, is it correct?
On EP side, you said it succeeded in link training, do you see the "Link is up." on EP side please?
On RC side, do you mean the it is stuck at the following place in the test case?
/* Wait for link to be up */
pcieWaitLinkUp(handle);
Could you check the DEBUG0 (0x21801728) register on both RC and EP devices to see what the LTSSM_STATE field looks like please?
The link training should be completed if both RC and EP reach the L0 state (LTSSM_STATE=0x11 in DEBUG0 register).
Hi Steven,
I configured SW5[3] to 1 (OFF) on both the 6678EVM. The example is in C:\ti\pdk_C6678_1_1_2_5\packages\ti\drv\exampleProjects\PCIE_exampleProject.
I configured two EVMs in no boot mode.
All SWs are ON excluding SW5[3] and SW3[1] that are OFF.
On EP side I can see "Link is up", see below:
**********************************************
* PCIe Test Start *
* EP mode *
**********************************************
Version #: 0x01000003; string PCIE LLD Revision: 01.00.00.03:Nov 5 2012:14:53:41
Power domain is already enabled. You probably re-ran without device reset (which is OK)
PCIe Power Up.
PLL configured.
Successfully configured Inbound Translation!
Successfully configured Outbound Translation!
Starting link training...
LTSSM state: 0x1
LTSSM state: 0x11
Link is up.
ON RC I obtained:
**********************************************
* PCIe Test Start *
* RC mode *
**********************************************
Version #: 0x01000003; string PCIE LLD Revision: 01.00.00.03:Nov 5 2012:14:53:41
Power domain is already enabled. You probably re-ran without device reset (which is OK)
PCIe Power Up.
PLL configured.
Successfully configured Inbound Translation!
Successfully configured Outbound Translation!
Starting link training...
LTSSM state: 0x1
LTSSM state: 0x2
LTSSM state: 0x7
LTSSM state: 0x7
LTSSM state: 0x2
LTSSM state: 0x7
LTSSM state: 0x7
LTSSM state: 0x2
LTSSM state: 0x7
I have the following configuration:
Crate: ELMA MicroTCA Blu!Box.
Switch: NAT-MCH PCIe configured as follow:
mch
INFO - MCH configuration CFG V2.12 for V2.12 Final (11:54:22 Mar 21 2012) ok
MCH global parameter:
---------------------
remote interfaces:
RMCP access: enabled
telnet access: enabled
WEB access: enabled
IP address source Mgmt: DHCP
RMCP session activity timeout minutes: 0 min
RMCP session activity timeout seconds: 60 sec
default fan level: 30 percent
MCH configuration flags:
Enable backward compatibility V2.4: no
Enable alternative cooling scheme: no
Control rear transition module fans: no
Allow SRIO 6.25GBaud/s Connections: no
Shelf manager parameter:
------------------------
ShM configuration flags:
allow shelf FRU invalid: yes
temperature management: enabled
emergency shutdown on critical ev.: disabled
emergency shutdown on non recov ev.: disabled
use external shelf manager: no
Carrier manager parameter:
--------------------------
carrier number default: 0
quiesced event timeout: 10 sec
CM configuration flags:
allow carrier FRU invalid: yes
overrule carrier FRU: no
shutdown system if MCH goes down: no
enable Clock E-keying: no
CM debug:
IPMI: disabled
FRU: disabled
E-keying: disabled
sensor: disabled
event: disabled
power module: disabled
cooling unit: disabled
CM/ShM interface: disabled
debugging FRU: 0
SEL parameter:
--------------
SEL configuration flags:
'keep on read': disabled
allocate SEL in non-volatile RAM: yes
ignore 'version change' sensor: yes
Ethernet switch parameter:
--------------------------
configuration source: no configuration
ignore backplane FRU info: no
CLK module parameter:
---------------------
CLK module configuration source: no configuration
PCIe parameter:
---------------
operating mode: default (transparent)
upstream transparent slot number: 2
upstream non-transp slot number: 2
upstream slot power up delay: 5 sec
PCIe configuration flags:
100 MHz spread spectrum: disabled
hot plug support: disabled
PCIe early ekey (before payload): disabled
PCIe clustering: disabled
NTP parameter:
--------------
NTP server IP: 0.0.0.0
NTP 'check for time' delay minutes: 0 min
NTP 'check for time' delay hours: 0 h
NTP local time offset: 0 h
NTP configuration flags:
NTP client: disabled
DHCP Configuration:
-------------------
host name: nat0
flags: 0x00
nat>
nat>
nat> show_fru
FRU Information:
----------------
FRU Device State Name
==========================================
0 MCH M4 NMCH-CM
3 mcmc1 M4 NAT-MCH-MCMC
6 AMC2 M4 TI-LC_EVM
9 AMC5 M4 TI-LC_EVM
41 CU2 M4 Cooling Unit
50 PM1 M4 Ericsson AB PM
60 Clk1 M4 MCH-Clock
61 Hub1 M4 MCH-PCIe
==========================================
nat>
nat>
nat>
nat> show_fruinfo 6
---------------------------------------
FRU Info for device 6:
---------------------------------------
Common Header : 0x01 0x00 0x00 0x0e 0x00 0x1e 0x00 0xd3
---------------------------------------
Internal Use Area : -
---------------------------------------
Chassis Info Area : -
---------------------------------------
Board Info Area : at offs=112, len=48
Manufacturer(17) : Texas Instruments
Board Name(06) : TI EVM
Serial Number(08) : Part Number(01) : ---------------------------------------
Product Info Area : -
---------------------------------------
Multi Record Area : at offs=240
Record(0): Type ID=0xc0, PICMG Record ID=0x19, offset=0x000, len=208
AMC Point-to-Point record:
AMC Slot 2, OEM GUID Count = 1
Record Type = AMC, len=208
Channel Descriptor count = 3
Channel(0): Port[0 1 - -]
Channel(1): Port[8 9 10 11]
Channel(2): Port[4 5 - -]
Link Descriptors: size=170
Link 0 of Channel 0: lanes[0..3]=[1000], Eth , 1000Base-BX, Grp=0x0, Match=0x0
Link 1 of Channel 0: lanes[0..3]=[0100], Eth , 1000Base-BX, Grp=0x0, Match=0x0
Link 2 of Channel 1: lanes[0..3]=[1111], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 3 of Channel 1: lanes[0..3]=[1111], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 4 of Channel 1: lanes[0..3]=[1111], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 5 of Channel 1: lanes[0..3]=[1111], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 6 of Channel 1: lanes[0..3]=[1100], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 7 of Channel 1: lanes[0..3]=[1100], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 8 of Channel 1: lanes[0..3]=[1100], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 9 of Channel 1: lanes[0..3]=[1100], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 10 of Channel 1: lanes[0..3]=[0011], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 11 of Channel 1: lanes[0..3]=[0011], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 12 of Channel 1: lanes[0..3]=[0011], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 13 of Channel 1: lanes[0..3]=[0011], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 14 of Channel 1: lanes[0..3]=[1000], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 15 of Channel 1: lanes[0..3]=[1000], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 16 of Channel 1: lanes[0..3]=[1000], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 17 of Channel 1: lanes[0..3]=[1000], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 18 of Channel 1: lanes[0..3]=[0100], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 19 of Channel 1: lanes[0..3]=[0100], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 20 of Channel 1: lanes[0..3]=[0100], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 21 of Channel 1: lanes[0..3]=[0100], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 22 of Channel 1: lanes[0..3]=[0010], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 23 of Channel 1: lanes[0..3]=[0010], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 24 of Channel 1: lanes[0..3]=[0010], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 25 of Channel 1: lanes[0..3]=[0010], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 26 of Channel 1: lanes[0..3]=[0001], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 27 of Channel 1: lanes[0..3]=[0001], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 28 of Channel 1: lanes[0..3]=[0001], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 29 of Channel 1: lanes[0..3]=[0001], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 30 of Channel 2: lanes[0..3]=[1100], PCIe, Gen 2, no SSC, Grp=0x0, Match=0x1
Link 31 of Channel 2: lanes[0..3]=[1100], PCIe, Gen 1, no SSC, Grp=0x0, Match=0x1
Link 32 of Channel 2: lanes[0..3]=[1000], PCIe, Gen 2, no SSC, Grp=0x0, Match=0x1
Link 33 of Channel 2: lanes[0..3]=[1000], PCIe, Gen 1, no SSC, Grp=0x0, Match=0x1
Record(1): Type ID=0xc0, PICMG Record ID=0x16, offset=0x0d0, len=11
Module Current Requirements Record:
Current Draw: 3.0 A
Record(2): Type ID=0xc0, PICMG Record ID=0x00, offset=0x0db, len=10
Header=buf 0x41d3c33b len 5
c0 82 05 75 44
Data =buf 0x41d3c340 len 5
5a 31 00 00 00
---------------------------------------
nat> show_fruinfo 6 9
---------------------------------------
FRU Info for device 9:
---------------------------------------
Common Header : 0x01 0x00 0x00 0x0e 0x00 0x1e 0x00 0xd3
---------------------------------------
Internal Use Area : -
---------------------------------------
Chassis Info Area : -
---------------------------------------
Board Info Area : at offs=112, len=48
Manufacturer(17) : Texas Instruments
Board Name(06) : TI EVM
Serial Number(08) : Part Number(01) : ---------------------------------------
Product Info Area : -
---------------------------------------
Multi Record Area : at offs=240
Record(0): Type ID=0xc0, PICMG Record ID=0x19, offset=0x000, len=208
AMC Point-to-Point record:
AMC Slot 5, OEM GUID Count = 1
Record Type = AMC, len=208
Channel Descriptor count = 3
Channel(0): Port[0 1 - -]
Channel(1): Port[8 9 10 11]
Channel(2): Port[4 5 - -]
Link Descriptors: size=170
Link 0 of Channel 0: lanes[0..3]=[1000], Eth , 1000Base-BX, Grp=0x0, Match=0x0
Link 1 of Channel 0: lanes[0..3]=[0100], Eth , 1000Base-BX, Grp=0x0, Match=0x0
Link 2 of Channel 1: lanes[0..3]=[1111], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 3 of Channel 1: lanes[0..3]=[1111], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 4 of Channel 1: lanes[0..3]=[1111], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 5 of Channel 1: lanes[0..3]=[1111], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 6 of Channel 1: lanes[0..3]=[1100], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 7 of Channel 1: lanes[0..3]=[1100], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 8 of Channel 1: lanes[0..3]=[1100], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 9 of Channel 1: lanes[0..3]=[1100], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 10 of Channel 1: lanes[0..3]=[0011], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 11 of Channel 1: lanes[0..3]=[0011], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 12 of Channel 1: lanes[0..3]=[0011], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 13 of Channel 1: lanes[0..3]=[0011], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 14 of Channel 1: lanes[0..3]=[1000], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 15 of Channel 1: lanes[0..3]=[1000], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 16 of Channel 1: lanes[0..3]=[1000], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 17 of Channel 1: lanes[0..3]=[1000], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 18 of Channel 1: lanes[0..3]=[0100], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 19 of Channel 1: lanes[0..3]=[0100], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 20 of Channel 1: lanes[0..3]=[0100], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 21 of Channel 1: lanes[0..3]=[0100], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 22 of Channel 1: lanes[0..3]=[0010], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 23 of Channel 1: lanes[0..3]=[0010], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 24 of Channel 1: lanes[0..3]=[0010], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 25 of Channel 1: lanes[0..3]=[0010], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 26 of Channel 1: lanes[0..3]=[0001], SRIO, 5.0 GBaud, Grp=0x0, Match=0x0
Link 27 of Channel 1: lanes[0..3]=[0001], SRIO, 3.125 GBaud, Grp=0x0, Match=0x0
Link 28 of Channel 1: lanes[0..3]=[0001], SRIO, 2.5 GBaud, Grp=0x0, Match=0x0
Link 29 of Channel 1: lanes[0..3]=[0001], SRIO, 1.25 GBaud, Grp=0x0, Match=0x0
Link 30 of Channel 2: lanes[0..3]=[1100], PCIe, Gen 2, no SSC, Grp=0x0, Match=0x1
Link 31 of Channel 2: lanes[0..3]=[1100], PCIe, Gen 1, no SSC, Grp=0x0, Match=0x1
Link 32 of Channel 2: lanes[0..3]=[1000], PCIe, Gen 2, no SSC, Grp=0x0, Match=0x1
Link 33 of Channel 2: lanes[0..3]=[1000], PCIe, Gen 1, no SSC, Grp=0x0, Match=0x1
Record(1): Type ID=0xc0, PICMG Record ID=0x16, offset=0x0d0, len=11
Module Current Requirements Record:
Current Draw: 3.0 A
Record(2): Type ID=0xc0, PICMG Record ID=0x00, offset=0x0db, len=10
Header=buf 0x41d4283f len 5
c0 82 05 75 44
Data =buf 0x41d42844 len 5
5a 31 00 00 00
---------------------------------------
nat>
nat>
nat> show_ekey
EKeying information - activated Links:
--------------------------------------
AMC FRU State Channel Type Port
=================================================
AMC2 6 M4 0 Eth 0 <-> MCH1 Fabric A 1000Base-BX
AMC5 9 M4 0 Eth 0 <-> MCH1 Fabric A 1000Base-BX
2 PCIe 4 <-> MCH1 Fabric D downstream Gen 2, no SSC
5 <-> MCH1 Fabric E downstream Gen 2, no SSC
=================================================
Any suggestions to solve the problem??
Regards
Massimiliano
Massimiliano,
For the NAT-MCH backplane, you are referring to the following one?
http://www.nateurope.com/products/amc/nat-mch-modules.htm
If so, there is one PCIe switch on it. And the RC should be plugged into the "Upstream" port and the EP should be plugged into the "Downstream" port.
It might be you plug both RC and EP into the Downstream ports that only EP could get link up.
So please check your backplane and PCIe switch data manual to make sure if the switch has been configured correctly and if RC is connected to Upstream port of the switch.
Hi Steven,
Yes, I refer to NAT-MCH - Mezzanine Modules with option PCIe.
So far as in my understanding the RC is already in the Upstream port since I set "upstream transparent slot number: 2" (2 means AMC 2 that is the slot where the RC is plugged) option in the PCIe parameter:
PCIe parameter: --------------- operating mode: default (transparent) upstream transparent slot number: 2 upstream non-transp slot number: 2 upstream slot power up delay: 5 sec PCIe configuration flags: 100 MHz spread spectrum: disabled hot plug support: disabled PCIe early ekey (before payload): disabled PCIe clustering: disabled
Any other suggestions please?
Regards
Massimiliano
Hi Steven,
Now both PCIe links are up. I modified the "Asymmetric Both" field in the MMC code of the RC. However RC and EP cannot exchange data. The RC complex software have to be modified to enumerate the bus where now the switch is present. I am following this forum: http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/193398.aspx?pi70912=2
Regards
Massimiliano Bitossi