ti8168 pcie link problem

Hi,

ge zhiming said:

    My design is ti8168(RC)+PEX8717(PCIe switch)+3 dm8168(EP).

    My problem is ti8168(RC) can not scan PEX8717, psp is TI816X-LINUX-PSP-04.00.01.13.

Is the RC EVM or custom board?

Also what is the power up and reset sequence you are using? Is the switch powered up before RC?

Thanks.

   Hemant

Hi,

    my board is custom board.

    RC powered up before switch. no reset operation with switch

Hi,

How is the  ref clock provided to the switch?

Is it possible to try with powering up the switch before RC?

Or if the clock is sourced from RC board, then can you try following:

1) Power up the RC board, so that the 100MHz clock will be available to the switch

2) Power up the switch

3) Apply reset to the DM8168 RC device

For 3rd step, if applying h/w reset to DM8168 is not possible, you can try one of the following:

a) Type 'reboot' at RC kernel prompt OR

b) Toggle reset to PCIe h/w on RC by first clearing bit-7 and then setting it to 1 again in register @0x48180B10 (you can use devmem2 utility at RC kernel prompt)

Of course you need to reinitialize link by setting PCIe register 0x51000004 with 0xa07.

After this, check the register @0x51001728

   Hemant

hi,

We use SL28SRC02BZIT and CY28800OXCT generate 100M clk to DM8168 and switch.

Power up sequence is AVS(1.0 core) , 0.9V(switch),1.5v(ddr3). I can not change the sequence.

I try to give a reset to switch in uboot or reboot kernel, but the switch also can not link up. register 0x51001728 is the same as before

other projects as ti8168+SIL3132 or ti8168+PEX8112, Pcie is OK.

we are now checking the PEX8717 sch.

ge zhiming said:

We use SL28SRC02BZIT and CY28800OXCT generate 100M clk to DM8168 and switch.

So the clocks are independent? Can you confirm none of them use SSC?

ge zhiming said:

Power up sequence is AVS(1.0 core) , 0.9V(switch),1.5v(ddr3). I can not change the sequence.

I was referring to changing power up sequence between RC and switch - power up switch before RC.

ge zhiming said:

I try to give a reset to switch in uboot or reboot kernel, but the switch also can not link up. register 0x51001728 is the same as before

Can you instead try resetting the RC (or PCIe h/w on RC) as mentioned in last post?

ge zhiming said:

other projects as ti8168+SIL3132 or ti8168+PEX8112, Pcie is OK.

we are now checking the PEX8717 sch.

Can you confirm the GEN difference between the PEX devices above? Is the one not working a GEN3 device?

Thanks.  

   Hemant

Hi,

Pcie link is OK, but ep pcie resource map is error, bar1 and bar2 length is 0

ti81xx_pcie_ep: Found TI81xx PCIe EP @0xc5c6f400, DEVICE ID = b800
pci 0000:03:00.0: This driver supports booting the first TI816x or TI814x target found on the bus
pci 0000:03:00.0: Major number 254 assigned
pci 0000:03:00.0: Added device to the sys file system
pci 0000:03:00.0: BAR Configuration -
           Start        |       Length  |       Flags
pci 0000:03:00.0:       0x20000000      |       4096    |       0x00040200
pci 0000:03:00.0:       0x00000000      |       0       |       0x00000000
pci 0000:03:00.0:       0x00000000      |       0       |       0x00000000
pci 0000:03:00.0: TI81XX registers mapped to 0xc681e000
pci 0000:03:00.0: Failed reserve ocmc resource
pci 0000:03:00.0: could not get resources

Thanks

gzm

Hi,

Pcie debug info is:

ti81xx_pcie: Invoking PCI BIOS...
ti81xx_pcie: Setting up Host Controller...
ti81xx_pcie: Register base mapped @0xc6820000
ti81xx_pcie: Starting PCI scan...
PCI: bus0: Fast back to back transfers disabled
PCI: bus1: Fast back to back transfers disabled
PCI: bus2: Fast back to back transfers disabled
pci 0000:03:00.0: Setting PCI class for 81xx PCIe device
PCI: bus3: Fast back to back transfers disabled
pci 0000:04:00.0: Setting PCI class for 81xx PCIe device
PCI: bus4: Fast back to back transfers disabled
PCI: bus5: Fast back to back transfers enabled
PCI: bus6: Fast back to back transfers enabled
PCI: bus7: Fast back to back transfers enabled
pci 0000:00:00.0: BAR 9: can't assign mem pref (size 0x1c000000)
pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x204fffff]
pci 0000:00:00.0: BAR 7: can't assign io (size 0x2000)
pci 0000:01:00.0: BAR 9: can't assign mem pref (size 0x1c000000)
pci 0000:01:00.0: BAR 8: assigned [mem 0x20000000-0x203fffff]
pci 0000:01:00.0: BAR 0: assigned [mem 0x20400000-0x2043ffff]
pci 0000:01:00.0: BAR 0: set to [mem 0x20400000-0x2043ffff] (PCI address [0x20400000-0x2043ffff])
pci 0000:01:00.1: BAR 0: assigned [mem 0x20440000-0x20441fff]
pci 0000:01:00.1: BAR 0: set to [mem 0x20440000-0x20441fff] (PCI address [0x20440000-0x20441fff])
pci 0000:01:00.2: BAR 0: assigned [mem 0x20442000-0x20443fff]
pci 0000:01:00.2: BAR 0: set to [mem 0x20442000-0x20443fff] (PCI address [0x20442000-0x20443fff])
pci 0000:01:00.3: BAR 0: assigned [mem 0x20444000-0x20445fff]
pci 0000:01:00.3: BAR 0: set to [mem 0x20444000-0x20445fff] (PCI address [0x20444000-0x20445fff])
pci 0000:01:00.4: BAR 0: assigned [mem 0x20446000-0x20447fff]
pci 0000:01:00.4: BAR 0: set to [mem 0x20446000-0x20447fff] (PCI address [0x20446000-0x20447fff])
pci 0000:01:00.0: BAR 7: can't assign io (size 0x2000)
pci 0000:02:01.0: BAR 9: can't assign mem pref (size 0x18000000)
pci 0000:02:02.0: BAR 9: can't assign mem pref (size 0x4000000)
pci 0000:02:01.0: BAR 8: assigned [mem 0x20000000-0x201fffff]
pci 0000:02:02.0: BAR 8: assigned [mem 0x20200000-0x203fffff]
pci 0000:02:01.0: BAR 7: can't assign io (size 0x1000)
pci 0000:02:02.0: BAR 7: can't assign io (size 0x1000)
pci 0000:03:00.0: BAR 4: can't assign mem pref (size 0x10000000)
pci 0000:03:00.0: BAR 1: can't assign mem pref (size 0x800000)
pci 0000:03:00.0: BAR 0: assigned [mem 0x20000000-0x20000fff]
pci 0000:03:00.0: BAR 0: set to [mem 0x20000000-0x20000fff] (PCI address [0x20000000-0x20000fff])
pci 0000:02:01.0: PCI bridge to [bus 03-03]
pci 0000:02:01.0:   bridge window [io  disabled]
pci 0000:02:01.0:   bridge window [mem 0x20000000-0x201fffff]
pci 0000:02:01.0:   bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 2: can't assign mem pref (size 0x4000000)
pci 0000:04:00.0: BAR 0: assigned [mem 0x20200000-0x20200fff 64bit]
pci 0000:04:00.0: BAR 0: set to [mem 0x20200000-0x20200fff 64bit] (PCI address [0x20200000-0x20200fff])
pci 0000:02:02.0: PCI bridge to [bus 04-04]
pci 0000:02:02.0:   bridge window [io  disabled]
pci 0000:02:02.0:   bridge window [mem 0x20200000-0x203fffff]
pci 0000:02:02.0:   bridge window [mem pref disabled]
pci 0000:02:03.0: PCI bridge to [bus 05-05]
pci 0000:02:03.0:   bridge window [io  disabled]
pci 0000:02:03.0:   bridge window [mem disabled]
pci 0000:02:03.0:   bridge window [mem pref disabled]
pci 0000:02:04.0: PCI bridge to [bus 06-06]
pci 0000:02:04.0:   bridge window [io  disabled]
pci 0000:02:04.0:   bridge window [mem disabled]
pci 0000:02:04.0:   bridge window [mem pref disabled]
pci 0000:02:05.0: PCI bridge to [bus 07-07]
pci 0000:02:05.0:   bridge window [io  disabled]
pci 0000:02:05.0:   bridge window [mem disabled]
pci 0000:02:05.0:   bridge window [mem pref disabled]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0:   bridge window [io  disabled]
pci 0000:01:00.0:   bridge window [mem 0x20000000-0x203fffff]
pci 0000:01:00.0:   bridge window [mem pref disabled]
pci 0000:00:00.0: PCI bridge to [bus 01-07]
pci 0000:00:00.0:   bridge window [io  disabled]
pci 0000:00:00.0:   bridge window [mem 0x20000000-0x204fffff]
pci 0000:00:00.0:   bridge window [mem pref disabled]
PCI: enabling device 0000:00:00.0 (0140 -> 0143)
PCI: enabling device 0000:01:00.0 (0140 -> 0143)
PCI: enabling device 0000:02:01.0 (0140 -> 0143)
PCI: enabling device 0000:02:02.0 (0140 -> 0143)
PCI: enabling device 0000:02:03.0 (0140 -> 0143)
PCI: enabling device 0000:02:04.0 (0140 -> 0143)
PCI: enabling device 0000:02:05.0 (0140 -> 0143)
pci 0000:00:00.0: BAR 9: assigned [mem 0x20500000-0x208fffff pref]
pci 0000:00:00.0: BAR 7: can't assign io (size 0x2000)
pci 0000:01:00.0: BAR 9: assigned [mem 0x20500000-0x208fffff 64bit pref]
pci 0000:01:00.0: BAR 7: can't assign io (size 0x2000)
pci 0000:02:01.0: BAR 9: assigned [mem 0x20500000-0x206fffff 64bit pref]
pci 0000:02:02.0: BAR 9: assigned [mem 0x20700000-0x208fffff 64bit pref]
pci 0000:02:01.0: BAR 7: can't assign io (size 0x1000)
pci 0000:02:02.0: BAR 7: can't assign io (size 0x1000)
bio: create slab <bio-0> at 0 

Hello,

So the setup is DM816x (RC)  --> PCIe Switch --> DM816x (EP), is this correct?

Can you confirm if the boot pin settings are done as mentioned in http://ap-fpdsp-swapps.dal.design.ti.com/index.php/PCI_Express_Endpoint_Boot_Driver_User_Guide#Setting_up_the_EP ?

More combinations for boot pin settings should be described in TRM.

   Hemant

Hi,

1. ti8168(RC) can scan pex8717 and ti8168(ep),but prefetch memory can not assign

2. we have checked our sch, confirm the boot pin is OK

pci 0000:03:00.0: BAR 4: can't assign mem pref (size 0x10000000)
pci 0000:03:00.0: BAR 1: can't assign mem pref (size 0x800000)
pci 0000:03:00.0: BAR 2: can't assign mem pref (size 0x800000)
pci 0000:03:00.0: BAR 0: assigned [mem 0x20000000-0x20000fff]

gzm

Hi,

Good to see that the resource assignment issue is fixed.

Regarding GEN1, can you check if the switch you are using supports directing the speed change?

In short, generally the link comes up in GEN1 and then the upstream (or downstream) can initiate a speed change depending upon link speeds possible - this can either be done by upstream/downstream h/w automatically or by s/w intervention. In case of DM8168 devices, the s/w needs to configure "speed change" but since we are EP (in this case), if the switch h/w could do this, the link should GEN2.

For example, the DM8168 RC driver does the speed change settings at the initialization time to ensure the links goes to GEN2 when endpoint supports it.

Hemant

Hemant,

Let me make sure I understand what you said here; because that may be the issue.  So in this system, the DM8168 IS the RC; so did I understand you to say that our driver doesn't support dynamically changing the link speed- only can be done at initialization? 

Hi Hemant:

 What does SSC mean as you refers?

We connect two pci devices in follow way

rc-bridge-ep,

I see the refer clock 100M and the power order is key to ensure ep device to be detect,

but i really fuse about this ,what is the details we have to pay attation to ?

 another problem is when  i insmod ti81xx_pcie_epdrv.ko module broken,
and log print as follow:

root@dm816x:/opt/dvr_rdk/ti816x/kermod# insmod ti81xx_pcie_epdrv.ko

[   63.390000] Unhandled fault: external abort on non-linefetch (0x1028) at 0xd3001004

[   63.400000] Internal error: : 1028 [#1]

[   63.400000] last sysfs file: /sys/kernel/uevent_seqnum

[   63.400000] Modules linked in: ti81xx_pcie_epdrv(+)

[   63.400000] CPU: 0    Not tainted  (2.6.37 #8)

[   63.400000] PC is at ti81xx_ep_pcie_init+0x1a8/0x28c [ti81xx_pcie_epdrv]

[   63.400000] LR is at kobject_put+0x48/0x5c

[   63.400000] pc : [<bf000b8c>]    lr : [<c01bbb34>]    psr: 60000013

[   63.400000] sp : cd779e98  ip : cd779db8  fp : cd779ebc

[   63.400000] r10: 0000001c  r9 : 00000013  r8 : 00000000

[   63.400000] r7 : bf000d95  r6 : 00000000  r5 : bf001158  r4 : bf001124

[   63.400000] r3 : d3000000  r2 : d3001000  r1 : c01bbbd0  r0 : bf001124

[   63.400000] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user

[   63.400000] Control: 10c5387d  Table: 8d43c019  DAC: 00000015
[   63.400000] Process insmod (pid: 451, stack limit = 0xcd7782e8)

[   63.400000] Stack: (0xcd779e98 to 0xcd77a000)

[   63.400000] 9e80:                                                       bf000d95 bf001158

[   63.400000] 9ea0: bf001000 00000000 00000001 bf0009e4 cd779ef4 cd779ec0 c00424f8 bf0009f0

[   63.400000] 9ec0: bf001000 00000000 00000001 cd41d240 cd779ef4 bf001000 00000000 00000001

[   63.400000] 9ee0: cd41d240 00000001 cd779fa4 cd779ef8 c00a10d4 c0042448 bf00100c cd779f08

[   63.400000] 9f00: c0042298 c009ebd0 00000000 00012008 bf001108 c039f424 c00de3d8 d08fe000

[   63.400000] 9f20: 00002ca3 d08ffc18 d08ffb5d d09008a8 cd412680 0000119c 0000135c 00000000

[   63.400000] 9f40: 00000000 00000011 00000012 00000009 00000007 00000005 00000000 00000000

[   63.400000] 9f60: 00000000 00000000 00000000 00000000 00000000 c04bc080 cd412c00 00008a2c

[   63.400000] 9f80: 00012008 be889eca 00000080 c004cde8 cd778000 00000000 00000000 cd779fa8

[   63.400000] 9fa0: c004cc40 c009fa14 00008a2c 00012008 00012018 00002ca3 00012008 40247248

[   63.400000] 9fc0: 00008a2c 00012008 be889eca 00000080 00004000 00000000 00000003 00000000

[   63.400000] 9fe0: 00002ca3 be889c4c 00008ca4 401e5684 60000010 00012018 00000000 00000000

[   63.400000] Backtrace:

[   63.400000] [<bf0009e4>] (ti81xx_ep_pcie_init+0x0/0x28c [ti81xx_pcie_epdrv]) from [<c00424f8>] (do_one_initcall+0xbc/0x190)

[   63.400000]  r7:bf0009e4 r6:00000001 r5:00000000 r4:bf001000

[   63.400000] [<c004243c>] (do_one_initcall+0x0/0x190) from [<c00a10d4>] (sys_init_module+0x16cc/0x1894)

[   63.400000]  r8:00000001 r7:cd41d240 r6:00000001 r5:00000000 r4:bf001000

[   63.400000] [<c009fa08>] (sys_init_module+0x0/0x1894) from [<c004cc40>] (ret_fast_syscall+0x0/0x30)

[   63.400000] Code: e5943028 e1a00004 e2832a01 e583618c (e5921004) 
[   63.640000] ---[ end trace f234e815e66268a3 ]---

Segmentation fault

i have set pci_mem=8M in boot params, 
to fix this problem i do insmod ti81xx_pcie_epdrv.ko on ti8168 evm board without any other pcie device connected,same problem also happens then, what is the problem then ?

I think ep should  start before rc board, but before i insmod ti81xx_pcie_epdrv.ko, can ep be found at rc board?
or it have to be after insmod ti81xx_pcie_epdrv.ko?

best regards
xavier