PCIe boot problems with C6678

First, please see the post http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/121758/433978.aspx#433978.  This post gives a pointer to a endpoint mode example.  There are two examples that are similar to your use-case.  First, there is an example (post) which downloads the POST image (power on self test) via PCIe.  This shows how to load a large portion of the L2 (more than 16KB) on a single core.  The second example is the HelloWorld, which downloads to DDR via PCIe, and executes the image on multiple cores.

The linux portion of the code which is in C:\Program Files\Texas Instruments\mcsdk_2_00_00_11\tools\boot_loader\examples\pcie\linux_host_loader after installing the MCSDK  This code programs the inbound address translation registers, and programs IB_BAR1 to point to the L2 in the same way as you are programing 0x80100300.  However, we are using IB_BAR1, not IB_BAR0.  We leave IB_BAR0 pointing at the PCIe registers such that we can program the rest of the IB_BAR registers.

It is important to note that the linux portion only binds to then immediately performs its IO with the 6678 card.  It does not attempt to establish anything in /dev nor define a communication protocol between user mode on linux and the 6678.

The example relies on updating the IBL in the I2C.  The IBL will program a few more registers than the bootrom.  Specifically, it is setting the number of fast training sequences to be a larger number in order to support spread spectrum clocks, or asynchronous (separate) clocks.

Depending on when your EVM was received, it may only support asynchronous (separate) clocks.  If this is the case, then the root complex's clock must NOT be spread spectrum because PCIe requires a common clock when using spread-spectrum.  How are your clocks configured?

John,

We are not using Spread spectrum clocks. Our PCIe clock is running at 125MHz. With this setup, we do not need to update the IBL in the I2C, correct?

The problem of getting target aborts over 16KB is not seen when doing single word PCI writes to all of PCI-mapped L2 SRAM. So, we will try writing to the DDR Init and Boot Tables, and verify PCIe boot. On some addresses single word reads are OK but burst reads return target abort.

We ran the PCI EP example project over XDS100 successfully i.e. configure the BAR0 and communicate to the DSP EP application. Our goal is to boot the device over PCIe and for this, we want to use the bootloader example from the latest MCSDK release cited above.

The PCIe user guide (section 2.6.2) seems to imply that the BAR0 is different from IB_BAR0 registers, and BAR0 is always configured. This is seen to be true when the EVM is set to PCIe boot mode; with this setup, PCIe config space is accessible even though the IB_BAR0 (and 1-3) is not setup for this access. Since we do not need all the 4 (IB_BAR) bars, it is OK for us to configure IB_BAR0 to correspond to BAR0 again.

But the default config of inbound translation registers when the device powers up (shown above) is not understood - is this configuration coming from the Boot ROM in the DSP? Perhaps this is not important since it is expected to be overwrriten by the host.

The other question is how to use the BAR_MASK bits to reduce the BAR window sizes. Can this be done from the RC host? Or does doing this require a I2C flash? In our custom board, we will not have a flash and we plan to boot over PCIe. Using the Boot ROM in the DSP and BARConfig of 0000b, is it correct that we will need a 256MB address window?

Thanks

First you mention a 125 mhz clock.  Assuming you are using the clock on the EVM it is 100mhz.

When you use the example from the MCSDK via the XDS100, are you able to avoid the target aborts even for large transfers?  Since the EVM will enumerate, it may be possible to apply the configuration change that allows the different clocks to work via the BAR0 access to the configuration registers.  If this is the case, no code would need to be loaded from a flash.  Which example are you downloading through the XDS100, so I can see which configurations changes are made by the example?

The BAR mask cannot be set remotely.  It must be set from the internal interface, which is SW running on the 6678 before enumeration starts.  However, it is OK if you see the 256MB window.  If your PC SW simply ignores this BAR, it can still use the others.  There is only a problem if you need to use all the BARs simutaneously.  We had verified access through the other BARs on a 32-bit machine where linux wouldn't allocate the 256MB BAR.  We found that linux does allocate 256MB on a 64-bit machine.

While I noted that the example programmed IB_BAR0 register to point at the configuration registers, this is not necessary for correct operation.  Thus you shouldn't change your configuration of the inbound address translation based on my previous post.  This was confirmed on our setup as well.

The configuration of the inbound address translation is put there by the boot rom.  Only the r/c knows the values that it will map into the BARs so it knows what should be put in the IB_BAR registers.  These values are no better or worse than any other values such as 0's.

The BAR masks can be set with the boot mode pins.   In http://focus.ti.com/lit/ds/symlink/tms320c6678.pdf, figure 2-6 and table 2-8 show how to configure various combinations of BAR masks that can be configured.

John,

Our PCIe clock is 100MHz, not 125 as stated before. 

Tried to boot the EVM over PCI using the DDR3 binary provided with no changes - as provided under

mcsdk_2_00_00_11\tools\boot_loader\examples\pcie\pcieboot_ddrinit\evmc6678l\bin .

The device does not seem to boot, boot magic address does not change back to 0 after waiting several seconds. On this EVM, the IBL update was *not* done. The first 16 words of Boot Internal Stats area read -

82073260: 647ed4d4 a27d4f43 fc51502a f851ac22
82073270: 0a627e65 e06e736f 0d7e992d 4a77d218
82073280: a779194b 798be16e 3c1e2011 3124a2ef
82073290: 974afb63 71bb4f9a 6be16296 a10fd75a

Any ideas on how this can be debugged further? XDS100 is still connected. But no serial console from EVM.

Thanks