RTOS/AM5728: PCIe - How to Retrieve EP BAR

Hi,

In your case, the AM572x is the PCIE RC running TI-RTOS code. The FPGA is the PCIE EP. You have the linked up successfully.

The data access (R/W) from AM572x side looks like below (I use PCIESS1 here for simplicity):

• The CPU access (R/W) the the PCIe_SS1 Configuration space address 0x2xxx_xxxx
• There is outbound translation setup to convert this 0x2xxx_xxxx to PCIE address. The pcie_sample.c code has two outbound regions:

1. OB region 0 used to access remote EP configuration space offset 0x1000 to 0x1FFC. So if you look at 0x2000_1000 you should be able to access FPGA side configuration space, starting from device id/vendor id. E.g., 0x2000_1010 you should be able to access BAR0 of FPGA.
2. OB region 1 used to access remote EP memory space offset 0x0100_0000 with 16MB window. So, if you want to access FPGA side data space, you need to look at 0x2100_0000 and onwards. This OB setup the outbound access to 0x7000_0000. It means when you look at 0x2100_0000 on RC side, the PCIE address cross the connection is 0x7000_0000. Then in your FPGA side, you need an inbound mapping of this 0x7000_0000 into some internal memory.

Given you know that there is a fixed mapping for FPGA side: 0x4_0000 as local to translate to 0x4000_0000.

What you need to do:

1. In the RC side for the OB1, you need to change 0x7000_0000 to 0x4_0000
2. In the RC side you need to do a Pcie_cfgBar() for the remote BAR with barCfg.base = 0x4_0000; From RC side 0x2000_1010 you can cross-check this value.

What happened is: on RC side, you W/R 0x2100_0000, it translates to 0x4_0000 over the PCIE interface. Then this address is accepted at the FPGA side with BAR matched. Then the FPGA converts the 0x4_0000 into 0x4000_0000. If you know the LED offset, you should be able to access it with 0x2100_0000 + offset (subject to earlier configured 16MB window size).  

Regards, Eric

Hi,

I interpret 0x2000_1011 as 0x2000_1014, because this is 32-bit address and BAR1 is at 0x2000_1014.

For your trial of writing 0xFFFF_FFFF to BAR0 and got 0xFFF0_0000, this confirmed the BAR size is 0x10_0000. That is 1MB, matched your document excerpt. Similarly, BAR1 is 64KB from your trial. This also matched the document excerpt.

For a BAR0 mask of 1MB, it masks bit 0-19. So you wrote a 0x4_0000 and got 0x0000_0000. It is expected. I don't understand "Local Address: Enter values as 0x40000 to translate the BAR0 address space to CoreGPIO address (0x4000_0000)" on the FPGA side." Maybe you can check the FPGA vendor for this.

Some thing we can try on AM57x side,

1) 

barCfg.location = pcie_LOCATION_REMOTE;

 barCfg.mode     = pcie_EP_MODE;

 barCfg.base     = 0x40000; =========> change this to 0, as it has no effects

 barCfg.prefetch = pcie_BAR_NON_PREF;

 barCfg.type     = pcie_BAR_TYPE32;

 barCfg.memSpace = pcie_BAR_MEM_MEM;

 barCfg.idx      = 0;

 if ((retVal = Pcie_cfgBar(handle, &barCfg)) != pcie_RET_OK)

 {

   PCIE_logPrintf ("Failed to configure FPGA BAR0 (%d)\n", (int)retVal);

   exit(1);

 }

2) In the pcie_sample.c, there is a function called pcieRet_e pcieObTransCfg(Pcie_Handle handle, uint32_t obAddrLo, uint32_t obAddrHi, uint8_t region)

You see there is an additional region configure for RC mode

if(PcieModeGbl == pcie_RC_MODE)
{
/*Configure OB region for remote configuration access space*/
regionParams.regionDir = PCIE_ATU_REGION_DIR_OUTBOUND;
regionParams.tlpType = PCIE_TLP_TYPE_CFG;
regionParams.enableRegion = 1;

regionParams.lowerBaseAddr = PCIE_WINDOW_CFG_BASE + resSize;
regionParams.upperBaseAddr = 0; /* only 32 bits needed given data area size */
regionParams.regionWindowSize = PCIE_WINDOW_CFG_MASK;

regionParams.lowerTargetAddr = 0U;
regionParams.upperTargetAddr = 0U;

if ( (retVal = Pcie_atuRegionConfig(
handle,
pcie_LOCATION_LOCAL,
(uint32_t) 0U,
&regionParams)) != pcie_RET_OK)
{
return retVal;
}
}

Where #define PCIE_WINDOW_CFG_MASK 0x00000FFFU

This is a configuration translation to access the FPGA side PCIE registers with a 0x1000 window. That is why from AM572x side (0x2000_1000 to 0x2000_2000) you can look into FPGA side BAR0 for 0x1000 memory range (e.g, device/vendor ID, BAR, etc...).

Please update the PCIE_WINDOW_CFG_MASK to a big number like  0x1F_FFFF (that is 2 MB window). Then from AM57 side, make sure the increase doesn't break your access of FPGA by looking into 0x2000_1000 region. Then try to look at 0x2004_1000 or 0x2004_0000 (just do it via CCS memory window) to see if the FPGA can map this 0x4_0000 address into your GPIO region.

If still no luck, please try to check the FPGA vendor how to setup the access of BAR0 with address 0x4_0000. If this is a CFG access or MEM access? 

Regards, Eric