Inbound Endpoint Address Translation; Confused by pcie_sample.c

Chris Gembarowski

I'm a newbie here so please be gentile.

I'm very confused by the pcie_sample.c file. The following code is found in main and it's an endpoint setting up his inbound address translation.

    /* Configure Address Translation */
    
    barCfg.location = pcie_LOCATION_LOCAL;
    barCfg.mode     = pcie_EP_MODE;
    barCfg.base     = PCIE_IB_LO_ADDR_S;
    barCfg.prefetch = pcie_BAR_NON_PREF;
    barCfg.type     = pcie_BAR_TYPE32;
    barCfg.memSpace = pcie_BAR_MEM_MEM;
    barCfg.idx      = PCIE_BAR_IDX_S;
    
    if ((retVal = Pcie_cfgBar(handle, &barCfg)) != pcie_RET_OK) 
    {
      System_printf ("Failed to configure BAR!\n");
      exit(1);
    }

    ibCfg.ibBar         = PCIE_BAR_IDX_S; /* Match BAR that was configured above*/
    ibCfg.ibStartAddrLo = PCIE_IB_LO_ADDR_S;
    ibCfg.ibStartAddrHi = PCIE_IB_HI_ADDR_S;
    ibCfg.ibOffsetAddr  = (uint32_t)pcieConvert_CoreLocal2GlobalAddr ((uint32_t)dstBuf.buf);
    ibCfg.region        = PCIE_IB_REGION_S;

I'm confused because the endpoint is setting up his ibStartAddrLo. However, this value should come from the Root Complex, right?

I guess that the sample code has been simplified for teaching purposes by using predefined PCIe address values: 0x7000_0000 for the EP and 0x9000_0000 for the RC. The problem is that the proper technique for an EP to learn his PCIe address is not documented.

Am I right? If so, can you point me to example code that illustrates this process?

Thanks a bunch.

over 9 years ago

0 Ganapathi Dhandapani95 over 9 years ago

TI__Mastermind 28085 points

Hi Chris,

TI doesn’t provide PCIE RC enumeration example/driver source. Normally the RC runs some operating system that performs the enumeration with the following steps.

If C66x device is configured as RC, then CFG_SETUP register is used to specify the target bus/device/function numbers for the target device (switch/EP) and try to read/write the remote device space in MMR (from offset 0x2000 in PCIe MMR space) to generate those configuration requests.

The MCSDK PCIe example project is used to test the PCIe driver for two EVMs. You need to configure one EvM as RC and another EVM as EP. Please take a look at Readme file (“\ti\pdk_C66xx_x_x_x_x\packages\ti\drv\pcie\example\sample\Readme.txt”) for more information.

Thanks,

0 Chris Gembarowski over 9 years ago in reply to Ganapathi Dhandapani95

Prodigy 50 points

Ganapathi,

Thanks for the response. I read the Readme file and I think that I understand the purpose of the test. I'm trying to extract the steps from the sample code that are necessary to properly configure a PCIe End-Point device.

In my case I'm working on a PCIe device that will plug into the PCIe bus of a typical desktop motherboard running Linux (i.e. a Linux PC). So, the Linux PC will be the RC and that my PCIe device will be an EP. The RC will enumerate the PCIe bus and it will tell the EP what the EP's PCIe address is. My configuration must be a very common scenario.

Does TI have sample code that illustrates what an EP needs to do in this situation? Certainly the EP doesn't choose his PCIe address like in the sample code.

Thanks,

Chris

0 Ganapathi Dhandapani95 over 9 years ago in reply to Chris Gembarowski

TI__Mastermind 28085 points

Hi Chris,

Please provide your DSP part number? eg: C6678, C6670, C6657.

Refer section 2.11.2 PCIe as End Point on PCIe user guide(SPRUGS6D). PCIe initialization sequence are clearly explained in PCIe user guide document.

http://www.ti.com/lit/sprugs6

Better to use DSP PCIe boot mode for your case. Please refer “\ti\mcsdk_x_xx_xx_xx\tools\boot_loader\examples\pcie\docs\Readme.pdf” for more infomation about PCIe boot mode.

Thanks,

0 Chris Gembarowski over 9 years ago

Prodigy 50 points

mcsdk_2_01_02_06/tools/boot_loader/ibl/src/device/c66x/c66xinit.c

void iblPCIeWorkaround()
{
    UINT32  v, flag_6678 = 0, flag_6670 = 0, MAGIC_ADDR;
    UINT32  i;

     /* Power up PCIe */
    devicePowerPeriph (TARGET_PWR_PCIE);
    for(i=0; i<1000; i++) asm (" NOP");

    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_SERDES_CFG0), 0x00062320);  /* ss clock */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_SERDES_CFG1), 0x00022320);  /* ss clock */

    /* Wait for PCIe PLL lock */
    while(!(DEVICE_REG32_R(PCIE_STS_REG) & 1));

        /* Determine 6670 or 6678 */
    v = *((Uint32 *)DEVICE_JTAG_ID_REG);
    v &= DEVICE_JTAG_ID_MASK;

    if (v == DEVICE_C6678_JTAG_ID_VAL) {
                MAGIC_ADDR = 0x87fffc;
                flag_6678 = 1;
        }
        if (v == DEVICE_C6670_JTAG_ID_VAL) {
        MAGIC_ADDR = 0x8ffffc;
                flag_6670 = 1;
        }

    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_CLASSCODE_REVID), 0x04800001);  /* class 0x04, sub-class 0x80, Prog I/F 0x00, Other multimedia device */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_LINK_STAT_CTRL), 0x10110080);  /* extended sync, slot_clk_cfg = 1 */

    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_VENDER_DEVICE_ID), 0xb005104c);  /* Vendor and Device ID */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_DEVICE_CAP), 0x288701); /* L0 = 4, L1 = 3 */

        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_OB_SIZE), 0x00000003);     /* OB_SIZE = 8M */
        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_PL_GEN2), 0x0000000F);   /* num_fts = 0xF*/

    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_CMD_STATUS), 0x0020); /* Set dbi_cs2 to allow access to the BAR registers */

        if (flag_6678)  {
                /* 6678 */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR0), 0x00000FFF);   /* 4K */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR1), 0x0007FFFF);   /* 512K */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR2), 0x003FFFFF);   /* 4M */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR3), 0x00FFFFFF);   /* 16M */
        }

        if (flag_6670)  {
                /* 6670 */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR0), 0x00000FFF);   /* 4K */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR1), 0x000FFFFF);   /* 1M */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR2), 0x001FFFFF);   /* 2M */
                DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR3), 0x00FFFFFF);   /* 16M */
    }

        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_CMD_STATUS), 0x0);    /* dbi_cs2=0 */

        DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_STATUS_CMD), 0x00100146); /* ENABLE mem access */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_DEV_STAT_CTRL), 0x0000281F); /* Error control */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_ACCR), 0x000001E0); /* Error control */
    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_BAR0), 0); /* non-prefetch, 32-bit, mem bar */

    DEVICE_REG32_W ((PCIE_BASE_ADDR + PCIE_APP_CMD_STATUS), 0x0000007);    /* enable LTSSM, IN, OB */
    while((DEVICE_REG32_R(PCIE_BASE_ADDR + PCIE_DEBUG0) & 0x11)!=0x11);    /* Wait for training to complete */

    /* Wait for the Boot from Host */
    DEVICE_REG32_W(MAGIC_ADDR, 0);
        waitForBoot(MAGIC_ADDR);

    /* Will never reach here */
    return;
}

0 Ganapathi Dhandapani95 over 9 years ago in reply to Chris Gembarowski

TI__Mastermind 28085 points

Also refer mcsdk_2_01_02_06\tools\boot_loader\examples\pcie\linux_host_loader\pciedemo.c

		/* Configure IB_BAR0 to BAR0 for PCIE registers; Configure IB_BAR1 to BAR1 for LL2 for core 0
		   Configure IB_BAR2 to BAR2 for MSMC; Configure IB_BAR3 to BAR3 for DDR */
		for (i = 0; i < 4; i++) {
			iowrite32(i, ptrReg + IB_BAR(i)/4);     
			iowrite32(PCIE_DEV->resource[i].start, ptrReg + IB_START_LO(i)/4);    
			iowrite32(0, ptrReg + IB_START_HI(i)/4);
		}     
		iowrite32(PCIE_BASE_ADDRESS, ptrReg + IB_OFFSET(0)/4);    
		iowrite32(LL2_START + (1 << 28), ptrReg + IB_OFFSET(1)/4);    
		iowrite32(MSMC_START, ptrReg + IB_OFFSET(2)/4);  
		iowrite32(DDR_START, ptrReg + IB_OFFSET(3)/4);

Thanks,

Processors

Processors forum

Inbound Endpoint Address Translation; Confused by pcie_sample.c