Communication problem between DSP and FPGA

Hi,

 

After the PCIE link is up, is it stable overtime or will drop out quickly? Have you using the same or different reference clock FPGA and DSP card?

Are you able to read/write the IB_BARn, IB_STARTn_LO, IB_STARTn_HI and IB_OFFSETn (n = 0, 1, 2, 3) registers of FPGA via PCIE link?

When you use a PC as RC, PC driver writes the BAR registers of the EP during enumeration.

If use the DSP as the RC, it writes the BAR registers of the EP (FPGA) during enumeration. The MCSDK DSP’s software DOESN’T configure the BAR of the remote-end. You need to add code to do it through PCIe Remote Configuration Space. I assume you use PDK package under MCSDK. In  packages\ti\drv\pcie\example\sample\readme.txt, it mentioned:

At startup, each EVM configures its PCIe subsystem:

• Serdes, clock, PLL

• PCIe Mode and Power domain

• Inbound/Outbound address translation and BAR registers

• Link training is triggered

Our MCSDK example works on two EVM boards, each EVM configures its own BAR registers like below.

/* In this example all addresses are 32bit */

/* Outbound Base Address for PCIe Master(RC) */

#define PCIE_OB_LO_ADDR_M    0x70000000

#define PCIE_OB_HI_ADDR_M    0

 

/* Inbound  Base Address for PCIe Master (RC)*/

#define PCIE_IB_LO_ADDR_M    0x90000000

#define PCIE_IB_HI_ADDR_M    0

 

/* Outbound Base Address for PCIe Slave (EP)*/

#define PCIE_OB_LO_ADDR_S    PCIE_IB_LO_ADDR_M

#define PCIE_OB_HI_ADDR_S    0

 

/* Inbound  Base Address for PCIe Slave (EP)*/

#define PCIE_IB_LO_ADDR_S    PCIE_OB_LO_ADDR_M

#define PCIE_IB_HI_ADDR_S    0

Thanks,

Ganapathi,

Instead of

Ganapathi Dhandapani said:

If use the DSP as the RC, it writes the BAR registers of the EP (FPGA) during enumeration.

I would write "DSP as RC has to configure EP registers by config space writes". Whenever one uses example provided that does not happen. As you mentioned, each DSP configures its own side. With FPGA as endpoint its DSP's job to configure FPGA and there is no reference for that.

ZAKARIA BOURZOUK said:

On FPGA side, I realized my design in Qsys

This information is not enough to understand your issue. First of all, you have to configure remote endpoint (FPGA) from DSP side. DO NOT use Pcie_readRegs/Pcie_witeRegs functions from PCIe LLD unless you sure their design matches your endpoint layout. For you to know, It was not so in case of Xilinx Spartan 6. Unfortunately, I cannot give you precise recipe for your FPGA. Just in my case I found end point config space layout in Xilinx user guide. You have to find something similar in your docs.

Next, you have to make config writes. That is, writes to addresses starting from 0x21802000. Write to 0x21802000 will land in endpoint cofig space at offset 0x000, which is VID/PID, write to 0x21802004 will land to endpoint Command/Status register at 0x004 and so on. You can make a use of CSL data types to help with registers manipulation. For example, I know that MSI capability in my FPGA is located at 0x048. Then I use following fragment to enable MSI:

volatile UInt32 *epCfg = (UInt32 *)0x21802000;
pcieMsiCapReg_t     MsiCap;

{
    pcieMsiCapReg_t *reg = &MsiCap;
    uint32_t val = reg->raw = epCfg[(0x48>>2)];

    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_64BIT_EN,     reg->en64bit);
    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MULT_MSG_EN,  reg->multMsgEn);
    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MULT_MSG_CAP, reg->multMsgCap);
    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MSI_EN,       reg->msiEn);
    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_NEXT_CAP,     reg->nextCap);
    pcie_getbits(val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_CAP_ID,       reg->capId);
} /* pcie_read_msiCap_reg */

MsiCap.msiEn    = 1;

{
    pcieMsiCapReg_t *reg = &MsiCap;
    uint32_t new_val = reg->raw;
    pcie_range_check_begin;

    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_64BIT_EN,    reg->en64bit);
    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MULT_MSG_EN, reg->multMsgEn);
    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MULT_MSG_CAP,reg->multMsgCap);
    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_MSI_EN,      reg->msiEn);
    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_NEXT_CAP,    reg->nextCap);
    pcie_setbits(new_val, CSL_PCIE_CFG_SPACE_ENDPOINT_MSI_CAP_CAP_ID,      reg->capId);

    epCfg[(0x48>>2)] = reg->raw = new_val;
} /* pcie_write_msiCap_reg */

Next, in your screenshot I see some references to dma at fpga side. To make use of it, you have to program dma controller registers. Again, I have similar situation, but my hardware is different, so my reference would not help you much. You have to know DMA controller registers layout and sequence of their programming, and that again to be found in you FPGA doc.

TI doesn’t provide PCIE RC enumeration example/driver source. Normally the RC runs some operating system that performs the enumeration with the following steps.

If C66x device is configured as RC, then CFG_SETUP register is used to specify the target bus/device/function numbers for the target device (switch/EP) and try to read/write the remote device space in MMR (from offset 0x2000 in PCIe MMR space) to generate those configuration requests.

Thanks,