This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS: Accessing both BAR0 and BAR1 of only one EP device from RC device

Other Parts Discussed in Thread: TMS320C6657

Tool/software: TI-RTOS

Hi Experts,

Regarding the PCIe Multiple devices setup, if we take DSP as RC and FPGA as EP, can we connect only one EP device for both BAR0 and BAR1. i.e i configure OB Region 0 and OB Region 1 for RC.

and i want to connect only one EP device in my case FPGA for both BAR0 and BAR1. OB Region 0 to BAR0  and OB Region 1 to BAR1?  And also is it possible at the same time to access from RC to EP BAR0 and BAR1?

Thanks,

Ramana.

  • Hi,

    I assume you are checking with Keystone I/II SOC. There is some confusion in the question. If you have a PCIE switch in between, then you can connect one RC with multiple EPs. If there is no switch, only one EP can be connected to RC.

    The OB in the RC is used to setup outbound translation, i.e., for access the address in the FPGA. The RC side has 2 BAR, it is used to setup accept/reject incoming packets. There is no directly relationship between BAR and OB.

    Please look at the PCIE user guide for details. Also, the RTOS wiki: processors.wiki.ti.com/.../Processor_SDK_RTOS_PCIe

    Regards, Eric
  • Hi,
    Thanks for the info.
    i am using only one EP which is connected to my RC.
    i know the OB to BAR relation.
    My question is , is it possible to communicate with RC with two BAR's(as you said) to one EP device with two BAR's.
    my target is, i want to write from RC to EP in two different address places at the same time. i.e i want to configure BAR0 and BAR1 on RC and communicate with two address ranges BAR0 and BAR1 on EP at the same time.

    Thanks,
    Ram.
  • Hi,

    Yes, you can do that. For example, on the RC side, you write to OB region 0 (e.g. 0x6000_0000 on C6678) and OB region 1 (e.g. 0x6010_0000 assume OB size is 1MB). At the EP size, you can setup two BARs, BAR0 accepts the address from RC OB region 0, and BAR1 accepts address from RC OB region 1.

    Regards, Eric
  • Hi,

    if So, can we communicate from RC to BAR0 and BAR1 on EP at the same time?

    Thanks,
    Ram.
  • Yes, you can.

    Regards, Eric
  • Hi Eric,

    Following up on the same topic opened by my colleague Ramana regarding BAR0 and BAR1 simultaneous access, I would like to give you a brief description about the issue we are facing with the PCIe interface; it would be great if you can provide any guidance.

    The configuration consists of a C6657 DSP as a RC connected to an Altera FPGA (EP). The RC remotely configures the FPGA as an EP.

    The FPGA has a custom DMA module with a configuration space, which can only be accessed through BAR0.The goal, is to remotely configure BAR0 and BAR1 such that BAR0 to be used for DMA inbound data transfer to and from the DSP and BAR1 to be used as an outbound data transfer to and from the FPGA.

    Currently only BAR1 is configured and can be used to transfer and receive data from the FPGA.

    Now configuring only BAR0 instead of BAR1 and attempting to access the DMA configuration space results in a corrupted data read.

    As an attempt to figure out the problem with BAR0 I used the pci_sample example on the two-evaluation boards (TMS320C6657) one as an RC and the other as an EP connected through a dual EVM Breakout card from TI.

    The example runs as expected.

    Now changing the PCIE_BAR_IDX_M and PCIE_BAR_IDX_S from 1 to 0 on both EVMs results in a test fail (data transmitted from RC but  never received by EP).

    Can this be an explanation why I can not use  BAR0  in my case ? If not what is the modification needed in the pci_sample to be able to use BAR0 to  successfully run the loop test between the two boards.

    Thanks in advance ,

    Hisham 

  • Hi,

    From your register dump, RC side 0x21800200 is setup for outbound translation, the size can be 1MB to 8MB, that covers the 0x6000_0000 + 1~8MB. From the PCIE remote configue space 0x21802010, this is the BAR0 of the FPGA side, it has 0x6000_0000, it looks correct. The BAR0 and BAR_MASK share the same register address, how big is the BAR_MASK for BAR0? Is it possible that BAR_MASK is too small so only 0x6000_0480 is accepted but 0x60000e80 is out of the range? Does FPGA have inbound translation? Or the 0x6000_xxxx is the real memory address?

    In C6657, the BAR0 is hard wired to access the PCIE application registers. See PCIE user guide, 2.7.2.2 BAR0 Exception for In-Bound Address Translation
    The memory space covered by BAR0 in inbound direction is completely dedicated to
    accessing the application registers, Address Space 0, in both RC and EP modes. It
    implies that the BAR0 cannot be remapped to any other location but to application
    registers. Any remote inbound access matching the BAR0 region will automatically be
    routed to these registers.

    So you can't change BAR1 to BAR0 for the C6657 to C6657 test.

    Regards, Eric
  • Hi Eric,

    Thank you for the reply.

    I can now access both BARs in outbound direction. The problem was not from the PCIe configuration but with the addresses being read from the FPGA.

    I still need to correctly set the inbound configuration to allow the FPGA (EP) to access the DSP (RC) using BAR1 and I have some questions regarding the matter.

    Currently during the remote configuration of the EP, I only configure the BAR registers of the remote device as follows to filter the packets between BAR0 and BAR1 as you suggested in this thread.

    The Inbound configuration of the RC is as follows:

    The EP does not have any Inbound or outbound translation but actual memory addresses, and the DMA registers can be configured to take a base address and block size, which can be used for inbound DSP access.

    If I initialized the memory address 0x90000000 at the DSP side.  Will the PCIe inbound address 0x90000000 set from the FPGA side access this actual memory location in the DSP?

    Is there any other inbound configuration needed for inbound DSP access which I am missing?

    Thanks,

    Hisham

  • Hi,

    For the DSP (RC), it has inbound translation and outbound translation. From your info, such inbound and outbound doesn't exist on the FPGA (EP) side.

    So, when you program remote BAR of EP (from RC) side with 0x6000_0000 and 0x6010_0000, any incoming address to the EP with that address ranges will be accepted by EP, it looks you already resolved the direction from RC to EP (read and write).

    Now for the EP to RC direction, assuming the EP can generate address like 0x9000_0000, you have a BAR setup at RC side to accept 0x9000_0000 this is OK. For the inbound translation, the address inside RC = incoming address (0x9000_0000) - ibStartLo (0x9000_0000) + iboffsetaddr (0) = 0x0. This is NOT a valid address in RC.

    For example, if you want this landed in MSMC, then iboffseraddr would be something like 0x0c00_0000, if you want this in DDR, then iboffsetaddr would be like 0x8000_0000. Then from EP side, you should be able to access this address in RC.

    Regards, Eric