• State Resolved
  • Locked Locked
  • Replies 3 replies
  • Answers 1 answer
  • Subscribers 34 subscribers
  • Views 2024 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PCIe switch - forwarding table

Varban Metodiev
Intellectual 690 points

Hi,

After going through the PCIe specification several times, I am still bewildered about the actual PCIe switch operation principles.

May someone please explain me what forwarding table(s) does a PCIe switch implement? Is there an analogue to the "MAC table" that an Ethernet switch would use after learning the MACs of the hosts attached to its corresponding  ports?

And if so, how do those table(s) get populated when the PCIe switch is forwarding in the different modes (let's say in a P2P example scenario - via DeviceID and via BAR addressing)?

Thank you in advance!

  • 0
    Guru 321640 points

    During enumeration, the host OS/firmware assings non-overlapping address ranges to each device on a bus, so when a switch handles a packet with a specific address, there will be at most one target bus/device.

  • 0 in reply to Clemens Ladisch
    Intellectual 690 points

    Thank you for your response!

    So, correct me if I am wrong, but this is how I understand the concept now:

    1. The PCIe is always strictly hierarchical, tree-like, addressing is done via a recursive, depth traversal of the tree.

    2. Due to point 1, the DeviceIDs and memory addresses are ascending is a "synchronous manner" - I mean, as the device IDs increases, so do the BARs.

    3. Due to point 2 and the fact that a switch port is always on top of some endpoint(s) in the tree hierarchy, we will consider the following example cases:

        3.1. If only one EP is connected to a downstream port P(n), it will see its BAR address (for example 0x00).

               This will mean that port P(n) will be configured with base address 0x00 and limit address 0x0F (for example).

        3.2. If two EPs are connected to the downstream P(n), they are expected to have different, but contiguous addresses.
               EP1 will have BAR 0x00

               EP2 will have BAR 0x10

               Because both EP1 and EP2 are beneath the P(n) in the hierarchy, the switch port will be configured with base address 0x00 and limit address 0xFF (again, just an example for brevity) - so that it will cover the whole range of the configured endpoint BARs beneath it. 

    So, whenever a TLP comes either from the upstream or another downstream port, the switch will use the per-port register configuration of the base-limit addresses and if the TLP address is within the range of port P(n), the switch will forward the request over that specific port.

  • +1 in reply to Varban Metodiev
    Guru 321640 points

    1. Yes. (Upstream transfers usually go to the root complex, i.e., to main memory, to addresses outside of any known range. There also are peer-to-peer transfers; if a switch supports this, it will check addresses of upstream packets.)

    2/3. In general, yes. Address ranges on a bus do not need to be contiguous (the OS might want to leave open ranges to allow hot-plugging). But the address range for a bus (i.e., for the switch in front of the bus) always covers all address ranges of the devices on the bus.