PCIe linux enumeration

Clement FR

Genius 4750 points

Hello everyone

(target is C6678)

I'm working on PCIe and researching the enumeration process.

I've looked at the Linux PCIe RC (Host) Driver for KeyStone PCIe Module configured as Root Complex

In the check_device function

/**
 * check_device() - Checks device availability
 * @bus: Pointer to bus to check the device availability on
 * @devfn: Device number and function
 *
 * Checks for the possibility of device being present. Relies on following
 * logic to indicate success:
 * - downstream link must be established to traverse PCIe fabric
 * - treating RC as virtual PCI bridge, first (and only) device on bus 1 will be
 *   numbered as 0
 * - don't check device number beyond bus 1 as device on our secondary side may
 *   as well be a PCIe-PCI bridge
 */
static int check_device(struct pci_bus *bus, unsigned int devfn)

I see this comment

		/*
		 * Apparently the C66x does not manage the error response
		 * to non-posted PCIe transactions with an explicit abort external 
		 * exception like on ARM with its MMU.
		 *
		 * So in order to check if a device is present or not when 
		 * accessing the PCIe remote configuration space, we try to
		 * read and modify the parity enable bit of the PCI command field.
		 * According to the PCI specification this bit is r/w and can be 
		 * modified for all EP and bridges.
		 *
		 * If the bit can be modified we come to the conclusion that
		 * device is present. Otherwise device is not present.
		 */

Can someone explains a little more about this ?

It would mean that the C6678 as a Root Complex doesn't respect the standard enumeration process as defined in the PCIe specification...

Regards,
Clement

over 10 years ago

0 Ganapathi Dhandapani95 over 10 years ago

TI__Mastermind 28085 points

Hi,

TI doesn’t provide PCIE RC enumeration example/driver source. Normally the RC runs some operating system that performs the enumeration with the following steps.

If C66x device is configured as RC, then CFG_SETUP register is used to specify the target bus/device/function numbers for the target device (switch/EP) and try to read/write the remote device space in MMR (from offset 0x2000 in PCIe MMR space) to generate those configuration requests.

Thanks,

0 Clement FR over 10 years ago in reply to Ganapathi Dhandapani95

Genius 4750 points

Hello Ganapathi

I know what you said.

Here the Root Complex runs Texas Instruments Linux operating system for C6x architecture

linux-c6x.org/.../Main_Page

It's Texas Instruments software so I think it's the right place to ask questions about it.

Clement

0 Rex Chang over 10 years ago in reply to Clement FR

TI__Guru 50170 points

Hi, Clement,

On C6678 we do not have a C66x exception triggered when accessing an invalid PCIe address (i.e. a device not present). Even if we do not have MMU with virtual addressing on C66x, the basic MMU could handle that but the PCIe IP logic is not connected to C66x external exception for this case. Even on K1/K2 ARM where we have exceptions for non-posted PCIe transaction it is not obvious due to Linux internal APIs and the fact that this exception could occurs for other reasons (Linux PCI framework was mainly designed for Intel).

The difference here is that instead of trying to access an “candidate” endpoint device address and see if we got an exception like on ARM and Intel, we try to read and modify a parity bit of the PCI command field for this “candidate” endpoint. If device is present, we can modify this bit. I would say this is in fact more standard to the PCI spec than using an exception which is implementation and architecture specific (AFAIK the PCI spec does not imply that you got an exception when accessing the address of a non-present device).

Rex

0 Clement FR over 10 years ago in reply to Rex Chang

Genius 4750 points

Hi Rex,

Thank you for the detailled answer.

It may not be written explicitely in the spec per se but it (getting an exception) is the expected behavior when reading the PCIe litterature. Maybe this information could be in the PCIe user guide.

Anyway speaking of enumeration.
I have a fixed PCIe topology (I know the number of endpoints, I setup the switch configuration, etc.)
Memory write and read are the expected transactions.

Is there a way to avoid doing a full enumeration ?

I've been playing around with configuration writes but I still get errors
when reading Root Complex memory from the Endpoint (but not the other way around)
suggesting that the bus device function of the Endpoint is not setup properly. (device and function should be 0 though)

Is there a way to read the bus device function "captured" by a Texas Instruments DSP C6678 endpoint ?

Clement

Processors

Processors forum

PCIe linux enumeration