J784S4XEVM: PCIe RC enable BAR 0 crash

Charles

Part Number: J784S4XEVM

Hi experts,

I am continuing this post : J784S4XEVM: PCIe RC enable BAR 0 crash - Processors forum - Processors - TI E2E support forums

Last answer was "it is not expected for BAR registers to change at runtime for RC "

But even if I config the BAR 0 from the device tree, it still crash.

pcie0_rc: pcie@2900000 {
		compatible = "ti,j784s4-pcie-host";
		reg = <0x00 0x02900000 0x00 0x1000>,
		      <0x00 0x02907000 0x00 0x400>,
		      <0x00 0x0d000000 0x00 0x00800000>,
		      <0x00 0x10000000 0x00 0x00001000>,
			  <0x40 0x00000000 0x01 0x00000000>;
		reg-names = "intd_cfg", "user_cfg", "reg", "cfg";
		interrupt-names = "link_state";
		interrupts = <GIC_SPI 318 IRQ_TYPE_EDGE_RISING>;
		device_type = "pci";
		ti,syscon-pcie-ctrl = <&pcie0_ctrl 0x0>;
		max-link-speed = <3>;
		num-lanes = <4>;
		power-domains = <&k3_pds 332 TI_SCI_PD_EXCLUSIVE>;
		clocks = <&k3_clks 332 0>;
		clock-names = "fck";
		#address-cells = <3>;
		#size-cells = <2>;
		bus-range = <0x0 0xff>;
		vendor-id = <0x104c>;
		device-id = <0xb012>;
		msi-map = <0x0 &gic_its 0x0 0x10000>;
		dma-coherent;
		ranges = 	<0x02000000 0x0 0x10001000 0x0 0x10001000 0x0 0x7fff000>,
			 		<0x42000000 0x40 0x0 0x40 0x0 0x1 0x0>;
		dma-ranges = <0x02000000 0x0 0xa8000000 0x0 0xa8000000 0x0 0x10000>,
					<0x02000000 0x0 0xA9000000 0x0 0xa9000000 0x0 0x10000>;  /* doesn't crash if I remove this line */
		status = "disabled";
	};

If I remove the second dma-range entry, (which config the BAR0) it doesn't crash.

I have patch the driver to use the BAR 0 when dma-range have only 1 entry. It still crashes

Every following cases crash :

BAR 0 32b pref
BAR 0 32b non-pref
BAR 0 64b pref
BAR 1 + BAR 0

Am I missing something ?

Have you successed on your side to enable BAR 0 + BAR 1 ?

Thanks for your help

Charles

6 months ago

0 Takuma Fujiwara 6 months ago

TI__Guru 61013 points

Hi Charles,

The dts is mapping 64KB of PCI address 0xA9000000 to system address 0xa9000000 in the line you are removing. I think default dts from TI maps PCIe address 0x0 to system address 0x0 and with large length, but is there a particular reason for the memory address in-use for your dts?

Regards,

Takuma

0 Charles 6 months ago in reply to Takuma Fujiwara

Prodigy 230 points

Hi Takuma,
indeed, default config a large windows.

For our application, we need to expose 2 small windows from the RC. We need to access specific data buffers in DDR and we want to prevent every other accesses.

Regards,

Charles

0 Takuma Fujiwara 6 months ago in reply to Charles

TI__Guru 61013 points

Hi Charles,

I see that you have a combination of PCI_DAT0 space and PCI_DAT1 space for the ranges property, while the dma-ranges does not seem to fall into those memory ranges.

We have not tested out mixing the 32-bit space and 64-bit space DAT0 and DAT1, as well as defining dma-ranges that is outside the PCI address space. But perhaps the combination is causing some unexpected behavior? Is it possible to use either DAT0 (i.e., the 0x10001000 range) or DAT1 (i.e., the 0x40 0x0 range)? And also define the dma-range within the PCI address space that is set in the ranges property?

Regards,

Takuma

0 Charles 6 months ago in reply to Takuma Fujiwara

Prodigy 230 points

Hi Takuma,

If my understanding is correct, the dma-range address is used to config the inbound translation address. Therefore it should be a local memory address (ex DDR) and not PCIe address ( PCI_DAT0/PCI_DAT1 space).

I have tested mixing the 32-bit space and 64-bit and it works correctly. My issue happened before mixing those 32b and 64b spaces.

When I configure 1 dma-range entry to DDR address (0xa800 0000), the driver configure the BAR 1 and it works correctly.

My issue seems to be related to the BAR 0.

With 1 dma-range entry to DDR :

The driver selects BAR 1 because it selects the smallest BAR which fit and BAR 1 max size < BAR 0 max size (this case works).

If I patch the driver reducing the BAR 0 max size, the driver select the BAR 0 first and then it crashes. even with only 1 entry for dma-range

Regards,

Charles

0 Takuma Fujiwara 6 months ago in reply to Charles

TI__Guru 61013 points

Hi Charles,

Would recommend this read: https://elinux.org/Device_Tree_Usage#PCI_Host_Bridge. But yes, I agree with your understanding that it is for inbound translation from PCIe device to local memory.

However, my understanding is that each entry is a mixture of both physical local memory address and PCI address. And what TI default dts does is map 0x0 PCI address to 0x0 physical address, and allocate a large memory window, effectively disabling translation between PCI address and physical address.

Technically, it should be possible to allocate smaller memory windows. But maybe there is some conflict on either the physical memory or PCI memory side?

Regards,

Takuma

0 Charles 5 months ago in reply to Takuma Fujiwara

Prodigy 230 points

Hi Takuma,

I couldn't achived to use BAR 0.

but I have found an alternative of using 2 small BAR.

I am using TI default config on the RC (RP_NO_BAR).
and instead of using BAR and IB translation, I am using 2 small OB region to the specific memories of the RC (outside the PCIe window).

Regards,

Charles