AM5748: Sysbios - PCIE driver issue

Hi Venkateswara,

I assume this thread is related to the previous thread here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1265044/am5706-how-to-change-vendor-id-or-device-id-of-pcie-in-am5706

To set some expectations, this will require some time from my end to root cause what the issue is. The original team that had worked on the device and driver is no longer at TI, so I will need some more time to familiarize myself with the registers, driver, and device in general. Thank you for your patience.

I will be running experiments in parallel with you, but to accelerate the debug, would you be able to try out some experiments for me?

For the outbound/inbound address, I see that there are several places within PDK that defines this. I assume it the definition in pcie_sample.h is the one that is used, but just in case, could you also modify the following locations within PDK directory as well?

• packages/ti/board/diag/pcie/src/pcie_test.h
• packages/ti/drv/pcie/example/Qos/src/pcie_qos_sample.h
• packages/ti/drv/pcie/example/sample/src/pcie_sample.h

Regards,

Takuma

Hi Takuma,

I have modified the outbound address in all files what you have suggested. But the state is same(not sending or receiving any data).

For your information:

I assume this thread is related to the previous thread here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1265044/am5706-how-to-change-vendor-id-or-device-id-of-pcie-in-am5706

YES.

Thanks and regards,

S. Venkateswara Reddy

Hi Venkateswara,

I looked through the code some more and found in pdk_am57xx*/packages/ti/csl/src/ip/pcie/V1/priv/pcie.c's PCIEBarConfig function that we are setting BAR like this:

Could you try hardcoding the desired BAR address instead and see if that affects BAR registers?

Regards,

Takuma

Hi Takuma,
I am able to write and read PCIe registers properly.
But I am unable send data from RC(HOST) to EP(FPGA). Only link is success.

As per the example code data send and receive from RC to EP using the following API
Cache_inv (ptr, size, Cache_Type_ALLD, TRUE);
How this API send and receive the data? I mean how this Cache API controls the PCIe to send and receive functionality.

Please share me, if you have any working code(using A15 core).

Just for your information, this driver is important and urgent to my project. Please help me.

Thanks and regards,
S. Venkateswara Reddy

Hi Venkateswara,

Apologies for the delay. Unfortunately, I will need some more time to look into the code to answer the latest questions.

However, as an alternative and to answer your question about working code using the A-core, there is a Linux driver for PCIe for AM57x that should run on the A15 core: https://software-dl.ti.com/processor-sdk-linux/esd/AM57X/08_02_01_00/exports/docs/linux/Foundational_Components/Kernel/Kernel_Drivers/PCIe/PCIe_Root_Complex.html.

I would assume this is similar to the Linux driver used with our current generation AM6x family of devices, so code should be maintained more - and I should be able to comment more on this driver as I am more familiar with the Linux driver.

Regards,

Takuma

Hi Takuma and Venkateswara , 

I am also facing the same issues as Venkateswara , My code is getting stuck in cache_invalidate. Also im not able to write any PCIe Registers.

I am using Sitara AM571x Processor as Root Complex and Kintex FPGA as Endpoint . 

/* Data sent to EP.
RC waits for the loopback to be completed and
receive data back from EP */

do {
cache_invalidate ((void *)dstBuf.buf, PCIE_EXAMPLE_DSTBUF_BYTES);
} while(dstBuf.buf[PCIE_BUFSIZE_APP] != PCIE_EXAMPLE_BUF_FULL);

looking forward for your assistance , 

Regards , 

Pariposh.

Hi Venkateswara , 

Can you please share how you are able to write and read PCIe registers properly?

Regards , 

Pariposh.

Hi Pariposh,

Is your BAR register also not changing from the default values like Venkateswara?

Regards,

Takuma

Hi Takuma, 

I need the default values only ,

But when i am writing data from RC to EP , its not getting reflected and getting stuck in do-while loop,

/* Write from RC to EP */
for (i=0; i<PCIE_BUFSIZE_APP; i++)
{
pciedstBufBase->buf[i] = srcBuf[i];
}

do {
cache_invalidate ((void *)dstBuf.buf, PCIE_EXAMPLE_DSTBUF_BYTES);
} while(dstBuf.buf[PCIE_BUFSIZE_APP] != PCIE_EXAMPLE_BUF_FULL);

when i am printing this pciedstBufBase->buf[i] its coming as 0xffffffff

Is there a way to disable this cache invalidation ?

Hi Pariposh,

I think Venkateswara is also facing the same issue.

Venkateswara Reddy Saripalli said:

I am able to write and read PCIe registers properly.

^ I think they are able to write the PCIe configuration registers such as vendor ID, device ID, etc, but not able to write actual data.

Expertise and prior knowledge in terms of PCIe RTOS code, especially for AM57x, is low, so I will need some time to build up an understanding on how this works. The cache invalidation is most likely to maintain cache coherency between A-cores/other remote cores of EP, DDR, and PCIe device.

Regards,

Takuma

Hi Takuma,

Good morning.

Any update on this. Share me if you have working code in sysbios.

Thanks and regards,

S. Venkateswara Reddy

Hi Venkateswara,

Unfortunately, I have not been able to get to experiments with the AM57x. I expect around a month or two of delay.

In the meantime, let me know if there was any progress on your end, or if there is any experiments and findings you have had.

Regards,

Takuma

Hi Takuma,

The following tests we have done:

Test1:
We have taken two EVM572x setups connected PCIEs back to back.
Tried to use one EVM as RC and another EVM as EP (used sample project - PCIE_idkAM572x_wSoCFile_armExampleProject). Try to send and receive the data.
RESULT - link is up and sending and receiving data from RC to EP to RC(loopback) is working fine.

Test2:
We have taken two same 574x custom board connected PCIEs back to back.
Tried to use one board as RC and another board as EP (used sample project - PCIE_idkAM572x_wSoCFile_armExampleProject) try to send and receive the data.
RESULT - link is up but data is not transferring.

Test3:
We have taken one EVM572x and 574x custom board connected PCIEs back to back.
Tried to use EVM572x as RC and 574x as EP.
RESULT - link is up but data is not transferring.

Test4:
We have taken one EVM572x and 574x custom board connected PCIEs back to back.
Tried to use 574x as RC and 572x as EP.
RESULT - link is up but data is not transferring.

Any suggestions??

Thanks and regards,

S. Venkateswara Reddy

Hi Venkateswara,

If two TI EVM works, and if custom board is involved the example fails, then my suspicion would be hardware related, or board pinmuxing related.

Are there any differences between the custom board and TI EVM in terms of which pins are connected/disconnected? For example, any differences in terms of these auxilary signals:

• Reference clock (REFCLK), must be supplied by the system (see Section 2.1.1)
• Add-in card presence detect pins (PRSNT1# and PRSNT2#), required
• PERST#, required
• JTAG, optional
• SMBus, optional
• Wake (WAKE#), required only if the device/system supports wakeup and/or the OBFF
mechanism
• +3.3Vaux, optional

Or differences in which PCIe controller instance, PCIe lane, or serdes being used?

Regards,

Takuma

Hi Takuma,
Thanks for your information. I will check it once.

For your information:
We connected PCIe between AM5748-custom board with TI EVM 5728(which is working).
DSP version is working fine. Data transferred RC to EP to RC(loopback).
ARM version is not working.

Any suggestion from your side???

Thanks and regards,
S. Venkateswara Reddy

Hi Venkateswara,

Thanks for the additional information. If the DSP version is working, then I assume all of the signals are correct. My next suspicion would be with interrupt routing. It might be that the interrupts routed to DSP is not routed to ARM.

• Are legacy interrupts being used?
• Would you happen to know if interrupts are set up similar to how interrupts are set up in Linux device tree: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm/boot/dts/dra7.dtsi?h=ti-linux-4.19.y#n335.

Regards,

Takuma

Unlocking the thread for further communication on the thread.

Regards

Karan