TDA2xx: ProcessorSDK 3.01 PDK PCIe example/PCIe optimization

Hi Nick,

TDA2xx is not a Keystone device.

As far as I know EDMA gives best throughput when ACount=16k, CCount=1, BCount=x.

Let me find if there is an EDMA configuration which gives better throughput.

There is lot of documentation freely available online which you can refer to see PCIe packet structure.

Also the numbers you are seeing are skewed as there are three UART prints in between.

Can you comment the prints as done below and try:

//UARTPuts("RC: Link is UP\r\n\r\n", -1);

/*Configure PCIe traffic*/
PcieAppConfigTrafficRemote();
/*Wait till mem space is enabled*/
while (memSpaceEnable != 1U) ;
//UARTPuts("RC: Mem space is enabled\r\n", -1);
//UARTPuts("RC: Sending data to EP\r\n", -1);
/*Init EDMA and enable transfer*/
PcieAppEdmaInit();

Regards,

Rishabh

Hi Rishabh,

Here's what I tried and the corresponding results:

DONE: declared the dataTxBuffer on RC side as below:
static volatile uint32_t dataTxBuffer[DATA_SIZE_IN_WORD] __attribute__((aligned(128)));
RESULT: Nothing changed regarding throughput.

DONE: enabled 2 lanes in PCIE_NUMBER_OF_LANES
RESULT: Nothing changed regarding throughput.

I am currently looking at this document: www.ti.com/.../sprac21.pdf. Table 42 on page 78 is measured performance for pcie operations in X2 GEN2 mode. For WR operation from RC to EP it is stated that measured throughput is 5966.86Mbps which is ~750MBps (for reference I get 370MBps only).
On page 76 there are configurations with which the performance was measured. The EDMA and PCIe configurations are exactly the same as in the test from PDK so I suppose that test was used when performance was measured. The only configuration that I am not sure is exactly the same for me is timer configuration, could you help me check that?

Also, I looked into PCIe protocol in more detail and would like to know if the performance measured in the document above is the throughput regarding payload only or the whole packets with 8b/10b encoding that, ultimately, travel through the PCIe link.

Also, I would be very grateful if you could run the test with one and two lanes and see if there is a difference in measured throughput.

Thank you in advance.

Regards,
Nick

Hi Nick,

The test used for this document is different from the one given in PDK.

In this document there are two tables. X1 is for single lane and X2 is two lanes.

With the current system you are able to get 370 MBps for single lane and the document says 3013.15Mbps (376.6 MBps).

For dual lane the performance should definitely improve (almost twice).

Can you one again confirm throughput with a clean build for PCIe Gen2 with two lanes.

Regards,

Rishabh

Hi Rishabh,

I just installed clean PROCESSOR_SDK_VISION_03_02_00_00, did a clean build of the pdk drivers, sbl and examples. I only changed the NUMBER_OF_LANES macro definition to 2 in pcie_app.h, nothing else. Still no success, it says that throughput is 370MBps.

Here's the console output from both cores below:

I have no idea what could be the problem.

Regards,
Nick

Hi Rishabh,

Actually I am testing on a custom board which connects the two SoCs via 2-lane PCIe, I checked our schematics. Are you trying to say that inside the SoC there is only one lane connected to the PCIe subsystem 1, no matter if there are 2 lanes externally connected to the pins?

Thank you for your help so far.

Regards,
Nick

Hi Rishabh,

I'm very happy to inform you that I have found what the problem was. Having in mind the hint about PHY you mentioned, I stumbled upon the Table 26-61. PCIePHY Subsystem Low-Level Programming Sequence in TDA2xx TRM (page 7274). There I found two additional steps that were required in order to utilize 2 lanes in PCIe Subsystem 1 that were not present in the original test from PDK. Those are: "Configure the PCIE PHYs to ×1 or ×2 mode", "Configure the PCIESS2 Power Control clock frequency to match SYS_CLK1 in MHz" and "Power up PCIESS2_PHY_TX and PCIESS2_PHY_RX".

After setting correct values in those registers in the beginning of the PlatformPCIESS1CtrlConfig() function in the pcie_app.c, I am getting steady 740 MBps which is close to the measured value from the TDA2xx performance document.

The PlatformPCIESS1CtrlConfig() now looks like this:

void PlatformPCIESS1CtrlConfig(void)
{
    uint32_t regVal;

    /*ENABLE x2 MODE*/
    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_CONTROL);

    HW_SET_FIELD32(regVal, CTRL_CORE_PCIE_CONTROL_PCIE_B1C0_MODE_SEL,
                    0x01U);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_CONTROL,
                regVal);

    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_CONTROL);

    HW_SET_FIELD32(regVal, CTRL_CORE_PCIE_CONTROL_PCIE_B0_B1_TSYNCEN,
                    0x01U);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_CONTROL,
                regVal);

    /*CONTROL MODULE PWR CTL REG status of PCIeSS1*/
    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_PCIESS1);

    HW_SET_FIELD32(regVal, CTRL_CORE_PHY_POWER_PCIESS1_PCIESS1_PWRCTL_CMD,
                    0x03U);

    HW_SET_FIELD32(regVal, CTRL_CORE_PHY_POWER_PCIESS1_PCIESS1_PWRCTL_CLKFREQ,
                    0x1AU);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_PCIESS1,
                regVal);

    /*CONTROL MODULE PWR CTL REG status of PCIeSS2*/
    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_PCIESS2);

    HW_SET_FIELD32(regVal, CTRL_CORE_PHY_POWER_PCIESS2_PCIESS2_PWRCTL_CMD,
                    0x03U);

    HW_SET_FIELD32(regVal, CTRL_CORE_PHY_POWER_PCIESS2_PCIESS2_PWRCTL_CLKFREQ,
                    0x1AU);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_PCIESS2,
                regVal);

    /*Set PCIeSS1 delay count*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_PCS,
                    CTRL_CORE_PCIE_PCS_PCIESS1_PCS_RC_DELAY_COUNT, 0xF1U);
    /*Set PCIeSS2 delay count*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CTRL_CORE_PCIE_PCS,
                    CTRL_CORE_PCIE_PCS_PCIESS2_PCS_RC_DELAY_COUNT, 0xF1U);
}

I would like to thank you again for all the help and I hope that people will find this post helpful if they have similar problems.

Best regards,
Nick