• Resolved

AM5728: PCIE Performance

Part Number: AM5728

Hello, TI Experts,

 

Our customer plan to use AM5728 for their product.

And they sent us the question about PCIE performance of AM5728 by using TI-RTOS(PROCESSOR-SDK-RTOS-AM57X).

We found related wiki site like below for K2G(It seems to use Linux).

http://processors.wiki.ti.com/index.php/Processor_SDK_Linux_Kernel_Performance_Guide#PCIe_Driver

 

Question:

Is there any other appropriate document/web-site for PCIE performance of AM5728 by using TI-RTOS(PROCESSOR-SDK-RTOS-AM57X).

 

If there are any difference/notice to use this wiki-site PCIE performance information to AM5728, please also tell us.

 

Best regards,

  • Guru 50180 points

    In reply to matusan:

    Hi,

    For Q1/2/3,
    When the RC reads data from EP, the memory address is determined by the inbound translation in the EP side. See below code in pcie_sample.c:

    ibCfg.ibBar = PCIE_BAR_IDX_EP; /* Match BAR that was configured above*/
    ibCfg.ibStartAddrLo = PCIE_IB_LO_ADDR_EP;
    ibCfg.ibStartAddrHi = PCIE_IB_HI_ADDR_EP;
    ibCfg.ibOffsetAddr = (uint32_t)pcieConvert_CoreLocal2GlobalAddr ((uint32_t)dstBuf.buf);
    ibCfg.region = PCIE_IB_REGION_EP;

    if ((retVal = pcieIbTransCfg(handle, &ibCfg)) != pcie_RET_OK)

    Set a breakpoint here and check the value of ibCfg.ibOffsetAddr. I believe is internal memory, please double check.

    Where the data write to the RC side is determined by EDMA read function,

    *totalTimePointer=0;
    totalDMATime = 0;

    edmaTransfer(hEdma,(EDMA3_Type) EDMA_TYPE, (unsigned int*) remoteBuf, (unsigned int*) source,
    ACount, BCount, CCount, EDMA3_DRV_SYNC_A,totalTimePointer);

    totalDMATime += *totalTimePointer;
    PCIE_logPrintf("EDMA read %d bytes with %d cycles\n", (PCIE_EXAMPLE_LINE_SIZE*PCIE_EXAMPLE_UINT32_SIZE), (unsigned int)totalDMATime);

    Set a breakpoint, check the remoteBuf value. I believe is DDR, please double check.

    For Q4, RC writes into the EP, the memory type used is the same as RC reads from EP.

    Regards, Eric
  • In reply to lding:

    Hi,

    Thank you very much for your detail explanation.

    I really appreciate your help.

    I'd like to share the checked result below. (please refer the pdf in detail.)

     - Set a breakpoint here and check the value of ibCfg.ibOffsetAddr.

        -> We found "ibCfg.ibOffsetAddr=0x81082CD0".

             The address seems to be "DDR".

     - Set a breakpoint, check the remoteBuf value.

       -> We found "remoteBuf=0x21000A00".

            The address seems to be "PCIE_SS1".

    So, our understanding is as follows;

    - pcie_sample.c : EDMA read demo

        - EP   read from DDR & the data is transferred to RC.

        - RC  write to PCIE_SS1.

    Question:

     - Is this understanding correct?

     - Could you tell us the next/final destination of the data of PCIE_SS1 written by RC like above.      

        -> Is Final destination DDR on RC?

    Best regards,

    break.pdf

  • Guru 50180 points

    In reply to matusan:

    Hi,

    Thanks for the test!

    For the first breakpoint, ibCfg.ibOffsetAddr, it is 2147813632 (integer) = 0x8005_0900 (Hex). So the EP side, the buffer is inside DDR.
    For the second breakpoint, seems I made a mistake, please check the value of source, I think it should be an address in DDR.

    Regards, Eric
  • In reply to lding:

    Hi,

    Thank you very much for your kindness.

    I really appreciate your help.

    I checked the value of "source" like below. (please refer attached pdf)

      - source=0x81069400 (it seems to be an address in DDR as you said.)

    Thank you!

    And our customer sent us additional questions like below;

    Question:

    - 1: Could you explain the below definitions?

       #define PCIE_IB_LO_ADDR_RC   0x90000000

       #define PCIE_IB_HI_ADDR_RC   0

    -2: Are there any document or guide of below registers?

         - They would like to know how to configure those registers and recommended configuration.

         PCIECTRL_PL_IATU_INDEX

         PCIECTRL_PL_IATU_REG_CTRL_1

         PCIECTRL_PL_IATU_REG_CTRL_2

         PCIECTRL_PL_IATU_REG_LOWER_BASE

         PCIECTRL_PL_IATU_REG_UPPER_BASE

         PCIECTRL_PL_IATU_REG_LIMIT

         PCIECTRL_PL_IATU_REG_LOWER_TARGET  

         PCIECTRL_PL_IATU_REG_UPPER_TARGET

    Best regards,

    bp.pdf

  • Guru 50180 points

    In reply to matusan:

    Hi,

    Thanks for checking this! Yes, the buffer is inside DDR3.

    A1.
    #define PCIE_IB_LO_ADDR_RC 0x90000000
    #define PCIE_IB_HI_ADDR_RC 0

    This translate an incoming PCIE address into SOC internal memory address. For 32-bit BAR, the PCIE_IB_HI_ADDR_RC is zero (this is used for 64-bit BAR). For the meaning of: PCIE_IB_LO_ADDR_RC, you can refer to www.ti.com/.../sprabk8.pdf section 3.2.3 PCIe Inbound Address Translation Examples. Note: this document is for Keystone I/II device, but the inbound translation concept also applies to AM57x and well explained.

    In your example, you have a incoming PCIE address of 0x9000_0000, you minus this PCIE_IB_LO_ADDR_RC, then add the ibCfg.ibOffsetAddr = (uint32_t)pcieConvert_CoreLocal2GlobalAddr ((uint32_t)dstBuf.buf); =====>you got your SOC internal memory address.

    A2. Those registers are explained in AM572x TRM, section 24.9.7.5.1 PCIe_SS_PL_CONF Register Summary

    Regards, Eric
  • In reply to lding:

    Hi,

    Thank you very much for your kindness.
    I really appreciate your help.
    I would like to answer to the customer.

    Our customer sent us additional questions about "pcie_sample.c".
    They try to connect TMDXIDK5728(as EP) to their Windows-PC(as RC) to measure performance by using "pcie_sample.c".
    - It seems to be success "link" to create windows driver by themselves.
    - For the next step, they would like to know "BARx address information" to access to the DDR on TMDXIDK5728 from Windows-PC.

    Question:
    - Could you tell us the BAR address information like below?
    BAR0:?
    BAR1:?
    BAR2:?
    BAR3:?
    BAR4:?
    BAR5:?

    - What BAR address should be set to RC which running on Windows PC?
    - Are there any source code modification of "pcie_sample.c" running on TMDXIDK5728(as EP) to their Windows-PC(as RC)?
    - Which part should be refer in the "pcie_sample.c" to understand "BAR address configuration" running on TMDXIDK5728(as EP)?

    Best regards,
  • Guru 50180 points

    In reply to matusan:

    Hi,

    We don't do any test of PCIE EP to work with any host PC, like Linux or Windows machine. So there is no code example for it.

    On the PC side, you need a Windows driver that enumerate the PCIE bus, then the driver reads the BAR mask programmed by the AM5728 and allocate memory. Based on the allocated memory, the driver program BAR0/1/... 5. On the AM5728 side, you only need to program BAR mask. The Windows PCIE driver needs to program the Inbound and outbound translation of AM5728.

    The reference code for BAR mask is inside: pcieCfgEP(Pcie_Handle handle)
    /* Configure BAR Masks */
    /* First need to enable writing on BAR mask registers */
    if ((retVal = pcieCfgDbi (handle, 1)) != pcie_RET_OK)
    {
    return retVal;
    }

    /* Configure Masks*/
    memset (&getRegs, 0, sizeof(getRegs));
    memset (&setRegs, 0, sizeof(setRegs));
    type0Bar32bitIdx.reg.reg32 = PCIE_BAR_MASK;
    setRegs.type0BarMask32bitIdx = &type0Bar32bitIdx;

    /* BAR 0 */
    type0Bar32bitIdx.idx = 0; /* configure BAR 0*/
    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    PCIE_logPrintf ("SET BAR MASK register failed!\n");
    return retVal;
    }

    /* BAR 1 */
    type0Bar32bitIdx.idx = 1; /* configure BAR 1*/
    if ((retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &setRegs)) != pcie_RET_OK)
    {
    PCIE_logPrintf ("SET BAR MASK register failed!\n");
    return retVal;
    }

    /* Disable DBI writes */
    if ((retVal = pcieCfgDbi (handle, 0)) != pcie_RET_OK)
    {
    return retVal;
    }

    Regards, Eric
  • In reply to lding:

    Hi,

    Thank you very much for your kindness.
    I really appreciate your help.
    I would like to send the answer to the customer.

    Best regards,