• State TI Thinks Resolved
  • Locked Locked
  • Replies 10 replies
  • Answers 1 answer
  • Subscribers 32 subscribers
  • Views 903 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM2432: PCIe driver doesn't work correctly on Gen 1 mode.

Hideaki Matsumoto
Genius 9600 points
Part Number: AM2432
Other Parts Discussed in Thread: TMDS64EVM, SYSCONFIG

Tool/software:

Hi,

 

My customer is considering to use AM243x for EtherCAT module, but they have a problem on the PCIe driver.

Could you help to answer their questions below ?

 

Q1) Could you make PCIe driver support PCIe Gen 1 as well ?

Q2) If Yes, when can they get the driver which PCIe works correctly ?

 

There is an known issue regarding PCIe in the Release note of mcu_plus_sdk_09_02_01_05.

 

 

From the above known issue, it seems that PCIe driver in SDK 9.2 cannot be used.

Therefore, the customer tried some PCIe drivers in some versions of SDK like below, but there is no driver which PCIe Gen 1 works correctly.

 

 

                                                     EP mode                          RC mode

09.01.00.41           Gen 1     Doesn’t exit from driver        Doesn’t exit from driver

                              Gen 2     Data Transfer is working.     Data transfer is working.

 

09.02.00.50           Gen 1     Doesn’t exit from driver        Doesn’t exit from driver

                              Gen 2     Data Transfer is working.     Data transfer is working.

 

09.02.01.05           Gen 1     Can link-up, but can’t transfer data     Can link-up, but can’t transfer data

                              Gen 2     Can link-up, but can’t transfer data     Data transfer is working

Again, please help to support them.

Q1) Could you make PCIe driver support PCIe Gen 1 as well ?

Q2) If Yes, when can they get the driver which PCIe works correctly ?

 

 

Thanks and regards,

Hideaki

  • 0
    TI__Mastermind 27490 points

    Hi Hideaki,

    Thanks for your query.

    Can you provide some background information?

    AM243x working as EP?

    What about RC (A windows/Linux PC or AM243x-RTOS-RC) ?

    Hideaki Matsumoto said:
    There is an known issue regarding PCIe in the Release note of mcu_plus_sdk_09_02_01_05.

    Yes, we have some known problems with SDK 9.2.1

    PCIE driver updation is in progress from SDK 9.2.1 onwards.

    Currently, only below example is based on new driver. 

    C:\ti\mcu_plus_sdk_am243x_09_02_01_05\examples\drivers\pcie\pcie_enumerate_ep\am243x-evm

    Other examples porting is pending acitivity.

    Regards
    Ashwani

  • 0 in reply to Ashwani Goel
    TI__Genius 9600 points

    Hi Ashwani,

    Thank you for your reply.

    Ashwani Goel said:

    Can you provide some background information?

    AM243x working as EP?

    What about RC (A windows/Linux PC or AM243x-RTOS-RC) ?

    They’re now using AM243x for both RootComplex and EndPoint. (Actually using AM64x EVM for now)

    TMDS64EVM (R5F) : RC  --------  EP : TMDS64EVM (R5F)

    Could you tell them when Gen 1 can be supported ?

                                                 AM243x R5F EndPoint                AM243x (R5F) RootComplex

    09.01.00.41           Gen 1     Doesn’t exit from driver                    Doesn’t exit from driver

                                  Gen 2     Data Transfer is working.                 Data transfer is working.

     

    09.02.00.50           Gen 1     Doesn’t exit from driver                    Doesn’t exit from driver

                                  Gen 2     Data Transfer is working.                 Data transfer is working.

     

    09.02.01.05           Gen 1     Can link-up, but can’t transfer data     Can link-up, but can’t transfer data

                                  Gen 2     Can link-up, but can’t transfer data     Data transfer is working

    Thanks and regards,

    Hideaki

  • 0 in reply to Hideaki Matsumoto
    TI__Mastermind 27490 points

    Hi Hideaki,

    • Gen 1 support is mandatory in any gen 2 hardware.
    • Gen 1 is just a serdes speed downgrade (3.125G vs 6.25G), no other major changes

    Can you try "pcie_msi_irq" test case from SDK 09.02.01.05 and update the results?

    Regards

    Ashwani

  • 0 in reply to Ashwani Goel
    TI__Genius 9600 points

    Hi Ashwani,

    The customer tried "pcie_msi_irq" test case from SDK 09.02.01.05 which you suggested, but the phenomenon has not been improved.

    Could you tell us if you or anyone confirmed the PCIe driver on AM243x worked with Gen 1 mode correctly ?

     

    They’re sure that there are some bugs in the PCIe driver like below.

    Could you please fix these bugs as soon as possible.

    • The driver doesn’t wait for PCIe link-up in the case of EP mode.
    • Region Windows Size of inbound cannot be changed on the SysConfig tool.
      => They are guessing that this is one of the root causes of the issue that data transfer cannot be operated.

     

    Could you check if there is no problem on their test procedure below ? 

        -  Use CCS ver. 12.7.1
        -  Use two AM243x (TMDS64EVM) connected via PCIe cable.
            One is operated on End-Point mode, and another is operated on Root-Complex mode.
        -  Same version of PCIe drivers are used on both EP and RC
        -  For changing between Gen1 and Gen2, only selecting Gen1 or Gen2 on the SysConfig.
        -  Tested in the following two cases.
        -  Just running program
        -  Set the break point in the function of pcie_msi_irq_xx_main() on both EP and RC, and read the register at the address 0x0F10_0014 to check Gen1 or Gen2.

     

    * [Status 1] (Almost same between RC and EP, but a few function are different.)

                                                             EP mode                                               RC mode

    09.01.00.41        Gen 1        Doesn’t exit from driver [Status 1*]       Doesn’t exit from driver [Status 1*]
                                                 reg (0xF100014) = 0x10000113           reg (0xF100014) = 0x10000113
                                                 No change from Gen2                          No change from Gen2

                               Gen 2        Data Transfer is working.                      Seems to be OK, because Device ID=100X and Vendor ID=17CDX are displayed.
                                                 reg (0xF100014) = 0x10000113           reg (0xF100014) = 0x10000113
                                                 ( Gen2 )                                                 ( Gen2 )

     09.02.00.50       Gen 1       Doesn’t exit from driver [Status 1*]        Doesn’t exit from driver [Status 1*]
                                                reg (0xF100014) = 0x10000113            reg (0xF100014) = 0x10000113
                                                No change from Gen2                           No change from Gen2
                               Gen 2       Data Transfer is working.                       Seems to be OK, because Device ID=100X and Vendor ID=17CDX are displayed.
                                                reg (0xF100014) = 0x10000113            reg (0xF100014) = 0x10000113
                                                 ( Gen2 )                                                ( Gen2 )

    09.02.01.05                      On the EP Example of this version, “Device in EP mode” is displayed on the console
    (Latest version)              even though in the stand-alone operation (no physical PCIe connection with RC).
                                              On the RC Example, Nothing is displayed in the stand-alone operation with no connection with EP.
                                               => Keep Status 1*. They think this is correct operation because it’s waiting for PCIe connection with EP.
                                              Since the PCIe driver of this version can set Vendor ID / Device ID, they set the different values between EP and RC like below.
                            
    　                                        EP mode                                                    RC mode
                                               Vendor ID = 0x17C4                                   Vendor ID = 0x17C5
                                               Device ID = 0xeee4                                    Device ID = 0xeee5

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                EP mode                                                      RC mode
                              Gen 1       Supposably NG                                        Supposably NG
                                               reg (0x0F100014) = 0x10000103             reg (0x0F100014) = 0x10000103
                                               Could change to Gen 1                           Could change to Gen 1

                              Gen 2       Unclear                                                    Seems to be OK, because Device ID = EEE4X and Vendor ID = 17C4X are displayed.
                                               reg (0x0F100014) = 0x10000113            reg (0x0F100014) = 0x10000113
                                               (Gen 2)                                                     (Gen 2)

     

    Thanks and regards,

    Hideaki

  • 0 in reply to Hideaki Matsumoto
    TI__Mastermind 27490 points
    Hideaki Matsumoto said:
    Could you tell us if you or anyone confirmed the PCIe driver on AM243x worked with Gen 1 mode correctly ?

    We have not tried or tested Gen 1 mode specifically.

    Hideaki Matsumoto said:
    They’re sure that there are some bugs in the PCIe driver like below.

    I will work with driver team and get back to you on this.

    Regards

    Ashwani

  • 0 in reply to Ashwani Goel
    Mastermind 7620 points

    Hello Hideaki,

    Hideaki Matsumoto said:
    The driver doesn’t wait for PCIe link-up in the case of EP mode.

    this is intentional, as an EP needs to perform additional configuration that is only possible once there is a reference clock. See the pcie_enumerate_ep for an example. If your customer wants to delay execution in the EP until a link is established they can call "Pcie_waitLinkUp()" before the call to "Pcie_cfgEP()":

        DebugP_log("Device in EP mode\r\n");

    status = Pcie_waitLinkUp(gPcieHandle[CONFIG_PCIE0]);
    DebugP_assert(SystemP_SUCCESS == status);

    /* Configure End Point */
    status = Pcie_cfgEP(gPcieHandle[CONFIG_PCIE0]

    Hideaki Matsumoto said:
    Region Windows Size of inbound cannot be changed on the SysConfig tool.
    => They are guessing that this is one of the root causes of the issue that data transfer cannot be operated.

    I believe they're talking about the EP here. In that case it is intentional, as for an EP you need to configure the BAR aperture. Configuring the BAR aperture also configures the inbound ATU accordingly.

    Regarding the main topic of this thread:

    I've just checked the pcie_msi_irq examples. It does work as expected when either the RC or both the RC and EP are configured for Gen1. It does indeed not work when only the EP is configured for Gen2.

    The reason for this behavior is that the RC was modified as little as possible for SDK 09.02.01.05, preserving existing behavior. The existing code checked for the configured link parameters, so that behavior was preserved for the RC code. For the EP case, it is left to the application to a) wait for link up (see above) and b) verify link parameters.

    Before 09.02.01.05, the code would only check if a link was established with the configured parameters.

    Since 09.02.01.05, the code actually limits its own link to Gen1, for both RC and EP. For the EP, the checks were removed, for the RC, the checks were kept.

    If they want the RC to optionally support Gen1 or Gen2, and only limit then EP, they would have to remove the following lines from the pcie_v0.c and recompile the MCU+ SDK:

            if (SystemP_SUCCESS == status && config->attrs->operationMode == PCIE_RC_MODE)
        {
            /* Verify Link width & speed */
            status = Pcie_checkLinkParams(object->handle);
        }

    Best Regards,

    Dominic

  • 0 in reply to Dominic Rath
    TI__Genius 9600 points

    Hi Dominic,

    Thank you so much for answering to some questions.

    Let me reconfirm, you did confirm below by using AM243x EVM or other boards ?

    Dominic Rath said:
    I've just checked the pcie_msi_irq examples. It does work as expected when either the RC or both the RC and EP are configured for Gen1.

    Not only link but also data communication work correctly ?

     

     

    Could you explain below more detail ? 09.02.01.05 only supports Gen 1 ?

    Dominic Rath said:
    Since 09.02.01.05, the code actually limits its own link to Gen1, for both RC and EP. For the EP, the checks were removed, for the RC, the checks were kept.

     

    The next release MCU+ SDK 10.1 also will be same as above ?

    If so, they have to remove some lines and recompile the MCU+ SDK ?

     

     

    Thanks and regards,

    Hideaki

  • 0 in reply to Hideaki Matsumoto
    Mastermind 7620 points

    Hello Hideaki,

    Hideaki Matsumoto said:
    Let me reconfirm, you did confirm below by using AM243x EVM or other boards ?

    I confirmed using one AM24x and one AM64x EVM.

    Hideaki Matsumoto said:
    Not only link but also data communication work correctly ?

    Yes, communication working correctly.

    I noticed one issue with the pcie_msi_irq example in 09.02.01.05 in a different context: Depending on the startup order and timing variations you could see the following output:

    Endpoint Device ID: FFFFX
Endpoint Vendor ID: 1X

    I had to re-try a couple of times until I got that issue. This should not happen if EP is started before RC. A proper fix is also possible:

            /*
         * retry reading vendor and device ID in case of CRS completion
         */
        do {
            status = Pcie_getVendorId(gPcieHandle[CONFIG_PCIE0], PCIE_LOCATION_REMOTE, &vndId, &devId);
        } while (status == SystemP_SUCCESS && vndId == 0x0001 && devId == 0xffff);

    Hideaki Matsumoto said:
    Could you explain below more detail ? 09.02.01.05 only supports Gen 1 ?

    Old behavior up to 09.02.00:

    SysConfig Gen 2: PCIe core can establish link at Gen 1 OR Gen 2, but software spins until link is Gen 2. If link can only be estbalished at Gen 1, e.g. because the remote end only supports Gen 1, the software spins forever.

    SysConfig Gen 1: PCIe core can establish link at Gen 1 OR Gen 2, but software spins until link is Gen 1. If link is established at Gen 2, the software spins forver.

    Old behavior is the same for RC and EP.

    New behavior starting with 09.02.01:

    SysConfig Gen 2: PCIe core can establish link at Gen 1 OR Gen 2. The EP driver doesn't verify which Gen was established, and leaves that to the application. The RC driver still verifies that a link was established at Gen 2.

    SysConfig Gen 1: PCIe core can establish link ONLY at Gen 1. The EP driver doesn't verify which Gen was established, and leaves that to the application. The RC driver still verifies that a link was established at Gen 1 (which should be true).

    New behavior is different for RC and EP.

    Hideaki Matsumoto said:

    The next release MCU+ SDK 10.1 also will be same as above ?

    If so, they have to remove some lines and recompile the MCU+ SDK ?

    I can't comment on upcoming MCU+ SDK releases. I believe it makes sense to modify the RC driver to NOT verify which Gen a link is, and leave that to the application.

    Until such a change is available in a MCU+ SDK release, they could remove some lines and recompile the MCU+ SDK to get the desired behavior.

    Regards,

    Dominic

  • 0 in reply to Dominic Rath
    TI__Genius 9600 points

    Hi Dominic,

    Thank you so much for your answer and sorry for my delayed response.

    The customer has been able to confirm that PCIe communication between two AM243x EVMs is working by some modifications.

    Could you check if there is no problem in their modification of SDK for PCIe Gen 1 usage ?

     

    • Version :         MCU+ SDK AM243x Ver.9.01.00.41
    • Library :          Drivers.am243x.r5f.ti-arm-clang.debug.lib (No modification)

     

    [ Modification ]

         1. Set the structure “Pcie_DevParams” .linkSpeed as the structure “Pcie_Attrs” .gen

         2. Call Pcie_open()     (In the example code, Drivers_open() is called.)

     

    < Details >

    Added the following five lines.

     

    extern Pcie_InitCfg Pcie_initCfg;
    extern Pcie_Attrs gPcieAttrs[];

    void func(void)
    {
        Pcie_DevParams  *p_Pcie_DevParams;
        p_Pcie_DevParams = Pcie_initCfg.dev.basesPtr[0]->devParams;
        p_Pcie_DevParams->linkSpeed = gPcieAttrs[0].gen;

        Drivers_open();

           . . .

    }

     

    [ Results ]

    They confirmed that the transfer time of Gen 1 became twice of that of Gen2.

    • How to confirm

                  In the Block transfer

                    Set GPIO to High when the first data of block is received at receiver.

                    Set GPIO to Low when the last data is received.

                    Measure the time between High and Low.

     

    • Confirmed Register

                  Address : 0x0F10_0014

    -  Bit [5:4] = 01 (at Gen 2 setting),  00 (at Gen 1 setting)

     

    Reference :

        PCIE0_LINKSTATUS Register (Offset = 14h) 

            bit[5:4] :  NEGOTIATED_SPEED 

                    11:  16  GT/s
                    10:   8   GT/s
                    01:   5   GT/s
                    00:  2.5 GT/s

      

    [ Details of Operation in Driver ]

     The structure “Pcie_DevParams” is declared with the initial value in the pcie_soc.c.

    “Pcie_DevParam” .linkSpeed is fixed as PCIE_GEN2. ---- 1)

     

    The structure “Pcie_Attrs” defined in the pcie.h is declared with the initial value in the ti_drivers_config.c.

    This initial value is set by SysConfig tool.

    • If Gen 1 is set by SysConfig,

               “Pcie_Attrs” .gen = PCIE_GEN1  ---- 2)

     

    When Gen 1 is set by SysConfig, the setting of 1) and 2) doesn’t match, False.

    This is the root cause why program counter cannot exit from Pcie_open() and Gen 1 didn’t work.

     

    If their modification mentioned above is applied, 1) can match with 2), the program can exit and Gen 1 can work.

     

     

    Please check if their modification is correct.

     

     

    Thanks and regards,

    Hideaki

     

     

     

  • 0 in reply to Hideaki Matsumoto
    Mastermind 7620 points

    Hello Hideaki,

    yes, that modification is what limits the PCIe speed to Gen 1. A device configured that way will only establish a Gen 1 connection, and then the 2) check succeeds.

    If they want to ensure their device only works at Gen 1, then yes, this modification is correct.

    Regards,

    Dominic