• State Resolved
  • Locked Locked
  • Replies 21 replies
  • Answers 1 answer
  • Subscribers 32 subscribers
  • Views 979 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

MCU-PLUS-SDK-AM243X: does not configuration PCIe endpoint configuration with mcu_plus_sdk_am243x_11_00_00_15

mitsuhiko igarashi
Prodigy 20 points
Part Number: MCU-PLUS-SDK-AM243X
Other Parts Discussed in Thread: SYSCONFIG

Tool/software:

Hi,

We found bug PCIe endpoint setting with mcu_plus_sdk_am243x_11_00_00_15.

Pcie_open() in mcu_plus_sdk_am243x_11_00_00_15\source\drivers\pcie\v0\pcie_v0.c calls Pcie_LtssmCtrl() with out calling Pcie_cfgEP().
Then this driver cannot configuration PCIe endpoint configuration.

Could you fix this issue?

And we want to know reason why removing “else” part of If(PCIE_RC_MODE).
Is this intentionally?

Regards,

  • 0
    TI__Mastermind 27440 points

    Hi mitsuhiko igarashi,

    Thanks for your query.

    Can you please help me with your setup ?

    • It is AM243x-RTOS-RC <=> AM243x-RTOS-EP ?
    • or PC-RC <=> AM243x-RTOS-EP ?

    Regards

    Ashwani

  • 0 in reply to Ashwani Goel
    Prodigy 20 points

    Hi Ashwani,

    PC-RC <=> AM243x-RTOS-EP.

    Regards,

  • 0 in reply to mitsuhiko igarashi
    TI__Mastermind 27440 points

    Thanks for confirmation.

    mitsuhiko igarashi said:
    And we want to know reason why removing “else” part of If(PCIE_RC_MODE).

    The updated PCIE drivers support only RTOS based generic EP not RTOS based generic RC

    Please refer below documentation 

    AM64x and AM243x: MCU+ SDK-Based PCIe End Point

    AM64x MCU+ SDK: PCIE EP Enumeration

    Regards

    Ashwani

  • 0
    Mastermind 7560 points

    That change is intentional. The pcie_enumerate_ep example should show how this is intended to be used. 

    By splitting cfgEP from the driver initialization, it is possible to modify the ep configuration as soon as a link is established. This requires a reference clock, which might not be available at driver initialization.

    Not all the existing examples were adapted to this changed approach for EP (only msi irq, iirc). 

    RC was kept as is. 

    Regards, 

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi,

    >it is possible to modify the ep configuration as soon as a link is established.

    We think “it must be  modify the ep configuration as soon as a link is established.”.

    According to link training status transition, it takes 1ms from Configuration of LTSSM to L0.

    Then we needs to call Pcie_cfgEP() in 1ms form calling Pcie_LtssmCtrl() with using SDK.

     

    We think Current sample takes over 1msec form Pcie_LtssmCtrl() to Pcie_cfgEP().

    We think it needs to separate L1790 “status = Pcie_LtssmCtrl(object->handle, TRUE);” form current SDK.

    About splitting cfgEP from the driver initialization, we needs sample program which works well.

    If possible, could you provide sample program or patch for EXT_SITMPUSW-22?

    Regards,

    igarashi

     

     

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Did you check the pcie_enumerate_ep example? That should work with a PC RC, with example drivers for windows and linux. 

    The ep is configured to respond with "configuration retry status". The ep has 1 second to call cfgEP after a link is detected. 

    Please note I haven't tested this with 11, but that was working well for us with 10.0 and 10.1 (ever since 09.02.01).

    Regards, Dominic 

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    Thank you for your information.

    We are checking pcie_msi_irq.  

    We will check pcie_enumerate_ep.

    pcie_msi_irq did not work well.

    If possible, please let us know which sample is work well.

    We are slightly confusing about SDK samples(6 samples) related PCIe.

    Regards, 

    igarashi

  • 0 in reply to mitsuhiko igarashi
    TI__Mastermind 27440 points
    mitsuhiko igarashi said:
    We will check pcie_enumerate_ep.

    As Dominic mentioned this is working example with updated drivers in latest SDK.

    Regards

    Ashwani

  • 0 in reply to Ashwani Goel
    Prodigy 20 points

    Hi Ashiwani,

    Thank you for your information. I understood.

    BYW, how about status of EXT_SITMPUSW-22?

    I want to know schedule of this issue.

    Regards,

    Igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hi igarashi,

    sorry, I was only on my mobile phone these last few days.

    The note regarding pcie_msi_irq being adapted to the "new" driver was only with regard to EXT_SITMPUSW-22. For all I know pcie_msi_irq_rc/ep should work. Please note that all of these "existing examples" (those listed in EXT_SITMPUSW-22) have only ever worked properly with an AM64x RC and an AM64x EP. See this commit message for an explanation of the changes necessary for these "existing" examples: https://github.com/TexasInstruments/mcupsdk-core/commit/aec57244bfff3183edcfa517901c5292f13627a1.

    IMHO you should just ignore these "existing examples", unless you want to understand how to use a specific feature like e.g. legacy IRQs, or you just want to verify that two AM64x can communicate with each other as RC+EP.

    For something that shows how to use an AM64x as an EP with a "generic" RC (e.g. Windows or Linux), you should look at the pcie_enumerate_ep example. This is a lot more complex, but that should cover most of the assumptions of e.g. a PC BIOS. It was tested successfully on a few x86 systems running Linux and Windows.

    Also make sure you read the documentation for the EP enumeration example: https://software-dl.ti.com/mcu-plus-sdk/esd/AM64X/11_00_00_15/exports/docs/api_guide_am64x/EXAMPLES_DRIVERS_PCIE_ENUMERATE_EP.html, especially the section "Important concepts in the example". If you're building a PCIe EP that should go into "any" off-the-shelf PC, you'll be facing some challenges to comply with the PCIe requirements descibed in the documentation. Either you're able to boot very fast (I haven't tested this, and I'm not sure if it's feasible to boot in < 120ms), or you need to rely on hot-plugging so that your EP can be detected "later", when the OS is running, which gives the EP plenty of time to initialize.

    Life is a lot easier when you're designing a closely coupled system of an x86 RC and AM64x EP. In that case you can make sure that the EP boots before the x86 reset gets released.

    Best Regards,

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    Thank you for your information.

    I understood current SDK status. About EXT_SITMPUSW-22, current SDK had tested an AM64x RC and an AM64x EP.

    Basically, SDK supports PCIe connection between PC RC and AMXX evaluation board.

    And if customer wants to connect other RC, customer need to modify according to reference to EP enumeration example.

    I will discuss internally.

    Regards,

    igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hello igarashi,

    mitsuhiko igarashi said:
    I understood current SDK status. About EXT_SITMPUSW-22, current SDK had tested an AM64x RC and an AM64x EP.

    Not exactly. In MCU+ SDK up to 09.02.00, there were only examples suitable for AM64x RC+EP.

    With MCU+ SDK 09.02.01 the pcie_enumerate_ep example was added. This is the only example suitable for x86 RC + AM64x EP.

    Adding the pcie_enumerate_ep example required some changes to the driver, and those driver changes required changes to the "existing" examples (those that were included before 09.02.01), but only pcie_msi_irq was adapted, the other examples were not updated, and don't work anymore starting with 09.02.01. That's what EXT_SITMPUSW-22 is about.

    With 09.02.01 there are also examples for host "drivers" for the pcie_enumerate_ep example for Windows on Linux, but they work only on x86. Using an ARM Linux RC instead would probably require modifications, due to the lack of an IOMMU on many ARM systems.

    Regards,

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    Thank you for your information.

    I got it.

    BTY, in case of combination of ARM RTOS RC and AM243x EP, could TI support this use case with SDK?

    We are planning this use case now.

    Regard,

    igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hello igarashi,

    Please note I'm not TI, so I can only comment from a technical POV.

    Do you have a specific ARM RTOS RC in mind? E.g. AM64x RC + AM243x EP both running MCU+ SDK is easy. This used to work, the MSI example should still work, and adapting the other examples is simple. 

    What do you have in mind?

    Regards,

    Dominic 

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    We are using ”pcie_msi_irq” with latest SDK(mcu_plus_sdk_am243x_11_00_00_15).

    Unfortunately, it does not work because Pcie_cfgEP() is not called in 1ms after calling Pcie_LtssmCtrl().

    Our environment as below.

    RC: Arm  RTOS 

    EP:TDMS243EVM(AM243x Evaluation Module)

    Regards,

    igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hello igarashi,

    mitsuhiko igarashi said:
    Unfortunately, it does not work because Pcie_cfgEP() is not called in 1ms after calling Pcie_LtssmCtrl().

    I'm not sure where this requirements comes from. I don't know of any such requirement. The PCIe driver in 09.02.01 (and later) sets the "configuration retry status" bit, and that gets the EP one whole second.

    If you're seeing a problem where

    Endpoint Device ID: FFFFX
Endpoint Vendor ID: 1X

    is printed on the RC when running AM64x/AM24x RC+EP with pcie_msi_irq_rc/ep, that's exactly because of CRS. In that case you could either

    If that's not your problem you need to explain in a bit more details what "RC: Arm RTOS" is exactly. If it is some other ARM controller (either TI or something else) running some other RTOS (i.e. not MCU+ SDK, but something like QNX or VxWorks or ...), the pcie_msi_irq_ep example is not useful, as it is really only to be used with pcie_msi_irq_rc.

    Regards,

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    >I'm not sure where this requirements comes from.

    This from PCI Express® Base Specification Revision 3.1a(NCB-PCI_Express_Base_r3.1a_December7-2015.pdf).

    Transition of Configuration state to L0 state is automatically in 1msec according to Link Training and Status State Machine.

    >the pcie_msi_irq_ep example is not useful, as it is really only to be used with pcie_msi_irq_rc.

    Thank you for your infromation. we will refer pcie_msi_irq_rc.

    Regard,

    Igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hello Igarashi,

    mitsuhiko igarashi said:

    >I'm not sure where this requirements comes from.

    This from PCI ExpressRegistered Base Specification Revision 3.1a(NCB-PCI_Express_Base_r3.1a_December7-2015.pdf).

    Transition of Configuration state to L0 state is automatically in 1msec according to Link Training and Status State Machine.

    yes, but I don't think cfgEP is necessary for that. From what I've seen the link is established and in L0 even before cfgEP gets called, but any configuration request should be replied with a "configuration retry status", see §2.3.1 in PCIe spec revision 3.1a:

    "For Configuration Requests only, following reset it is possible for a device to terminate the request but indicate that it is temporarily unable to process the Request, but will be able to process the Request in the future – in this case, the Configuration Request Retry Status (CRS) Completion Status is used"

    See §2.3.2 for how this can be detected in the RC:

    "For a Configuration Read Request that includes both bytes of the Vendor ID field of a device Function’s Configuration Space Header, the Root Complex must complete the Request to the host by returning a read-data value of 0001h for the Vendor ID field and all ‘1’s for any additional bytes included in the request"

    When you call Pcie_cfgEP, a number of configuration registers in the EP get written, and finally the bit that causes the AM64x to respond with CRS gets cleared, to allow the RC to enumerate the EP.

    You can test this yourself with a setup using two AM64x/AM24x as RC+EP and the pcie_msi_irq_rc/ep examples. Let the EP run first up to the point before it calls Pcie_cfgEP, then let the RC run to the point where it reads the vendor and device ID. You'll get 0xffff/0x0001 returned, indicating that the EP returned CRS. If you let the EP run until after Pcie_cfgEP gets called and then start the RC, you'll read the vendor and device ID configured via SysConfig.

    Regards,

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hi Dominic,

    Thank you for your advice.

    We will try to check "pcie_enumerate_ep " sample.

     >Let the EP run first up to the point before it calls Pcie_cfgEP, then let the RC run to the point where it reads the vendor and device ID

    In our case (rtos RC <->AM24x EP), we can read correct vendor and device ID after calling Pcie_cfgEP. However, cannot configure BAR settings.

    In your case, can RC set BAR after calling  Pcie_cfgEP?

    And I want to confirm about MSI interrupt functionality.

    Which sample does work well in latest SDK samples?

    Best regards,

    igarashi

  • 0 in reply to mitsuhiko igarashi
    Mastermind 7560 points

    Hello igarashi,

    I'm not sure I understand exactly what you're trying to do.

    mitsuhiko igarashi said:
    In our case (rtos RC <->AM24x EP), we can read correct vendor and device ID after calling Pcie_cfgEP. However, cannot configure BAR settings.

    Can you explain what "rtos RC" is, and what example you're refering to? Are you talking about the pcie_msi_irq_rc/ep examples? These examples assume matching configuration on the RC and EP, and are NOT suitable for anything except the matching rc/ep example on AM24x/AM64x. Specifically the RC doesn't configure BARs in this example.

    If you're using pcie_msi_irq_rc on the RC, can you explain how you're trying to configure the BAR in the EP? The PCIe driver API is NOT suitable for this. RC functionality in the MCU+ SDK is severely limited, and is basically only useful for the pcie_msi_irq_rc/ep test. You can work around those issues, but for that you need to implement some functionality yourself, directly accessing PCIe registers.

    mitsuhiko igarashi said:
    In your case, can RC set BAR after calling  Pcie_cfgEP?

    The pcie_enumerate_ep example support configuration of BARs. This has been tested using Windows and Linux as an RC.

    mitsuhiko igarashi said:

    And I want to confirm about MSI interrupt functionality.

    Which sample does work well in latest SDK samples?

    The pcie_enumerate_ep example supports MSIs and works with Windows and Linux RC. The pcie_msi_irq_rc/ep examples use MSIs and work, but again they're only useful in that exact combination. If your RC is something else, the pcie_msi_irq_ep example is not useful, and you should look into the pcie_enumerate_ep example.

    Regards,

    Dominic

  • 0 in reply to Dominic Rath
    Prodigy 20 points

    Hello Dominic,

    Thank you for your kindly support and sorry for delay reply.

    >Can you explain what "rtos RC" is, and what example you're refering to? 

    We are using pcie_enumerate_ep. We understand other samples are not working with latest SDK .

    Is this correct?

    "rtos RC" uses our original driver for our custom SoC.

     

    >The pcie_enumerate_ep example support configuration of BARs. This has been tested using Windows and Linux as an RC.

    Thanks. I got it.

    >The pcie_enumerate_ep example supports MSIs and works with Windows and Linux RC. The pcie_msi_irq_rc/ep examples use MSIs and work, but again they're only useful in that exact combination. If your RC is something else, the pcie_msi_irq_ep example is not useful, and you should look into the pcie_enumerate_ep example.

    Thanks. We will re-check pcie_enumerate_ep again.

    Best regards,

    Igarashi