• State Resolved
  • Locked Locked
  • Replies 12 replies
  • Subscribers 99 subscribers
  • Views 1451 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

problem of c6678 DSP MSI

TMS320C6678: multiple MSI interrupts generation problem.

Daniel Costin
Intellectual 610 points

Hi,

     I have a question related to the multiple MSI activatations.

    I read the https://e2e.ti.com/support/processors/f/791/t/649117?problem-of-c6678-DSP-MSI, but the things become more confusing and also the observed behavior seems to be extra confusing.

    We have a basic setup with 2 EVMc6678 connected though PCIe and we want to generate various MSIs on both directions.

    We configured the BAR0 and outbound regions on both sides such that to be able to access app registers for MSIs generation.

    1. The confusion is related to MSI_CAP, MSI_LOW32, MSI_UP32 and MSI_DATA registers configuration. If we want to use all 32 MSIs I understand that we need to config MSI_CAP[MSI_EN]=1, MSI_CAP[MULT_MSG_EN]=5.

     Why in your pdk examples you set MSI_CAP[MULT_MSG_CAP] field as this is read only?

     2. Do we need to respect any order in terms of RC and EP configuration for these MSI related registers?

     3. What values should we set for MSI_LOW32, MSI_UP32 and MSI_DATA for RC and EP on C6678?

   4.  The description of MULT_MSG_EN field says that "must not be greater than multiple message capable value", but MULT_MSG_CAP is read only. How could be this valid? Based on what is set the MULT_MSG_CAP by PCIe?

Thank you,

   Daniel

    

  • 0
    Intellectual 610 points

    + What could be wrong if I write 0x8 in MSI_IRQ over PCIe and I read from interrupt handler the status corresponding to MSI0? 

    + Even if all MSI are enabled, the core1 to core7 will never have interrupt handler triggered. It seems only the MSI0 for core 0 is working as expected...

  • 0
    Guru 13202 points

    Hello,

    For the #1 yes, you have to set 1 to MSI_CAP to enable MSI functionality. Next, Endpoint advertises its MSI multi-vector capability in multMsgCap, then it's up to Root Complex to enable all of them of limited subset. To do that RC writes power of two to multMsgEn. Ultimately, reading multMsgCap and writing it back to multMsgEn allows use of all advertised interrupts.

    There is a cheat in TI's way of PCIe setup. In regular computer systems EPs expose their config space to RC, and then it's responsibility of RC to write appropriate values to remote (EP's) configuration registers. This presumes that EPs are all ready at RC configuration time, so RC can discover them, poll and configure. That is known as enumeration process and not implemented in TI's examples. Instead, in TI's examples you will find that processor operating as EP still writes in its config registers. I would name that self-configuration. Because both RC and EP in our hands, we know what RC is going to write to EP and write that ourselves.

    Hope this clarifies #2, 4 as well. 

    As to #3, MSI_LOW32, MSI_UP32 define address of MSI_DATA register on remote party. Normally, MSI propagate from EP to RC, so by intention MSI_DATA is register withing RC address space, where EP writes value of MSI vector. Think of it as RC tells to EP "if you want interrupt, write to [MSI_LOW32:MSI_UP32] address". Note, there are two registers for address, that is to support 64 addressing on PCIe. If you live within 32 bit address space, upper is zero.

    CORRECTION:

    MSI writes go to MSI_IRQ register, not MSI_DATA.

    The value written to MSI_DATA is MSI vector, i.e. if you write 0, that is MSI0, if you write 13, that is MSI13. 

    TI's implementation of PCIe subsystems extends PCIe spec in way, that RC can also trigger MSI interrupts to EP, assuming the latter is capable of receiving them.

    Hope this helps.

  • 0 in reply to Victor Kazmirenko
    Intellectual 610 points

    Hi,

         Thank you for your reply.

        1. According to manual, if I want to generate all MSIs IDS in both directions, I understood that I need first to enable them (write 0xF into every MSIx_IRQ_ENABLE_SET, where x=0to7) then I should configure BAR0 and one outbound remotely corresponding to this BAR0 (ex: RC: BAR0 base = 0x9000.0000 and in EP: OB=0x9000.0000 wih region_id=0), enable MSIs as general (MSI_CAP[MSI_EN]=1), then I could write from EP at address 0x6000.0054 the value of the MSI vector that is according to my above configuration mapped to MSI_IRQ register from RC that should trigger the MSI handler and should set the status bit in the corresponding MSIx_IRQ_STATUS register.

       With the above configuration, regardless what I write to MSI_IRQ (0,8,16,24) I always get bit0 set from MSI0_IRQ_STATUS, on core0.

       2. What is the role of MSI_DATA in the scenario described above and how should I write for generating a basic MSI? From your respond it seems that I should write to MSI_DATA instead of MSI_IRQ or am I wrong? I think the manual is not specifying this....

      3. If the MSI_IRQ address is 0x2180.0054 in C6678 device, does this mean I need to configure MSI_LOW32=0x0x2180.0054 and MSI_UP32=0 (if 32 bits used) ?

    Daniel

  • 0 in reply to Daniel Costin
    Intellectual 610 points

    You said: "MSI_LOW32, MSI_UP32 define address of MSI_DATA register on remote party. "

    I really don't understand. What is the connection of these registers with writing into MSI_IRQ for activating a MSI?

  • 0 in reply to Daniel Costin
    Guru 13202 points

    Hello,

    My bad, I meant to say that writes go to MSI_IRQ register. And its address is what to be in MSI_LOW32, MSI_UP32 registers, so 

    3. If the MSI_IRQ address is 0x2180.0054 in C6678 device, does this mean I need to configure MSI_LOW32=0x0x2180.0054 and MSI_UP32=0 (if 32 bits used) ?

    is correct if there is no extra address translation between parties. It looks that MSI_DATA is what supposed to be sent, MSI vector to be signalled, though purpose of that register is unclear to me as well. In my case, I run DSP as RC, and that register was set zero.

    About your issue with generating interrupts other than 0, please check setting of multMsgEn register.

    Finally, you may manually write to MSI_IRQ register by DSP to self-trigger MSI to other cores and see whether appropriate ISR was invoked.

  • 0 in reply to Victor Kazmirenko
    Intellectual 610 points

    Hi,

       We tested MSI with self trigger mechanism (basically we activated locally) and seems to work ok. Based on MSI_IRQ value we found the correct status in MSIx_IRQ_STATUS.

       But when we try to activate MSIs over PCIe using outbound write we have these problems, so my suspected part is the PCIE transfer itself and the role of MSI_LOW32, MSI_UP32, MSI_DATA registers.

        We use translation for accesing remote app registers, so if we have mapped the outbound region 0 to 0x9000.0000(EP) that is connected to BAR0 (RC) what we should set in  MSI_LOW32? Should we set 0x6000.0054 as this is the address from PCIe data where we write msi vector (0 to 31) for activating the interrupt remote?

    /Daniel

  • 0 in reply to Daniel Costin
    Intellectual 610 points

    EP: outbound region0 -  0x9000.0000

    RC BAR0 -  0x9000.0000

    We tried to configure MSI_LOW32=0x6000.0054 , but when we write 8 to 0x6000.0054 from EP  (use outbound region 0) we got msi0 (status bit set) on RC

    We tried to configure MSI_LOW32=0x2180.0054 , but when we write 8 to 0x6000.0054 from EP (use outbound region 0) we got msi0 (status bit set) on RC.

    So, none of the solution works.

    My question is - is someone from TI that validated the MSI generation from EP <->RC in the past?

    On the other side, I think is not ok to have some registers without any description and with direct impact over functionality.

    Look to the MSI_LOW32 register in the sprugs6d.pdf manual: "Lower 32 bits address". Address for what? To be set for what?

    It's quite frustrating to have 4-6 registers and a so nonsense in the explanations.

  • 0 in reply to Daniel Costin
    Intellectual 610 points

    I did exactly what TI specified here: https://e2e.ti.com/support/processors/f/791/p/194406/841403?tisearch=e2e-sitesearch&keymatch=MSI_LOW32#841403

    "For example, if you set BAR0 in RC as 0x70000000 (PCIe address over the PCIe link as you defined) and you should setup the outbound translation in EP to use the same PCIe address (0x70000000) over the link. Then you can write 0x0 to the PCIe data space (e.g. write 0x0 to 0x60000054 in EP). The write will be translated to 0x70000054 over the PCIe link and could be accepted by BAR0 in RC, which is 0x70000000.  SInce BAR0 is mapped to Application Registers region (from 0x21800000), the writing to 0x70000054 (from EP) will be targeted to 0x21800054 in RC and the MSI_IRQ in RC will be written into 0x0, which will trigger MSI_0 in the RC."

    The behavior is not as expected.

  • 0 in reply to Daniel Costin
    TI__Guru**** 161600 points

    Let me check this.

    Best Regards,
    Yordan

  • 0 in reply to Yordan Kovachev
    Intellectual 610 points

    Hi,

      I see the ticket is closed - Can you please reopen it until we clarify it?

      I am using processor-sdk-rtos-c667x-evm-05.03.00.07-Linux-x86-Install.

      We tried all the possible ways for configuration. for the simplicity we activated all 4 MSIs for core0 (MSI0_IRQ_ENABLE_SET) , MSI_EN=1 , INTX_DIS=1.

      If we send through PCIe to write MSI_IRQ=0 we got interrupt handler and bit 0 of MSI0_IRQ_STATUS set. If we send through PCIe to write MSI-IRQ=8 we got the interrupt handler , BUT also bit0 is set instead of bit1 in MSI0_IRQ_STATUS.

       Please provide us a pseudo code or part of source for how did you validated the MSIs and what configuration did you do on BAR0, BAR1 and outbound regions.

    /Daniel

  • 0 in reply to Daniel Costin
    TI__Guru**** 161600 points

    Hi,

    Let me check this with the design team.

    Best Regards,
    Yordan

  • 0 in reply to Daniel Costin
    Intellectual 610 points

    Hi,

       We found the root cause. We have both EVMs configured in big endian mode. We made a trick and wrote the same value for rmeote MSI_IRQ into a GPR register also and we observed that on remote was find the value inverted. So basically when we wrote 0x000.0008 we found remote 0x8000.0000.

      It seems we need to config Endian Mode Register (The offset is 38h).