Generating MSI interrupts

There are 8 groups of MSI interrupts (MSI 0~7) and each group has 4 vectors (MSI vector 0-31). You should write MSI vector value into MSI_IRQ to trigger corresponding interrupt, such as write 0x0 to trigger MSI vector 0 (MSI_0 event), write 0x1 to trigger MSI vector (MSI_1 event), etc.

Section 2.14.2.2 describes the general rule of generating MSI interrupt from EP to RC. RC will put the MSI_IRQ offset to the MSI Address register and assign vector value to MSI DATA register for each EP during the enumeration/initialization process. 

There are several threads discussing about PCIe MSI interrupt. Please take a look and hope they could cover your questions.

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/166202.aspx

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/194406/717007.aspx#717007

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/199151/711351.aspx#711351

Thank you for that information.

I would be grateful if you would let me know exactly where in the User Guide it makes clear that ' You should write MSI vector value into MSI_IRQ to trigger corresponding interrupt, such as write 0x0 to trigger MSI vector 0 (MSI_0 event), write 0x1 to trigger MSI vector (MSI_1 event), etc.'

Peter,

It is based on our experiments. And we will make it clear in the MSI_IRQ description in the next release of PCIe user guide.Thanks.

Unfortunately, there are far too many occasions when the only way to discover of how a TI device works is to experiment. Your users would be saved a great deal of wasted effort if the documentation were completely rewritten with the aim of making it possible for users to understand unambiguously how to use the hardware. Far too much is assumed or described in an inadequate way.

On further reflection, I am not much further forward than when I started.

You wrote 'There are 8 groups of MSI interrupts (MSI 0~7) and each group has 4 vectors (MSI vector 0-31)'. I assume that you are using the term 'vector' in two contexts: 'vector' meaning one of the 4 items in each group and 'MSI vector' meaning the (unspecified) combination of group and 'vector'.

So we have 32 possible values.

You then write 'You should write MSI vector value into MSI_IRQ to trigger corresponding interrupt, such as write 0x0 to trigger MSI vector 0 (MSI_0 event), write 0x1 to trigger MSI vector (MSI_1 event), etc.'.

The value written to the MSI_IRQ register is therefore a value in the range 0..31. This would suggest that only 5 bits of the register are used, yet the User Guide says 'This register is written to by the remote device (value = 0-FFFFFFFFh)'.  Is this another error in the User Guide?

What happens if I write the value 14 to MSI_IRQ? Is that group 3 vector 2 or group 6 vector 1?

I also note this gem in section 2.14.5.1: 'In addition, the software drivers for downstream devices must ensure that the data transaction that is expected to complete before interrupt is triggered in RC device's CPU has completed. Because PCIe write transactions are posted, it is not necessary that a write from EP to system memory in RC has completed before a write to MSI interrupt generation register has completed. This could create a potential race condition.'

As interrupts are the mechanism usually used to indicate the completion of an operation, how would you suggest that software ensures the transaction has completed? I note that I have been unable to find an example of PSIe transfers that uses interrupts and polling is used instead. This would be intolerable in real code.

Please take a look at Table 2-10 in PCIe user guide. If you are writing value 14 (0xE) into MSI_IRQ, the MSI6_IRQ will be generated since MSI 6 includes vectors 6/14/22/30.

The section 2.14.5.1 talks about the possibility that the PCIe Root Complex or switches could reorder the memory write transactions just posted ahead of previously posted memory write transactions or message transactions. Similarly, message transactions just posted may be ordered ahead of previously posted memory write or message transactions due to the relax ordering feature in PCIe.

 Memory write and Message transactions are treated the same in most respects: both are handled as posted operations, both are received into the same Posted buffer, and both are subject to the same ordering requirements.

 So if the “relaxed ordering” is enabled, the Root Complex or switches are given the permission to move this transaction ahead of others, whereas the action is normally prohibited in strongly ordering case.

 The relaxed ordering could improve the performance of the post transactions while it could potentially cause issues if the reordering is not permitted in some circumstances.

 In the RC mode, the user could disable the relaxed ordering feature by clearing the “RELAXED” field (bit4) to 0 in “DEV_STAT_CTRL” register. The RC could also clear “RELAXED” field (bit1) to 0 in “TLPCFG” register to disable the reordering request for all outgoing TLPs.

 For more details, please take a look at the PCIe standard specification from PCISIG, such as section 2.4 "Transaction Ordering" in PCI_Express_Base_Rev_2.0 document.

Hope the above info helps.

Thank you for the explanation; your life could have meen made much easier had the documentation been adequate.

I find it difficult to see how I could be expected to work out what you wrote from the descriptions in the User Guide.

Table 2-10 includes a column simply labelled 'Interrupt'; does this column and its values have any meaning at all?

Your description shows that only the bottom 3 bits of MSI_IRQ are relevant. Why doesn't the documentation say this? Is the actual value written to MSI_IRQ (all 32 bits) readable by the interrupted processor? In other words, if I generate interrupt 1 by writing 1 or 9 to MSI_IRQ, can the interrupted processor read MSI_IRQ to determine which was used? Also, why make any mention of interrupts 8-31 as from a practical point of view they don't exist?

Similarly, how am I supposed to make the leap from 'software drivers for downstream devices must ensure...' to 'clear the RELAXED bit in DEV_STAT_CTRL'? Don't you think it might be a little bit helpful to reference that bit in the place where its use is crucial? Even then, the description of RELAXED isn't particularly helpful. The actual description is probably in yet another document, but it wouldn't take much thought to make the description of the race condition and the solution quite clear.

Not only is the documentation incomplete, it is actually misleading.

1. The interrupt column number in Table 2-10 is used in IRQ_EOI register to clear the interrupt. For example, in order to clear interrupt event triggered by MSI0 vector 0, the user needs to write 1 to bit 0 in MSI0_IRQ_STATUS register and also write 4 (MSI_0 is interrupt 4 in table 2-10) to IRQ_EOI register to indicate the end of this interrupt, which will enable the same event to be triggered again.

2. MSI_IRQ will be written only, read will return all zeros. Please read the MSIn_IRQ_STATUS register for the status of the interrupt event. Again, we will update the description of MSI_IRQ register in the user guide. Sorry for the confusion.

3. We will try to connect the race condition and "relaxed" bit description in the user guide as well. You may notice that there are lots of PCIe standard terms or register descriptions are not expanded in this TI PCIe user guide. The user will have to refer to the PCIe standard specification as well. But we will try to describe TI specific PCIe usage clearly in this document and make the user guide more complete and user friendly. Thanks for your suggestions.