RTOS/AM6548: MSI Interrupt Generation in EP Mode

Dusan Corlija

Part Number: AM6548
Other Parts Discussed in Thread: AM6546,

Tool/software: TI-RTOS

Hello,

I am trying to configure an EP (FPGA) to be able to trigger MSI interrupts on the AM6546 processor (RC). Unfortunately, I am not sure which address should the EP write TLP target.

In the SPRUHY8H for 66AK2G1x processor (Multicore DSP+Arm ® KeyStone II System-on-Chip (SoC)) for example, stands following:

The memory write transactions to generate MSI interrupts in RC are actually targeted at PCIE_MSI_IRQ
register. The MSI interrupt is generated as a result of the one of 32 events that is triggered by a write of
MSI vector value to PCIE_MSI_IRQ register in RC.

The address of the PCIE_MSI_IRQ is available as well (2180 0054h). However, in the SPRUID7B for AM65x/DRA80xM Processors (my RC) there is no mention of such register.

I would appreciate any help.

Best regards,

Dušan

over 7 years ago

0 lding over 7 years ago

TI__Guru* 95265 points

Hi,

The 0x2180_0000 offset 0x54 is a PCIE application registers for TI Keystone I/II device (like the K2G you mentioned here). The AM654x doesn't have this register.

The way generating MSI or MSI-X interrupt from EP to RC and RC reception can be found on the PCIE Spec. You can also refer to TI AM654x TRM for description:
12.2.2.4.4.1.2 MSI Interrupt Generation in EP Mode

Although your EP is a FPGA device, the concept and registers are the same, because they are the standard PCIE registers. The interrupt generating is a memory mapped write transaction. Mostly you need to program register offset 0x1050, 0x1054, 0x1058, 0x105c.
- in capacity, you have MSI enabled (legacy disabled)
- then your write a MSI data vector at MSI address

On the RC side, please check 12.2.2.4.4.4.2 MSI Interrupts Reception in RC Mode,
- the MSI address needs to match the EP side
- you need have BAR setting to allow this incoming PCIE MSI address
Then I expect one of the MSI bit in PCIE_EP_MMR0/1/2/3... 7_IRQ_STATUS_RAW Register is set, based on the MSI vector used in EP side. This means the MSI interrupt come in.
- You also need ISR code to process PCIE interrupt, clear the MSI status bits and write EOI to allow the next MSI come in.

Your work is ahead of our PCIE RTOS driver development. We have a simple RTOS PCIE driver showing the data transfer with CPU R/W. The interrupt code is not developed yet. I am not sure what OS you will be used on RC side, Linux or RTOS? Linux should have MSI supported.

As you asked how the EP side address you should write, I would say you need to program the MSI address match on both sides, there is not a specific address like Keystone I/II PCIE you can write to.

Regards, Eric

0 Dusan Corlija over 7 years ago in reply to lding

Prodigy 50 points

Dear Eric,

thank You for Your answer. Unfortunately, I was unable to get the MSIs working. At the moment RC (AM6548 on the GPEVM_AM65x evaluation board) is able to read/write from/to EP (FPGA evaluation board). Also EP is able to read/write from/to CPUs (AM6548) RAM. Following your advice i programmed the EP registers using the PCIe Core Memory Sub-Region - PCIe remote configuration space (Remote PF0). If I understood You correctly, I have to write inside MSI PCIE_RC_MSI_CAP_OFF_04H_REG registers on both EP and RC side the same address - doesn't matter which. I found the MSI capability structure at offset 0x70 from the 0x055A0000 and programmed the following registers ( I am using acronyms defined in AM65x/DRA80xM Processors Technical Reference Manual, SPRUID7B):
PCIE_RC_PCI_MSI_CAP_ID_NEXT_CTRL_REG register = 0x00B7 9005 in 32-Bit Hex - Ti Style,
PCIE_RC_MSI_CAP_OFF_04H_REG = 0x700B 0000 - some location obtained via malloc(),
PCIE_RC_MSI_CAP_OFF_08H_REG = 0x0000 0000,
PCIE_RC_MSI_CAP_OFF_0CH_REG = 0x0000 00A0. Here I used the acronyms for RC but they are the EP registers actually.

Then, I programmed PCIE_RC_MSI_CAP_OFF_04H_REG = 0x700B 0000 at offset of 0x05501054 accordingly. But to no avail. I read extensively the TRMs for 66AK2G1x Multicore DSP+Arm ® KeyStone II System-on-Chip (SoC) SPRUHY8H as well as TRM for my processor but was not able to get a clear understanding of has to be done in the case of the new processor. The PCIe reference guide was not very helpful so far.

My goal is to get the MSIx working as the final solution. I thought it would be a good start to get MSIs working but I experienced great dificulities. I will use this opportunity to ask you a couple of additional questions:
1. What is supposed to happen after the reception of TLP with the address specified in the local configuration space of RC PCIE_RC_MSI_CAP_OFF_04H_REG? Is the hardware (PCIESS responsible for writing to the PCIE_RC_MMR[0:7]_IRQ_STATUS register)?
2. Are SPRUID7B (AM65x TRM) or the PCIe driver (and the example in the PDK) going to be updated soon?

Thanks in advance

0 lding over 7 years ago in reply to Dusan Corlija

TI__Guru* 95265 points

Hi,

In EP side, PCIE_RC_PCI_MSI_CAP_ID_NEXT_CTRL_REG you used bit16=1 (MSI enabled) and bit23=1 (64-bit MSI), then you wrote a vector to offset 0xC. This should work.

On RC side, I am not sure if you have any BAR matching the incoming address 0x700B_0000. Do you have BAR0/1 programmed as 64-bit BAR (0x550_1010/14) for this? Also, I think RC side BAR 0 is dedicated for address space 0.

Probably you can use 32-bit MSI address and writes the MSI vector to address offset 0x8. Then in RC side if you can setup BAR1 in 32-bit mode to accept incoming address 0x700B_0000.

The expectation is that one of the MSI bit in PCIE_EP_MMR0/1/2/3... 7_IRQ_STATUS_RAW Register is set.

We don't have any recent plan for RTOS PCIE driver with MSI support. But I think the Linux driver already supports the MSI on AM65x. You may look at that.

Regards, Eric

Processors

Processors forum

RTOS/AM6548: MSI Interrupt Generation in EP Mode