• State
  • Locked Locked
  • Replies 6 replies
  • Subscribers 100 subscribers
  • Views 637 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PCIe MSI issue on TI8168

AngelaS
Prodigy 30 points

Hi

I'm testing MSI on a development board(not TI's EVM). We are using PSP04.00.00.11 PSP release, and I know MSI was not supported in this release, but it is supported in the later release. So I have applied the following patch to include the support.

: http://arago-project.org/git/projects/?p=linux-omap3.git;a=commit;h=ef702ecea55c904f144206eea3bf4eddf32d4e93

Now my pci_enable_msi() and request_irq(dev->irq) calls return ok(request_irq() used to return -22 before applying the patch). But  I don't think kernel sees the interrupt as my interrupt handler was not called. ( we know for sure the PCIe endpoint is sending MSI interrupt). Could someone please give me some pointers where to look to debug this issue?

Here are lspci and /proc/interrupts info.

"lspci -xvv" produces this message( I only listed the device which I'm interested).

07:00.0 Class ff00: Device 1172:0004 (rev 01)
        Subsystem: Device 1172:0004
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 368
        Region 0: Memory at 2d000000 (64-bit, prefetchable) [size=2M]
        Region 2: Memory at 2d800000 (32-bit, non-prefetchable) [size=2M]
        Capabilities: [50] MSI: Mask- 64bit+ Count=1/1 Enable+
                Address: 0000000000000000  Data: 0000
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] Express (v1) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 unlimited, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100] Virtual Channel <?>
00: 72 11 04 00 46 05 10 00 01 00 00 ff 10 00 00 00
10: 0c 00 00 2d 00 00 00 00 00 00 80 2d 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 72 11 04 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 30 01 00 00

"cat /proc/interrupts" produce the followings(highlighted in RED is what we interested in)

           CPU0
 12:        695        INTC  edma
 14:          0        INTC  edma_error
 16:          0        INTC  ahci
 17:          0        INTC  cppi41_dma
 18:          8        INTC  musb-hdrc.0
 19:          8        INTC  musb-hdrc.1
 20:          0        INTC  gpmc
 40:          0        INTC  eth0
 41:      54409        INTC  eth0
 42:         83        INTC  eth0
 43:          0        INTC  eth0
 44:          0        INTC  eth1
 45:       4190        INTC  eth1
 46:      43408        INTC  eth1
 47:          0        INTC  eth1
 64:       2466        INTC  mmc0
 67:     211103        INTC  gp timer
 70:          0        INTC  omap_i2c
 71:          0        INTC  omap_i2c
 72:          0        INTC  serial idle
 73:          0        INTC  serial idle
 74:       4098        INTC  serial idle, OMAP UART2
122:          0        INTC  omap-iommu.1
123:          0        INTC  omap-iommu.0
189:          1        GPIO  fpga0
368:          0    PCIe-MSI  fpga1
Err:          0

Thanks a lot for your support,

Angela

 

  • 0
    TI__Expert 5685 points

    Angela,

    I see one problem here:

    Capabilities: [50] MSI: Mask- 64bit+ Count=1/1 Enable+
                    Address: 0000000000000000  Data: 0000

    Ideally we should see something like:

     Capabilities: [50] MSI: Mask- 64bit+ Count=1/1 Enable+
                    Address: 0000000051000054  Data: 0000

     

    I suspect the lspci dump you provided was gathered before loading the driver (or some issue with the driver otherwise). Can you please provide the output of "lspci -kvv" *after* the driver is loaded (and has enabled the MSI)?

       Hemant

  • 0 in reply to HemantPedanekar
    Prodigy 30 points

    Thanks for your quick reply,  Hemant.

    I didn't call pci_enable_device(), so the device state was PCI_UNKNOWN,  and __write_msi_msg () didn't write the address field.

    After I enabled the device, I see address field is now 0x5100054,  I still don't get the interrupt though. Where else should I look?

    here is "lspci -kvv" dump after I enabled the device.

    07:00.0 Class ff00: Device 1172:0004 (rev 01)
            Subsystem: Device 1172:0004
            Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
            Latency: 0, Cache Line Size: 64 bytes
            Interrupt: pin A routed to IRQ 368
            Region 0: Memory at 21000000 (64-bit, prefetchable) [size=2M]
            Region 2: Memory at 21800000 (32-bit, non-prefetchable) [size=2M]
            Capabilities: [50] MSI: Mask- 64bit+ Count=1/1 Enable+
                    Address: 0000000051000054  Data: 0000
            Capabilities: [78] Power Management version 3
                    Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                    Status: D0 PME-Enable- DSel=0 DScale=0 PME-
            Capabilities: [80] Express (v1) Endpoint, MSI 00
                    DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                    DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                            MaxPayload 128 bytes, MaxReadReq 512 bytes
                    DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                    LnkCap: Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 unlimited, L1 unlimited
                            ClockPM- Surprise- LLActRep- BwNot-
                    LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                    LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
            Capabilities: [100] Virtual Channel <?>

  • 0 in reply to AngelaS
    TI__Expert 5685 points

    Angela,

    You mentioned that you are certain about the MSI being sent from the EP - can you please explain how are you confirming this? Are you checking some status register on EP or using bus analyzer to see the TLP?

    Also, can you try setting the Max Read Request field on the EP device to 256 Bytes as described in http://processors.wiki.ti.com/index.php/DM816x_C6A816x_AM389x_PCI_Express_Root_Complex_Driver_User_Guide#Using_PCIe_Endpoint

       Hemant

     

  • 0 in reply to HemantPedanekar
    Prodigy 30 points

    I confirmed it by checking the status register on EP. I don't have bus analyzer to check TLP.

    Can you tell me whether MSI has been tested on EVM? Which EP was tested?

    Thanks.
    Angela

     

  • 0 in reply to AngelaS
    TI__Expert 5685 points

    Yes MSI support is available since 04.00.00.12 release onwards but a few have already got it working on top on 04.00.00.10 PSP release by picking up MSI patch from Arago.

    I have tested MSI with Realtek 8168 based Ethernet card.

       Hemant

  • 0 in reply to HemantPedanekar
    Prodigy 190 points

    Hi Hermant,

    Can you perhaps assist with a PCI msi issue, I need to allocate multiple msi's to a card for use on multiple streams from the card, any idea howto as currently I see that only one MSI is available per driver.

    Regards,

    Johan