AM5718: PCIe question

user4759774

Intellectual 745 points

Part Number: AM5718
Other Parts Discussed in Thread: AM5728

Dear All,

I do have a question similar to posted here:

I do work on system with AM5728 and AM5718 being interchangeable (pin compatible).

On my AM5728 PCIe[0] and PCIe[1] work correctly

I do use PCIe_SS1 (AH13, AG13, AG14, AH14 - on IDK schematic: PCIE_TX{P|N} and PCIE_RX{P|N} connected to PCIe connector)

and PCIe_SS2 connected to (AD11, AC11, AE12, AF12 - on IDK those are "USB1_3.0"):

root@bb:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments (rev 01)
0001:00:00.0 PCI bridge: Texas Instruments (rev 01)

Problem starts with AM5718.

According to

Point 3.5.4 on AM571x:

AM571x: AM572x PCIe lane 1 balls are N.C. Lane 1 can be mapped on USB3.0 balls by software. Default
configuration is USB3.0 with the same pinout as in AM572x, that is, PCIe lane 1 is not available by
default.

Have I understood correctly that to have PCIe_SS2 output (PCIe lane1?) I do need to change mapping of usb3.0_tx, usb3.0_rx to pcie_tx{p|n} and pcie_rx{p|n}?

I've followed following document:

Point 24.9.2 (PCIe Controller Environment)

Table 24-661. PCIe_SS Port Configuration

Indeed at 0x4A003c3c (CTRL_CORE_PCIE_CONTROL) there is 0x8 value - PCIe_SS: Port 0 (PCIESS lane 0) and Port USB (USB3.0). This is correct and expected on AM5718.

However, changing it to 0x0 don't bring PCIe1 being available:

root@bb:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments (rev 01)

Also in the table "Table 24-661. PCIe_SS Port Configuration" for port USB I can only set PCIESS2 lane 0 (PCIE_B1C0_MODE_SEL=0) or PCIESS1 lane 1(PCIE_B1C0_MODE_SEL=0x1).

Is it at all possible to set PCIESS2 with lane 1 to reuse AM5728 configuration?

Thanks for replying,

Łukasz

over 7 years ago

0 Biser Gatchev-XID over 7 years ago

TI__Guru**** 393215 points

Hi,

The PCIe experts have been notified. They will respond here.

0 user4759774 over 7 years ago in reply to Biser Gatchev-XID

Intellectual 745 points

Hi,

Are there any news regarding this issue?

Best regards,
Łukasz

0 Rogerio Almeida over 7 years ago in reply to user4759774

TI__Mastermind 26025 points

Thanks for posting you question here....most of our apps are coming back from vacation this week, so you can expect an answer soon.

0 lding over 7 years ago in reply to Rogerio Almeida

TI__Guru* 95265 points

Hi,

Sorry for the late response, I was just back from vacation today.

For AM571x, it still has two PCIE lanes, for your usage case, PCIE_SS1 controls lane 0 and PCIE_SS2 controls lane 1 (by default it is USB 3.0)? Or you wants to use PCIE_SS1 to control both lanes?

For TI AM571x IDK EVM, we only have 1 PCIEx4 connector on the board. So we was able to test PCIESS1x2 mode, not PCIESS1x1 and PCIESS2x1, which needs customized board.

The special configuration on RTOS code is under pdk_am57xx_1_0_4\packages\ti\drv\pcie\example\sample\am57x\src\pcie_sample_board.c, function PlatformPCIESSSetPhyMode():

#if defined(SOC_AM571x)
/* OCP2SCP1_USB3TX_PHY_USB.MEM_ENTESTCLK = 1 */
*(volatile unsigned int*)0x4a08482c |= (1<<31); //PCIE mode
/* USB3PHYRX_ANA_PROGRAMMABILITY_REG1.MEM_EN_PLLBYP = 1 */
*(volatile unsigned int*)0x4a08440c |= (1<<7); //PCIE mode
#endif
/* Set the PHY mode as X2 */
#if defined(PCIESS1_X2)
{
uint32_t regVal;
/* CTRL_CORE_CONTROL_IO_2 */
*(volatile unsigned int*)0x4a002558 |= (1<<13); /* 0: 1-lane; 1: 2-lane */

/* B1C0 Mode selection, bit 3:2 of 0x4a003c3c, reset = 0x2?
* 0: PCIESS1x1 and/or PCIESS2x1
* 1: PCIESS2x2, PCIESS2 not used */
regVal = *(volatile unsigned int *)0x4a003c3cU;
regVal &= 0xFFFFFFF2U;
regVal |= (1U << 2); //PCIESS1 X2 mode
regVal |= (1U << 0); //PCIE_B0_B1_TSYNCEN
*(volatile unsigned int *)0x4a003c3cU = regVal;
}
#endif

The AM571x Linux PCIE code was derived from above, when Linux boot up, you can check if 0x4a08482c and 0x4a08440c is correct. Then look at AM571x TRM for 0x4a002558 and 0x4a003c3c, depending on your use two seperate controllers or one controller with 2 lanes.

If this still doesn't work, I will ask Linux team for help. Please clarify if you use PCIESS1x2 or PCIESS1x1 and PCIESS2x1. Are you using a customized board?

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

Thank you for your reply.

First of all - we are using a custom board with PCIe[0] and PCIe[1] - used to control separate devices.
Please find below "big picture" explanation:

From my understanding we are using PCIESS1x1 and PCIESS2x1 which have following declarations in arch/arm/boot/dts/dra7.dtsi (some code removed for readibility):

axi@0 {
compatible = "simple-bus";
ranges = <0x51000000 0x51000000 0x3000
0x0 0x20000000 0x10000000>;
pcie1: pcie@51000000 {
compatible = "ti,dra7-pcie";
reg = <0x51000000 0x2000>, <0x51002000 0x14c>, <0x1000 0x2000>;
ranges = <0x81000000 0 0 0x03000 0 0x00010000
0x82000000 0 0x20013000 0x13000 0 0xffed000>;
ti,hwmods = "pcie1";
phys = <&pcie1_phy>;
phy-names = "pcie-phy0";
}
}

and

axi1: axi@1 {
compatible = "simple-bus";
ranges = <0x51800000 0x51800000 0x3000
0x0 0x30000000 0x10000000>;
status = "disabled";
pcie2: pcie@51800000 {
compatible = "ti,dra7-pcie";
reg = <0x51800000 0x2000>, <0x51802000 0x14c>, <0x1000 0x2000>;
ranges = <0x81000000 0 0 0x03000 0 0x00010000
0x82000000 0 0x30013000 0x13000 0 0xffed000>;
ti,hwmods = "pcie2";
phys = <&pcie2_phy>;
phy-names = "pcie-phy0";
}
}

Definitions of pcie2_phy and pcie1_phy can be found in the same file.

The pcie1 -> is PCIe[0] and pcie2 -> is PCIe[1] for both AM5718 and AM5728.

When configuring values of 0x4a002558 and 0x4a003c3c registers [*] at bootloader to setup PCIe[1] lines to not serve for USB3.0 on AM5718, I do have Linux crash when I try to enable the lane for PCIe[1].

The only workaround, which I've found is to disable PCIe[1] phy for AM5718:

&pcie2_phy {
status = "disabled";
};

However, we need both PCIe ports on AM5718 (and of course the setup works with AM5728).

I will check on my HW following snippet (I did not know about those registers before your reply):
/* OCP2SCP1_USB3TX_PHY_USB.MEM_ENTESTCLK = 1 */
*(volatile unsigned int*)0x4a08482c |= (1<<31); //PCIE mode
/* USB3PHYRX_ANA_PROGRAMMABILITY_REG1.MEM_EN_PLLBYP = 1 */
*(volatile unsigned int*)0x4a08440c |= (1<<7); //PCIE mode

if it fixes the problem and let you know about the result.

Best regards,
Łukasz

0 user4759774 over 7 years ago in reply to user4759774

Intellectual 745 points

Hi Eric,

Yes, I'm sure that my current code uses PCIe_SS1 x1 (lane 0 -> port 0) and PCIe_SS2 x1 (lane 0 -> port1 shared with usb3.0) (the state of 0x4A00 2558 = 0 and 0x4A003c3c = 0).

First question:

Are below registers documented somewhere? AM572x TRM embraces them as a "reserved" region. Only from dts I know that those registers are part of USB3_PHY1 (probably Designware IP proprietary block).

OCP2SCP1_USB3TX_PHY_USB.MEM_ENTESTCLK = 1
USB3PHYRX_ANA_PROGRAMMABILITY_REG1.MEM_EN_PLLBYP = 1

Setting them up at bootloader did not solve the issue (the pcie driver hangs at dra7xx_pcie_probe() when modifying LTSSM_EN bit)

What do I set during AM5718 bootup:

mm 0x4a0093E0 0x1; Enable OCP2SCP1 clock
mm 0x4a08482c 0x80000000; undocumented
mm 0x4a08440c 0x80; undocumented
mm 0x4a003c3c 0x0; Set PCIeSS1 x 1 and PCIeSS2 x 1.

And no luck - the same problem - works on AM5728 but not at AM5718.

Side question:
Are 0x4A00 2558 and 0x4A00 3c3c only registers responsible for MUXING port 1 between lane-0 from PCIe_SS2 and lane-1 from PCIe_SS1 ? Are there any other one to switch between USB and PCIe PHYs?

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi Lukasz,

Thanks for confirming the setup is two seperate controllers each with one device and the Linux code works on AM5728, also thanks for the trial of AM5718.

Testing AM571x with two seperate PCIE controllers is a probelm for us, as we don't have the EVM supporting this. We only tested PCIESS1X2 lanes on AM571x either use RTOS or Linux. Then, recommend some check and code change here for your usage.

As you mentioned:

mm 0x4a0093E0 0x1; Enable OCP2SCP1 clock

mm 0x4a08482c 0x80000000; undocumented ====> Will be updated in next TRM

mm 0x4a08440c 0x80; undocumented ====> Will be updated in next TRM

mm 0x4a003c3c 0x0; Set PCIeSS1 x 1 and PCIeSS2 x 1. ====> yes

0x4A00 2558 = 0 ====> make sure bit 13 = 0

On AM571x: USB3.0 and PCIE_SS2 is OCP2SCP1, On AM572x: both PCIE_SS1 and SS2 are OCP2SCP3.

I am not sure if "mm 0x4a0093E0 0x1; Enable OCP2SCP1 clock" enough. I attached the Processor SDK RTOS code, as you can see the call sequence (in pcie_sample.c) is:

/*Set PCIE_PERSTn to out of reset state*/

PlatformPCIE_GPIO_Init();

PlatformPCIE_PERSTn_Reset(0);

PlatformPCIESS1ClockEnable();

PlatformPCIESS2ClockEnable();

PlatformPCIESS1PllConfig();

PlatformPCIESSSetPhyMode();

PlatformPCIESS1CtrlConfig();

PlatformPCIESS2CtrlConfig();

PlatformPCIESS1Reset();

PlatformPCIESS2Reset();

PlatformPCIESS1PhyConfig();

PlatformPCIESS2PhyConfig();

The function implementation is done in pcie_sample_board.c. The first two functions are GPIO setting for the PCIE_PERSTn pins, that is HW dependent. As you can see there are more operations on OCP2SCP1 base address. I don't know in your dtsi you enabled PCIE_SS2 entry and set "mm 0x4a0093E0 0x1" are enough or not. I added our Linux team in this thread.

Are 0x4A00 2558 and 0x4A00 3c3c only registers responsible for MUXING port 1 between lane-0 from PCIe_SS2 and lane-1 from PCIe_SS1 ? ========>Yes.

Are there any other one to switch between USB and PCIe PHYs? =====> Those two undocumented registers.

Regards, Eric

pcie_sample_board.c

/*  ============================================================================
 *   Copyright (c) Texas Instruments Incorporated 2015-2016
 * 
 *  Redistribution and use in source and binary forms, with or without 
 *  modification, are permitted provided that the following conditions 
 *  are met:
 *
 *    Redistributions of source code must retain the above copyright 
 *    notice, this list of conditions and the following disclaimer.
 *
 *    Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the 
 *    documentation and/or other materials provided with the   
 *    distribution.
 *
 *    Neither the name of Texas Instruments Incorporated nor the names of
 *    its contributors may be used to endorse or promote products derived
 *    from this software without specific prior written permission.
 *
 *  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
 *  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
 *  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 *  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 
 *  OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
 *  SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 
 *  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 *  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 *  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 
 *  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
 *  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
*/


/**  
 * @file pcie_example_board.h
 *
 */

#include <stdint.h>

#include <ti/csl/soc.h>
#define MEM_BARRIER_DISABLE
#include <ti/csl/hw_types.h>

#include "pcie_sample.h"
#include "pcie_sample_board.h"
#include <ti/csl/soc.h>
#include <ti/osal/osal.h>
#include <ti/drv/pcie/pcie.h>
#ifdef _TMS320C6X
#include <ti/csl/csl_chipAux.h>
#endif
#include <string.h>

#if defined(IDK_AM572x)
#include <ti/drv/gpio/GPIO.h>
#include <ti/drv/gpio/soc/GPIO_v1.h>
#endif
#include <ti/csl/cslr_device.h>
#include <ti/csl/soc/am572x/src/cslr_control_core_pad_io.h>

#if defined(IDK_AM572x)
#define GPIO_PIN_VAL_LOW        (0U)
#define GPIO_PIN_VAL_HIGH       (1U)

/* Port and pin number mask for GPIO_PCIE_RSTDRVn and GPIO_PCIE_SWRSTn.
   Bits 7-0: Pin number  and Bits 15-8: Port number */
#define GPIO_PCIE_RSTDRVn         (0x0316) /*VIN1A_D18/GPIO3_22*/
#define GPIO_PCIE_SWRSTn          (0x0317) /*VIN1A_D19/GPIO3_23*/

#define CTRL_CORE_PAD_GPIO_PIN    (0x20000 | 0x0E)



/* GPIO Driver call back functions */
GPIO_CallbackFxn gpioCallbackFunctions[] = {
    NULL,
    NULL
};

/* GPIO Driver board specific pin configuration structure */
GPIO_PinConfig gpioPinConfigs[] = {
	GPIO_PCIE_RSTDRVn | GPIO_CFG_OUTPUT,
	GPIO_PCIE_SWRSTn | GPIO_CFG_OUTPUT,
};

/* GPIO Driver configuration structure */
GPIO_v1_Config GPIO_v1_config = {
    gpioPinConfigs,
    gpioCallbackFunctions,
    sizeof(gpioPinConfigs) / sizeof(GPIO_PinConfig),
    sizeof(gpioCallbackFunctions) / sizeof(GPIO_CallbackFxn),
    0,
};
#endif


void PlatformPCIESS1ClockEnable(void)
{
    uint32_t regVal;

    /*OCP2SCP1 enables accessing the PCIe PHY serial configuration*/
    HW_WR_FIELD32(SOC_L3INIT_CM_CORE_BASE + CM_L3INIT_OCP2SCP1_CLKCTRL,
                  CM_L3INIT_OCP2SCP1_CLKCTRL_MODULEMODE,
                  CM_L3INIT_OCP2SCP1_CLKCTRL_MODULEMODE_AUTO);

    /*OCP2SCP3 enables accessing the PCIe PHY serial configuration*/
    HW_WR_FIELD32(SOC_L3INIT_CM_CORE_BASE + CM_L3INIT_OCP2SCP3_CLKCTRL,
                  CM_L3INIT_OCP2SCP3_CLKCTRL_MODULEMODE,
                  CM_L3INIT_OCP2SCP3_CLKCTRL_MODULEMODE_AUTO);

    /*PCIeSS CLKSTCTRL SW WakeUp*/
    HW_WR_FIELD32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_CLKSTCTRL,
                  CM_PCIE_CLKSTCTRL_CLKTRCTRL,
                  CM_PCIE_CLKSTCTRL_CLKTRCTRL_SW_WKUP);

    /*L3 Init PCIeSS1 CLKCTRL SW Enable*/
    HW_WR_FIELD32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS1_CLKCTRL,
                  CM_PCIE_PCIESS1_CLKCTRL_MODULEMODE,
                  CM_PCIE_PCIESS1_CLKCTRL_MODULEMODE_ENABLED);

    while ((HW_RD_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS1_CLKCTRL) &
            CM_PCIE_PCIESS1_CLKCTRL_IDLEST_MASK) !=
           CM_PCIE_PCIESS1_CLKCTRL_IDLEST_FUNC)
    {
        ;
    }

    /*Enable PCIe PHY optional clk*/
    regVal = HW_RD_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS1_CLKCTRL);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_DIV,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_DIV_FCLK_EN);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_FCLK_EN);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_32KHZ,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_32KHZ_FCLK_EN);

    HW_WR_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS1_CLKCTRL, regVal);
}

void PlatformPCIESS1PllConfig(void)
{
    uint32_t regVal;

    /*OCP2SCP_SYSCONFIG[1] Soft Reset*/
    regVal = HW_RD_REG32(SOC_OCP2SCP3_BASE + 0x10U) & 0xFFFFFFFDU;

    regVal |= 0x02U;

    HW_WR_REG32(SOC_OCP2SCP3_BASE + 0x10U, regVal);

    /*OCP2SCP_SYSSTATUS[0] Reset Done*/
    while ((HW_RD_REG32(SOC_OCP2SCP3_BASE + 0x14U) & 0x01U) != 0x01U)
    {
        ;
    }

    /*OCP2SCP_TIMING[9:7] Division Ratio = 1*/
    regVal = HW_RD_REG32(SOC_OCP2SCP3_BASE + 0x18U) & 0xFFFFFC7FU;

    regVal |= (uint32_t) 0x8U << 4U;

    HW_WR_REG32(SOC_OCP2SCP3_BASE + 0x18U, regVal);

    /*OCP2SCP_TIMING[3:0] (SYNC2) = 0xF*/
    regVal = HW_RD_REG32(SOC_OCP2SCP3_BASE + 0x18U) & 0xFFFFFFF0U;

    regVal |= 0xFU;

    HW_WR_REG32(SOC_OCP2SCP3_BASE + 0x18U, regVal);

#ifdef SOC_AM571x
    /*OCP2SCP_SYSCONFIG[1] Soft Reset*/
    regVal = HW_RD_REG32(SOC_OCP2SCP1_BASE + 0x10U) & 0xFFFFFFFDU;

    regVal |= 0x02U;

    HW_WR_REG32(SOC_OCP2SCP1_BASE + 0x10U, regVal);

    /*OCP2SCP_SYSSTATUS[0] Reset Done*/
    while ((HW_RD_REG32(SOC_OCP2SCP1_BASE + 0x14U) & 0x01U) != 0x01U)
    {
        ;
    }

    /*OCP2SCP_TIMING[9:7] Division Ratio = 1*/
    regVal = HW_RD_REG32(SOC_OCP2SCP1_BASE + 0x18U) & 0xFFFFFC7FU;

    regVal |= (uint32_t) 0x8U << 4U;

    HW_WR_REG32(SOC_OCP2SCP1_BASE + 0x18U, regVal);

    /*OCP2SCP_TIMING[3:0] (SYNC2) = 0xF*/
    regVal = HW_RD_REG32(SOC_OCP2SCP1_BASE + 0x18U) & 0xFFFFFFF0U;

    regVal |= 0xFU;

    HW_WR_REG32(SOC_OCP2SCP1_BASE + 0x18U, regVal);
#endif

    /*PCIe DPLL - M&N programming; CLKSEL*/
    regVal = HW_RD_REG32(SOC_CKGEN_CM_CORE_BASE + CM_CLKSEL_DPLL_PCIE_REF);

    HW_SET_FIELD(regVal, CM_CLKSEL_DPLL_PCIE_REF_DPLL_DIV, 0x09U);

    HW_SET_FIELD(regVal, CM_CLKSEL_DPLL_PCIE_REF_DPLL_MULT, 0x2EEU);

    HW_WR_REG32(SOC_CKGEN_CM_CORE_BASE + CM_CLKSEL_DPLL_PCIE_REF, regVal);

    /*SigmaDelta SD DIV programming */
    HW_WR_FIELD32(SOC_CKGEN_CM_CORE_BASE + CM_CLKSEL_DPLL_PCIE_REF,
                  CM_CLKSEL_DPLL_PCIE_REF_DPLL_SD_DIV, 0x06U);

    /*PCIe DPLL - M2 programming*/
    HW_WR_FIELD32(SOC_CKGEN_CM_CORE_BASE + CM_DIV_M2_DPLL_PCIE_REF,
                  CM_DIV_M2_DPLL_PCIE_REF_DIVHS, 0x0FU);

    /*DPLL Enable*/
    HW_WR_FIELD32(SOC_CKGEN_CM_CORE_BASE + CM_CLKMODE_DPLL_PCIE_REF,
                  CM_CLKMODE_DPLL_PCIE_REF_DPLL_EN,
                  CM_CLKMODE_DPLL_PCIE_REF_DPLL_EN_DPLL_LOCK_MODE);

    /* Check for DPLL lock status */
    while (((HW_RD_REG32(SOC_CKGEN_CM_CORE_BASE + CM_IDLEST_DPLL_PCIE_REF) &
             CM_IDLEST_DPLL_PCIE_REF_ST_DPLL_CLK_MASK) <<
            CM_IDLEST_DPLL_PCIE_REF_ST_DPLL_CLK_SHIFT) !=
           CM_IDLEST_DPLL_PCIE_REF_ST_DPLL_CLK_DPLL_LOCKED)
    {
        ;
    }

    /*PCIe Tx and Rx Control of ACSPCIe*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_SMA_SW_6,
                  CSL_CONTROL_CORE_SEC_SMA_SW_6_PCIE_TX_RX_CONTROL, 0x02U);

    /*Locking APLL to 2.5GHz with 100MHz input*/
    regVal = HW_RD_REG32(SOC_CKGEN_CM_CORE_BASE + CM_CLKMODE_APLL_PCIE);

    HW_SET_FIELD(regVal, CM_CLKMODE_APLL_PCIE_CLKDIV_BYPASS,
                 CM_CLKMODE_APLL_PCIE_CLKDIV_BYPASS_PCIEDIVBY2_BYPASS_1);

    HW_SET_FIELD(regVal, CM_CLKMODE_APLL_PCIE_REFSEL,
                 CM_CLKMODE_APLL_PCIE_REFSEL_CLKREF_ADPLL);

    HW_WR_REG32(SOC_CKGEN_CM_CORE_BASE + CM_CLKMODE_APLL_PCIE, regVal);

    HW_WR_FIELD32(SOC_CKGEN_CM_CORE_BASE + CM_CLKMODE_APLL_PCIE,
                  CM_CLKMODE_APLL_PCIE_MODE_SELECT,
                  CM_CLKMODE_APLL_PCIE_MODE_SELECT_APLL_FORCE_LOCK_MODE);

    /*Wait for APLL lock*/
    while (((HW_RD_REG32(SOC_CKGEN_CM_CORE_BASE + CM_IDLEST_APLL_PCIE) &
             CM_IDLEST_APLL_PCIE_ST_APLL_CLK_MASK) <<
            CM_IDLEST_APLL_PCIE_ST_APLL_CLK_SHIFT) !=
           CM_IDLEST_APLL_PCIE_ST_APLL_CLK_APLL_LOCKED)
    {
        ;
    }
}

void PlatformPCIESSSetPhyMode (void)
{
/* Set the PHY mode as PCIE and not USB */
#if defined(SOC_AM571x)
  /* OCP2SCP1_USB3TX_PHY_USB.MEM_ENTESTCLK = 1 */
  *(volatile unsigned int*)0x4a08482c |= (1<<31);   //PCIE mode
  /* USB3PHYRX_ANA_PROGRAMMABILITY_REG1.MEM_EN_PLLBYP = 1 */
  *(volatile unsigned int*)0x4a08440c |= (1<<7);    //PCIE mode
#endif
/* Set the PHY mode as X2 */
#if defined(PCIESS1_X2)
  {
    uint32_t regVal;
    /* CTRL_CORE_CONTROL_IO_2 */
    *(volatile unsigned int*)0x4a002558 |= (1<<13);   /* 0: 1-lane; 1: 2-lane */

    /* B1C0 Mode selection, bit 3:2 of 0x4a003c3c, reset = 0x2?
     * 0: PCIESS1x1 and/or PCIESS2x1
     * 1: PCIESS2x2, PCIESS2 not used */
    regVal  = *(volatile unsigned int *)0x4a003c3cU;
    regVal &= 0xFFFFFFF2U;
    regVal |= (1U << 2);   //PCIESS1 X2 mode
    regVal |= (1U << 0);   //PCIE_B0_B1_TSYNCEN
    *(volatile unsigned int *)0x4a003c3cU = regVal;
  }
#endif
}

void PlatformPCIESS1Reset(void)
{
    /*Reset PCIeSS1*/
    HW_WR_FIELD32(SOC_L3INIT_PRM_BASE + RM_PCIESS_RSTCTRL,
                  RM_PCIESS_RSTCTRL_RST_LOCAL_PCIE1,
                  RM_PCIESS_RSTCTRL_RST_LOCAL_PCIE1_CLEAR);

    /* Wait till PCIeSS1 is out of reset */
    while (((HW_RD_REG32(SOC_L3INIT_PRM_BASE + RM_PCIESS_RSTST) &
             RM_PCIESS_RSTST_RST_LOCAL_PCIE1_MASK) <<
            RM_PCIESS_RSTST_RST_LOCAL_PCIE1_SHIFT) !=
           RM_PCIESS_RSTST_RST_LOCAL_PCIE1_RESET_YES)
    {
        ;
    }
}

void PlatformPCIESS1CtrlConfig(void)
{
    uint32_t regVal;

    /*CONTROL MODULE PWR CTL REG status of PCIeSS1*/
    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS1);

    HW_SET_FIELD(regVal, CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS1_PCIESS1_PWRCTL_CMD,
                 0x03U);

    HW_SET_FIELD(regVal, CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS1_PCIESS1_PWRCTL_CLKFREQ,
                 0x1AU);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS1,
                regVal);

    /*Set PCIeSS1/2 delay count*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PCIE_PCS,
                  CSL_CONTROL_CORE_SEC_PCIE_PCS_PCIESS2_PCS_RC_DELAY_COUNT, 0x96U);
}

void PlatformPCIESS1PhyConfig(void)
{
    uint32_t regVal;

    /*Program for Analog circuits in the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU);
    regVal &= 0x07FFFFFFU;
    regVal |= ((uint32_t) 0x10U << 24U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU);
    regVal &= 0xFFFC3FFFU;
    regVal |= ((uint32_t) 0x10U << 12U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU, regVal);

    /*Program for digital section of the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x28U);
    regVal &= 0xE30007FFU;
    regVal |= 0x001B3000U;
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x28U, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU);
    regVal &= 0xFFFFFF9FU;
    regVal |= ((uint32_t) 0x0U << 4U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x0CU, regVal);

    /*Determines which of the 4 EFUSE registers. Selects dll_rate2_coarsetrim*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x1CU);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0x8U << 28U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x1CU, regVal);

    /*
     * Programs the DLL and the Phase Interpolator analog RW 0x3
     * circuits to work with different clock frequencies
     */
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x24U);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0xCU << 28U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x24U, regVal);

    /*Program IP Equalizer*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x38U);
    regVal &= 0x0U;
    regVal |= 0x0000F80FU;
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE1_BASE + 0x38U, regVal);
}

static uint32_t msi_ints = 0, intx_ints = 0, unknown_ints = 0;
SemaphoreP_Handle semaphoreHandle;
static void PlatformMsiIntxIsr (uintptr_t vhandle)
{
    pcieTiConfIrqStatusMsiReg_t pendingBits;
    pciePlconfMsiCtrlIntStatusReg_t msiBits[8];
    pciePlconfMsiCtrlIntStatusReg_t *clearMsiBits = NULL;
    int32_t clearMsiBitsSize = 0;
    Pcie_Handle handle = (Pcie_Handle)vhandle;
    int32_t foundInt = 0;
    /* Figure out which type if interrupt */
    if (Pcie_getPendingFuncInts (handle, &pendingBits, sizeof(msiBits), msiBits) == pcie_RET_OK)
    {
        if (pendingBits.msi)
        {
            int32_t msiError = 0, i;
            /* Check if its "right" msi */
            for (i = 1; i < 8; i++)
            {
                if (msiBits[i].msiCtrlIntStatus)
                {
                    msiError = 1; /* Unexpected MSI */
                    break;
                }
            }
            if (msiBits[0].msiCtrlIntStatus != 1)
            {
                msiError = 1; /* Unexpected MSI */
            }
            if (! msiError)
            {
                /* Got "right" msi */
                foundInt = 1;
                msi_ints++;
                /* clear it */
                clearMsiBitsSize = sizeof(msiBits);
                clearMsiBits = msiBits;
            }
        }
        if (pendingBits.inta)
        {
            intx_ints++;
            foundInt = 1;
        }
        if (pendingBits.intb)
        {
            intx_ints++;
            foundInt = 1;
        }
        if (pendingBits.intc)
        {
            intx_ints++;
            foundInt = 1;
        }
        if (pendingBits.intd)
        {
            intx_ints++;
            foundInt = 1;
        }
    }
    if (! foundInt)
    {
        /* Couldn't decode the interrupt */
        unknown_ints ++;
    }
    else
    {
        /* Tell user task ISR happend */
        SemaphoreP_postFromISR (semaphoreHandle);
    }
    /* Clear/acknowledge the interrupt */
    Pcie_clrPendingFuncInts (handle, &pendingBits, clearMsiBitsSize, clearMsiBits);
}


HwiP_Handle pcieHwi;
SemaphoreP_Handle PlatformSetupMSIAndINTX (Pcie_Handle handle)
{
    HwiP_Params                        hwiInputParams;
    CSL_XbarIrqCpuId                   cpu;
    uint32_t                           cpuEvent;
    uint32_t                           xbarIndex;
    int32_t                            vector;
    pcieRet_e                          retVal;
    pcieRegisters_t                    regs;
    pcieRegisters_t                    epRegs;
    pcieTiConfIrqStatusMsiReg_t        rcMsiStat;
    pcieTiConfIrqEnableSetMsiReg_t     rcMsiEn;
    pciePlconfMsiCtrlAddressReg_t      rcMsiLowAddress;
    pciePlconfMsiCtrlUpperAddressReg_t rcMsiUpAddress;
    pciePlconfMsiCtrlIntEnableReg_t    rcMsiIntEnable;
    pciePlconfMsiCtrlIntStatusReg_t    rcMsiBits[8];
    pcieMsiCapReg_t                    epMsiCap;
    pcieMsiLo32Reg_t                   epMsiLowAddress;
    pcieMsiUp32Reg_t                   epMsiUpAddress;
    pcieMsiDataReg_t                   epMsiDataVal;
    int32_t                            i;

    /* Create a semaphore for user task to wait for interrupt */
    semaphoreHandle = SemaphoreP_create (0, NULL);
    if (!semaphoreHandle)
    {
        PCIE_logPrintf("Failed to create semaphore\n");
        exit(1);
    }
    memset (&regs, 0, sizeof(regs));
    memset (&epRegs, 0, sizeof(epRegs));
    memset (&rcMsiEn, 0, sizeof(rcMsiEn));
    memset (&rcMsiLowAddress, 0, sizeof(rcMsiLowAddress));
    memset (&rcMsiStat, 0, sizeof(rcMsiStat));
    memset (&rcMsiUpAddress, 0, sizeof(rcMsiUpAddress));
    memset (&rcMsiIntEnable, 0, sizeof(rcMsiIntEnable));
    memset (&epMsiCap, 0, sizeof(epMsiCap));
    memset (&epMsiLowAddress, 0, sizeof(epMsiLowAddress));
    memset (&epMsiUpAddress, 0, sizeof(epMsiUpAddress));
    memset (&epMsiDataVal, 0, sizeof(epMsiDataVal));

    /* Read existing EP registers */
    epRegs.msiCap = &epMsiCap;
    epRegs.msiLo32 = &epMsiLowAddress;
    epRegs.msiUp32 = &epMsiUpAddress;
    epRegs.msiData = &epMsiDataVal;
    retVal = Pcie_readRegs (handle, pcie_LOCATION_REMOTE, &epRegs);
    if (retVal != pcie_RET_OK)
    {
        PCIE_logPrintf ("read of EP interrupt regs failed (%d)\n", retVal);
        exit(1);
    }

    /* Enable MSI on EP */
    epMsiCap.msiEn = 1;
    epMsiDataVal.data = PCIE_WINDOW_MSI_DATA;
    epMsiUpAddress.addr = 0;
    epMsiLowAddress.addr = PCIE_WINDOW_MSI_ADDR>>2; /* because lld wants upper 30 bits only */
    retVal = Pcie_writeRegs (handle, pcie_LOCATION_REMOTE, &epRegs);
    if (retVal != pcie_RET_OK)
    {
        PCIE_logPrintf ("write of EP interrupt regs failed (%d)\n", retVal);
        exit(1);
    }

    /* Clear any pending interrupts inside PCIE */
    regs.tiConfIrqStatusMsi = &rcMsiStat;
    rcMsiStat.inta = rcMsiStat.intb = rcMsiStat.intc = rcMsiStat.intd = 1;
    rcMsiStat.msi = 1;
    /* Clear any MSIs */
    for (i = 0; i < 8; i++)
    {
        rcMsiBits[i].msiCtrlIntStatus = ~0U;
        regs.plconfMsiCtrlIntStatus[i] = &rcMsiBits[i];
    }
    /* Enable the interrupts inside PCIE */
    regs.tiConfIrqEnableSetMsi = &rcMsiEn;
    rcMsiEn.inta = rcMsiEn.intb = rcMsiEn.intc = rcMsiEn.intd = 1;
    rcMsiEn.msi = 1;
    /* Set RC's data address */
    regs.plconfMsiCtrlAddress = &rcMsiLowAddress;
    regs.plconfMsiCtrlUpperAddress = &rcMsiUpAddress;
    rcMsiLowAddress.msiCtrlAddress = PCIE_WINDOW_MSI_ADDR;
    rcMsiUpAddress.msiCtrlUpperAddress  = 0;
    /* Set RC's interrupt enable in plconf */
    regs.plconfMsiCtrlIntEnable[0] = &rcMsiIntEnable;
    rcMsiIntEnable.msiCtrlIntEnable = 1;
    retVal = Pcie_writeRegs (handle, pcie_LOCATION_LOCAL, &regs);
    if (retVal != pcie_RET_OK)
    {
        PCIE_logPrintf ("write of RC interrupt regs failed (%d)\n", retVal);
        exit(1);
    }

#ifdef _TMS320C6X
    if (CSL_chipReadDNUM() == 0)
    {
        cpu = CSL_XBAR_IRQ_CPU_ID_DSP1;
    }
    else
    {
        cpu = CSL_XBAR_IRQ_CPU_ID_DSP2;
    }
    cpuEvent = 48;
    xbarIndex = cpuEvent - 31;
    vector = 12;
#elif __ARM_ARCH_7A__
    cpu = CSL_XBAR_IRQ_CPU_ID_MPU;
    vector = 100;
    cpuEvent = vector - 37;
    xbarIndex = vector - 37;
#else
    /* Map IPU Intr 49 to PCIe_SS1_IRQ_INT1 event id 233 */
    cpu = CSL_XBAR_IRQ_CPU_ID_IPU1;
    vector = 49;
    cpuEvent = 49;
    xbarIndex = cpuEvent - 22;
#endif

    /* Configure xbar */
    CSL_xbarIrqConfigure (cpu, xbarIndex, CSL_XBAR_PCIe_SS1_IRQ_INT1);

    /* Construct Hwi object for this UART peripheral. */
    HwiP_Params_init (&hwiInputParams);
    hwiInputParams.name = "PCIE_MSI_AND_INTX";
    hwiInputParams.arg  = (uintptr_t)handle;
#if defined (_TMS320C6X) || defined (__ARM_ARCH_7A__)
    hwiInputParams.priority = 0;
#else  /* M4 */
    hwiInputParams.priority = 0x20;
#endif
    hwiInputParams.evtId = cpuEvent;
    pcieHwi = HwiP_create(vector, PlatformMsiIntxIsr, &hwiInputParams);

    if (! pcieHwi)
    {
        PCIE_logPrintf("Hwi create failed\n");
        exit(1);
    }

    return semaphoreHandle;
}

void PlatformGetInts (uint32_t *msis, uint32_t *intx, uint32_t *unknowns)
{
    if (msis)
    {
        *msis = msi_ints;
    }
    if (intx)
    {
        *intx = intx_ints;
    }
    if (unknowns)
    {
        *unknowns = unknown_ints;
    }
}

void PlatformPCIE_GPIO_Init(void)
{
#if defined(IDK_AM572x)
	GPIO_init();
#endif
}

/*
 * API: PlatformPCIE_PERSTn_Reset
 * Variable: resetOnOffF - Reset On/OFF Flag 1-ON/RESET , 0- OFF/Out of RESET
 * Set PCIE_PERSTn to out of reset state
 * GPIO_PCIE_RSTDRV - LOW to select the Reset driver
 * GPIO_PCIE_SWRST - HIGH to set to out of RESET state
 */ 
void PlatformPCIE_PERSTn_Reset(uint32_t resetOnOffF)
{
#if defined(IDK_AM572x)
	GPIO_write(0, GPIO_PIN_VAL_HIGH); //Select Reset Driver

	if(resetOnOffF)	
	{
		GPIO_write(1, GPIO_PIN_VAL_LOW);
	}
	else
	{
		GPIO_write(1, GPIO_PIN_VAL_HIGH);
	}

#endif
}

void PlatformPCIESS2ClockEnable(void) 
{
    /* This example enables portions of PCISS2 to use its serdes lane
     * for dual operation of PCIESS1 */
#ifdef PCIESS1_X2
    uint32_t regVal;

    /*L3 Init PCIeSS2 CLKCTRL SW Enable*/
    HW_WR_FIELD32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS2_CLKCTRL,
                  CM_PCIE_PCIESS2_CLKCTRL_MODULEMODE,
                  CM_PCIE_PCIESS2_CLKCTRL_MODULEMODE_ENABLED);

    while ((HW_RD_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS2_CLKCTRL) &
            CM_PCIE_PCIESS2_CLKCTRL_IDLEST_MASK) !=
           CM_PCIE_PCIESS2_CLKCTRL_IDLEST_FUNC)
    {
        ;
    }

    /*Enable PCIe PHY optional clk*/
    regVal = HW_RD_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS2_CLKCTRL);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS2_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_DIV,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_DIV_FCLK_EN);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS2_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_PCIEPHY_CLK_FCLK_EN);

    HW_SET_FIELD(regVal, CM_PCIE_PCIESS2_CLKCTRL_OPTFCLKEN_32KHZ,
                 CM_PCIE_PCIESS1_CLKCTRL_OPTFCLKEN_32KHZ_FCLK_EN);

    HW_WR_REG32(SOC_L3INIT_CM_CORE_BASE + CM_PCIE_PCIESS2_CLKCTRL, regVal);
#endif
}


void PlatformPCIESS2Reset(void)
{
    /* This example enables portions of PCISS2 to use its serdes lane
     * for dual operation of PCIESS1 */
#ifdef PCIESS1_X2
    /*Reset PCIeSS2*/
    HW_WR_FIELD32(SOC_L3INIT_PRM_BASE + RM_PCIESS_RSTCTRL,
                  RM_PCIESS_RSTCTRL_RST_LOCAL_PCIE2,
                  RM_PCIESS_RSTCTRL_RST_LOCAL_PCIE2_CLEAR);

    /* Wait till PCIeSS2 is out of reset */
    while (((HW_RD_REG32(SOC_L3INIT_PRM_BASE + RM_PCIESS_RSTST) &
             RM_PCIESS_RSTST_RST_LOCAL_PCIE2_MASK) >>
            RM_PCIESS_RSTST_RST_LOCAL_PCIE2_SHIFT) !=
           RM_PCIESS_RSTST_RST_LOCAL_PCIE2_RESET_YES)
    {
        ;
    }
#endif
}

void PlatformPCIESS2CtrlConfig(void)
{
    /* This example enables portions of PCISS2 to use its serdes lane
     * for dual operation of PCIESS1 */
#ifdef PCIESS1_X2
    uint32_t regVal;

#ifdef SOC_AM572x
    /*CONTROL MODULE PWR CTL REG status of PCIeSS2*/
    regVal = HW_RD_REG32(
        SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS2);

    HW_SET_FIELD(regVal, CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS2_PCIESS2_PWRCTL_CMD,
                 0x03U);

    HW_SET_FIELD(regVal, CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS2_PCIESS2_PWRCTL_CLKFREQ,
                 0x1AU);

    HW_WR_REG32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PHY_POWER_PCIESS2,
                regVal);

    /*Set PCIeSS2 delay count*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PCIE_PCS,
                  CSL_CONTROL_CORE_SEC_PCIE_PCS_PCIESS2_PCS_RC_DELAY_COUNT, 0x96U);
#endif

#ifdef SOC_AM571x
    /*CONTROL MODULE PWR CTL REG status of CTRL_CORE_PHY_POWER_USB: 0x4A002370 */
    regVal = HW_RD_REG32(
    		SOC_CTRL_MODULE_CORE_CORE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_USB);

    HW_SET_FIELD(regVal, CTRL_CORE_PHY_POWER_USB_USB_PWRCTL_CLK_CMD,
                 0x13U);

    HW_SET_FIELD(regVal, CTRL_CORE_PHY_POWER_USB_USB_PWRCTL_CLK_FREQ,
                 0x1AU);

    HW_WR_REG32(SOC_CTRL_MODULE_CORE_CORE_REGISTERS_BASE + CTRL_CORE_PHY_POWER_USB,
                regVal);

    /*Set PCIeSS2 delay count*/
    HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_PCIE_PCS,
                  CSL_CONTROL_CORE_SEC_PCIE_PCS_PCIESS2_PCS_RC_DELAY_COUNT, 0x96U);
#endif
#endif
}

void PlatformPCIESS2PhyConfig(void)
{
#ifdef PCIESS1_X2
    uint32_t regVal;

#ifdef SOC_AM572x
    /*Program for Analog circuits in the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU);
    regVal &= 0x07FFFFFFU;
    regVal |= ((uint32_t) 0x10U << 24U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU);
    regVal &= 0xFFFC3FFFU;
    regVal |= ((uint32_t) 0x10U << 12U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU, regVal);

    /*Program for digital section of the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x28U);
    regVal &= 0xE30007FFU;
    regVal |= 0x001B3000U;
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x28U, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU);
    regVal &= 0xFFFFFF9FU;
    regVal |= ((uint32_t) 0x0U << 4U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x0CU, regVal);

    /*Determines which of the 4 EFUSE registers. Selects dll_rate2_coarsetrim*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x1CU);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0x8U << 28U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x1CU, regVal);

    /*
     * Programs the DLL and the Phase Interpolator analog RW 0x3
     * circuits to work with different clock frequencies
     */
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x24U);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0xCU << 28U);
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x24U, regVal);

    /*Program IP Equalizer*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x38U);
    regVal &= 0x0U;
    regVal |= 0x0000F80FU;
    HW_WR_REG32(SOC_OCP2SCP3_USB3RX_PHY_PCIE2_BASE + 0x38U, regVal);
#endif

#ifdef SOC_AM571x
    /*Program for Analog circuits in the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU);
    regVal &= 0x07FFFFFFU;
    regVal |= ((uint32_t) 0x10U << 24U);
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU);
    regVal &= 0xFFFC3FFFU;
    regVal |= ((uint32_t) 0x10U << 12U);
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU, regVal);

    /*Program for digital section of the IP.*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x28U);
    regVal &= 0xE30007FFU;
    regVal |= 0x001B3000U;
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x28U, regVal);

    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU);
    regVal &= 0xFFFFFF9FU;
    regVal |= ((uint32_t) 0x0U << 4U);
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x0CU, regVal);

    /*Determines which of the 4 EFUSE registers. Selects dll_rate2_coarsetrim*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x1CU);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0x8U << 28U);
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x1CU, regVal);

    /*
     * Programs the DLL and the Phase Interpolator analog RW 0x3
     * circuits to work with different clock frequencies
     */
    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x24U);
    regVal &= 0x3FFFFFFFU;
    regVal |= ((uint32_t) 0xCU << 28U);
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x24U, regVal);

    /*Program IP Equalizer*/
    regVal  = HW_RD_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x38U);
    regVal &= 0x0U;
    regVal |= 0x0000F80FU;
    HW_WR_REG32(SOC_OCP2SCP1_USB3RX_PHY_USB_BASE + 0x38U, regVal);
#endif
#endif
}

3240.pcie_sample.c

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

With two PCIe_SS1 (lane 0 -> PCIe port 0) and PCIe_SS2 (lane 0 -> PCIe port 1) I do see following output from lspci:
root@bb:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0001:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)

root@bb:~# lspci -t
-+-[0001:00]---00.0-[01]--
\-[0000:00]---00.0-[01]----00.0

where the PCI bridge is a driver for pcie{1|2} Linux devices. Such output is from AM5728.

I've tried today to reuse your configuration:
PCIe_SS1 x2 (lane - 0 -> port 0 , lane - 1 -> port 1 - MUXed) ?

Then I only had one entry in lspci
root@bb:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)

with devices connected to port 0 and 1. The advantage of such approach is that you have it working on AM5718 on linux (which SDK version supports it?).

As I said, I gave it a shoot, but it also hanged when I disabled PCIe_SS1 lane-0 phy1 (port0) and only tried to use lane-1 phy2 (port1).

Questions for this approach (to you and Linux team :-) ):

- How in Linux check if pcie2_phy (port 2) has been properly initialized? (I did not find any "good" way for this in /sysfs /procfs debugfs). I would like to know that both lanes are operational without adding some "debug" code to Linux source code.

- I would NOT need to enable and configure OCP2SCP1 in this case, since I will only use PCIe_SS1 which is accessible via OCP2SCP3 (which is setup in the same way as in AM5728). I do only need to modify four mentioned above registers to setup muxing.

- Is there any significant performance penalty between using (PCIe_SS1 x 1 + PCIe_SS2 x 1) and (PCIe_SS1 x 2) ?

Best regards,
Łukasz

0 user4759774 over 7 years ago in reply to user4759774

Intellectual 745 points

Hi Eric,

If I could amend above question - have you tested on AM5718 PCIe_SS1 x2 (with lane 0 and lane 1) on a configuration where PCIe device was connected to lane-1 (port 1) and lane-0 (port 0) was not connected?

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Lukasz,

"As I said, I gave it a shoot, but it also hanged when I disabled PCIe_SS1 lane-0 phy1 (port0) and only tried to use lane-1 phy2 (port1)."

"have you tested on AM5718 PCIe_SS1 x2 (with lane 0 and lane 1) on a configuration where PCIe device was connected to lane-1 (port 1) and lane-0 (port 0) was not connected?" ====> No, the test we did is only on TI EVM, AM571x IDK EVM has a PCIEx4 connector, we tested it with several Generic PCIE-SATA cards (x2 lanes). We can't test a device with lane 0 disconnected but lane 1 connected.

It seems your board is flexible in terms of PCIE connector? You can either has two seperate PCIEx1 connectors and each connect with a x1 devices? Or you can have a PCIE x n connector and connect two a x2 devices?

"With two PCIe_SS1 (lane 0 -> PCIe port 0) and PCIe_SS2 (lane 0 -> PCIe port 1) I do see following output from lspci:
root@bb:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0001:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)

root@bb:~# lspci -t
-+-[0001:00]---00.0-[01]--
\-[0000:00]---00.0-[01]----00.0

where the PCI bridge is a driver for pcie{1|2} Linux devices. Such output is from AM5728.

I've tried today to reuse your configuration:
PCIe_SS1 x2 (lane - 0 -> port 0 , lane - 1 -> port 1 - MUXed) ? "

So, with AM5728, you have SS1x2 and SS1x1 + SS2x1 worked already?

For AM5718, the latest software-dl.ti.com/.../index_FDS.html Linux should have SS1x2 support. Can you try this release for SS1x2 as a starting point? Then we can try how to do SS1x1+SS2x1?

- Is there any significant performance penalty between using (PCIe_SS1 x 1 + PCIe_SS2 x 1) and (PCIe_SS1 x 2) ? =====>You have 2 EP or 1 EP? If you have 1 EP only, you can only do SS1x2, you can't use two set of PCIE control signals (from SS1 and SS2) to the same EP.

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

Thanks for reply.

lding said:

"As I said, I gave it a shoot, but it also hanged when I disabled PCIe_SS1 lane-0 phy1 (port0) and only tried to use lane-1 phy2 (port1)."

"have you tested on AM5718 PCIe_SS1 x2 (with lane 0 and lane 1) on a configuration where PCIe device was connected to lane-1 (port 1) and lane-0 (port 0) was not connected?"

====> No, the test we did is only on TI EVM, AM571x IDK EVM has a PCIEx4 connector, we tested it with several Generic PCIE-SATA cards (x2 lanes). We can't test a device with lane 0 disconnected but lane 1 connected.

This would explain why you used PCIe_SS1 with two lanes. Such setup does not cover my problem..... (since I do use PCIe_SS2 controller).

lding said:

It seems your board is flexible in terms of PCIE connector? You can either has two seperate PCIEx1 connectors and each connect with a x1 devices? Or you can have a PCIE x n connector and connect two a x2 devices?

I do connect devices directly to PCIe ports (port 0 and port 1 - exact pinout mentioned earlier in this thread). Those are x1 PCIe devices (they do require only one lane).

And yes - this is quite flexible.

Ach......

Please correct me if I'm wrong.

It seems like with my setup I DO need PCIe_SS1 -> lane 0 (port 0) and PCIe_SS2 -> lane 0 (port 1).

I cannot use PCIe_SS1 with two lanes connected to two ports (and different devices), since such configuration is for PCIe x2 devices (which can use two lanes for transmission to single device).

lding said:

So, with AM5728, you have SS1x2 and SS1x1 + SS2x1 worked already?

The _question_ here is if I can have one PCIe_SS1 controller and two different devices (PCIEx1) connected to lane-0 and lane-1 ?

Best regards,

Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi,

Yes, you can only use PCIESS1 to one device, either 1 lane or 2 lanes. If you what to connect to two devices, you have to use PCIESS1 and PCIESS2 seperatedly each with one lane/port.

Due to our EVM limitation, we can only give you some code suggestion how to make it work, but we can't verify/debug it for your scenario that two seperate controllers to two devices. Sorry for this!

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

lding said:

Hi,

Yes, you can only use PCIESS1 to one device, either 1 lane or 2 lanes. If you what to connect to two devices, you have to use PCIESS1 and PCIESS2 seperatedly each with one lane/port.

Ok, so now its clear how things should be configured.

lding said:

Due to our EVM limitation, we can only give you some code suggestion how to make it work, but we can't verify/debug it for your scenario that two seperate controllers to two devices. Sorry for this!

Ok, I see. Please prepare such suggestions, especially with all undocumented registers taken into account.

Thanks for support.

Łukasz

0 user4759774 over 7 years ago in reply to user4759774

Intellectual 745 points

Hi Eric,

What I've read (or understood ?) so far:

- PCIe_SS2 controller for AM5718 is placed at 0x5180 0000 (pcie@51800000)
- The PCIe2 PHY for AM5718 is placed at 0x4a084400 (phy@4a084400) - OCP2SCP1 and according to spruhz7c.pdf (FIG. 26-18) it is USB3_PHY_TX.
- What IP block is put under 0x4a084400 address at AM5718? Is it modified USB3_PHY? Where can I find description for it?

Solution:
- I think that I've found solution for the problem. However, it is quite hackish and not ready for mainline.

I do stop execution of the code at dra7xx_pcie_probe() function and for PCIe2 (pcie@51800000) I change via JTAG debugger values of registers:

mmahb 0x4a0093E0 0x1
mmahb 0x4a08482c 0x80000000
mmahb 0x4a08440c 0x80
mmahb 0x4a003c3c 0x0

DTS modification for AM5718:

&pcie2 {
status = "okay";

phys = <&usb3_phy1>;
};

And then lspci shows proper output (I however do need to test it on real HW).

I will share patch for this when I make it looking like one acceptable for posting it to main line.

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi,

"Please prepare such suggestions, especially with all undocumented registers taken into account"====>The suggestion was the update I made 01-03-2017 1:48 PM.

0x4a08440c 0x80 bit 7:
0: USB Mode
1: PCIe Mode

0x4a08482c bit 31:
0: USB Mode
1: PCIe Mode

pdk_am57xx_1_0_4\packages\ti\csl\soc\am571x\src\cslr_soc_dsp_baseaddress.h

#define CSL_DSP_OCP2SCP1_REGS (0x4a080000U)
#define CSL_DSP_OCP2SCP1_SIZE (0x1cU)
#define CSL_DSP_OCP2SCP1_USB_PHY1_CORE_REGS (0x4a084000U)
#define CSL_DSP_OCP2SCP1_USB_PHY1_CORE_SIZE (0x7cU)
#define CSL_DSP_OCP2SCP1_USB3RX_PHY_USB_REGS (0x4a084400U)
#define CSL_DSP_OCP2SCP1_USB3RX_PHY_USB_SIZE (0x68U)
#define CSL_DSP_OCP2SCP1_USB3TX_PHY_USB_REGS (0x4a084800U)
#define CSL_DSP_OCP2SCP1_USB3TX_PHY_USB_SIZE (0x38U)
#define CSL_DSP_OCP2SCP1_DPLLCTRL_USB_OTG_SS_REGS (0x4a084c00U)
#define CSL_DSP_OCP2SCP1_DPLLCTRL_USB_OTG_SS_SIZE (0x400U)
#define CSL_DSP_OCP2SCP1_USB_PHY2_CORE_REGS (0x4a085000U)
#define CSL_DSP_OCP2SCP1_USB_PHY2_CORE_SIZE (0x7cU)

Thanks very much for your trial and please share your results how to make PCIESS1x1 and PCIESS2x1 working if possible, that helps us and the whole community!

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

It is strange, but I do need to write following value:

mmahb 0x4a08482c 0x80000000

twice to read it back as 0x80000000.

Is this the expected behaviour?

Best regards,

Łukasz

0 user4759774 over 7 years ago in reply to user4759774

Intellectual 745 points

Hi Eric,

Any feedback on this register write?
Is it somewhat special (since it is not documented)?

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi,

Sorry for the late response! Since the thread is inactive for a month, I missed that. I tried the Processor SDK RTOS code, writing 0x4a08482c to 0x80000000 happens instantly and can be read back. No double write is required. I am not sure why you need do twice in Linux.

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

Thanks for confirmation.

To be 100% sure - there is no "status" bit for this register, so it could be easily read back?

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi,

Yes, that is right. From TRM Table 26-33, BIT 0-25 is reserved. BIT 26-30 are explained. BIT 31 we will update it. There is "status" bit and the register can be easily readback.

Regards, Eric

Processors

Processors forum

AM5718: PCIe question