DM8168 EVM as PCIe endpoint

I'm seeing a similar failure on a DM8148 EVM, with the kernel module included in the TI814X-LINUX-PSP-04.01.00.07 release.

I'm also following the instructions on the wiki:  http://processors.wiki.ti.com/index.php/DM81xx_AM38xx_PCI_Express_Endpoint_Driver_User_Guide

Has anyone successfully built and tested this module with a recent release?

Can anyone @ TI verify that this should at least build and load out-of-the-box?

Thanks.

-Cory

root@dm814x-evm:/opt/dvr_rdk/ti814x/kermod# insmod ./ti81xx_pcie_epdrv.ko
Unhandled fault: external abort on non-linefetch (0x1028) at 0xcd001004
Internal error: : 1028 [#1]
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in: ti81xx_pcie_epdrv(+)
CPU: 0 Not tainted (2.6.37 #1)
PC is at ti81xx_ep_pcie_init+0x1b8/0x290 [ti81xx_pcie_epdrv]
LR is at kobject_put+0x48/0x5c
pc : [<bf000c1c>] lr : [<c01ade44>] psr: 60000013
sp : c36d7f18 ip : cd001050 fp : c36d7f44
r10: 00000000 r9 : c36d6000 r8 : bf000e37
r7 : bf0011a8 r6 : 00000000 r5 : bf001198 r4 : cd000000
r3 : 00000000 r2 : cd001004 r1 : bf001210 r0 : bf001200
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 83460019 DAC: 00000015
Process insmod (pid: 1122, stack limit = 0xc36d62e8)
Stack: (0xc36d7f18 to 0xc36d8000)
7f00: bf000e37 bf0011a8
7f20: c0085638 00000000 bf001074 00012008 00012018 bf000a64 c36d7f7c c36d7f48
7f40: c003942c bf000a70 bf001074 00012008 00012018 c0043fa8 00000000 bf001074
7f60: 00012008 00012018 c0043fa8 00000000 c36d7fa4 c36d7f80 c0095930 c0039368
7f80: c00c2428 c00c2340 00000000 00004000 00012018 00000080 00000000 c36d7fa8
7fa0: c0043e00 c00958a0 00000000 00004000 00012018 00002d43 00012008 00000001
7fc0: 00000000 00004000 00012018 00000080 00012008 00000000 00000003 00000000
7fe0: bed3fec5 bed3fc5c 00008f5c 401bc7d4 60000010 00012018 00000000 00000000
Backtrace:
[<bf000a64>] (ti81xx_ep_pcie_init+0x0/0x290 [ti81xx_pcie_epdrv]) from [<c003942c>] (do_one_initcall+0xd0/0x1a4)
r8:bf000a64 r7:00012018 r6:00012008 r5:bf001074 r4:00000000
[<c003935c>] (do_one_initcall+0x0/0x1a4) from [<c0095930>] (sys_init_module+0x9c/0x1bc)
[<c0095894>] (sys_init_module+0x0/0x1bc) from [<c0043e00>] (ret_fast_syscall+0x0/0x30)
r7:00000080 r6:00012018 r5:00004000 r4:00000000
Code: e284cd41 e2822004 e584618c e28cc010 (e5923000)
---[ end trace 8cb5e381e1be0009 ]---
Segmentation fault
root@dm814x-evm:/opt/dvr_rdk/ti814x/kermod#

Hello,

As mentioned in the driver user guide, you need to follow the execution sequence extactly.

In the first port, I see that SD boot mode is used, this means that the PCIe is not set in endpoint mode which EP driver expects.

I have the same problem with the first port,Is the endpoint must run in boot driver mode??

Can anyone  help??

hi,

   hope this wiki will help you:

http://processors.wiki.ti.com/index.php/TI81XX_PSP_U-Boot_PCIe_Boot_User_Guide

Jack

Hi Hemant:

I want to use EP as non-pcie boot mode, should i have to add the

EP initial sequences  rferred in TRM in the UBoot ?

As i find that the ep uninitial at all, the reg 0x48140640 value is 0,

no ep device is found at rc part.

And i see no ep initial code to set PLL, mode , and so on,

This should be done by ourself then?

best regards

xavier

Xavier said:

I want to use EP as non-pcie boot mode, should i have to add the EP initial sequences  rferred in TRM in the UBoot ?

That's what we did.  I believe newer U-Boot builds may already include some of the necessary setup (i.e. PCIe operating mode, PLL, etc.).  We don't actually map BARs, but we do set up the size and attributes - enough so that our host OS can enumerate the card as a PCI device and allocate appropriate resources.

Hi Cory:

I find the pcie init code ,but it is ready for pcie_boot mode, in file

ti_tools\linux_lsp\uboot\u-boot-dvr-rdk\arch\arm\cpu\arm_cortexa8\ti81xx\pcie.c

whitout the moroc CONFIG_TI81XX_PCIE_BOOT and CONFIG_TI816X_MIN_CONFIG defined

pcie_init will not be called,

and if i enable this in \ti_tools\linux_lsp\uboot\u-boot-dvr-rdk\board\ti\ti8168\pcie.c

this is pcie load ep mode, 

not no-pcie load ep mode  what i really want ,

so i try to dircect call this fun ,but it can not be complied for no pcie_init found even i 

has include file #include <..\pcie.h>

Code

int misc_init_r (void)
{
#if defined(CONFIG_TI81XX_PCIE_BOOT) && defined(CONFIG_TI816X_MIN_CONFIG)
extern int pcie_init(void);
printf("\nSetting up for pcie boot...\n");
pcie_init();
return 0;
#endif

}

 

int pcie_init(void)

{
int bar_enable_mask = 0;
#ifndef TI81XX_NO_PIN_GPMC
unsigned long long bar_size[6] = {0};
#endif

trig = 1;
get_pcie_wdt_base_reload(&wdt_base, &wdt_reload);
wdt_set_reload(wdt_base, wdt_reload);

bootflag = (unsigned int) get_bootflag_addr();
__raw_writel(0, bootflag);

pcie_hw_setup();
pcie_pll_setup();
pcie_enable_module();
app_retry_enable();
DEBUGF("app_retry_enable() done\n");
set_basic_config();
DEBUGF("set basic config() done\n");

write_pcie_lcfg_reg_bits(STATUS_COMMAND,
CFG_REG_CMD_STATUS_MEM_SPACE_ENB,
CFG_REG_CMD_STATUS_MEM_SPACE);
enable_dbi_cs2();
set_ti81xx_device_id();
#ifdef CONFIG_TI81XX_PCIE_32
#ifdef TI81XX_NO_PIN_GPMC
set_size_config_32(&bar_enable_mask);
#else
get_size_boot_mode_pin_32(&bar_enable_mask, bar_size);
set_size_pin_32(bar_size);
#endif
#endif

#ifdef CONFIG_TI81XX_PCIE_64
#ifdef TI81XX_NO_PIN_GPMC
set_size_config_64(&bar_enable_mask);
#else
get_size_boot_mode_pin_64(&bar_enable_mask, bar_size);
set_size_pin_64(bar_size);
#endif
#endif

DEBUGF("set_bar_size() done\n");
disable_dbi_cs2();
DEBUGF("disable_dbi_cs2() done\n");
#ifdef CONFIG_TI81XX_PCIE_32
set_bar_config_32(&bar_enable_mask);
#endif

#ifdef CONFIG_TI81XX_PCIE_64
set_bar_config_64(&bar_enable_mask);
#endif

DEBUGF("set_bar_config() done\n");
config_appl_regs();
DEBUGF("config_appl_regs() done\n");
if (get_ti81xx_device_id() == TI8148_DEVICEID) {
config_force_x1(); /* Not for TI816X */
DEBUGF("config_force_x1() done\n");
}
pcie_enable_link();
install_exceptions();
wait_for_host();
return 0;
}

best regards

xavier

Xavier-

We'd taken the U-Boot board support files (board/ti/ti8148/*), and "cloned" them to our own board type, allowing us a free hand to hack in exactly what was needed for our particular hardware and use cases.  I'm honestly not familiar with what (if any) magic combination of CONFIG_* macros will work with the default board support files to accomplish what you're trying to do.

Below is just the applicable sequence of register writes we use for our hardware - this may prove a useful starting point if you head down a similar "custom board" path.  We simply call pcie_init() from s_init() following DDR setup every time.  Note also that we've tweaked the EP setup somewhat for our particular closed system - ours is a PCI card in a relatively "old", fixed, host system.  

/* --- PCIe configuration functionality ------------------------------------- */

static void pcie_hw_setup(void)
{
    /* Set up Endpoint mode */
    __raw_writel(0x00000000, 0x48140480);  /* PCIE_CFG */
}


static void pcie_pll_setup(void)
{
    __raw_writel(0x00000002, 0x48140E24);  /* SERDES_REFCLK_CTRL */
    delay(500);

    __raw_writel(0x00000000, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(500);
    __raw_writel(0x00640000, 0x481406DC);  /* PCIE_PLLCFG1 */
    delay(500);
    __raw_writel(0x00000000, 0x481406E0);  /* PCIE_PLLCFG2 */
    delay(500);
    __raw_writel(0x004008E0, 0x481406E4);  /* PCIE_PLLCFG3 */
    delay(500);
    __raw_writel(0x0000609C, 0x481406E8);  /* PCIE_PLLCFG4 */
    delay(500);

#if 0
    /* WARNING!  WARNING!  WARNING!  WARNING!  WARNING!  WARNING!  WARNING!
     *
     * There is no reference to this particular register anywhere in either
     * the DM816x or DM814x technical literature.  In the original U-Boot
     * PCIe configuration, the corresponding write was commented out.
     *
     * For the time being, we'll leave it commented out and see what (if
     * anything) breaks.
     *
     * WARNING!  WARNING!  WARNING!  WARNING!  WARNING!  WARNING! WARNING!
     */
    __raw_writel(0x00000E7B, 0x48141318);  /* PCIE_SERDES_CFG_MISC */
    delay(500);
#endif

    __raw_writel(0x00000004, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(500);
    __raw_writel(0x00000014, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(500);
    __raw_writel(0x00000016, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(500);
    __raw_writel(0x30000016, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(500);
    __raw_writel(0x70007016, 0x481406D8);  /* PCIE_PLLCFG0 */
    delay(2000);
    __raw_writel(0x70007017, 0x481406D8);  /* PCIE_PLLCFG0 */

    while ((__raw_readl(0x481406EC) & 0x00000001) == 0x00000000)
    {
        /* NOP */
    }
}


static void pcie_enable_module(void)
{
    __raw_writel(0x0000009C, 0x48180B10);  /* RM_DEFAULT_RSTCTRL */
    delay(500);

    __raw_writel(0x00000000, 0x48180578);  /* CM_DEFAULT_PCI_CLKCTRL */
    __raw_writel(0x00000000, 0x48180510);  /* CM_DEFAULT_PCI_CLKSTCTRL */
    delay(500);

    __raw_writel(0x00000002, 0x48180510);  /* CM_DEFAULT_PCI_CLKSTCTRL */
    __raw_writel(0x00000002, 0x48180578);  /* CM_DEFAULT_PCI_CLKCTRL */
    delay(500);

    __raw_writel(0x0000001C, 0x48180B10);  /* RM_DEFAULT_RSTCTRL */
    delay(500);
}


static void pcie_setup_ep(void)
{
    /* Set "Application Request Retry Enable" while we configure EP.
     * (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) | (1 << 4), 0x51000004);

    /* Set vendor & device ID. (VENDOR_DEVICE_ID) */
    __raw_writel(0x104C | (0xB801 << 16), 0x51001000);

    /* Enable device for memory accesses. (STATUS_COMMAND) */
    __raw_writel(__raw_readl(0x51001004) | (1 << 1), 0x51001004);

    /* Set CLASSCODE. (CLASSCODE_REVID) */
    __raw_writel(0x04000001, 0x51001008);

    /* Disable 64-bit MSI capability. */
    __raw_writel(__raw_readl(0x51001050) & ~(1 << 23), 0x51001050);

    /* Enable DBI_CS2 - allows BAR mask setup. (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) | (1 << 5), 0x51000004);
    while((__raw_readl(0x51000004) & (1 << 5)) == 0);

    /* Set BAR masks prior to enumeration by RC. */
    __raw_writel(0x00000FFF, 0x51001010);  /* BAR0: 4 KiB */
    __raw_writel(0x00007FFF, 0x51001014);  /* BAR1: 32 KiB */
    __raw_writel(0x00000000, 0x51001018);  /* BAR2: disabled */
    __raw_writel(0x00000000, 0x5100101C);  /* BAR3: disabled */
    __raw_writel(0x00000000, 0x51001020);  /* BAR4: disabled */
    __raw_writel(0x00000000, 0x51001024);  /* BAR5: disabled */

    /* Disable DBI_CS2. (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) & ~(1 << 5), 0x51000004);

    /* Set BAR access type and attributes after DBI_CS2 disable. */
    __raw_writel(0x00000000, 0x51001010);  /* BAR0: 32-bit, !pre-fetch */
    __raw_writel(0x00000000, 0x51001014);  /* BAR1: 32-bit, !pre-fetch */
    __raw_writel(0x00000000, 0x51001018);  /* BAR2: disabled */
    __raw_writel(0x00000000, 0x5100101C);  /* BAR3: disabled */
    __raw_writel(0x00000000, 0x51001020);  /* BAR4: disabled */
    __raw_writel(0x00000000, 0x51001024);  /* BAR5: disabled */

    /* Disable (skip over) exposing power management capabilities.
     *
     * Host system BIOS will place card into the D3 power state upon system
     * boot.  This results in the card not showing up in Windows - generally
     * considered a BadThing.  In order to avoid this unhappy state of affairs,
     * we're removing the PM capabilities entry from the extended configuration
     * section of the Local Configuration registers by pointing to the
     * descriptor immediately thereafter.
     */
    __raw_writel(0x00000050, 0x51001034);

    /* Enable [in/out]-bound address translation. (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) | (3 << 1), 0x51000004);

    /* Disable legacy (inbound), enable MSI (inbound), interrupts. */
    __raw_writel(0x0000000F, 0x5100018C);  /* IRQ_ENABLE_CLR */
    __raw_writel(0xFFFFFFFF, 0x51000108);  /* MSI0_IRQ_ENABLE_SET */
    __raw_writel(__raw_readl(0x51001050) | (1 << 16), 0x51001050);
#if 0  /* Support for "Disable INTx" is PCI 2.3 or later. */
    __raw_writel(__raw_readl(0x51001004) | (1 << 10), 0x51001004);
#endif

    /* Set x1 mode: LINK_CAP, PL_GEN2 & PL_LINK_CTRL */
    __raw_writel((__raw_readl(0x5100107C) & ~(0x3F <<  4)) | (1 <<  4), 0x5100107C);
    __raw_writel((__raw_readl(0x5100180C) & ~(0xFF <<  8)) | (1 <<  8), 0x5100180C);
    __raw_writel((__raw_readl(0x51001710) & ~(0x3F << 16)) | (1 << 16), 0x5100180C);
}


static void pcie_init(void)
{
    pcie_hw_setup();
    pcie_pll_setup();
    pcie_enable_module();
    pcie_setup_ep();

    /* Enable LTSSM. (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) | 0x00000B01, 0x51000004);
    /* EP configuration done, disable app retry. (CMD_STATUS) */
    __raw_writel(__raw_readl(0x51000004) & ~(1 << 4), 0x51000004);
}