Board hangs after "ti81xx_pcie: Register base mapped @0xcb838000" message

Hi Carl,

Are you on EZSDK? If yes, make sure you are using the latest version 5.05.

This is what I have with the 5.05 EZSDK version:

ti81xx_pcie: Invoking PCI BIOS...
ti81xx_pcie: Setting up Host Controller...
ti81xx_pcie: Register base mapped @0xd7020000
ti81xx_pcie: Starting PCI scan...
PCI: bus0: Fast back to back transfers enabled

Regards,

Pavel

Thank you for your reply, but this is not a software problem.  I have four other boards that work fine with the version of the SDK I'm using.  I'm just hoping that someone can give me an idea about what hardware problem this board has because of the symptom that it hangs when attempting to read the PL_GEN2 register.  For example, is it because there's no PCIe reference clock (I don't know if this is true or not, it's just an example).

Hi Carl,

Can we first double check that the issue is in the PCIe for sure (there is a possibility that the linux kernel memory is not enough to proceed the boot-up process). For that purpose can you disable the PCIe support (just for the test) and check whether the boot-up process will proceed to the end. You can disable PCIe thus:

 $ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm menuconfig

Bus support  --->

            [ ] PCI support   - exclude the PCI support

$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm uImage

Then boot with the new uImage.

Other test that we can try is for the DDR memory:

http://processors.wiki.ti.com/index.php/DM816x_C6A816x_AM389x_DDR3_Init#Run_mtest

Run the mtest, to check whether the DDR memory is OK.

Regards,

Pavel

I tried removing PCI support and it worked.  However, please remember that I have 4 other identical systems that all run fine with PCI support.

I tried running mtest and got the following for 0x80000000:

--------------------------------------------------------------------------------------------------------------

# mtest 0x80000000 0xa0000000 0xaa55aa55 3
Testing 80000000 ... a0000000:
Iteration:      1
FAILURE (read/write) @ 0x806edde0: expected 0x001bb779, actual 0xffe44887)

FAILURE (read/write) @ 0x806edde4: expected 0x001bb77a, actual 0xffe44887)

FAILURE (read/write) @ 0x806eddec: expected 0xffe44885, actual 0x08000000)

FAILURE (read/write) @ 0x806eddf0: expected 0xffe44886, actual 0x001bb77c)

FAILURE (read/write) @ 0x806eddec: expected 0x001bb779, actual 0x001bb77a)

FAILURE (read/write) @ 0x806eddf0: expected 0x001bb77a, actual 0x001bb77c)

FAILURE (read/write) @ 0x7f912218: expected 0x001bb77b, actual 0xefefffff)

FAILURE (read/write): @ 0x806eda88: expected 0xffe4495c, actual 0x00000020)

FAILURE (read/write): @ 0x806eda8c: expected 0xffe4495b, actual 0x00000000)

FAILURE (read/write): @ 0x806eda90: expected 0xffe4495a, actual 0x0000000f)

FAILURE (read/write): @ 0x806eda94: expected 0xffe44959, actual 0x00000004)

FAILURE (read/write): @ 0x806eda9c: expected 0xffe44957, actual 0x35344957)

FAILURE (read/write): @ 0x806edaa0: expected 0xffe44956, actual 0x33343439)

FAILURE (read/write): @ 0x806edaa4: expected 0xffe44955, actual 0xffe43333)

FAILURE (read/write): @ 0x806edae0: expected 0xffe44946, actual 0x806eddd0)

FAILURE (read/write): @ 0x806edae4: expected 0xffe44945, actual 0x00000020)

FAILURE (read/write): @ 0x806edae8: expected 0xffe44944, actual 0x80000000)

FAILURE (read/write): @ 0x806edaec: expected 0xffe44943, actual 0x00000020)

FAILURE (read/write): @ 0x806edaf0: expected 0xffe44942, actual 0x806edbd7)

FAILURE (read/write): @ 0x806edaf4: expected 0xffe44941, actual 0x806eddcc)

FAILURE (read/write): @ 0x806edaf8: expected 0xffe44940, actual 0xffffffff)

FAILURE (read/write): @ 0x806edafc: expected 0xffe4493f, actual 0x807211d0)

FAILURE (read/write): @ 0x806edb00: expected 0xffe4493e, actual 0x00000010)

FAILURE (read/write): @ 0x806edb04: expected 0xffe4493d, actual 0xffffffff)

FAILURE (read/write): @ 0x806edb08: expected 0xffe4493c, actual 0x00000008)

FAILURE (read/write): @ 0x806edb0c: expected 0xffe4493b, actual 0x00000020)

FAILURE (read/write): @ 0x806edb10: expected 0xffe4493a, actual 0x806edb94)

FAILURE (read/write): @ 0x806edb18: expected 0xffe44938, actual 0x00000008)

FAILURE (read/write): @ 0x806edb1c: expected 0xffe44937, actual 0x8072cc6b)

FAILURE (read/write): @ 0x806edb68: expected 0xffe44924, actual 0x0000004d)

FAILURE (read/write): @ 0x806edb6c: expected 0xffe44923, actual 0x00000001)

FAILURE (read/write): @ 0x806edb70: expected 0xffe44922, actual 0x806edb94)

FAILURE (read/write): @ 0x806edb74: expected 0xffe44921, actual 0x8070c2d0)

FAILURE (read/write): @ 0x806edb78: expected 0xffe44920, actual 0x806edb94)

FAILURE (read/write): @ 0x806edb7c: expected 0xffe4491f, actual 0xffffffff)

FAILURE (read/write): @ 0x806edb80: expected 0xffe4491e, actual 0x80000000)

FAILURE (read/write): @ 0x806edb84: expected 0xffe4491d, actual 0x8071437c)

FAILURE (read/write): @ 0x806edb88: expected 0xffe4491c, actual 0x806edb94)

FAILURE (read/write): @ 0x806edb8c: expected 0xffe4491b, actual 0x807145ec)

FAILURE (read/write): @ 0x806edb94: expected 0xffe44919, actual 0x4941460a)

FAILURE (read/write): @ 0x806edb98: expected 0xffe44918, actual 0x4552554c)

FAILURE (read/write): @ 0x806edb9c: expected 0xffe44917, actual 0x65722820)

FAILURE (read/write): @ 0x806edba0: expected 0xffe44916, actual 0x772f6461)

FAILURE (read/write): @ 0x806edba4: expected 0xffe44915, actual 0x65746972)

FAILURE (read/write): @ 0x806edba8: expected 0xffe44914, actual 0x40203a29)

FAILURE (read/write): @ 0x806edbac: expected 0xffe44913, actual 0x38783020)

FAILURE (read/write): @ 0x806edbb0: expected 0xffe44912, actual 0x64653630)

FAILURE (read/write): @ 0x806edbb4: expected 0xffe44911, actual 0x3a306262)

FAILURE (read/write): @ 0x806edbb8: expected 0xffe44910, actual 0x70786520)

FAILURE (read/write): @ 0x806edbbc: expected 0xffe4490f, actual 0x65746365)

FAILURE (read/write): @ 0x806edbc0: expected 0xffe4490e, actual 0x78302064)

FAILURE (read/write): @ 0x806edbc4: expected 0xffe4490d, actual 0x34656666)

FAILURE (read/write): @ 0x806edbc8: expected 0xffe4490c, actual 0x64303934)

FAILURE (read/write): @ 0x806edbcc: expected 0xffe4490b, actual 0x6361202c)

FAILURE (read/write): @ 0x806edbd0: expected 0xffe4490a, actual 0x6c617574)

FAILURE (read/write): @ 0x806edbd4: expected 0xffe44909, actual 0x36783020)

FAILURE (read/write): @ 0x806edbd8: expected 0xffe44908, actual 0x33383736)

FAILURE (read/write): @ 0x806edbdc: expected 0xffe44907, actual 0x29363337)

FAILURE (read/write): @ 0x806edbe0: expected 0xffe44906, actual 0xffe4000a)

FAILURE (read/write): @ 0x806eddb4: expected 0xffe44891, actual 0x806eddc4)

FAILURE (read/write): @ 0x806eddb8: expected 0xffe44890, actual 0x00000000)

FAILURE (read/write): @ 0x806eddbc: expected 0xffe4488f, actual 0x8070f6bc)

FAILURE (read/write): @ 0x806eddc0: expected 0xffe4488e, actual 0x807430e8)

FAILURE (read/write): @ 0x806eddc4: expected 0xffe4488d, actual 0x807146ac)

FAILURE (read/write): @ 0x806eddc8: expected 0xffe4488c, actual 0x00000000)

FAILURE (read/write): @ 0x806eddcc: expected 0xffe4488b, actual 0x8070f6c0)
Tested 3 iteration(s) with 66 errors.

-----------------------------------------------------------------------

I then tried the same test for 0xa0000000 (mtest 0xa0000000 0xc0000000 0xaa55aa55 3) and let it run for several minutes, but it never returned after giving me the message: 

Testing a0000000 ... c0000000:
Iteration:      1

However, I tried these same tests on a known working system and got very similar results, so I'm not certain that this reflects a problem with the DDR3 memory.  One question I have is where U-BOOT resides in memory and what memory is U-BOOT using?  Obviously I should not be trying to mtest U-BOOT's memory since there'll be conflicts.

Hello,

Yes, the u-boot is in address range 0x8000_0000 - 0x8080_0000. So the test should be:

TI8168_EVM#mtest 0x80800000 0xA0000000 0xaa55aa55 1
Pattern AA55AA55  Writing...  Reading...Tested 1 iteration(s) with 0 errors.
TI8168_EVM#mtest 0xA0000000 0xC0000000 0xaa55aa55 1
Pattern AA55AA55  Writing...  Reading...Tested 1 iteration(s) with 0 errors.
TI8168_EVM#

I will further check for the PCIe reference clock.

Regards,

Pavel

The PCIe clock status can be check in these two registers:

CM_DEFAULT_PCI_CLKSTCTRL (at addr 0x48180510), bit [8] CLKACTIVITY_PCI_GCLK should be 0x1 (enable)

CM_DEFAULT_PCI_CLKCTRL (at addr 0x48180578) bits [1:0] MODULEMODE = 0x2 (module enable)

Also, for Hardware Design Check, see:

http://processors.wiki.ti.com/index.php/Hardware_Design_Checklist

DM816x datasheet, section 8.14 Peripheral Component Interconnect Express (PCIe) - Please refer to the routing, design and layout specifications

One additional question:

Do your flow stuck at this line:

__raw_writel(DIR_SPD | __raw_readl(
                reg_virt + SPACE0_LOCAL_CFG_OFFSET + PL_GEN2),
            reg_virt + SPACE0_LOCAL_CFG_OFFSET + PL_GEN2);

Or at this line:

val = __raw_readl(reg_virt + SPACE0_LOCAL_CFG_OFFSET + PL_GEN2);

What if you comment this line of code, does the kernel move forward with the boot-up process?

BR,

Pavel

I ran both mtests successfully.  There were no memory errors.

The flow gets stuck at the first line:

__raw_writel(DIR_SPD | __raw_readl(
                reg_virt + SPACE0_LOCAL_CFG_OFFSET + PL_GEN2),
            reg_virt + SPACE0_LOCAL_CFG_OFFSET + PL_GEN2);

When I comment that line out it then gets stuck at:

 __raw_writel(LTSSM_EN_VAL | __raw_readl(reg_virt + CMD_STATUS),
             reg_virt + CMD_STATUS);

It seems to be having trouble reading the PCIe registers.

I checked the PCI clock status by reading the registers you mentioned.  The CLKACTIVITY_PCI_GCLK is set to 0x1 and the MODULEMODE is set to 0x2.

Hi Carl,

The PCIe functional/interface clock (pcie_ck) is fine, this is what the CLKACTIVITY_PCI_GCLK and MODULEMODE bits stated.

For the PCIe reference clock (100MHz, serdes_clkp/n), you can check:

DM816x datasheet, section 7.3.2 SERDES_CLKN and SERDES_CLKP Input Clock

http://processors.wiki.ti.com/index.php/DM816x_C6A816x_AM389x_PCIe_Clocking_Schemes

Could you please provide me your boot arguments, and the value of the PCIE_CFG (addr 0x48140640) and SERDES_CTRL (addr 0x481406FC) registers, just before the "stuck" line of code.

Regards,

Pavel

Boot args are:

Kernel command line: console=ttyO2,115200 mem=176M mem=324M@0x9F900000 vmalloc=500M vram=128M earlyprintk z3dram=1024M notifyk.vpssm3_sva=0xBF900000 ti816xfb.vram=0:43M,1:43M,2:42M noinitrd root=/dev/nfs nfsroot=10.0.2.5:/home/cdb/work/z3/filesys/fs,nolock,udp rw rootdelay=4 ip=10.0.3.109:10.0.2.5:10.0.1.129:255.255.0.0:Z3-Netra::off

PCIE_CFG = 0x01c90002

SERDES_CTRL = 0x00000000

I've just been told by Z3 (the manufacturer of our DM8168 board) that their board is configured to be a PCIe endpoint rather than a PCIe root complex.  Do you know if this might cause a problem?  I assume this means that the PCIe hardware interface is expecting to see a PCIe reference clock on a PCIe connector which is then sent to the PCIe interface on the DM8168.  Would the PCIe interface on the DM8168 hang if there were no PCIe reference clock?

Hello,

The value of the PCIE_CFG register at my EVM, at this point is 0x01C90102. The difference is in bit [8] LOCK, which in your case is 0x0 (PCIe PLL has NOT locked). This is indication for PCIe clock issue, and that is why you can not read/write the PCIe registers.

Please have a look in our PCIe EP driver user guide:

http://processors.wiki.ti.com/index.php/DM81xx_AM38xx_PCI_Express_Endpoint_Driver_User_Guide

I will further check with our PCIe expert, for a possible workaround.

Regards,

Pavel