Hi Folks,
I am trying to bring up a TI XIO2213A 1394B interface via the PCIe slot on the EVM. I am using TI's XIO2213ZAY reference board for the FireWire card.
I have rebuilt the kernel using the Linux PSP included in the ti-ezsdk_c6a816x-evm_5_00_00_56 (e.g. /psp/linux-2.6.34-psp04.00.00.07) and enabled the new FireWire stack.
The OHCI controller is recognized on the PCIe bus and the driver is loaded.
If I connect a FW HD to the Firewire bus, the device is recognized, the SBP-2 driver is loaded and the FAT filesystem on the drive is mounted.
If I then do a ls on the filesystem, I get a kernel panic:
root@c6a816x-evm:~# ls /media/sda1/
Unhandled fault: Precise External Abort on non-linefetch (0x1008) at 0xca87a080
Internal error: : 1008 [#1]
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in: bufferclass_ti omaplfb pvrsrvkm TI81xx_hdmi(P) ti81xxfb vpss syslink ipv6
CPU: 0 Tainted: P (2.6.34 #1)
PC is at at_context_transmit+0x460/0x510
LR is at ___dma_single_cpu_to_dev+0x60/0x70
pc : [<c0200ae8>] lr : [<c00379d0>] psr: 60000093
sp : c6bb59e0 ip : c6bb59e0 fp : c6bb5a54
r10: c7011000 r9 : 00000093 r8 : c7011000
r7 : c7011534 r6 : c6ad1014 r5 : ffc61d60 r4 : c6ad106c
r3 : ca87a000 r2 : 00000003 r1 : 86ad106c r0 : 00000003
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 8682c019 DAC: 00000015
Process ls (pid: 1233, stack limit = 0xc6bb42e8)
The PC where the exception occurs is a register access in the Firewire-OHCI driver drivers/firewire/ohci.c line 1036:
reg_read(ohci, OHCI1394_IntEventSet) & OHCI1394_busReset) {
Hello,
Please provide following details:
1) Output of "lspci -xvvv" (you may need to build/install this if the filesystem doesn't have this utility).
2) cat /proc/iomem
3) cat /proc/ioports
Also, I haven't looked much at the firewire driver but is the crash observed on the *first* access to the register or a few accesses work before the crash?
Thanks.
Hemant
--- Hemant
Hi Hermant,
Thank you for the reply.
I will do these things tomorrow when I have access to the board and post them.
As far as question about the accesses, I can't tell you off hand if this is the first access to this register or not, but it is definitely not the first access to registers in the OHCI chip as a substantial number of FW transactions have occurred by this point (those required to id the FW device, mount the disk, etc.). Since the register being accessed is the pending Interrupts register, I rather think that this is *not* the first access to this register, but I need to check the code more closely to be sure.
Best regards,
B.J.
lspci -xvvv --- you may need to build/install this if the filesystem doesn't have this utility....
What is the easiest way to get the pciutils onto the EVM?
opkg install pciutils
does not work.
I was able to build pciutils with arago-oe, but I have no idea how to get the ipkg stuff to expand on the EVM, or how to get opkg to look at my arago repository. I also don't really see how to manually do the cross compile.
Any suggestions would be appreciated!
"opkg install pciutils" should work provided conf files inside /etc/opkg directory are set correctly. Make sure you have at least 'armv7a' entry inside /etc/opkg/arch.conf and respective config files for feeds are present (e.g., /etc/opkg/arago-armv7a-feed.conf).
If you are still not able to get ispci, then you can provide dump of entries from /sys/bus/pci. I particularly wanted dump of resources and size.
In addition, please also provide the dump of 0x51001728 register *before* and *after* the crash.
HemantPedanekar "opkg install pciutils" should work provided conf files inside /etc/opkg directory are set correctly. Make sure you have at least 'armv7a' entry inside /etc/opkg/arch.conf and respective config files for feeds are present (e.g., /etc/opkg/arago-armv7a-feed.conf).
This does not appear to work. /etc/opkg/arago-armv7a-feed.conf does not exist on the rootfs that comes with the EVM. There is a sample file for that, so I copied the sample to the .conf file, and then did opkg update. But it still does not find the package for pciutils. I finally simply built a rootfs with arago, and then copied the pciutils*.ipkg files to the root image and did the opkg install from the ipkg files.
So, I have lspci installed, but it seems like something is still not set up correctly for using the opkg tool.
When I do lspci, I get something that does not look right (this is with a NEC USB3 card [which I used to try to eliminate the PCIe <-> PCI bridge that is on the FireWire card], but the Firewire card looks the same -- with an additional entry for the bridge)
# lspci -xvvv
00:00.0 PCI bridge: Texas Instruments Device 8888 (rev ff) (prog-if ff)
!!! Unknown header type 7f
00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
10: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
20: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
30: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
01:00.0 USB Controller: NEC Corporation Device 0194 (rev ff) (prog-if ff)
as far as dumping the "resource" file from /sys, here is the result:
root@c6a816x-evm:~# more /sys/devices/pci0000\:00/0000\:00\:00.0/resource
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000020000000 0x00000000200fffff 0x0000000000000200
root@c6a816x-evm:~# more /sys/devices/pci0000\:00/0000\:00\:00.0/0000\:01\:00.0/resource
0x0000000020000000 0x0000000020001fff 0x0000000000140204
Hi Hemant,
HemantPedanekar In addition, please also provide the dump of 0x51001728 register *before* and *after* the crash.
I'm sorry -- I am new to the TI world and still trying to get my bearings. Can you explain exactly how to do this?
Thanks!
B.J. Buchalter
Here is the other info you requested:
root@c6a816x-evm:~# cat /proc/iomem
00000000-00000000 : omap2-nand
20000000-2fffffff : pcie-nonprefetch
47401000-474017ff : musb_hdrc.0
47401800-47401fff : musb_hdrc.1
48020000-4802001f : serial
48022000-4802201f : serial
48024000-4802401f : serial
48028000-4802803f : i2c_omap.1
48028000-4802803f : i2c_omap
4802a000-4802a03f : i2c_omap.2
4802a000-4802a03f : i2c_omap
48030100-480301ff : omap2_mcspi.1
48050000-48052fff : mcasp
48050000-48052fff : davinci-mcasp
48060100-480700ff : mmci-omap-hs.0
48060100-480700ff : mmci-omap-hs
49000000-49007fff : edma_cc0
49000000-49007fff : edma
49800000-498003ff : edma_tc0
49900000-499003ff : edma_tc1
49a00000-49a003ff : edma_tc2
49b00000-49b003ff : edma_tc3
4a100000-4a103fff : davinci_emac.0
4a100000-4a103fff : eth0
4a120000-4a123fff : davinci_emac.1
4a120000-4a123fff :
4a140000-4a150fff : ahci
51000000-51003fff : pcie-regs
80000000-ffffffff : pcie-inbound0
80000000-8a5fffff : System RAM
80031000-803e4fff : Kernel text
803e6000-80447f5b : Kernel data
root@c6a816x-evm:~# cat /proc/ioports
40000000-402fffff : PCI I/O
40000000-402fffff : pcie-io
So, I managed to build a 2.6.37 kernel and get it booted; it does not fully work because the modules don't match -- still trying to work out how to build a complete kernel and root-filesystem for the newer kernel.
That being said, PCIe seems to work much better with the newer kernel; if I boot with the USB3 card installed, lspci -vv actually shows information about the configuration space. If I boot with a FireWire card installed, the kernel panics on boot because it cannot mount the root filesystem. But I suspect that has something to do with not properly building the kernel.
So, can someone provide the steps required to build a more modern kernel/root file system with arago? The one that is built from the arago checkout seems to be consistent with the current ezsdk which is 2.6.34.
Hi,
B.J. Buchalter [which I used to try to eliminate the PCIe <-> PCI bridge that is on the FireWire card], but the Firewire card looks the same -- with an additional entry for the bridge)
[which I used to try to eliminate the PCIe <-> PCI bridge that is on the FireWire card], but the Firewire card looks the same -- with an additional entry for the bridge)
Does the lspci dump for firewire card at least show correct values (instead of 0xff 's)?
B.J. Buchalter It seems like something must be screwed up with the PCI RC as all the reads seem to be coming back as 0xffffffff. Any clue, or anything else you want me to check?
It seems like something must be screwed up with the PCI RC as all the reads seem to be coming back as 0xffffffff.
Looks like for some reason, the PCIe link went down after successful enumeration. That's why the "/sys/devices/pci0000\:00/0000\:01\:00.0/resource" dump shows correct values but "lspci -x" doesn't.
You can cross compile and use devmem2 like utility to dump the register I mentioned earlier (0x51001728). The last 5 bits should be 0x11 to indicate the link is established.
But if the firewire card at least shows correctly in lspci, then I would suggest to use it to debug rather than the USB card.
B.J. Buchalter Here is the other info you requested: root@c6a816x-evm:~# cat /proc/iomem 00000000-00000000 : omap2-nand 20000000-2fffffff : pcie-nonprefetch
I guess this dump is before the driver is initialized? I was expecting an entry for driver resource (e.g., BAR0) here.
B.J. Buchalter So, I managed to build a 2.6.37 kernel and get it booted; it does not fully work because the modules don't match -- still trying to work out how to build a complete kernel and root-filesystem for the newer kernel.
Yes, the SDK with latest kernel is not yet out so you cannot use pre-built modules.
B.J. Buchalter That being said, PCIe seems to work much better with the newer kernel; if I boot with the USB3 card installed, lspci -vv actually shows information about the configuration space.
That being said, PCIe seems to work much better with the newer kernel; if I boot with the USB3 card installed, lspci -vv actually shows information about the configuration space.
Did you use "lspci -vv" or "lspci -xvv"?
B.J. Buchalter If I boot with a FireWire card installed, the kernel panics on boot because it cannot mount the root filesystem. But I suspect that has something to do with not properly building the kernel.
If I boot with a FireWire card installed, the kernel panics on boot because it cannot mount the root filesystem. But I suspect that has something to do with not properly building the kernel.
If you are able to boot the kernel with PCIe-USB card, then it should boot fine with PCIe-Firewire card too as long as you use the same rootfs (e.g., NFS). Are you using the firewire driver as module or built into kernel? It may be better if you use it as module (if possible).
No. It is the same as the USB card.
Looks like for some reason, the PCIe link went down after successful enumeration.
Thats sort of what I was thinking.
I did that; when I ran devmem2 against that physical address, I got the same message that I reported when I started this thread (about a precise exception) and the program segfaulted.
It seems like, with the 2.6.34 kernel, that the PCIe RC fails after some number of accesses; in each case I checked it had failed by the time I could log in on the console to run lspci or any other tests.
As I said in a later post, I have tried to build 2.6.37 from the linux-omap3 git repository, and it seems to get me a bit further, but I am having problems getting the system up by just replacing the uImage (for example none of the driver modules are available, and the inittab references the wrong tty); I am really unclear on the process of moving to a newer kernel and building a working boot image and root filesystem.
With the "sort-of-working" 2.6.37 kernel, if I use the USB card, I can get reasonable output from lspci. Unfortunately, if I boot with the FIreWire card installed, the kernel panics when it tries to mount the root partition from the sdcard; I am unclear on why having the FireWire card plugged in at boot would cause the sdcard to not be able to mount.
With the 2.6.37 kernel, and no FireWire card, the relevant portion of the boot sequence is:
usb 1-1: new low speed USB device using musb-hdrc and address 2
mmc0: new SDHC card at address e699
mmcblk0: mmc0:e699 SD08G 7.40 GiB
mmcblk0: p1 p2 p3
ata1: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
EXT3-fs: barriers not enabled
usb 1-1: New USB device found, idVendor=13ee, idProduct=0003
usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-1: Product: Optical Mouse
usb 1-1: Manufacturer: MosArt
input: MosArt Optical Mouse as /devices/platform/musb-ti81xx/musb-hdrc.0/usb1/1-1/1-1:1.0/input/input0
generic-usb 0003:13EE:0003.0001: input: USB HID v1.10 Mouse [MosArt Optical Mouse] on usb-musb-hdrc.0-1/input0
kjournald starting. Commit interval 5 seconds
EXT3-fs (mmcblk0p2): warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs (mmcblk0p2): using internal journal
EXT3-fs (mmcblk0p2): recovery complete
EXT3-fs (mmcblk0p2): mounted filesystem with writeback data mode
VFS: Mounted root (ext3 filesystem) on device 179:2.
With the 2.6.37 kernel and the FireWire card installed, the same portion looks like:
Root-NFS: no NFS server address
VFS: Unable to mount root fs via NFS, trying floppy.
VFS: Cannot open root device "mmcblk0p2" or unknown-block(2,0)
Please append a correct "root=" boot option; here are the available partitions:
1f00 64 mtdblock0 (driver?)
1f02 64 mtdblock2 (driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(2,0)
Backtrace:
[<c0043b44>] (dump_backtrace+0x0/0x110) from [<c0360558>] (dump_stack+0x18/0x1c)
r7:c6c13000 r6:00000000 r5:c002b87c r4:c04cbd50
[<c0360540>] (dump_stack+0x0/0x1c) from [<c03605bc>] (panic+0x60/0x17c)
[<c036055c>] (panic+0x0/0x17c) from [<c0009254>] (mount_block_root+0x1e0/0x220)
r3:00000000 r2:00000000 r1:c6c23f58 r0:c0414b4c
[<c0009074>] (mount_block_root+0x0/0x220) from [<c0009340>] (mount_root+0xac/0xcc)
[<c0009294>] (mount_root+0x0/0xcc) from [<c00094d0>] (prepare_namespace+0x170/0x1d4)
r4:c04cb6e4
[<c0009360>] (prepare_namespace+0x0/0x1d4) from [<c0008784>] (kernel_init+0x114/0x154)
r5:c0008670 r4:c04cb680
[<c0008670>] (kernel_init+0x0/0x154) from [<c0063b00>] (do_exit+0x0/0x5e4)
r5:c0008670 r4:00000000
Note that it does not seem to load the driver for the mmc.
Any ideas, or is there anywhere that really describes how to rebuild the complete ezsdk kernel and filesystem, hopefully with instructions that would support moving to the newer kernel?
Hermant, have you tried using PCIe on the kernel that ships with the ezSDK? Because it doesn't seem to work at all here. If you are working with a newer kernel, can you point to the proper bits to be using?
HemantPedanekar If you are able to boot the kernel with PCIe-USB card, then it should boot fine with PCIe-Firewire card too as long as you use the same rootfs (e.g., NFS). Are you using the firewire driver as module or built into kernel? It may be better if you use it as module (if possible).
Look at my previous message for details, but, at least for now, I am trying to use the rootfs that comes on the SD card. The kernel is built with the FireWire drivers built in. I did not build in the drivers for the USB3 card, so that could be the difference, but the problem is that when the FireWire card is intstalled, the linux kernel driver for the SD card is not loading and the root file system does not mount at all, so it is not even to the point that a module could load.
Perhaps doing it via NFS would alleviate the problem or change the order in such a way that the MMC driver loads, but I still don't really have a good recipe to see how to actually rebuild the root file system for NFS mount; in particular, I am not sure how to get the GPU drivers installed in to the image.
I think we are getting into too many combination here! Can you please first try the following and provide your observations? Also, let's stick to 2.6.34 kernel for now (till we see that it is not at all working).
1) Use the 2.6.34 kernel but *do not* build firewire support into it. Then boot this kernel with firewire card plugged into the PCIe slot.
2) With the above, please provide the "lspci -xvvv" dump.
3) If you see 0xff 's, then enable the PCI debugging in kernel (make menuconfig --> Bus support --> PCI Debugging) and append "debug" to the bootargs (you may also want to turn on low level debugging). Please provide the log you will get (specifically PCI enumeration part).
4) If you see lspci in (2) fine without 0xff 's, then build the firewire drivers as modules and load them. Now check the "lspci -xvvv" again.