Hello guys !
I am trying to understand how to use PCIe and EDMA in order to transmit data from Linux Host to EVMC6678 DDR3 and vice versa. Now I have some problems, and probably they are because of bad understandting of linux memory management, DMA technology and Linux device drivers (I am not professional device driver writer). So here the story goes:
My current test setup
- At my Host side I use Ubuntu 12.04 (32-bit) in order to make, insert and execute pciedemo.ko module from example provided by TI in MCSDK;
- EVMC6678 with AMC-PCIe adaptor is installed in PCIe slot at Host side, boot mode SWitches are set to PCIE boot;
- Prebuilt IBL is flashed vie EEPROM writer. When HOST is on it recognizes Board as Multimedia controller: Texas Instruments Device b005; BARs are also configured correctly (I suppose);
- Pciedemo.c is modified by me in the following way:
- Have added device registration in the system in order to use it to get data in user space for futher displaying and processing;
- Have removed pushing of interrupt demo code into DSP. I load it via my 560v2 emulator
- .mmap field of file operations driver structure calls my mmap routine which uses dma_mmap_coherent API;
- Interrupt demo code is modified by me in the following way:
- “start_boot” is deleted;
- Bit reverse operation is skipped. DSP just forwards data it recieves from HOST back to HOST.
My workflow description
1) Turn on Host machine with DSP board in PCIe slot; Check that Host recognizes Board and BARS;
2) Turn on my Windows machine and built interrupt demo code in CCS; initalize DDR, Load modified interrupt demo code and run it;
3) Insert modified pciedemo.ko module;
4) Open registered device and use mmap in order to get data into user space;
My results
I can see the 4 MB of data which I send from HOST at the HOST side via printk. It seems to be correct.
I can also see this 4 MB of data in DDR3 of my DSP board starting from 0x80000000 and until 0x80400000. It seems to be absolutely correct;
I can also see data at the HOST side received from DSP via printk. It seems to be correct also.
Linux says that my full sized DMA buffer is mapped and I can receive data in the user space and here the problem goes:
Problem description and questions
- Linux says that size of mapped buffer is 4 MB (4 MB is DMA buffer size in .ko module). As far as I understand DSP DDR3 memory is somehow mapped via IBL to BAR3 of 16 MB size, created by IBL. And then BAR mapping is made to DMA buffer somehow via pciedemo.ko. Is my understanding correct ? At least approximately ( I will repeat again that I am not profi in DMA, PCIe interface and Linux memory management);
- So in my mmap API I ask to mmap available 4 MB of received DMA buffer to user workspace and it does. But in the end when I try to read more than 1 MB of data using created file descriptor I always have segmentation error. How to fix this problem and to map the whole DMA buffer to user space ?
- This 1 MB is filled with correct data for sure, the data which have been send from host, to DDR3 starting from 0x80000000, which have been modified (not modified) by DSP and finally received back to host. I can read some Kb of data after 1 MB and I see some trash there which I don’t expect to see (not DDR3 data). And if I try to read lets say 2 MB – welcome to segmentation error. And the most strange that with several iterrations I finally once or twice received correct data from the end of this 4 MB DMA buffer in my user space. But I asked only for one 32 bit register at the end of the buffer and not the whole buffer. How could it be ? I suppose my problem is with mmaping DMA buffer from kernel to user space. Because at the host side in kerenl all 4 MB of data is correct. How to solve this problem?
I will really appriciate any help and explanations ! Thank you !
