Hi All,
I am doing a PCIE throughput test via a kernel module, the test result numbers are quite strange (write is 210MB/s but read is just 60MB/s for PCIE gen1 x1). I would like to ask for your suggestions and correction if there are wrong approaches in my test configuration.
My test configuration is as follow:
- One board is configured as the Root Port, one board is configured as the Endpoint. PCIE link is gen 1, width x1, MPS 128B. Both boards run Linux OS
- At Root Port side, we allocate a memory buffer and its size is 4MB. We map the inbound PCIE memory transaction to this buffer.
- At Endpoint side, we do DMA read/write to the remote buffer and measure throughput. With this test the Endpoint will always be the initiator of transactions.
- The test result is 214MB/s for EP Write test and it is only 60MB/s for EP Read test. The Write test throughput is reasonable for PCIe Gen1 x1, but the EP Read throughput is too low.
For the RP board, I tested it with PCIE Ethernet e1000e card and get maximum throughput ~900Mbps. I just wonder in the case of Ethernet TX path, the Ethernet card (plays Endpoint role) also does EP Read request and can get high throughput (~110MB/s) with even smaller DMA transfer, so there must be something wrong with my DMA EP Read configuration.
The detail of the DMA Read test can be summarized with below pseudo code:
dest_buffer = kmalloc(1MB)
memset(dest_buffer, 0)
dest_phy_addr = dma_map_single(destination_buffer)
source_phy_addr = outbound region of Endpoint
get_time(t1)
Loop 100 times
Issue DMA read from source_phy_addr to dest_phy_addr
wait for DMA read completion
get_time(t2)
throughput = (1MB * 100)/(t2 - t1) Any recommendations and suggestion are appreciated. Thanks in advanced!
