Hello,
I have developed a linux device driver to use a C6678 as a PCIe peripheral. In my driver, I need access to a wide range of the C6678 address space, and to accomplish this, I periodically "re-map" the PCIe BAR1 for the device to target different areas of memory. In my case, I have configured BAR1 to be 32MB in size, and if I need to access memory that is not currently in this 32MB "window", I re-write the IB_OFFSET1 register of the C6678 PCIe peripheral via BAR0.
I recently discovered and 'fixed' a problem which was very difficult for me to find. I found that in certain situations, a PCIe read/write seemed to target the 'old' mapping, if issued immediately after the write to IB_OFFSET1 (to create the 'new' mapping). That is, it appears like the old address translation rules were applied, even though the transaction was performed after the write to IB_OFFSET1 which changed to translation rules. This happened very rarely (my application which transfers many MB/sec could run for many hours before this occurred (and even then, very non-deterministically). Due to the nature of what I was reading/writing, this issue showed up as a very subtle 'memory corruption'. I found that simply doing a single read of IB_OFFSET1 immediately after writing this register fixed this issue.
I am posting this here for two purposes:
1) in case this helps anyone else experiencing intermittent 'memory corruption' issues.
2) It would be much appreciated if TI could confirm that this is possible at the HW level, and what the recommended programming sequence is for re-mapping a PCIe BAR.
To me, it makes sense that the HW may require some finite non-zero amount of time before the new address mapping rule is in effect, however I have not found that to be documented in the user-guide.
Thanks,
Joel