SK-AM62: Need QoS help with GPMC Contention between A53s and DMA

Hello Victor ,

Can you please confirm you are running which OS on A53 core ?

The A53 core and DMA use both GPMC.

If both cores are accessing at the same time , how did you handle this situation? Are you using any synchronization in between them ?

Regards,

Anil.

I am running Linux on the A53s.

I expect the interconnect on the AM62 to handle the traffic to the GPMC from A53s and DMA. I thought the QoS when set accordingly will give A53s higher priority over the DMA

While CPU0 starts the DMA transfer, it has no idea when it finishes sucking in the data.  Also, CPU 1 to CPU3 do not know if DMA transfers are in progress.  So rather difficult to mutex lock the GPMC.

We have this same functionality working in the AM4376.  However, traffic on the GPMC bus increased substantially on the AM62 due to newer higher capacity ASIC.

Thanks,

Victor

Hello Victor,

If core and DMA are accessing the same GPMC bus, we should use synchronization techniques, either Spin lock or Mutex techniques.

On the SOC level, there is no arbitration for the GPMC in AM62X devices.

Without using these synchronizations, using the same bus with DMA and Core is not recommended at the SOC level.

Anyway, I am routing your query to a Linux expert and let's see what they can give the feedback.

Regards,

Anil.

Hi Victor,

Please see the AM62x TRM (revB) Table 14-5 "QOS, QOS Registers". The address of A53 read QoS register is 0x45d20500, and its write QoS register is 0x45d20900.

By default out of reset, the register value is 0x7000, which sets the priority to 7 - the lowest.

Can you please try to set both registers to value 0x6000 to see if solve the A53 access GPMC starving problem?

Hello,

I finally did enough testing to know that setting the QoS register does not work.  I tried 0x6000, 0x4000 and 0x0000.  I believe the problem is that once the DMA starts reading from the GPMC bus, it does not relinquish it until it finish reading it regardless if the CPU with the higher QoS reading is waiting.

We are trying to fix the problem by breaking up the DMA transfer to smaller chunks which will then allow the CPU to come in after a DMA completion and take back the GPMC bus.

Thanks for the assistance,

Victor

Hi Victor,

thanks for doing these experiments. I an out of office for the next two weeks, I will talk to our SoC designers once I am back in office to see if what you observed is expected. 

Hi Victor,

When the CPU and DMA access GPMC simultaneously, do you see how long the CPU is blocked by the DMA access?

I talked to our SoC design, potentially the biggest data size in DMA transfer between QoS scheduling is 320 bytes, even if the application programs the DMA to transfer much more data.

Here is a rough math for the CPU delay estimate:

Assuming the FPGA runs at 100MHz, which is 0.01us per cycle. Also assuming FPGA data bus width is 16bit. 320 bytes would be 320*8/16 = 160 cycles, which is 1.6us. So it would be normal if the CPU is blocked by a few usec. But if you see much longer delay, it seems something is incorrectly configured... Let me know if you see my math is completely off.

Another thought - is it possible the CPU access is blocked by kernel interrupt context? Have to tried to use interrupt affinity to move DMA interrupt handling to other CPU than CPU0 to see if it improves the CPU access GPMC?

Hi Victor,

I don't deal with secure boot, but by secure boot, do you mean now you signed the boot image differently or you converted the HS-FS devices to HS-SE?

I guess maybe firewall blocks it. Please start a new post, and I will work with our security expert to figure it out.