Hello,
I'm working on a DSP program on the AM5726 where we're attempting to DMA data from a FIFO provided by an FPGA on the GMPC to L2 memory in the DSP. My understanding is that the SAM and FWID bits in the PaRAM options (and therefore, the EDMA options register) need to be configured for constant addressing with the width of our FIFO. When we try to perform the DMA, the SoC becomes overloaded and unusable, with many errors printed out to the serial debug port. However, if we replace the DMA operation with a manual copy from the GPMC address to L2 memory, the operation is successful and we're able to do a second DMA from L2 memory to a different shared memory buffer to pass down our processing pipeline. This suggests that something with the DMA from the GPMC isn't configured correctly, but I'm not sure what we're doing wrong.
Here is the error output generated:
[ 66.302116] ------------[ cut here ]------------ [ 66.302123] WARNING: CPU: 0 PID: 23 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x330/0x380 [ 66.302127] 44000000.ocp:L3 Standard Error: MASTER DSP1_DMA TARGET GPMC (Write Non-Posted): At Address: 0x10000000 : Opcode Fetch in User mode during Debug access [ 66.302129] Modules linked in: rpmsg_proto rpmsg_rpc ti_prueth pru_rproc snd_soc_simple_card snd_soc_simple_card_utils pruss irq_pruss_intc snd_soc_omap_hdmi pvrsrvkm(O) omap_aes_driver omap_sham pruss_soc_bus omap_wdt ti_vpe ti_sc ti_csc ti_vpdma dwc3_omap rtc_omap extcon_palmas rtc_palmas omap_des omap_hdq des_generic at24 rtc_ds1307 snd_soc_tlv320aic3x wire crypto_engine omap_crypto omap_remoteproc virtio_rpmsg_bus remoteproc sch_fq_codel uio_module_drv(O) uio ftdi_sio usbserial usbcore usb_common jailhouse(O) gdbserverproxy(O) cryptodev(O) cmemk(O) [ 66.302178] CPU: 0 PID: 23 Comm: kworker/0:1 Tainted: G W O 4.19.94-gbe5389fd85 #1 [ 66.302180] Hardware name: Generic DRA74X (Flattened Device Tree) [ 66.302194] Workqueue: events rproc_crash_handler_work [remoteproc] [ 66.302197] Backtrace: [ 66.302204] [<c020ca34>] (dump_backtrace) from [<c020cd6c>] (show_stack+0x18/0x1c) [ 66.302209] r7:c0dfe4a4 r6:60000193 r5:00000000 r4:c125071c [ 66.302216] [<c020cd54>] (show_stack) from [<c0a15f64>] (dump_stack+0x9c/0xb0) [ 66.302222] [<c0a15ec8>] (dump_stack) from [<c022e100>] (__warn+0xdc/0xf8) [ 66.302226] r7:c0dfe4a4 r6:00000009 r5:00000000 r4:dfa2d9dc [ 66.302231] [<c022e024>] (__warn) from [<c022dd38>] (warn_slowpath_fmt+0x50/0x6c) [ 66.302236] r9:dfa05bc0 r8:e0880158 r7:c0dfe3c4 r6:c0dfe7ac r5:c0dfe474 r4:c1204c48 [ 66.302243] [<c022dcec>] (warn_slowpath_fmt) from [<c056711c>] (l3_interrupt_handler+0x330/0x380) [ 66.302246] r3:dfa3a840 r2:c0dfe474 [ 66.302249] r5:00000005 r4:80080001 [ 66.302256] [<c0566dec>] (l3_interrupt_handler) from [<c02895ec>] (__handle_irq_event_percpu+0x68/0x140) [ 66.302261] r10:c1252338 r9:dfa37f00 r8:00000017 r7:dfa2db08 r6:00000000 r5:dfa37f68 [ 66.302264] r4:dfa3ac40 [ 66.302270] [<c0289584>] (__handle_irq_event_percpu) from [<c02896f8>] (handle_irq_event_percpu+0x34/0x88) [ 66.302275] r10:c1257cd0 r9:dfa2c000 r8:df80c000 r7:00000001 r6:dfa37f00 r5:dfa37f68 [ 66.302278] r4:c1204c48 [ 66.302284] [<c02896c4>] (handle_irq_event_percpu) from [<c028978c>] (handle_irq_event+0x40/0x64) [ 66.302288] r6:c120a434 r5:dfa37f68 r4:dfa37f00 [ 66.302294] [<c028974c>] (handle_irq_event) from [<c028d4dc>] (handle_fasteoi_irq+0xc0/0x168) [ 66.302298] r7:00000001 r6:c120a434 r5:dfa37f68 r4:dfa37f00 [ 66.302304] [<c028d41c>] (handle_fasteoi_irq) from [<c02887e8>] (generic_handle_irq+0x2c/0x3c) [ 66.302308] r7:00000001 r6:00000000 r5:00000000 r4:c1068cfc [ 66.302315] [<c02887bc>] (generic_handle_irq) from [<c0288e4c>] (__handle_domain_irq+0x64/0xbc) [ 66.302321] [<c0288de8>] (__handle_domain_irq) from [<c0565458>] (gic_handle_irq+0x44/0x80) [ 66.302326] r9:dfa2c000 r8:fa213000 r7:dfa2dbd0 r6:fa212000 r5:fa21200c r4:c1205104 [ 66.302331] [<c0565414>] (gic_handle_irq) from [<c02019f8>] (__irq_svc+0x58/0x8c) [ 66.302335] Exception stack(0xdfa2dbd0 to 0xdfa2dc18) [ 66.302339] dbc0: 60000093 1eed0000 00000000 60000013 [ 66.302343] dbe0: 00000000 00000000 00000084 c1204c7c c12589f0 dfa2c000 c1257cd0 dfa2dc6c [ 66.302347] dc00: dfa2dc20 dfa2dc20 c028615c c0286164 60000013 ffffffff [ 66.302352] r9:dfa2c000 r8:c12589f0 r7:dfa2dc04 r6:ffffffff r5:60000013 r4:c0286164 [ 66.302358] [<c0285e1c>] (console_unlock) from [<c0287810>] (vprintk_emit+0x17c/0x210) [ 66.302363] r10:00000001 r9:60000013 r8:00000032 r7:00000000 r6:0000502b r5:ffffe000 [ 66.302366] r4:c1257cd0 [ 66.302371] [<c0287694>] (vprintk_emit) from [<c06c1760>] (dev_vprintk_emit+0xec/0x208) [ 66.302376] r10:dfa2dda4 r9:c0e22ab8 r8:bf0d11ec r7:d3203020 r6:c1204c48 r5:dfa2dccc [ 66.302378] r4:00000015 [ 66.302383] [<c06c1674>] (dev_vprintk_emit) from [<c06c18b4>] (dev_printk_emit+0x38/0x54) [ 66.302388] r10:d3203020 r9:d3203000 r8:bf0d09f4 r7:00000100 r6:d3203208 r5:d37b3600 [ 66.302391] r4:c1204c48 [ 66.302396] [<c06c1880>] (dev_printk_emit) from [<c06c1978>] (__dev_printk+0x50/0x90) [ 66.302399] r3:bf0d11ec r2:c0e22ab8 [ 66.302401] r4:c1204c48 [ 66.302406] [<c06c1928>] (__dev_printk) from [<c06c1bac>] (_dev_err+0x50/0x6c) [ 66.302419] [<c06c1b60>] (_dev_err) from [<bf0ccc94>] (rproc_resource_cleanup+0x298/0x5d8 [remoteproc]) [ 66.302423] r3:00000000 r2:01000000 r1:bf0d09f4 [ 66.302428] r4:d37b3680 [ 66.302448] [<bf0cc9fc>] (rproc_resource_cleanup [remoteproc]) from [<bf0cd784>] (rproc_shutdown+0xfc/0x3c4 [remoteproc]) [ 66.302453] r10:00000001 r9:d3203020 r8:d32031d0 r7:d320321c r6:d320321c r5:00000000 [ 66.302455] r4:d3203000 [ 66.302475] [<bf0cd688>] (rproc_shutdown [remoteproc]) from [<bf0cdb74>] (rproc_trigger_recovery+0x50/0x68 [remoteproc]) [ 66.302480] r10:00000000 r9:d3203240 r8:00000000 r7:dff3c000 r6:00000000 r5:d3203250 [ 66.302483] r4:d3203000 [ 66.302503] [<bf0cdb24>] (rproc_trigger_recovery [remoteproc]) from [<bf0cdc00>] (rproc_crash_handler_work+0x74/0x84 [remoteproc]) [ 66.302507] r7:dff3c000 r6:d3203000 r5:d32031d0 r4:d320323c [ 66.302520] [<bf0cdb8c>] (rproc_crash_handler_work [remoteproc]) from [<c02464d0>] (process_one_work+0x1f4/0x404) [ 66.302524] r7:dff3c000 r6:dff38d80 r5:dfa05380 r4:d320323c [ 66.302529] [<c02462dc>] (process_one_work) from [<c02472a4>] (worker_thread+0x58/0x568) [ 66.302534] r10:dff38d80 r9:c1203d00 r8:dff38d98 r7:00000008 r6:dff38d80 r5:dfa05394 [ 66.302536] r4:dfa05380 [ 66.302542] [<c024724c>] (worker_thread) from [<c024c7b4>] (kthread+0x160/0x168) [ 66.302547] r10:df885e74 r9:c024724c r8:dfa05380 r7:dfa2c000 r6:00000000 r5:dfa06200 [ 66.302550] r4:dfa07080 [ 66.302556] [<c024c654>] (kthread) from [<c02010d8>] (ret_from_fork+0x14/0x3c) [ 66.302559] Exception stack(0xdfa2dfb0 to 0xdfa2dff8) [ 66.302563] dfa0: 00000000 00000000 00000000 00000000 [ 66.302567] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 66.302571] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 66.302575] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c024c654 [ 66.302578] r4:dfa06200 [ 66.302581] ---[ end trace 4ae3fa7fbdac4053 ]--- [ 66.311643] omap_i2c 48070000.i2c: controller timed out [ 66.314297] ------------[ cut here ]------------
The address in the "L3 Standard Error" message is the address we're attempting to read from, but I'm confused as to why it's identified as a "write" operation when we're trying to read from the GPMC.
Here is a function we're using to start the DMA transfer:
static void dma_xfer(volatile uint32_t *dma_src, volatile uint32_t *dma_dst, uint32_t xfer_size, bool src_is_fifo)
{
uint32_t retSuccess;
EDMA3CCPaRAMEntry edmaParam;
// Fill out the Parameter RAM settings to use
edmaParam.opt = 0;
edmaParam.srcAddr = (uint32_t) dma_src;
edmaParam.destAddr = (uint32_t) dma_dst;
edmaParam.aCnt = xfer_size;
edmaParam.bCnt = edma_bCnt;
edmaParam.cCnt = edma_cCnt;
edmaParam.srcBIdx = xfer_size;
edmaParam.destBIdx = xfer_size;
edmaParam.srcCIdx = xfer_size;
edmaParam.destCIdx = xfer_size;
edmaParam.linkAddr = 0xFFFFU; // no linking
edmaParam.opt |= (EDMA3_OPT_TCINTEN_MASK | ((edma_tcc_num << EDMA3_OPT_TCC_SHIFT) & EDMA3_OPT_TCC_MASK));
if (src_is_fifo)
{
edmaParam.opt |= ((EDMA3_ADDRESSING_MODE_FIFO_WRAP << EDMA3_OPT_SAM_SHIFT) & EDMA3_OPT_SAM_MASK);
edmaParam.opt |= ((EDMA3_FIFO_WIDTH_32BIT << EDMA3_OPT_FWID_SHIFT) & EDMA3_OPT_FWID_MASK);
}
retSuccess = EDMA3RequestChannel(edma_base_address, EDMA3_CHANNEL_TYPE_DMA, edma_ch_num, edma_tcc_num, edma_evtq_num);
if (!retSuccess)
{
Log_print0(Diags_STATUS, "EDMA3RequestChannel called with invalid params");
}
EDMA3SetPaRAM(edma_base_address, edma_ch_num, &edmaParam);
retSuccess = EDMA3EnableTransfer(edma_base_address, edma_ch_num, EDMA3_TRIG_MODE_MANUAL);
if (!retSuccess)
{
Log_print0(Diags_STATUS, "EDMA3EanbleTransfer called with invalid params");
}
Semaphore_pend(sem_dma_xfer_complete, BIOS_WAIT_FOREVER);
}
And here's how the function is being called to read from the GPMC FIFO into L2:
dma_xfer(xrayFrameAddr, l2pixBuf, (pixels_per_line + XRAY_META_PIX) * bytes_per_pixel, TRUE);
Here's a successful call that copies from L2 to a shared buffer, where neither are FIFOs:
dma_xfer(l2pixBuf, bPix + (line_num * pixels_per_line), pixels_per_line * bytes_per_pixel, FALSE);
And here is how our devicetree is set up:
gpmc@50000000 {
compatible = "ti,am3352-gpmc";
ti,hwmods = "gpmc";
reg = <0x50000000 0x37c>;
interrupts = <0x0 0xf 0x4>;
dmas = <0xf6 0x4 0x0>;
dma-names = "rxtx";
gpmc,num-cs = <0x8>;
gpmc,num-waitpins = <0x2>;
#address-cells = <0x2>;
#size-cells = <0x1>;
interrupt-controller;
#interrupt-cells = <0x2>;
gpio-controller;
#gpio-cells = <0x2>;
status = "okay";
depth = <0x1>;
pinctrl-names = "default";
pinctrl-0 = <0xf7>;
ranges = <0x3 0x0 0x10000000 0x1000000>;
phandle = <0x1fe>;
fpga {
status = "okay";
pinctrl-names = "default";
reg = <0x3 0x0 0x1000000>;
bank-width = <0x2>;
gpmc,cs-on-ns = <0x4>;
gpmc,cs-rd-off-ns = <0x3c>;
gpmc,adv-on-ns = <0xf>;
gpmc,adv-rd-off-ns = <0x13>;
gpmc,oe-on-ns = <0x17>;
gpmc,oe-off-ns = <0x3c>;
gpmc,access-ns = <0x38>;
gpmc,rd-cycle-ns = <0x3c>;
};
};
As far as I can tell, we've changed the appropriate options to do the first DMA from the FIFO but it's not working correctly. Any help would be appreciated!
Thanks,
Jon
