This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3358: Porting GPMC edma solution from 3.2.X kernel to 6.12.X

Part Number: AM3358

Tool/software:

Dear colleagues,

On this moment I'm working for porting source code from 3.2 linux kernel to 6.12 (Yes it's quite big jump :) ).

The kernel I use from: git.ti.com/.../

In my device I use GPMC bus for communicate with FPGA. For speed up transfer process I use DMA.

Below presented code snippet from 3.2 kernel:

dma_ch = edma_alloc_channel(EDMA_CHANNEL_ANY, dma_complete_callback, NULL, EVENTQ_DEFAULT);

edma_set_src (dma_ch, (dma_addr_t)(FPGA_BASE_DMA), INCR, W16BIT);
edma_set_dest(dma_ch, dmaphysdest + (bufnum * frame_size) +
                      (portion * dma_transfer_size), INCR, W16BIT);
edma_set_src_index(dma_ch, dma_transfer_size, dma_transfer_size);
edma_set_dest_index(dma_ch, dma_transfer_size, dma_transfer_size);
// Use A-Sync Transfer Mode
edma_set_transfer_params(dma_ch, dma_transfer_size,
                            dma_bcount,
                            dma_ccount,
                            dma_bcount_reload, ASYNC);


 edma_read_slot(dma_ch, &param_set);
// disable the Intermediate Int's
param_set.opt &= ~(ITCINTEN);
if ((dma_bcount > 1) || (dma_ccount > 1)){
    // enable the Intermediate Int's
     //param_set.opt |= (ITCINTEN);
     // enable chaining
     param_set.opt |= (ITCCHEN);
}else{
    // disable the Intermediate Int's
    param_set.opt &= ~(ITCINTEN);
    // disble chaining
    param_set.opt &= ~(ITCCHEN);
}
   // enable the final int tc
param_set.opt |= (TCINTEN);
param_set.opt |= EDMA_TCC((dma_ch & 0x3f));
param_set.opt |= EDMA_TCC(EDMA_CHAN_SLOT(dma_ch));

edma_write_slot(dma_ch, &param_set);

edma_start(dma_ch);

The code on 3.2 works well.

On the 6.12 I have to use dmaengine.

My device tree related gpmc look like:

&gpmc {
	pinctrl-names = "default";
	pinctrl-0 = <&gpmc_pins>;
    compatible = "test,fpga";
    interrupts = <100>;
    gpmc,num-cs = <2>;
	gpmc,num-waitpins = <2>;

	status = "okay";
    dmas = <&edma 52 0>;
    dma-names = "rx";

    interrupt-controller;
	#interrupt-cells = <2>;

	#address-cells = <2>;
	#size-cells = <1>;
	ranges = <0 0 0x08000000 0x01000000   /* CS0 @addr 0x08000000, size 0x1000000 */
	          1 0 0x09000000 0x01000000>; /* CS1 @addr 0x09000000, size 0x1000000 */

	    fpga_control@0,0 {
		
			status = "okay";
			#address-cells = <1>;
			#size-cells = <1>;
			reg = <0 0 0x01000000>;
			rb-gpios = <&gpmc 0 GPIO_ACTIVE_HIGH>; /* gpmc_wait0 */
	    };
 
		fpga_stream@1,0 {
		
			status = "okay";
			#address-cells = <1>;
			#size-cells = <1>;
			reg = <1 0 0x01000000>;
			rb-gpios = <&gpmc 0 GPIO_ACTIVE_HIGH>; /* gpmc_wait0 */

	};
};

I'm configure GPMC registers from source code like it was before on 3.2 and it looks like:

CS[0] GPMC_CONFIG1 0xA9401001
CS[0] GPMC_CONFIG2 0x00060800
CS[0] GPMC_CONFIG3 0x00020200
CS[0] GPMC_CONFIG4 0x04020600
CS[0] GPMC_CONFIG5 0x0204080A
CS[0] GPMC_CONFIG6 0x040702C0
CS[0] GPMC_CONFIG7 0x00000F48

CS[1] GPMC_CONFIG1 0xE9401001
CS[1] GPMC_CONFIG2 0x00060800
CS[1] GPMC_CONFIG3 0x00020200
CS[1] GPMC_CONFIG4 0x04020600
CS[1] GPMC_CONFIG5 0x0204080A
CS[1] GPMC_CONFIG6 0x040702C0
CS[1] GPMC_CONFIG7 0x00000F49

The source code responsible for launch DMA:

void __iomem* dma_virt_addr = dma_alloc_coherent(dev->parent, 1024*1024, &dma_addr, GFP_KERNEL);
  

memset(&dma_cfg, 0, sizeof(dma_cfg));
dma_cfg.direction = DMA_DEV_TO_MEM;

dma_cfg.src_addr = fpga_stream.gpmc_base.start; //0x09000000
dma_cfg.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
dma_cfg.src_maxburst = 16;

ret = dmaengine_slave_config(fpga_stream.dma_ch, &dma_cfg);
if (ret) {
	DRV_MSG("DMA engine slave config failed: %d\n", ret);
	return -1;
}  
 
 
dma_desc = dmaengine_prep_slave_single(fpga_stream.dma_ch, dma_addr, 1024, DMA_DEV_TO_MEM, DMA_PREP_INTERRUPT | DMA_CTRL_ACK);

if (!dma_desc)
{
	DRV_MSG("DMA busy\n");
	return -1;
}

dma_desc->callback = dma_complete_callback;
dma_desc->callback_param = NULL;

dma_cookie = dmaengine_submit(dma_desc);
if (dma_submit_error(dma_cookie)) {
	DRV_MSG("DMA transaction submission FAILED\n");
	return -1;
}

DRV_MSG("DMA transaction submission SUCCESS\n");
    

dma_async_issue_pending(fpga_stream.dma_ch);
DRV_MSG("DMA start\n");

On the dmesg I see that DMA channel has configured:

[   67.974832] edma 49000000.dma: Got eDMA channel 52 for virt channel 50 (HW trigger)
[   68.056257] edma 49000000.dma: vchan a74ee4d7: txd fd8f725f[2]: submitted
[   68.062970] edma 49000000.dma: 
[   68.062970]  pset[0]:
[   68.062970]   chnum  52
[   68.062970]   slot   49
[   68.062970]   opt    00134004
[   68.062970]   src    09000000
[   68.062970]   dst    95100000
[   68.062970]   abcnt  00100004
[   68.062970]   ccnt   00000010
[   68.062970]   bidx   00040000
[   68.062970]   cidx   00400000
[   68.062970]   lkrld  ffffffff
[   68.062995] edma 49000000.dma: first transfer starting on channel 52
[   68.063008] edma 49000000.dma: ER1 00000000
[   68.063021] edma 49000000.dma: EER1 00100000

My problem that I don't see DMA complete callback. On the 3.2 kernel callback executed properly without any issues.

I will be appreciate for any advice about the reason of this behavior.

Thanks in advance!

  • Hi Viktor,

    Ii won't be possible to point out what exact is mis-configured by just reviewing the code segments you provided. You might want to compare the EDMA registers to see what is configured differently.

  • Hello,

    The problem that the configuration related EDMA the same.

    Maybe in 6.12 I have to explicitly configure some interrupts which was implicitly  configured in 3.2 ?

  • After more deep investigation I found that in case of 3.2 kernel PARAM_SET OPT register looks like:

    EDMA_OPT 0x80102000

    The bit 31 (PRIV) has value 1. In case of 6.12 kernel this bit is 0.

    Can it be  the reason?

    Thank you!

  • In the end I have launched the DMA transaction from GPMC to memory via EDMA.

    The problem was that I has configured my DMA in DMA_DEV_TO_MEM mode. How it's recommended in most documents pages. The DMA_DEV_TO_MEM mode REQIRED the HW trigger. Honestly about it mentioned in debug message:

    edma 49000000.dma: Got eDMA channel 52 for virt channel 50 (HW trigger)

    But in my case I don't have a HW trigger. The source code which I have in 3.2 kernel describes a DMA_MEM_TO_MEM concept.

    Below presented code snippet which I worked on my side:

    dma_cap_zero(mask);
    dma_cap_set(DMA_MEMCPY,  mask);
    dma_ch = dma_request_channel(mask, NULL, NULL);
        
    dma_virt_addr = dma_alloc_coherent(dev->parent, 1024*1024, &dma_addr, GFP_KERNEL);
    dma_desc = dmaengine_prep_dma_memcpy(dma_ch, dma_addr, gpmc_base.start,	2*0x2480, DMA_PREP_INTERRUPT);
     
    if (!dma_desc)
    {
    	DRV_MSG("DMA busy\n");
        return -1;
    }
    
    dma_desc->callback = dma_complete_callback
    dma_desc->callback_param = dma_virt_addr;
    
    dma_cookie = dmaengine_submit(dma_desc);
    if (dma_submit_error(dma_cookie)) {
    	DRV_MSG("DMA transaction submission FAILED\n");
    	return -1;
    }
    
    DRV_MSG("DMA transaction submission SUCCESS\n");
        
    dma_async_issue_pending(fpga_stream.dma_ch);
    DRV_MSG("DMA start\n");	

    After launch my code I see following message in dmesg:

    edma 49000000.dma: Got eDMA channel 20 for virt channel 0 (SW trigger)

    In this case I've a question. Is it correct way to use DMA_MEM_TO_MEM instead of  DMA_DEV_TO_MEM  related GPMC bus?

  • Hi Viktor,

    Sorry I was debugging a different issue today and didn't get time to review your message. I will try to review it tomorrow and get back to you.

  • Hi Viktor,

    I am configured in the way you use dma on kernel 6.12.

    In your original post:

    dma_cfg.src_addr = fpga_stream.gpmc_base.start; //0x09000000
    dma_cfg.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
    dma_cfg.src_maxburst = 16;

    ret = dmaengine_slave_config(fpga_stream.dma_ch, &dma_cfg);
    if (ret) {
        DRV_MSG("DMA engine slave config failed: %d\n", ret);
        return -1;
    }  
     
    dma_desc = dmaengine_prep_slave_single(fpga_stream.dma_ch, dma_addr, 1024, DMA_DEV_TO_MEM, DMA_PREP_INTERRUPT | DMA_CTRL_ACK);

    which uses DEV_TO_MEM to transfer 1024 bytes with bus width of 4 bytes. But in your post on 6/19:

    dma_cap_set(DMA_MEMCPY,  mask);
    dma_ch = dma_request_channel(mask, NULL, NULL);
        
    dma_virt_addr = dma_alloc_coherent(dev->parent, 1024*1024, &dma_addr, GFP_KERNEL);
    dma_desc = dmaengine_prep_dma_memcpy(dma_ch, dma_addr, gpmc_base.start,    2*0x2480, DMA_PREP_INTERRUPT);

    which uses MEM_TO_MEM to transfer 2*0x2480 bytes.

    Which version is the final one? How exactly are your FPGA data presented to the GPMC interface? Is it in FIFO mode with 4 bytes FIFO width or the entire data frame is memory mapped to GPMC address space? You would need DEV_TO_MEM for the former case but MEM_TO_MEM for latter case.

  • Hello Bin,

    Both sizes 1024 and 2*0x2480 are valid. It's just a different frames which FPGA can send to CPU via GPMC.

    In case of DMA_MEM_TO_MEM both sizes are working properly. In case of DMA_DEV_TO_MEM also both doesn't work.

    We use the data mapping to GPMC memory space (at leas ACNT=FPGA_FRAME_SIZE)

    Look like that DMA_MEM_TO_MEM correct. But I'm bit worry about future. 

    Physically we are talking about DEV - > MEMORY data transfer via GPMC bus.

    In our project we have to carry about following important points:

    - security aspect

    - possibilities for migration on the next revision of Linux kernel

    I'm afraid that in future the approach MEM -> MEM can be deprecated and prohibited in case if we are talking about data transfer from device.

  • Hi Viktor,

    The DMA_MEM_TO_MEM doesn't not mean that it is only for memory and cannot be used on device. MEM_TO_MEM is for DMA transfers that have the same size in source and target address. For example, to read from GPMC NAND, DMA_DEV_TO_MEM is used, while to read from GPMC NOR, DMA_MEM_TO_MEM is used.

    I'm afraid that in future the approach MEM -> MEM can be deprecated

    Where did you get the information that MEM_TO_MEM will be deprecated?

  • Hello Bin,

    Where did you get the information that MEM_TO_MEM will be deprecated?

    It's just a my private doubt.

    About NAND interface I have a good example for usage DMA interface:

    https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/mtd/nand/raw/omap2.c?h=ti-linux-6.12.y

    Could you please provide the link on the GPMC NOR implementation with DMA.

    Thank you!

  • Hi Viktor,

    I am not aware of any GPMC NOR driver, but following is the kernel OSPI driver which has serial NOR device attached to.

    drivers/spi/spi-cadence-quadspi.c

    its function cqspi_direct_read_dma() uses dmaengine_prep_dma_memcpy().