• State Resolved
  • Replies 12 replies
  • Subscribers 101 subscribers
  • Views 197 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

AM5748: How to change PCIe TLP Dword length from default value?

Phaneesh A
Prodigy 180 points
Part Number: AM5748
Other Parts Discussed in Thread: AM5728

Tool/software:

Hi ,

                       We are transferring data between in AM5748 (RC) and xilinx FPGA (EP) via PCIe port , so after the enumeration the address space is getting detected , Region 0: Memory at 20400000 (32-bit, non-prefetchable) [disabled] [size=4M] ,  so if I write data of 64 byte of uint32_t type buffer to this address space  on the End point the on ILA we are seeing data as shown below.

Here the upper 32 bit is address and lower 32 bit is the data that we are sending and if we see the packet header we can see F4000001 the last digit is 1 so the DWORD length of the packet is 1 so , my question is how to increase the length of the DWORD ,  I was not able to find any register for this configuration on the ARM side so please provide us with some pointers regarding how to set the DWORD length on the TLP header to 2 from 1(default).

Thanks,

Phaneesh A Kashyap

  • 0
    TI__Mastermind 33942 points

    Hi Phaneesh,

    What is the operating system used on the AM5728? And are these PCIe configuration writes that are being tested?

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 180 points

    Hi  Takuma

                      The operating system used in our board is linux Kernel 4.14 , and my issue is that if I send 64 bytes of data it's sending it in 16 packets of 64bit data is being sent where in each 64 bit packet contains 32 bit of address which is our system mem space address and 32 bit of data as you can see in the above picture of ILA which is how default protocol works , but my requirement was to fill the 64 bit packet completely with data , so when I checked in browser I got to know the the each and ever TLP will have a header so I checked that 64bit  header's value which is  00000000F4000001 , so here the last 9 bits represents the payload size or the DWORD length which is 1 so the data in packet is stored  in upper 32 bit only , so I need to set that DWORD length value to 2, since the TLP will be generated on the initiator side which AM5748 arm (RC ),  I want to know what changes required ,  I checked in the PCIe register manual section on TRM I was not able to find so I need some pointers here.

    Thanks

    Phaneesh

  • 0 in reply to Phaneesh A
    TI__Mastermind 33942 points

    Hi Phaneesh,

    TLP is created at the PCIe controller level instead of the CPU, and it is done by the hardware automatically. There are no registers or knobs that can be tweaked to change the TLP length other than MPS and MRRS which the smallest maximum payload size and max read request size are usually in the range of at least 128Bytes, so nothing to limit to 1 DWORD.

    I suspect issues with the user application being used. Is this using the pci_endpoint_test.c Linux driver, or some custom driver to send the data?

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 180 points

    Hi  Takuma ,

                      I'm different example , not "pci_endpoint_test" I'll check with this and will  update you, but how to write 64 bit data is the question.

    Regards,

    Phaneesh

  • 0 in reply to Phaneesh A
    TI__Mastermind 33942 points

    Hi Phaneesh,

    If only 1 DWORD is being sent, some things I can think of is then either the application itself is only sending 1 DWORD, some alignment issues in memory is fragmenting the TLP, or the API or function used to send the data is fragmenting the TLP, or the FPGA has some configuration or setting to only receive 1 DWORD.

    Regards,

    Takuma

  • 0
    Prodigy 180 points

    Hi  Takuma,

                     FPGA is ready to accept max payload up to 128 bytes and there's no config to receive only  1 DW , but this has to be initiated from the Root complex side, So I was browsing through this issue, I found out that we can use edma burst to send 64 bit data, but I am not getting proper example to check the edma transfer it's failing at dma_chan_create, so how to proceed with this issue need some pointers.

    Regards

    Phaneesh

  • 0 in reply to Phaneesh A
    TI__Mastermind 33942 points

    Hi Phaneesh,

    Linux has the pci-epf-test which should have some functionality built in to send data via DMA: https://software-dl.ti.com/processor-sdk-linux/esd/docs/04_03_00_05/linux/Foundational_Components.html#pcie-end-point

    Otherwise, we have an old PCIe RTOS driver which has been deprecated that uses the EDMA. Source code is still available by going back to an older RTOS SDK: https://www.ti.com/tool/download/PROCESSOR-SDK-RTOS-AM57X. Below screenshot is from 5.3 SDK version.

    But yes, DMA might help with constructing a longer TLP packet, so I think you are on the right track.

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 180 points

    Hi Takuma,                  

                         I've enabled the EDMA for AM5748 in linux kernel 4.14, and I've verified the in kernel prints which is as shown below.

    root@AirlinkBCP:~# dmesg | grep "edma"
    [    3.543716] edma 43300000.edma: memcpy is disabled
    [    3.546975] edma 43300000.edma: TI EDMA DMA engine driver

    But we have tried with a user application example to transfer data from one memory location to another and application is getting crashed each time please check the prints below.

    edma_pcie.c 
    #include <linux/module.h>
#include <linux/init.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/completion.h>



#define SRC_BUF_SIZE 64  // Size of the source buffer
#define DMA_TRANSFER_SIZE 64  // Size of each transfer in bytes
#define DEST_ADDR	0x20400000U

static struct dma_chan *dma_chan;
static dma_addr_t src_dma_addr, dst_dma_addr;
static char *src_buf, *dst_buf;
static struct completion dma_complete;

static void dma_callback(void *completion)
{
    complete(completion);  // Signal the completion of the DMA transfer
}

static int edma_transfer_test(void)
{
    struct dma_async_tx_descriptor *tx;
    struct dma_slave_config config;
    enum dma_ctrl_flags flags = DMA_CTRL_ACK | DMA_PREP_INTERRUPT;
    struct dma_device *dma_dev;
    dma_cookie_t cookie;
    int ret;

    // Step 1: Allocate DMA-capable memory for source and destination buffers
    src_buf = kzalloc(SRC_BUF_SIZE, GFP_KERNEL);
    if (!src_buf)
        return -ENOMEM;

/*    dst_buf = kzalloc(SRC_BUF_SIZE, GFP_KERNEL);
    if (!dst_buf) {
        kfree(src_buf);
        return -ENOMEM;
    } */

//    memset(src_dma_addr, 0xAA, SRC_BUF_SIZE);
    memset(src_buf, 0xAA, SRC_BUF_SIZE);

    // Step 2: Map the buffers to DMA addresses
    src_dma_addr = dma_map_single(dma_chan->device->dev, src_buf, DMA_TRANSFER_SIZE, DMA_TO_DEVICE);
//    dst_dma_addr = dma_map_single(dma_chan->device->dev, dst_buf, DMA_TRANSFER_SIZE, DMA_FROM_DEVICE);
/*
    if (dma_mapping_error(dma_chan->device->dev, src_dma_addr) ||
        dma_mapping_error(dma_chan->device->dev, dst_dma_addr)) {
        kfree(src_buf);
        kfree(dst_buf);
        return -ENOMEM;
    }
*/

    // Step 3: Configure the DMA transfer (source and destination addresses, burst sizes)
    memset(&config, 0, sizeof(config));
    config.src_addr = src_buf;
    config.dst_addr = DEST_ADDR;
    config.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
    config.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
    config.src_maxburst = 16;
    config.dst_maxburst = 16;

    ret = dmaengine_slave_config(dma_chan, &config);
    if (ret) {
        pr_err("Failed to configure DMA channel\n");
        goto unmap_dma;
    }

    // Step 4: Prepare the DMA transaction
    tx = dmaengine_prep_dma_memcpy(dma_chan, dst_dma_addr, src_dma_addr, DMA_TRANSFER_SIZE, flags);
    if (!tx) {
        pr_err("Failed to prepare DMA transaction\n");
        ret = -EINVAL;
        goto unmap_dma;
    }

    // Set up a completion callback for the DMA transfer
    tx->callback = dma_callback;
    tx->callback_param = &dma_complete;

    // Step 5: Submit the DMA transaction and start the transfer
    init_completion(&dma_complete);
    cookie = dmaengine_submit(tx);
    if (dma_submit_error(cookie)) {
        pr_err("Failed to submit DMA transaction\n");
        ret = -EINVAL;
        goto unmap_dma;
    }

    // Start the DMA transfer
    dma_async_issue_pending(dma_chan);

    // Wait for the DMA transfer to complete
    wait_for_completion(&dma_complete);

    pr_info("DMA transfer completed successfully\n");

unmap_dma:
    dma_unmap_single(dma_chan->device->dev, src_dma_addr, DMA_TRANSFER_SIZE, DMA_TO_DEVICE);
//    dma_unmap_single(dma_chan->device->dev, dst_dma_addr, DMA_TRANSFER_SIZE, DMA_FROM_DEVICE);

    kfree(src_buf);
    kfree(dst_buf);

    return ret;
}

static int __init edma_test_init(void)
{
    pr_info("edma_test_init\n");
    // Step 1: Request a DMA channel
 
    dma_chan = dma_request_chan_by_mask((dma_cap_mask_t *)DMA_SLAVE);
    pr_info("DMA request chan\n");
    if (IS_ERR(dma_chan)) {
        pr_err("Failed to request DMA channel\n");
        return PTR_ERR(dma_chan);
    }

    pr_info("EDMA transfer test module loaded\n");
    return edma_transfer_test();
}

static void __exit edma_test_exit(void)
{
    if (dma_chan)
        dma_release_channel(dma_chan);

    pr_info("EDMA transfer test module unloaded\n");
}

module_init(edma_test_init);
module_exit(edma_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("EDMA Transfer Test on AM5748");

    EDMA_crash_logs.txt 
       61.652808] edma_pcie: loading out-of-tree module taints kernel.
[   61.660522] Unable to handle kernel NULL pointer dereference at virtual address 0000000a
[   61.670345] pgd = ed2c9c00
[   61.673067] [0000000a] *pgd=ac919003, *pmd=fd1b7003
[   61.678024] Internal error: Oops: 207 [#1] PREEMPT SMP ARM
[   61.683534] Modules linked in: edma_pcie(O+) xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4l
[   61.726104] CPU: 0 PID: 965 Comm: insmod Tainted: G        W  O    4.14.40-g4796173fc5 #39
[   61.734402] Hardware name: Generic DRA74X (Flattened Device Tree)
[   61.740521] task: ec9286c0 task.stack: ec922000
[   61.745077] PC is at find_candidate+0x20/0x17c
[   61.749540] LR is at __dma_request_channel+0x50/0x90
[   61.754525] pc : [<c060db9c>]    lr : [<c060ddb4>]    psr: 200f0013
[   61.760816] sp : ec923d08  ip : ec923d30  fp : ec923d2c
[   61.766061] r10: ed2ca064  r9 : 00000000  r8 : 00000000
[   61.771307] r7 : 00000000  r6 : 00000000  r5 : eeac624c  r4 : c0e28880
[   61.777860] r3 : 00000000  r2 : 00000000  r1 : 0000000a  r0 : eeac624c
[   61.784415] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   61.791580] Control: 30c5387d  Table: ad2c9c00  DAC: fffffffd
[   61.797349] Process insmod (pid: 965, stack limit = 0xec922210)
[   61.803293] Stack: (0xec923d08 to 0xec924000)
[   61.807668] 3d00:                   c0e28880 00000000 00000000 0000000a c0e28870 00000000
[   61.815881] 3d20: ec923d54 ec923d30 c060ddb4 c060db88 ffffe000 00000000 bf244000 ed348140
[   61.824094] 3d40: 00000001 00000000 ec923d64 ec923d58 c060e69c c060dd70 ec923dc4 ec923d68
[   61.832307] 3d60: bf244018 c060e688 ed2c9c40 200f0013 ed2c9c40 00000001 8040003e 600f0013
[   61.840518] 3d80: 00000000 c0e7e80c ec923dec 8040003e f62d6fff f62d7000 00000002 00000000
[   61.848731] 3da0: 8040003f fffff000 ffffe000 00000000 bf244000 ed348140 ec923e34 ec923dc8
[   61.856943] 3dc0: c02019fc bf24400c ffffe000 ed2c9c40 ef414444 ee801e40 ffffe000 00019a9b
[   61.865156] 3de0: 00000000 ed2ca064 ec923e24 ec923df8 c032c2c4 c032bec4 00000001 f62d1000
[   61.873369] 3e00: ec923e34 ec923e10 c032d400 c0336fbc bf241000 bf241000 00000001 ed2ca040
[   61.881581] 3e20: ed348140 00000001 ec923e5c ec923e38 c02b1e98 c02019bc ec923e5c ec923e48
[   61.889792] 3e40: ec923f40 00000001 ed2ca040 bf241000 ec923f1c ec923e60 c02b0d3c c02b1e3c
[   61.898005] 3e60: bf24100c 00007fff bf241000 c02addd8 bf000000 c0b47dd4 c0b47dac bf241114
[   61.906218] 3e80: c0b47efc c0b92b18 bf241130 c0a045d0 bf241100 00000000 bf241048 bf241000
[   61.914429] 3ea0: c03446a0 c04fc7c4 00000003 00000005 ec923edc 00000000 00000000 00000000
[   61.922640] 3ec0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   61.930852] 3ee0: 00000000 00000000 00000000 00000000 7fffffff 00000000 00000003 000281f0
[   61.939066] 3f00: 0000017b c0207e64 ec922000 00000000 ec923fa4 ec923f20 c02b14ec c02aee3c
[   61.947278] 3f20: 7fffffff 00000000 00000003 ec923f3c ec923f40 f62d1000 00004c88 00000000
[   61.955490] 3f40: f62d14d6 f62d1000 00004c88 f62d5670 f62d54f4 f62d4cdc 00003000 00003180
[   61.963703] 3f60: 00000000 00000000 00000000 00001710 00000024 00000025 0000000c 0000000a
[   61.971915] 3f80: 00000008 00000000 00000000 0003b008 00000000 00025eb0 00000000 ec923fa8
[   61.980127] 3fa0: c0207c80 c02b1454 0003b008 00000000 00000003 000281f0 00000000 00000000
[   61.988339] 3fc0: 0003b008 00000000 00025eb0 0000017b 00000000 00000002 b6f8b000 00000000
[   61.996552] 3fe0: be93cc00 be93cbf0 0001f888 b6ee2a82 800b0030 00000003 ffffffff ffffffff
[   62.004761] Backtrace: 
[   62.007228] [<c060db7c>] (find_candidate) from [<c060ddb4>] (__dma_request_channel+0x50/0x90)
[   62.015791]  r9:00000000 r8:c0e28870 r7:0000000a r6:00000000 r5:00000000 r4:c0e28880
[   62.023573] [<c060dd64>] (__dma_request_channel) from [<c060e69c>] (dma_request_chan_by_mask+0x20/0x30)
[   62.033009]  r9:00000000 r8:00000001 r7:ed348140 r6:bf244000 r5:00000000 r4:ffffe000
[   62.040794] [<c060e67c>] (dma_request_chan_by_mask) from [<bf244018>] (edma_test_init+0x18/0x1000 [edma_pcie])
[   62.050845] [<bf244000>] (edma_test_init [edma_pcie]) from [<c02019fc>] (do_one_initcall+0x4c/0x170)
[   62.060018]  r7:ed348140 r6:bf244000 r5:00000000 r4:ffffe000
[   62.065708] [<c02019b0>] (do_one_initcall) from [<c02b1e98>] (do_init_module+0x68/0x1f8)
[   62.073833]  r8:00000001 r7:ed348140 r6:ed2ca040 r5:00000001 r4:bf241000
[   62.080567] [<c02b1e30>] (do_init_module) from [<c02b0d3c>] (load_module+0x1f0c/0x24ac)
[   62.088605]  r7:bf241000 r6:ed2ca040 r5:00000001 r4:ec923f40
[   62.094291] [<c02aee30>] (load_module) from [<c02b14ec>] (SyS_finit_module+0xa4/0xbc)
[   62.102156]  r10:00000000 r9:ec922000 r8:c0207e64 r7:0000017b r6:000281f0 r5:00000003
[   62.110019]  r4:00000000
[   62.112567] [<c02b1448>] (SyS_finit_module) from [<c0207c80>] (ret_fast_syscall+0x0/0x4c)
[   62.120778]  r6:00025eb0 r5:00000000 r4:0003b008
[   62.125417] Code: e1a05000 e1a07002 e1a08003 0a000005 (e5913000) 

Broadcast message from systemd-journald@AirlinkBCP (Thu 2021[   62.132772] ---[ end trace ce8b24a45630791d ]---
-10-07 06:04:02 UTC):

kernel[561]: [   61.678024] Internal error: Oops: 207 [#1] PREEMPT SMP ARM


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.797349] Process insmod (pid: 965, stack limit = 0xec922210)


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.803293] Stack: (0xec923d08 to 0xec924000)


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.807668] 3d00:                   c0e28880 00000000 00000000 0000000a c0e28870 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.815881] 3d20: ec923d54 ec923d30 c060ddb4 c060db88 ffffe000 00000000 bf244000 ed348140


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.824094] 3d40: 00000001 00000000 ec923d64 ec923d58 c060e69c c060dd70 ec923dc4 ec923d68

Segmentation fault

Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.832307] 3d60: bf244018 c060e688 ed2c9c40 200f0013 ed2c9c40 00000001 8040003e 600f0013


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:02 UTC):

kernel[561]: [   61.840518] 3d80: 00000000 c0e7e80c ec923dec 8040003e f62d6fff f62d7000 00000002 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.848731] 3da0: 8040003f fffff000 ffffe000 00000000 bf244000 ed348140 ec923e34 ec923dc8


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.856943] 3dc0: c02019fc bf24400c ffffe000 ed2c9c40 ef414444 ee801e40 ffffe000 00019a9b


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.865156] 3de0: 00000000 ed2ca064 ec923e24 ec923df8 c032c2c4 c032bec4 00000001 f62d1000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.873369] 3e00: ec923e34 ec923e10 c032d400 c0336fbc bf241000 bf241000 00000001 ed2ca040


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.881581] 3e20: ed348140 00000001 ec923e5c ec923e38 c02b1e98 c02019bc ec923e5c ec923e48


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.889792] 3e40: ec923f40 00000001 ed2ca040 bf241000 ec923f1c ec923e60 c02b0d3c c02b1e3c


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.898005] 3e60: bf24100c 00007fff bf241000 c02addd8 bf000000 c0b47dd4 c0b47dac bf241114


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.906218] 3e80: c0b47efc c0b92b18 bf241130 c0a045d0 bf241100 00000000 bf241048 bf241000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.914429] 3ea0: c03446a0 c04fc7c4 00000003 00000005 ec923edc 00000000 00000000 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.922640] 3ec0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.930852] 3ee0: 00000000 00000000 00000000 00000000 7fffffff 00000000 00000003 000281f0


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.939066] 3f00: 0000017b c0207e64 ec922000 00000000 ec923fa4 ec923f20 c02b14ec c02aee3c


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.947278] 3f20: 7fffffff 00000000 00000003 ec923f3c ec923f40 f62d1000 00004c88 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.955490] 3f40: f62d14d6 f62d1000 00004c88 f62d5670 f62d54f4 f62d4cdc 00003000 00003180


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.963703] 3f60: 00000000 00000000 00000000 00001710 00000024 00000025 0000000c 0000000a


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.971915] 3f80: 00000008 00000000 00000000 0003b008 00000000 00025eb0 00000000 ec923fa8


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.980127] 3fa0: c0207c80 c02b1454 0003b008 00000000 00000003 000281f0 00000000 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.988339] 3fc0: 0003b008 00000000 00025eb0 0000017b 00000000 00000002 b6f8b000 00000000


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   61.996552] 3fe0: be93cc00 be93cbf0 0001f888 b6ee2a82 800b0030 00000003 ffffffff ffffffff


Broadcast message from systemd-journald@AirlinkBCP (Thu 2021-10-07 06:04:03 UTC):

kernel[561]: [   62.125417] Code: e1a05000 e1a07002 e1a08003 0a000005 (e5913000)

    above files attached has EDMA user application driver and it's crash log please let us know if any mistake are there in this code.

    Also I disabled the driver in menu config and I only programmed the registers according to this section from TRM "17.2.5.5 Setting Up an EDMA Transfer" for PARAMs set number 0 ,still it got crashed at  init step.

    Code snippet

    -------------------------------------------------------------*/

    void edma_init(dma_addr_t src, dma_addr_t dest){

             edma_cc = ioremap(EDMA_CC_BASE, EDMA_CC_SIZE);
             if (!edma_cc) {
                    pr_err("Failed to map EDMA CC base address\n");
                    return;
             }

            //DMA chan maping
    //      *(EDMA_CC_BASE+0x100U) = 0x0;
            writel(0x00, edma_cc + 0x100); // opt

            void __iomem *param_base = edma_cc + EDMA_PARAM_BASE ;
            // PARAMS
            writel(0x00100004, param_base + 0x00); // opt
            writel(src, param_base + 0x04); //SRC
            writel(0x04000A00, param_base + 0x08); // ACNT & BCNT
            writel(dest, param_base + 0x0C); // DST
            writel(0x04000400, param_base + 0x10); // SRCBIDX & DSTBIDX     
            writel(0xFFFF000A, param_base + 0x14); // LINK & BCNTRLD        
            writel(0x00000000, param_base + 0x18); // SRCCIDX & DSTCIDX      
            writel(0x00000001, param_base + 0x1C); // CCNT

    }

    ------------------------------------------------------------*/

    Thanks and regards,

    Phaneesh

        

  • 0 in reply to Phaneesh A
    TI__Mastermind 33942 points

    Hi Phaneesh,

    Two questions:

    Regards,

    Takuma

  • 0 in reply to Takuma Fujiwara
    Prodigy 180 points

    Hi ,

          

    Takuma Fujiwara said:
    Does the crash only happen when writing to PCIe memory space? Or any memory?

             The crash is occuring for random  memory locations ,when I call "dma_request_chan" .

    1. Here is my device tree node for the EDMA .

    /*--------------------------------------------------------------------------------*/

                    edma: edma@43300000 {
                            compatible = "ti,edma3-tpcc";
                            ti,hwmods = "tpcc";
                            reg = <0x43300000 0x100000>;
                            reg-names = "edma3_cc";
                            ti,edma-channels = <64>;
                            ti,edma-slots = <512>;
                            ti,edma-tc = <2>;
                            ti,edma-queue = <2>;
                            ti,edma-xbar-event-map = <0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15>;
                            ti,edma-memcpy-channels = <20 21>;
                            interrupts = <GIC_SPI 361 IRQ_TYPE_LEVEL_HIGH>,
                                         <GIC_SPI 360 IRQ_TYPE_LEVEL_HIGH>,
                                         <GIC_SPI 359 IRQ_TYPE_LEVEL_HIGH>;
                            interrupt-names = "edma3_ccint", "edma3_mperr",
                                              "edma3_ccerrint";
                            dma-requests = <64>;
                            #dma-cells = <2>;

                            ti,tptcs = <&edma_tptc0 7>, <&edma_tptc1 0>;
                            status = "okay";
                            /*
                             * memcpy is disabled, can be enabled with:
                             * ti,edma-memcpy-channels = <20 21>;
                             * for example. Note that these channels need to be
                             * masked in the xbar as well.
                             */
                    };

                   edma_tptc0: tptc@43400000 {
                            compatible = "ti,edma3-tptc";
                            ti,hwmods = "tptc0";
                            reg =   <0x43400000 0x100000>;
                            interrupts = <GIC_SPI 370 IRQ_TYPE_LEVEL_HIGH>;
                            interrupt-names = "edma3_tcerrint";
                            status = "okay";
                    };

                    edma_tptc1: tptc@43500000 {
                            compatible = "ti,edma3-tptc";
                            ti,hwmods = "tptc1";
                            reg =   <0x43500000 0x100000>;
                            interrupts = <GIC_SPI 371 IRQ_TYPE_LEVEL_HIGH>;
                            interrupt-names = "edma3_tcerrint";

    /*--------------------------------------------------------------------------------*/

    2. Kernel boot prints.

    /*--------------------------------------------------------------------------------*/
    [    3.550260] edma 43300000.edma: eDMA3 CC HW configuration (cccfg: 0x03315445):
    [    3.550267] edma 43300000.edma: num_region: 8
    [    3.550274] edma 43300000.edma: num_channels: 64
    [    3.550281] edma 43300000.edma: num_qchannels: 8
    [    3.550287] edma 43300000.edma: num_slots: 512
    [    3.550294] edma 43300000.edma: num_tc: 2
    [    3.550300] edma 43300000.edma: chmap_exist: yes
    [    3.553855] edma 43300000.edma: TI EDMA DMA engine driver

    /*--------------------------------------------------------------------------------*/

    3. List of edma channels which got detected in /sys/class/dma/

    /*--------------------------------------------------------------------------------*/

    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan10 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan10
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan11 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan11
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan12 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan12
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan13 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan13
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan14 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan14
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan15 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan15
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan16 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan16
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan17 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan17
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan18 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan18
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan19 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan19
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan2 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan2
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan20 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan20
    lrwxrwxrwx    1 root     root           0 Oct  7 06:03 dma1chan21 -> ../../devices/platform/44000000.ocp/43300000.edma/dma/dma1chan21

    upto 64 chan.

    /*--------------------------------------------------------------------------------*/

    and example program .

    EDMA_tranfer.c 
    #include <linux/module.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include <linux/edma.h>
#include <linux/slab.h>
#include <linux/completion.h>
#include <linux/interrupt.h>

#define TRANSFER_SIZE 64 // 4KB transfer size

static dma_addr_t src_dma, dst_dma;
static char *src_buffer, *dst_buffer;
static struct dma_chan *dma_chan;
static struct completion dma_complete;

// Callback function for the EDMA transfer
static void edma_callback(void *completion)
{
    complete(completion);
}

// Function to set up and initiate EDMA transfer
static int edma_memcpy_transfer(void)
{
    struct dma_async_tx_descriptor *tx;
    dma_cap_mask_t mask;
    int ret;

    // Allocate memory for source and destination buffers
    src_buffer = kmalloc(TRANSFER_SIZE, GFP_KERNEL);
    if (!src_buffer) {
        pr_err("Failed to allocate source buffer\n");
        return -ENOMEM;
    }

    dst_buffer = kmalloc(TRANSFER_SIZE, GFP_KERNEL);
    if (!dst_buffer) {
        pr_err("Failed to allocate destination buffer\n");
        kfree(src_buffer);
        return -ENOMEM;
    }

    // Initialize source buffer with test data
    memset(src_buffer, 0xAA, TRANSFER_SIZE);
    memset(dst_buffer, 0x00, TRANSFER_SIZE);

    // Map memory for DMA (EDMA needs physical addresses)
    src_dma = dma_map_single(NULL, src_buffer, TRANSFER_SIZE, DMA_TO_DEVICE);
    dst_dma = dma_map_single(NULL, dst_buffer, TRANSFER_SIZE, DMA_FROM_DEVICE);

    if (dma_mapping_error(NULL, src_dma) || dma_mapping_error(NULL, dst_dma)) {
        pr_err("Failed to map DMA buffers\n");
        ret = -ENOMEM;
        goto free_buffers;
    }

    // Initialize completion structure
    init_completion(&dma_complete);

    // Request EDMA channel
    dma_cap_zero(mask);
    dma_cap_set(DMA_MEMCPY, mask);
//dma_chan = dma_request_chan(NULL, "edma_memcpy0");
    dma_chan = dma_request_chan(NULL, "dma1chan20");
    if (IS_ERR(dma_chan)) {
        pr_err("Failed to request EDMA channel\n");
        ret = PTR_ERR(dma_chan);
        goto unmap_dma;
    }

    // Prepare the EDMA transfer
    tx = dmaengine_prep_dma_memcpy(dma_chan, dst_dma, src_dma, TRANSFER_SIZE, DMA_CTRL_ACK | DMA_PREP_INTERRUPT);
    if (!tx) {
        pr_err("Failed to prepare EDMA transfer\n");
        ret = -EIO;
        goto release_channel;
    }

    // Set the callback function
    tx->callback = edma_callback;
    tx->callback_param = &dma_complete;

    // Submit the EDMA transfer
    ret = dma_submit_error(dmaengine_submit(tx));
    if (ret) {
        pr_err("Failed to submit EDMA transfer\n");
        goto release_channel;
    }

    // Start the EDMA transfer
    dma_async_issue_pending(dma_chan);

    // Wait for the transfer to complete
    wait_for_completion(&dma_complete);

    pr_info("EDMA transfer completed. Verifying data...\n");

    // Verify if the destination buffer now contains the source buffer data
    if (memcmp(src_buffer, dst_buffer, TRANSFER_SIZE) == 0) {
        pr_info("EDMA transfer verified successfully!\n");
    } else {
        pr_err("EDMA transfer verification failed\n");
    }

release_channel:
    dma_release_channel(dma_chan);

unmap_dma:
    dma_unmap_single(NULL, src_dma, TRANSFER_SIZE, DMA_TO_DEVICE);
    dma_unmap_single(NULL, dst_dma, TRANSFER_SIZE, DMA_FROM_DEVICE);

free_buffers:
    kfree(src_buffer);
    kfree(dst_buffer);

    return ret;
}

// Module initialization
static int __init edma_test_init(void)
{
    pr_info("Initializing EDMA test module\n");
    return edma_memcpy_transfer();
}

// Module exit
static void __exit edma_test_exit(void)
{
    pr_info("Exiting EDMA test module\n");
}

module_init(edma_test_init);
module_exit(edma_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("EDMA memcpy test for AM5748");

    And sure I'll check the pci-epf-test.c and will update you.

    Thanks & regards,

    Phaneesh.

  • 0 in reply to Takuma Fujiwara
    Prodigy 180 points

    Hi Takuma ,

    Takuma Fujiwara said:
    Could you try referencing the pci-epf-test's pci_epf_test_write function? https://github.com/torvalds/linux/blob/v4.14/drivers/pci/endpoint/functions/pci-epf-test.c#L198. Not quite sure if it uses EDMA underneath, but this may be a better reference to go off of, since it works with BAR memory address and handles memory alignment through pci_epc_mem_alloc_addr and other functions.

                       Here they're not using any DMA , in "pci_epf_test_write"  also he's using memcpy  to write data .

    Thanks regards

    Phaneesh

  • 0 in reply to Phaneesh A
    TI__Mastermind 33942 points

    Hi Phaneesh,

    My apologies. I was more familiar with the newer version of the driver, and linked the older 4.14 kernel assuming some DMA functionality existed there.

    In the newer kernels, there is DMA functionality that is added to the pci-epf-test. Linked is the 6.1 kernel: https://github.com/torvalds/linux/blob/v6.1/drivers/pci/endpoint/functions/pci-epf-test.c. I think this will be a good reference to use.

    Regards,

    Takuma