AM355x DMA Not Working

Hi Irwin,

Please provide more information: what software/varsion are you using, what device are you accessing over the GPMC?

Hi Biser

1. The GPMC access NAND flash, CPLD, FPGA each has its own chip select. FPGA has two chip selects one for asynch access one is for synch access.
2. u-boot : ti-u-boot-2013.01.01-amsdk-06.00.00.00, kernel v3.2_AM335xPSP_04.06.00.11

thanks

iw

Hi, Biser:

I am the software engineer working with Irwin on this project. I did make the GPMC synch access to FPGA with 4 words working so far. Next we plan to use EDMA to assist GPMC to access FPGA with 16 words or 32 words working. Can you provide some document and example how to make it work. Thank you very much!

Jin

Check these threads:

http://e2e.ti.com/support/arm/sitara_arm/f/791/t/348712

http://e2e.ti.com/support/arm/sitara_arm/f/791/t/105271

Hi, Biser:

I read the two post, I think I am still not clear how to set up the DMA in the FPGA device driver. Can you give an example?

Hi ,IT LOOKS NOT GOOD,I NEED TO TEST THE HARDWARE

Can somebody help to answer my question?

I set up the DMA to do burst of 4 words to FPGA through GPMA.

I got this error

[ 229.383026] Unable to handle kernel paging request at virtual address 44000000
[ 229.405303] pgd = ddef4000
[ 229.413635] [44000000] *pgd=00000000
[ 229.424682] Internal error: Oops: 805 [#1]
[ 229.437286] Modules linked in:
[ 229.446685] CPU: 0 Not tainted (3.2.02-00117-g839e174-dirty #90)
[ 229.466308] PC is at v7_dma_inv_range+0x30/0x48
[ 229.480255] LR is at ___dma_page_dev_to_cpu+0x58/0xa0
[ 229.495819] pc : [<c001cae0>] lr : [<c0019a00>] psr: 60000013
[ 229.495819] sp : dded3e58 ip : ff080000 fp : dded3e74
[ 229.531250] r10: c04d693c r9 : c0479d80 r8 : c0496f08
[ 229.547332] r7 : 00000100 r6 : 00000002 r5 : 00000000 r4 : bf568000
[ 229.567474] r3 : 0000003f r2 : 00000040 r1 : 44000100 r0 : 44000000
[ 229.587585] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 229.609588] Control: 10c5387d Table: 9def4019 DAC: 00000015
[ 229.627319] Process peekpoke (pid: 871, stack limit = 0xdded22f0)
[ 229.646087] Stack: (0xdded3e58 to 0xdded4000)
[ 229.659515] 3e40: df0e7700 00000000
[ 229.684722] 3e60: ddeb2ac0 00000100 dded3eec dded3e78 c01c2da0 c00199b4 00000000 00000000
[ 229.709960] 3e80: 00000000 ddef5008 dded3ecc c04e8000 000804e2 00000000 80000000 00000b14
[ 229.735198] 3ea0: 00000100 dded2000 00000014 c04f1c40 00000100 bef616fc 00000000 04000000
[ 229.760406] 3ec0: dded3f54 de8ec8a0 bef616fc 80144d01 00000015 00000000 dded2000 00000000
[ 229.785644] 3ee0: dded3f74 dded3ef0 c00b3d04 c01c27f0 de801798 de8ec8a0 00000101 00000004
[ 229.810852] 3f00: 00000000 00000000 00000000 df010b00 dc7d2c48 00000001 00101002 00000000
[ 229.836090] 3f20: 00000000 00000000 c00bc290 db8dc000 dc7d2c40 00000015 c0476254 db8dc000
[ 229.861297] 3f40: dc7d2c40 00000015 dded3f94 dded3f58 dc7d2c40 bef616fc 80144d01 00000015
[ 229.886535] 3f60: 00000000 dded2000 dded3fa4 dded3f78 c00b4224 c00b3c88 00000000 00000001
[ 229.911743] 3f80: c0014328 bef61780 000ab298 00000000 00000036 c0014328 00000000 dded3fa8
[ 229.936981] 3fa0: c0014180 c00b41b8 bef61780 000ab298 00000015 80144d01 bef616fc 00000000
[ 229.962219] 3fc0: bef61780 000ab298 00000000 00000036 00095ed4 00000015 40046000 bef61734
[ 229.987426] 3fe0: 404426f0 bef616cc 000638e4 404426fc 60000010 00000015 00000000 00000000
[ 230.012664] Backtrace:
[ 230.020202] [<c00199a8>] (___dma_page_dev_to_cpu+0x0/0xa0) from [<c01c2da0>] (FPGA_dma_ioctl+0x5bc/0x7cc)
[ 230.049713] r7:00000100 r6:ddeb2ac0 r5:00000000 r4:df0e7700
[ 230.067199] [<c01c27e4>] (FPGA_dma_ioctl+0x0/0x7cc) from [<c00b3d04>] (do_vfs_ioctl+0x88/0x530)
[ 230.094024] [<c00b3c7c>] (do_vfs_ioctl+0x0/0x530) from [<c00b4224>] (sys_ioctl+0x78/0x80)
[ 230.119232] r9:dded2000 r8:00000000 r7:00000015 r6:80144d01 r5:bef616fc
[ 230.139373] r4:dc7d2c40
[ 230.147460] [<c00b41ac>] (sys_ioctl+0x0/0x80) from [<c0014180>] (ret_fast_syscall+0x0/0x30)
[ 230.173217] r8:c0014328 r7:00000036 r6:00000000 r5:000ab298 r4:bef61780
[ 230.193908] Code: 1e070f3e e1110003 e1c11003 1e071f3e (ee070f36)
[ 230.212829] ---[ end trace fd6e645dde833dfc ]---

Any idea?

Post the source code of the setup of the DMA controller.

I assume that you messed up virtual and physical addresses.

regards

Wolfgang

struct FPGA_dma_device
{
char *name;
struct device *dev;
int major;
unsigned long phys_base;
int size;
char *dma_addr;
int dma_len;
struct resource *private;
int dma_ch;
struct completion dma_done;
};

static struct FPGA_dma_device *FPGA_dma;
static struct class *FPGA_dma_class;
static DEFINE_MUTEX(FPGA_mutex);

#define MAX_FPGA_DMA_BUF_SIZE 0x100000 /*1M, random no reason */

static ssize_t no_read(struct file *filp, char *data, size_t count, loff_t *offp)
{
return -EINVAL;
}

static ssize_t no_write(struct file *filep, const char *data, size_t count, loff_t *offp)
{
return -EINVAL;
}

static int FPGA_dma_open(struct inode *inode, struct file *filp)
{
int minor = iminor(inode);
int major = imajor(inode);

if ( major != FPGA_dma->major || minor != 0)
return -ENXIO;
if (FPGA_dma->dev == NULL)
return -ENXIO;
filp->private_data = FPGA_dma;
return 0;
}

static void mrv_FPGA_dma_cb(int channel, u16 ch_status, void *data)
{
struct FPGA_dma_device *c = (struct FPGA_dma_device *)data;
complete(&c->dma_done);
}

static inline int mrv_FPGA_dma_transfer(struct FPGA_dma_device *info, void *addr, unsigned int len, int offset, int is_write)
{
enum dma_data_direction dir = is_write ? DMA_TO_DEVICE : DMA_FROM_DEVICE;
dma_addr_t dma_src, dma_dst;
unsigned long timeout;
volatile unsigned *done;

if (is_write) {
dma_dst = info->phys_base + offset;
dma_src = dma_map_single(info->dev, addr, len, dir);
}
else {
dma_src = info->phys_base + offset;
dma_dst = dma_map_single(info->dev, addr, len, dir);
}
if (dma_mapping_error(info->dev, dma_src)) {
dev_err(info->dev, "Couldn't DMA map a %d byte buffer\n", len);
return -EFAULT;
}

omap_set_dma_transfer_params(info->dma_ch, OMAP_DMA_DATA_TYPE_S16,
len/2, 1, OMAP_DMA_SYNC_FRAME, OMAP24XX_DMA_GPMC,
is_write ? OMAP_DMA_DST_SYNC : OMAP_DMA_SRC_SYNC);
omap_set_dma_src_params(info->dma_ch, 0, OMAP_DMA_AMODE_POST_INC,
dma_src, 0, 0);
omap_set_dma_dest_params(info->dma_ch, 0, OMAP_DMA_AMODE_POST_INC,
dma_dst, 0, 0);
INIT_COMPLETION(info->dma_done);

omap_start_dma(info->dma_ch);

/*
* There's no exact timeout values at Spec.
* In real case it takes under 1 msec.
* So 20 msecs are enough.
*/
timeout = jiffies + msecs_to_jiffies(20);
done = &info->dma_done.done;
while (time_before(jiffies, timeout))
if (*done)
break;

dma_unmap_single(info->dev, dma_src, len, dir);

if (!*done) {
dev_err(info->dev, "timeout waiting for DMA\n");
return -EFAULT;
}

return 0;
}

static long FPGA_dma_ioctl(struct file *file, u_int cmd, u_long arg)
{

void __user *argp = (void __user *)arg;
struct FPGA_dma_device *FPGA_info;
int err = 0;
int len, left;
struct FPGA_dma_req *req;
int size = (cmd & IOCSIZE_MASK) >> IOCSIZE_SHIFT;

printk(KERN_ERR PFX "FPGA synch access command is %d\n", cmd);

if (!file || !arg)
return -1;

if (cmd & IOC_IN) {
if (!access_ok(VERIFY_READ, argp, size))
return -EFAULT;
}
if (cmd & IOC_OUT) {
if (!access_ok(VERIFY_WRITE, argp, size))
return -EFAULT;
}

FPGA_info = file->private_data;
req = kmalloc(sizeof(struct FPGA_dma_req), GFP_KERNEL);
if (!req) {
printk(KERN_ERR PFX "no mempry");
return -ENOMEM;

}
err = copy_from_user(req, argp, sizeof(struct FPGA_dma_req)) ? -EFAULT : 0;
if (err < 0) {
kfree(req);
return err;
}

mutex_lock(&FPGA_mutex);
switch (cmd) {
int offset;
case FPGA_DMA_READ:
left = req->size;
offset = 0;
while (left > 0) {
len = req->size > MAX_FPGA_DMA_BUF_SIZE ? MAX_FPGA_DMA_BUF_SIZE : req->size;
err = mrv_FPGA_dma_transfer(FPGA_info, FPGA_dma->dma_addr, len, offset, 0);
if (err >= 0)
err = copy_to_user(req->data + offset, FPGA_dma->dma_addr, len) ? -EFAULT : 0;
left -= len;
offset += len;
printk(KERN_ERR PFX "FPGA synch read from %p to %p size=%d, left=%d, offset=%d, total=%d\n",
FPGA_dma->dma_addr, req->data, len, left, offset, req->size);
if (err < 0)
break;
};
req->ret_size = req->size - left;
break;
case FPGA_DMA_WRITE:
left = req->size;
offset = 0;
while (left > 0) {
len = req->size > MAX_FPGA_DMA_BUF_SIZE ? MAX_FPGA_DMA_BUF_SIZE : req->size;
err = copy_from_user(FPGA_dma->dma_addr, req->data+offset, len) ? -EFAULT : 0;
if (err >= 0)
err = mrv_FPGA_dma_transfer(FPGA_info, FPGA_dma->dma_addr, len, offset, 1);
left -= len;
offset += len;
if (err < 0)
break;
};
req->ret_size = req->size - left;
break;
default:
printk(KERN_ERR PFX "FPGA synch access command is wrong %d\n", cmd);
err = -1;
}
mutex_unlock(&FPGA_mutex);
err = copy_to_user(argp, req, size) ? -EFAULT : 0;
kfree(req);
return err;
}

static const struct file_operations FPGA_dma_fops = {
.owner = THIS_MODULE,
.open = FPGA_dma_open,
.read = no_read,
.llseek = no_llseek,
.write = no_write,
.unlocked_ioctl = FPGA_dma_ioctl,
};

static int request_dma_channel(struct fpga_driver_info *di)
{
int err = 0;

if (di->dma_res) {
FPGA_dma->private = di->dma_res;
FPGA_dma->dev = di->dev;
err = omap_request_dma(AM33XX_DMA_GPM, "FPGA access", mrv_FPGA_dma_cb, FPGA_dma, &FPGA_dma->dma_ch);
if (err < 0) {
FPGA_dma->dma_ch = -1;
dev_err(di->dev, "DMA request failed!\n");
err = -ENXIO;
goto out_release_mem_region;
}
else {
omap_set_dma_dest_burst_mode(FPGA_dma->dma_ch,
OMAP_DMA_DATA_BURST_4); /* change it later to 16 */
omap_set_dma_src_burst_mode(FPGA_dma->dma_ch,
OMAP_DMA_DATA_BURST_4); /* change it later to 16 */
}
}
out_release_mem_region:
return err;

}

static int release_dma_channel(void)
{
omap_free_dma(FPGA_dma->dma_ch);
return 0;

}

static int init_FPGA_dma_device(struct fpga_driver_info *di)
{

int major=0, err = 0;

if (di == NULL)
return -EINVAL;

FPGA_dma = kzalloc(sizeof(struct FPGA_dma_device), GFP_KERNEL);
if (!FPGA_dma) {
printk(KERN_ERR PFX "kzalloc failed\n");
err = -ENOMEM;
goto __err_kmalloc;
}
FPGA_dma->name = "synch_fpga";
FPGA_dma->private = di->res;
FPGA_dma->phys_base = di->res->start;
FPGA_dma->size = di->res->end - di->res->start+1;
FPGA_dma->dma_addr = kmalloc(MAX_FPGA_DMA_BUF_SIZE, GFP_DMA | GFP_KERNEL);
if (!FPGA_dma->dma_addr) {
printk(KERN_ERR PFX "can't request mem region!\n");
goto __err_request_mem;
}

major = register_chrdev(0, FPGA_dma->name, &FPGA_dma_fops); /* fsd -- FPGA synch access with DMA */
FPGA_dma->major = major;
if (major < 0) {
printk(KERN_ERR PFX "register cdev(%s) failed, errno = %d\n", FPGA_dma->name, major);
err = major;
goto __err_register_chrdev;
}

/* create class */
FPGA_dma_class = class_create(THIS_MODULE, FPGA_dma->name);
err = PTR_ERR(FPGA_dma_class);
if (IS_ERR(FPGA_dma_class)) {
printk(KERN_ERR PFX "class_create(%s) failed\n", FPGA_dma->name);
goto __err_class;
}

/* add the char device with file operation to the system*/
FPGA_dma->dev = device_create(FPGA_dma_class, NULL,
MKDEV(FPGA_dma->major, 0), NULL, "sfpga");
err = PTR_ERR(FPGA_dma->dev);
if (IS_ERR(FPGA_dma->dev))
goto __err_device;

/* acquire the DMA */
if ((err = request_dma_channel(di)) < 0 )
goto __err_dma_channel;

return 0;

__err_dma_channel:
device_destroy(FPGA_dma_class, MKDEV(FPGA_dma->major, 0));

__err_device:
class_destroy(FPGA_dma_class);

__err_class:
unregister_chrdev(FPGA_dma->major, FPGA_dma->name);

__err_register_chrdev:
kfree(FPGA_dma->dma_addr);

__err_request_mem:
kfree(FPGA_dma);

__err_kmalloc:
release_resource(di->res);

return err;

}

static int fini_FPGA_dma_device(void)
{
if (FPGA_dma) {
release_dma_channel();
device_destroy(FPGA_dma_class, MKDEV(FPGA_dma->major, 0));
unregister_chrdev(FPGA_dma->major, FPGA_dma->name);
class_destroy(FPGA_dma_class);
kfree(FPGA_dma->dma_addr);
release_resource(FPGA_dma->private);
kfree(FPGA_dma);
}
return 0;
}

I found the crushing problem. Anyway it seems DMA still not read from GPMC, I keep get timeout message.

it is the dma_unmap_single() it should be:
if (is_write)
dma_unmap_single(info->dev, dma_src, len, dir);
else
dma_unmap_single(info->dev, dma_dst, len, dir);

Can somebody answer my question, I really need this to be solved quickly.
Thank you!
Jin

Now when I do a DMA transfer, I got this. Any idea?
Thank you!
Jin

[ 171.653625] omap_set_dma_dest_burst_mode: un-supported SDMA wrapper
[ 171.653656] omap_set_dma_src_burst_mode: un-supported SDMA wrapper
[ 171.673004] gpmc/fpga: dma transfer from 4010000 to 9b900000 size 256
[ 171.673034] omap2-fpga omap2-fpga.1: timeout waiting for DMA

Is anyone from TI monitoring this post? I never get a straight answer about how to access FPGA with synch burst mode from GPMC though EDMA.
Thank you!
Jin

Hi Irwin,

I don't have an option to physically check this use case for data transfers with FPGA, so will continue with suggestions to narrow down the problem on your side.

As I am understanding now you are able to write successfully needed content to GPMC/FPGA with EDMA transfers, but still not able to read content with EDMA.

Could you confirm that statement ?

One thing that is good to be checked is the exact EDMA channel associated with your transfers and corresponding GPMC Event (I guess it is GPMCEVT in your case) settings that is supposed to trigger/synchronize your GPMC Read transfers from the GPMC/FPGA. I am asking for such check because my first impression is that reading is not properly triggered...

You could refer "Event-Triggered Transfer Request" section in the processor TRM for details about the EDMA Trigger Events handling.

Dobry

Actually it is Jin, my own account is blocked out I just use Irwin's for now.

First let me update what is status.

I never test write yet. I start with read. I set it is burst of 4 words. It seems work in some degree. There still things are not quite right. When I read multiple of 4 words. The first four words always be repeated. I have no idea why.

The FPGA is GPMC burst 4 words synch access.

Here is the analyzer's output: See the addr line up6_addr, it starts with 0x20, when the second burst it starts with 0x20 again then the third one is 0x24, fourth one is 0x28...

Any idea?

Thank you!

Jin

I try to upload my analyzer image, it doesn't show here. If you need it I can email to you if I know the email address or tell me how to send to you.

Jin

You can send it to x0081122@ti.com

My points here are related to the event triggering settings used in your environment for EDMA handling.

In order to get reads from outside you will need proper EDMA triggering and event handling....

The read is working now, only thing I am not clear is that in the dma_callback function.

I keep get the ch_status is

OMAP1_DMA_TOUT_IRQ
or OMAP_DMA_DROP_IRQ

Then in the next read, first I get timeout in the transfer then I keep get OMAP_DMA_DROP_IRQ until the burst is done.

What should I do to get this right?

Thank you!

Jin

Hi, There:

The read is working now. the reason for the timeout is that it seems I have to enable the TCC interrupts every time before I issue a transfer.

Now I just have a simple question. based on the Tech Manual, the GPMC prefetch is only one chip select at a time. my board have one chip select connect to NAND device, one chip select connected to FPGA. The NAND device is using the GPMA prefetch mode I assume. Does it mean I have to make some lock system to protect it. In that sense does it still make sense to use prefetch in the FPGA access?

Thank you!

Jin

Hi, there:

The DMA access is working fine, but I see a different problem, the other GPMC chip select stop working. Any idea?

Thank you!

Jin

Jin, you will want to configure the GPMC to use the prefetch engine for your NAND device.  The prefetch engine is tied to a specific CS using GPMC_PREFETCH_CONFIG.ENGINECSSELECTOR, so you don't have to do any sort of lockout for accesses to/from your FPGA.

As for your CS problem, ensure that the CS and memory map is correctly configuring in the CONFIG7 register for the associated chip select that you are using.  You should be able to test this with simple single read/writes to each chip select.  If your DMAs don't seem to be working to your FPGA, then most likely you don't have the proper source or destination address setup in the DMA configuration.

Regards,

James

Hi, James:

Thank you for your answer.

From what I understand what you said I only can connect the PREFETCH to the NAND device. For FPGA access I can't use PREFETCH. Is it correct?

Is there is a way I can use PREFETCH for NAND device and FPGA DMA access?

Jin

Jin, i have only ever seen the prefetch and write posting engine use in the context of a NAND device. I suppose you could use it with an FPGA, but the fact that you can only set it up in one mode (prefetch read or write posting) at a time, it would seem to be cumbersome to use. DMAs to your FPGA will still be efficient without it.
So, yes, I would setup the prefetch engine for the CS on your NAND, and use the EDMA when burst accesses are required on your FPGA.

REgards,
James