Linux/PRU-ICSS-INDUSTRIAL-SW: What is the best way to access PRU remoteproc carveouts from userspace?

Greetings Nick,

Thanks for the reply. From uname -a:

Linux beaglebone 4.14.71-ti-r80 #1 SMP PREEMPT Fri Oct 5 23:50:11 UTC 2018 armv7l GNU/Linux

and from my env command:

PRU_SUPPORT=/usr/lib/ti/pru-software-support-package

PRU_CGT=/usr/share/ti/cgt-pru

Writing a driver was what I was anticipating doing, but I can't figure out how to get a reference to an rproc instance where the carveout instance list lives for a given PRU. I could work my way to one if I could get a reference to a platform_device, or just a device, or some other entity in the tree/web of structures that the driver and device tree code builds up, but I just can't seem to find a hook to start at. I thought about writing a driver and have it 'of' match on the same things that pru_rproc does, but I don't know if the kernel supports that. I'm kind of a kernel hacking noob, so I don't know if Linux supports having more than one driver associated with a device. I wish they would export the remoteproc device information through sysfs, unfortunately the only thing there is the firmware attribute that tells you the file name loaded in the PRU, and the state attribute. Any other guidance you might have would be greatly appreciated.

Thanks,

Bill 

Greetings Nick,

All of this is being done on a Beaglebone Green. I am not using a custom driver for the PRU, I am using sysfs. So for PRU0, I'm writing the requested firmware path and file to:

/sys/class/remoteproc/remoteproc1/firmware

and am starting it by writing 'start' to:

/sys/class/remoteproc/remoteproc1/state

My project is taking an 8 bit parallel video feed in GRB422 format and writing it to a frame buffer, which is what the carveout I created is. I am currently passing the frame buffer as is to OpenCV for display and doing various ML functions on it. It all works, I just mmap the carveout into userspace and pass it to the various OpenCV functions. But I have to parse the resource_table file in debugfs:

/sys/kernel/debug/remoteproc/remoteproc1/resource_table

to get the carveout address to use with mmap to map to userspace. I'm just not sure if that is the 'recommended' way to do it, or consistent with the way remoteproc might change in the future. I have started working on my own 'driver' and am able to get at the rproc like this:

struct device_node *np;

struct platform_device * pdev;

struct rproc *rproc;

np = of_find_node_by_path("/ocp/pruss_soc_bus@4a326004/pruss@0/pru@34000");

if(!np)

{

printk(KERN_INFO "pru device node could not be acquired.\n");

return -ENODEV;

}

printk(KERN_INFO "pru device acquired. full_name: %s.\n", np->full_name);

pdev = of_find_device_by_node(np);

of_node_put(np);

if (!pdev) return -EPROBE_DEFER;

if (!strstr(dev_name(&pdev->dev), "pru") && !strstr(dev_name(&pdev->dev), "rtu"))

{

put_device(&pdev->dev);

return -ENODEV;

}

rproc = platform_get_drvdata(pdev);

put_device(&pdev->dev);

if (!rproc) return -EPROBE_DEFER;

printk(KERN_INFO "rproc acquired. name: %s.\n", rproc->name);

And I will probably try to use this in a very 'hacky' driver and expose it through sysfs. I think I was also hoping that there were long term plans to expose this kind of information so people could write userspace drivers, perhaps by using UIO.

I'm still kind of new to Linux development in general, and I want to do things the 'right' way, but I also want to try to keep things simple, and want to keep my project moving along. This is a personal project for my robotics hobby, so nothing many would care about but me at this point. Thanks for all of your attention!

Greetings Suman,

Many thanks for your input. I've actually been tearing through your code for remoteproc, pru_rproc, etc. to try to get a better idea of which way to take things, so having your input is fantastic. I'm acquiring the memory like so:

...

{

TYPE_CARVEOUT, //type

FW_RSC_ADDR_ANY, //Device address

0, //Physical address

sizeof(PRU_INTEROP_1_DATA), //Len

0, //IOMMU flags

0, //reserved

"PRU_VIDEO_CARVEOUT",

},

...

where PRU_INTEROP_1_DATA is a structure I created that is sized for my BGR image (320x240x24bit_color), and a single byte flag to communicate to the Linux side when a frame buffer is full (so I don't access it mid-frame and get 'tearing'). So my PRU firmware doesn't care where the memory is, and relies on remoteproc to allocate it and give it the address. Also, it is not that I want to limit access to the memory region of the video frame, actually the opposite. I am a hobbyist, so am not terribly well versed in hardware programming or Linux (I do web as my day job), but was under the impression that Linux is moving towards tightening restrictions on how userspace can mmap memory, especially kernel memory, to enforce greater security. I also want to do things the most 'correct' way. I had previously implemented this project using UIOPRUSS and that did work well, but I was also under the impression that UIOPRUSS was falling by the wayside in favor of remoteproc. Also, mapping the memory between the PRU and my userspace program was a bit convoluted and required an awkward build process to map addresses, and remoteproc is SO much cleaner. I also want to keep things as 'mainline' as possible, avoiding modifying the 'stock' kernel so I won't build up any dependencies that will break as the code not written by me changes. I think option 2 is what I am working towards, but am not sure if I understand what you mean. Are you talking about abandoning remoteproc altogether? Or do you mean a complimentary driver that is able to work with the existing remoteproc infrastructure? If the latter, how can I get a reference to the DDR carveout?

Are there any plans down the road to make more information available about the resourceTable through sysfs, as opposed to debugfs? I learned from some driver development videos that sysfs is more structured, with one value per file in ASCII format, whereas debugfs is more 'wild west' and no standard is enforced. It seems like it would be nice to be able to develop userspace drivers for remoteproc applications by accessing resourceTable info from sysfs.

Anyway, thanks a million again for all of your help!

Greetings Nick,

Thanks again for all the help you and Suman provided. I am glad the idea is to create a complimentary driver, without having to change any of the 'standard' code. This does lead me to another question, though. How can I write my driver so that it is probed when the PRU is started, and removed when the PRU stops? I've looked at the example for the RPMsg character device driver, but it relies on RPMsg to probe it, and by extension the PRU firmware to request a RPMsg channel in its resource table. I would prefer not to have to do something 'kludgey' like create a bogus RPMsg resource in my PRU firmware to get RPMsg to probe my driver. Any thoughts on how to tackle that? Again, thanks a million for all of your help.

Best regards,

Bill

Greetings Nick,

Since my question has kind of mutated, should I start another thread? I'm now wondering how I can write my driver so that it is probed when the PRU is started, and removed when the PRU stops? I think all of this information answered my question as it stood, so I consider it resolved. If I should start another thread or look elsewhere for guidance, please let me know, and thanks a million again for all of the help from you and Suman.

Thanks,

Bill

Hello Bill,

Sorry for the delay - catching up after vacation. My initial suggestion would be to modify the existing drivers in the system that are called when you start/stop the PRU, but I haven't really thought about your usecase - maybe remoteproc_sysfs.c > state_store or one of the associated functions?

I have seen people call initialization functions for separate drivers within the probe function of other drivers, but it does not sound like that is a match for your usecase (a messy example is cpts.c getting initialized in the cpsw.c probe function).

If you wanted a more in-depth discussion on how to write your driver to start/stop when you want it to, go ahead and create a new thread (and link it back here if you want).

Regards,

Nick