This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/PROCESSOR-SDK-AM335X: PRU UIO client support

Part Number: PROCESSOR-SDK-AM335X
Other Parts Discussed in Thread: AM3358

Tool/software: Linux

Our goal is to read a digital stream using PRU0 and its GPIO read support within AM335x Linux PDK 02.00.00.00. 

We have setup CCS and are building and running PRU firmware ok. We followed the detailed instructions on this page to setup TI PRU ICSS

processors.wiki.ti.com/.../

We were able to confirm pruss_remoteproc and RPMsg support to our kernel (menuconfig) and everything builds fine. We followed this page:

If we renamed our firmware to the default:

/lib/firmware/am335x-pru0-fw ... remoteproc loaded and ran it at device start - no problem. In the terminal (after insmod pruss_remoteproc) we saw:

[16704.983625] pruss-rproc 4a300000.pruss: creating platform devices for PRU cores
[16705.006421]  remoteproc1: 4a334000.pru0 is available
[16705.011439]  remoteproc1: Note: remoteproc is still under development and considered experimental.
[16705.051932]  remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[16705.092678] pru-rproc 4a334000.pru0: booting the PRU core manually
[16705.098957]  remoteproc1: powering up 4a334000.pru0
[16705.128052]  remoteproc1: Booting fw image am335x-pru0-fw, size 30944
[16705.135149]  remoteproc1: remote processor 4a334000.pru0 is now up
[16705.141430] pru-rproc 4a334000.pru0: PRU rproc node /ocp/pruss@4a300000/pru@4a334000 probed successfully

We then started to look into Linux user-level API support for PRU0 interaction and came to understand that we may need to switch to UIO. Examples provided in the hands on lab:

http://processors.wiki.ti.com/index.php/PRU_Training:_Hands-on_Labs

.. directly referenced additional modules to press_remoteproc that we could not get to work (RPMsg). 

During startup of pruss_remoteproc and other modules there were warnings that the released version of these modules were "experimental". We considered updating to 02.00.01.07 - where pruss_remoteproc and other PRU modules were "built-in" (no special setup required as per the link above). Migrating up to 02.00.01.07 is not trivial for us. We also noticed that most online examples used the "older" option of UIO support for manual app load and start of PRU firmware and quite a simple API interface to reading and writing PRU memory. So, we pursued getting UIO support for PRU load/start/interaction.

To add software support for UIO we were able to use menuconfig to setup "uio.ko" and "uio_pdrv_genirq.ko" (manual load). However, we could not build uio_pruss as the "Texas Instruments PRUSS driver" option was not visible under menuconfig. uio_pruss was expected under a "Userspace I/O drivers" group (or similar). Reviewing the kconfig file showed:

config UIO_PRUSS
        tristate "Texas Instruments PRUSS driver"
        depends on ARCH_DAVINCI_DA850
        select GENERIC_ALLOCATOR
        help
          PRUSS driver for OMAPL138/DA850/AM18XX devices
          PRUSS driver requires user space components, examples and user space
          driver is available from below SVN repo - you may use anonymous login

          gforge.ti.com/.../

          More info on API is available at below wiki

          processors.wiki.ti.com/.../PRU_Linux_Application_Loader

          To compile this driver as a module, choose M here: the module
          will be called uio_pruss.

... which indicated no AM335x support, but reviewing other sources:

https://github.com/hvaibhav/am335x-linux/blob/master/drivers/uio/uio_pruss.c,

.. indicated that we could use the uio_pruss.c with our PDK for am335x. Therefore, we manually changed our Makefile to always generate uio_pruss.ko (from sources already within the PDK).

We can issue insmod on uio.ko and then we can then issue insmod on the uio_press.ko and uio_pdrv_genirq.ko and all modules load and run without any error indication. lsmod shows:

Module                  Size  Used by                            
uio_pruss               2583  0                             
uio_pdrv_genirq         3209  0                             
uio                     8926  2 uio_pruss,uio_pdrv_genirq

We also built the libprussdrv driver (online sources) and we can link and make calls into this prussdrv lib just fine, but it accesses:

/dev/uio/uio0 and this /dev entry is not available. We have also read that /sys/class/uio should be visible after UIO support is up - but we cannot see it either.

We wonder if there is an error in how the driver supporting /dev is accessing our modules (i.e. module code issue) or if the /dev folder support has an issue with our device tree settings? uio_pruss was built from sources within the PDK, not from sources obtained elsewhere. We just had to change Makefile data. uio and uio_pdrv_genirq were built by simply changing menuconfig. So, we expect that the sources for modules are ok. Thier loading without error backs this up. To update the device tree, we made the following changes to am335x-bone-common.dtsi:

{
	cpus {
		cpu@0 {
			cpu0-supply = <&dcdc2_reg>;
		};
	};

	memory {
		device_type = "memory";
		reg = <0x80000000 0x10000000>; /* 256 MB */
	};


	vmmcsd_fixed: fixedregulator@0 {
		compatible = "regulator-fixed";
		regulator-name = "vmmcsd_fixed";
		regulator-min-microvolt = <3300000>;
		regulator-max-microvolt = <3300000>;
	};
	
	pruss_uio: pruss@4a300000 {
		status = "okay";
		compatible = "ti,pruss-v2";
		ti,hwmods = "pruss";
		ti,deassert-hard-reset = "pruss", "pruss";
		reg = <0x4a300000 0x080000>;
		ti,pintc-offset = <0x20000>;
		interrupt-parent = <&intc>;
		interrupts = <20 21 22 23 24 25 26 27>;
	};	

};

... and:

&am33xx_pinmux {
	pinctrl-names = "default";
	pinctrl-0 = <&clkout2_pin>;
...

        adc_pru_pins: pinmux_pru_pru_pins {
 	   	pinctrl-single,pins = <
        		0x1a0 ( PIN_INPUT | MUX_MODE6 ) /* (B12) ADC SCLK (B12) can be mapped (mode 6) to support pr1_pru0_pru_R31_4 (PRU0 Input) */
        		0x1a4 ( PIN_INPUT | MUX_MODE6 ) /* (C13) ADC FSYNC (C13) can be mapped (mode 6) to support pr1_pru0_pru_R31_5 (PRU0 Input) */
        		0x1a8 ( PIN_INPUT | MUX_MODE6 ) /* (D13) ADC FSYNC (C13) can be mapped (mode 6) to support pr1_pru0_pru_R31_5 (PRU0 Input) */
    		>;
	};

};

... and:

&pruss {
    status = "disabled";
};

&pruss_uio {
    pinctrl-names = "default";
    pinctrl-0 = <&adc_pru_pins>;
    status = "okay"; 
};

I originally specified "pruss: pruss@4a300000" within my dtsi file and I got a devicetree error notification on compile: "duplicate pruss object". I then saw that there was a pruss node within am33xx.dtsi This suggested a problem with having 2 "pruss@4a300000" entries. So, I changed my reference to "pruss_uio: pruss@4a300000" and adding the &pruss { status = "disabled"; }; as shown above. The "duplicate" error was fixed.

Another important note: we have not removed pruss_remoteproc support from the kernel as we expect that it will be needed for other modules. For example,we are getting the following upon device start:

[    5.236866]  remoteproc0: powering up wkup_m3
[    5.277917]  remoteproc0: Booting fw image am335x-pm-firmware.elf, size 219803

Could this be an issue or is there an issue with our devicetree entries, or are we missing a step to getting /dev/uio0?

Thanks,

Mark J

  • One important clarification ....
    We are using the TI kernel and not the "bone" kernel. See the following reply to "uname -r": 4.1.6-g52c4aa7

    Thanks,

    Mark J
  • Hello Mark,

    Quick check - PDK is usually associated with RTOS, you are using a Linux SDK right?

    We do not support UIO on the e2e forums - if RPMsg does not work for your use case, you may need to check the beaglebone forums for support.

    As per this e2e thread it looks like there is probably a community provided patch to enable uio_pruss on your SDK - However I am not sure what the exact steps are (note that particular patch might be for a different version of the SDK). Also note the comments in that thread about device tree changes, menuconfig options, etc.

    Regards, 

    Nick

  • Hi Nick,

    I did not realize that PDK described the RTOS DK. We are using TI Arago/Yocto Linux distribution packaged with the T Processor SDK. I had thought that this compressed to PDK (my misunderstanding). Example downloads here:

    software-dl.ti.com/.../index_FDS.html

    We read many Beaglebone forums and all relate to the Debian distribution and not the TI Arago/Yocto Linux distribution we are using. So, supporting UIO is much more difficult for us.

    Late yesterday, we updated to TI SDK version 02.00.01.11 (linked above) and we are getting both pruss_remoteproc and RPMsg modules loading clearny at device start. Next, we will work through the hands on labs.

    The only piece missing (at least not in the labs) is ARM access to PRU RAM memory (PRU-specific or shared memory). However, there are some examples that might support our using RPMsg for this. We will find one and I will append to this posting.

    Thanks,

    Mark J

  • Hello Mark,

    glad to hear remoteproc/RPMsg is working for you.

    Direct access to the PRU memory is one thing that UIO offers and RPMsg has yet to address. We can still get it working with Remoteproc/RPMsg, but it might be tricky depending on your use case.
    1) Is this a kernel driver accessing the memory, or is it userspace code that is accessing the memory?
    2) Does the ARM access to PRU memory only need to happen in a debug environment with elevated permissions, or also in a distribution environment?
    3) any other helpful info about the ARM accessing memory use case?

    Regards,
    Nick
  • Hi Nick,

    Our intention is that only userspace code will access the PRU RAM memory. If requried, however, we can build a kernel module to support this.

    Access must happen in both debug and distribution code.

    Our PRU code is simply watching a digital steam on PRU GPI pins, building a buffer of words using stream data (we were planning on 4kB-8kB buffer size) and Linux code would periodically read the buffer. The buffer will be filled at a rate of 24 kB/sec. PRU to ARM signalling is handled by hands on lab 5 (i.e. to tell the Linux code to read a buffer). Direct access to RAM is not covered in a lab (i.e. for the Linux code to read the buffer). Any examples that you can reference would be great.

    Thanks,

    Mark J
  • Hello Mark,

    A kernel module would be the easiest way to go, since the kernel can directly access the PRU memory. PRU Hands-On Labs Lab 5 uses a kernel module to do RPMsg communication (the module code is at /ti-processor-sdk-linux-.../board-support/linux-.../samples/rpmsg/rpmsg_client_sample.c). The prueth driver gives an example of directly accessing PRU memory from the kernel.

    From userspace, if you have sudo permissions you can use /dev/mem to directly access the PRU memory - however, that won't work for distribution code. There may be a way to hack together something using the mmap interface, but I have not looked into that option.

    Regards, 

    Nick

  • Hi Nick,

    The prueth might be a good reference. I will look into it.

    We will have root access for both dev and deployed devices. I probably misunderstood what you meant by distribution. Deployed devices are headless and will have no support for physical connection.

    If accessing /dev/mem is the easiest option, we should pursue that. Is there an example that you would suggest?

    Thanks,

    Mark J
  • Hi Nick,

    There is way too much going on within prueth.c. It is a tough reference. I think that we just need a good example showing how PRU0 code writes to PRU0 DRAM and how ARM Linux code reads from PRU0 DRAM. If mmap() at user-space handles this, that works for us. I see many UIO examples, but not examples using the pruss or pru_rproc modules that we have running. Can you recommend one?

    Thanks,

    Mark J
  • Hi Nick,

    I now have both RPMsg comms and direct PRU0 DRAM access working within both PRU firmware and user-space Linux code.

    I just wanted to add some notes and code bits to support future ref.

    The "PRU_RPMsg_Echo_Interrupt0" example was used as a source for RPMsg comms support. This example shows bi-directional comms, but I expect to only use PRU to ARM. Importantly - it works well for both PRU to ARM and ARM to PRU comms.

    It is critical that the pruss header and lib support used for developing PRU firmware comes from the PRUSS support within the SDK used to build the Linux OS and SDK for app builds. Multiple versions of PRUSS support exist and they are not compatible. I am using Processor SDK 02.00.02.11, so the PRU lib support used is at: /ti-processor-sdk-linux-am335x-evm-02.00.02.11/example-applications/pru-icss-4.0.2

    This link details how RPMsg is used to send/receive messages. At the user level in linux, we simply read/write from /dev/rpmsg_pru30: 

    Note that the first 3 steps in the page are misleading and relate to older pruss_remoteproc handling. We cannot interact with the sysfs "state" member as it (and even "/sys/class/remoteproc") do not exist in our implementation of RPMsg and Remoteproc (we can see "/sys/class/rpmsg_pru" ... which has no "state" element for control). This is not an issue as the Remoteproc and RPMsg modules that load at kernel start can start our firmware automatically (see symbolic link note above).

    Rather than accessing memory through RPMsg, we can directly access memory in the ARM using mmap. PRUSS memory access in ARM (Global Mem) is from a base of 0x4A30 0000. This relates to PRU0 DRAM at 0x0000 0000 (when accessed locally -which is must faster within a PRU than global access). Each PRU has DRAM limit of 8kB (0x2000), so the working range for shared mem access is  0x4A30 0000 -  0x4A30 2000 in the ARM which equals 0x0000 0000 to 0x0000 2000 in the PRU. This is all described well by a TI in this forum posting: Note that this describes a device with different host processor. For the AM3358, the TRM shows PRU_ICSS base ad 0x4A30 0000. See the end of Section 2.1 - Table 2-1.

    PRU and Linux Code for RPMsg is quite well described within the "RU_RPMsg_Echo_Interrupt0" example.

    For PRU DRAM access, simply a buffer within your PRU code. Local DRAM was used for many things and I could not find a way to setup a cost address for the buffer, so I elected to follow an online example and setup my buffer within PRU shared memory. 

    The example explained how to setup a new section that used shared mem (PRU_SHAREDMEM):

    /* Specify the sections allocation into memory */
    SECTIONS {
    	/* Forces _c_int00 to the start of PRU IRAM. Not necessary when loading
    	   an ELF file, but useful when loading a binary
    	   shared_arm added by DWorx to alloc DRAM space for shared array. */
    	.text:_c_int00*	>  0x0, PAGE 0
    
    	.text		>  PRU_IMEM, PAGE 0
    	.stack		>  PRU_DMEM_0_1, PAGE 1
    	.bss		>  PRU_DMEM_0_1, PAGE 1
    	.cio		>  PRU_DMEM_0_1, PAGE 1
    	.data		>  PRU_DMEM_0_1, PAGE 1
    	.switch		>  PRU_DMEM_0_1, PAGE 1
    	.sysmem		>  PRU_DMEM_0_1, PAGE 1
    	.cinit		>  PRU_DMEM_0_1, PAGE 1
    	.rodata		>  PRU_DMEM_0_1, PAGE 1
    	.rofardata	>  PRU_DMEM_0_1, PAGE 1
    	.farbss		>  PRU_DMEM_0_1, PAGE 1
    	.fardata	>  PRU_DMEM_0_1, PAGE 1
    	.resource_table > PRU_DMEM_0_1, PAGE 1
    	.shared_arm >  PRU_SHAREDMEM, PAGE 2
    }

    ... and then add the following DATA_SECTION and REATAIN pragma statements to c code to ensue that the linker would block off this mem for my access:

    #pragma DATA_SECTION(SharedARMMem, ".shared_arm")
    
    #pragma RETAIN(SharedARMMem)
    
    volatile far uint8_t SharedARMMem[SHARED_MEM_SIZE];
    
    /* Setup a buffer to function as RPMsg payload store.			*/
    uint8_t RPMsgPayload[SMALL_RPMSG_BUF_SIZE];

    After doing this and building PRU code, the output .map file shows the "retained" section:

    GLOBAL SYMBOLS: SORTED ALPHABETICALLY BY Name 
    
    page  address   name                         
    ----  -------   ----                         
    0     00000534  C$$EXIT                      
    2     00026000  CT_CFG                       
    2     00020000  CT_INTC                      
    2     480c8000  CT_MBX                       
    1     0000017c  RPMsgPayload                 
    2     00010000  SharedARMMem   
    ...

    ... and then setup Linux access to this mem:

    #define OM3358_MEM_DEVNAME		"/dev/mem"			// Direct access to OM3358 memory.
    ...
    
            struct pollfd OM3358MemFD;
    	INT32 result;
    	char readBuf[32];
    	UINT8* pPRUMem;
    
    	memset(readBuf, 0, 32);
    	
    	OM3358MemFD.fd = open(OM3358_MEM_DEVNAME, O_RDWR | O_SYNC);
    
    	if(OM3358MemFD.fd < 0)
    	{
    		 printf("Failed to open OM3358 device mem at %s. Error: %d\r\n", OM3358_MEM_DEVNAME, OM3358MemFD.fd);
    		 return;
    	}
    
            /* map to PRU0 DRAM starting at 0x4A31 0000 with size of 4kB (0x1000). */
    	pPRUMem = (UINT8*) mmap(0, 0x1000, PROT_WRITE | PROT_READ, MAP_SHARED, OM3358MemFD.fd, 0x4A310000);
    
    	if(pPRUMem == MAP_FAILED)
    	{
    		 printf("Failed to map OM3358 device \r\n");
    		 close(OM3358MemFD.fd);
    		 return;
    	}
    	else
    	{
    		printf("Got mem pointer at %p\r\n", pPRUMem);
    		printf("Mem uint8 0: %x\r\n", pPRUMem[0]);    /* SharedARMMem[0] in PRU0 */
    		printf("Mem uint8 1: %x\r\n", pPRUMem[1]);    /* SharedARMMem[1] in PRU0 */
    	}
    
    	close(OM3358MemFD.fd);

    Mark J

     

  • Hello Mark,

    Thanks for posting your resolution to the issue!

    For your FYI, the sysfs interface is used to load PRU firmware in current versions of Processor SDK. The wiki you linked applies to versions of Processor SDK which are more recent than SDK 2.0 (but it still needs to be updated to match the fact that mailboxes have been replaced by interrupts in current code samples from TI).

    Automatic booting also works differently in more recent versions of the SDK, so this post may be a good reference for you if you update to a later version of Processor SDK in the future. 

    Regards, 

    Nick