Linux/TDA2PXEVM: Offscreen GPU rendering to userspace mapped buffer

Yavor Vidolov

Part Number: TDA2PXEVM

Tool/software: Linux

I'm using tda2px platform with vision sdk: software-dl.ti.com/processor-sdk-vision/esd/TDAx/vision-sdk/latest/index_FDS.html

Currently trying to setup a background processing pipeline sharing the workload between GPU and CPU. The GPU needs to render some offscreen data, which the CPU can process further.

In order to do this I require a userspace mapped buffer to be passed as an OpenGL resource.

Usually on the GPU side this is achieved by GL/EGL: EGLImage -> eglImageTargetRenderbufferStorage -> glFramebufferRenderbuffer -> glDraw*

I found utilities in vision_sdk/links_fw/src/hlos/system/system_egl_context.h being used in the ADAS demos for mapping the camera frame as a GPU resource.

System_EglWindowObj* pEglObj;
EGLCompatBuffer eglCBuf = pEglObj->eglInfo.get_egl_attach_buffer(buf_width, buf_height, SYSTEM_DF_RGB24_888 /*buf_format*/, buf_memory_pointer);
EGLImageKHR img = eglCreateImageKHR(pEglObj->display, EGL_NO_CONTEXT, EGL_NATIVE_PIXMAP_KHR, (EGLNativePixmapType)(eglCBuf.eglPixmap), NULL);

So now the question is how to get a contiguous memory region shared between the GPU/CPU and map it to our process.

I have gotten this far:

Use Generic Buffer Manager: ti_components/os_tools/linux/targetfs/usr/include/gbm/gbm.h to create the shared buffer

Use Direct Rendering Manager: ti_components/os_tools/linux/targetfs/usr/include/xf86drm.h to map the buffer to process userspace

Here is some pseudocode with error checking omitted:

struct gbm_device* gbmDev = (gbm_device*)pEglObj->nativeDisplay;

int fd = drmOpen("omapdrm", NULL); // drm handle
struct gbm_bo* bo = gbm_bo_create(gbmDev, buf_width, buf_height, GBM_BO_FORMAT_ARGB8888 /*buf_format*/, GBM_BO_USE_SCANOUT | GBM_BO_USE_RENDERING);
int bo_fd = gbm_bo_get_fd(bo);
union gbm_bo_handle bo_handle = gbm_bo_get_handle(bo);

struct drm_mode_map_dumb mreq = {0};
mreq.handle = bo_handle.ptr;
int ret = drmIoctl(fd /*bo_fd?*/, DRM_IOCTL_MODE_MAP_DUMB, &mreq);
void* map = mmap(0, buf_width*buf_height*4 /*creq.size*/, PROT_READ | PROT_WRITE, MAP_SHARED, bo_fd, mreq.offset);

The GBM buffer object seems to be properly created, having a handle and file descriptor, but I cannot get it mapped:

drmIoctl - returns -1, so we can't map the buffer object in order to provide the pointer to get_egl_attach_buffer.

Is this the right approach to this particular problem or is there a better way of achieving this functionality?

A follow up question will be about cache control, what tools are provided to invalidate/flush the shared buffer when switching processing from CPU to GPU and vice versa?

Thank you in advance,

Best Regards,

Yavor

over 6 years ago

0 Subhajit Paul over 6 years ago

TI__Expert 7015 points

can you try using "fd = open("/dev/dri/card0", O_RDWR)" instead of drmOpen and let me know?

0 Yavor Vidolov over 6 years ago in reply to Subhajit Paul

Prodigy 70 points

drmOpen("omapdrm", NULL) seems to use /dev/dri/card1
I tried with open("/dev/dri/card0", O_RDWR), but nothing changed.
MAP_DUMB still fails so I can't get the proper offset to mmap the buffer.

0 Subhajit Paul over 6 years ago in reply to Yavor Vidolov

TI__Expert 7015 points

which version of processor_sdk_vision are you using?

0 Yavor Vidolov over 6 years ago

Prodigy 70 points

Further development:

I tried to convert the gbm_bo to omap_bo, so I can use omap_bo_map function:

struct omap_device* omap_dev = omap_device_new(fd);
struct omap_bo *omap_bo_ = omap_bo_from_dmabuf(omap_dev, bo_fd);
void *omap_bo_userspace = omap_bo_map(omap_bo_);

however passing this pointer to get_egl_attach_buffer results in:

SI code = 2 (Invalid permissions for mapped object)

0 Yavor Vidolov over 6 years ago in reply to Subhajit Paul

Prodigy 70 points

The version of the rootFS and Processor SDK I am using are as follows:

tisdk-rootfs-image-dra7xx-evm_vsdk_3_5
PROCESSOR_SDK_VISION_03_05_00_00_setuplinux

0 Subhajit Paul over 6 years ago in reply to Yavor Vidolov

TI__Expert 7015 points

Are you getting a valid userspace address when you use omap_bo_map()? What address are you getting?
Where is the function get_gl_attach_buffer implemented?

0 Yavor Vidolov over 6 years ago in reply to Subhajit Paul

Prodigy 70 points

I tried creating a small temporal buffer on the heap, addresses seem to be relevant:
omap_bo_map pointer: 0x8a167000
process malloc pointer: 0x8a136008

CPU Test writing 48KB to both memory regions passes without fault:
[GBM] written to malloc memory
[GBM] written to omap_bo_userspace

get_gl_attach_buffer has a declaration in: vision_sdk/links_fw/src/hlos/system/system_egl_context.h
The implementation is hidden, the function receives a runtime pointer to the definition.
The pointer to the function seems correct, and is being used by the sdk in vision_sdk/links_fw/src/hlos/system/system_gl_egl_utils.c

0 Subhajit Paul over 6 years ago in reply to Yavor Vidolov

TI__Expert 7015 points

You should not use get_egl_attach_buffer function, unless you are running on QNX.

0 Yavor Vidolov over 6 years ago in reply to Subhajit Paul

Prodigy 70 points

I switched to using EGL_LINUX_DMA_BUF_EXT extension: (successful for YUV buffer)

// Generate Buffer Object with userspace mapping
int fd = drmOpen("omapdrm", NULL);
struct omap_device* omap_dev = omap_device_new(fd);
struct omap_bo* omap_bo_ = omap_bo_new(omap_dev, 256*256*4, 0* OMAP_BO_SCANOUT | OMAP_BO_MEM_CONTIG /* flags */);
void *omap_bo_userspace = omap_bo_map(omap_bo_);
int dma_fd = omap_bo_dmabuf(omap_bo_);

// Generate YUV texture using our buffer object
EGLint attr[] = {
            EGL_LINUX_DRM_FOURCC_EXT, DRM_FORMAT_NV12,
            EGL_WIDTH, 256,
            EGL_HEIGHT, 256,
            EGL_DMA_BUF_PLANE0_PITCH_EXT, 256,
            EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
            EGL_DMA_BUF_PLANE0_FD_EXT, dma_fd,
            EGL_DMA_BUF_PLANE1_PITCH_EXT, 256,
            EGL_DMA_BUF_PLANE1_OFFSET_EXT, 0,
            EGL_DMA_BUF_PLANE1_FD_EXT, dma_fd,
            EGL_NONE, EGL_NONE
        };

EGLImageKHR img = eglCreateImageKHR(pEglObj->display, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, attr);

glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture); CGLW();
glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, img); CGLW();

// Generate RGBA EGLImage from our buffer object
EGLint attr[] = {
            EGL_LINUX_DRM_FOURCC_EXT, DRM_FORMAT_ARGB8888,
            EGL_WIDTH, 256,
            EGL_HEIGHT, 256,
            EGL_DMA_BUF_PLANE0_PITCH_EXT, 256*4,
            EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
            EGL_DMA_BUF_PLANE0_FD_EXT, dma_fd,
            EGL_NONE, EGL_NONE
        };

EGLImageKHR img = eglCreateImageKHR(pEglObj->display, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, attr);

The buffer object, mapped address and dma fd are valid.

EGLImage with DRM_FORMAT_NV12 is created successfully, and the EGL resource can be mapped to GL_TEXTURE_EXTERNAL_OES texture. However this texture cannot be attached to framebuffer as a color attachement for rendering.

EGLImage with DRM_FORMAT_ARGB8888 format returns EGL_BAD_MATCH, although gbm_device_is_format_supported returns true for a 32bit ARGB format.

I also tried creating the dma buffer using:

struct gbm_bo* bo = gbm_bo_create(gbmDev, 256, 256, GBM_FORMAT_ARGB8888 /*format*/, GBM_BO_USE_RENDERING /*flags*/);
struct omap_bo *omap_bo_ = omap_bo_from_dmabuf(omap_dev, gbm_bo_get_fd(bo));

But still cannot create the RGBA EGLImage needed for rendering output.

0 Ramprasad over 6 years ago in reply to Yavor Vidolov

TI__Mastermind 30480 points

Hi Yavor,
Please refer create_texture() function in this source file.
git.ti.com/.../display-kmscube.c
This demonstrates adding NV12/YUYV/ARGB texture to an eglImage.

But this is applicable for only gbm display. For wayland-display PIXMAP is not supported.

Thanks
Ramprasad

0 Yavor Vidolov over 6 years ago in reply to Ramprasad

Prodigy 70 points

Thank you for the feedback, here is a working solution using PIXMAP instead of EGL_LINUX_DMA_BUF_EXT:

// Offscreen Zero-Copy GPU to Userspace Rendering
// Note: error checking and closing of handles omitted!

// 1. Create and map a shared buffer object

// We must have a valid GBM device, in my case this is the EGLNativeDisplayType configured by the pipeline
struct gbm_device* gbmDev = (EGLNativeDisplayType) nativeDisplay;

// Open DRM handle used by omap_device
int drm = drmOpen("omapdrm", NULL); // opened device is /dev/dri/card#

// Create a GenericBufferManager Buffer Object. We will use this object to map the EGL image
struct gbm_bo* bo = gbm_bo_create(gbmDev, 256, 256, GBM_FORMAT_ARGB8888 /*format*/, GBM_BO_USE_RENDERING /*flags*/);

// Use Omap Device to map the gbm buffer to userspace
struct omap_device* omap_dev = omap_device_new(drm); // open omap device
struct omap_bo *omap_bo_ = omap_bo_from_dmabuf(omap_dev, gbm_bo_get_fd(bo)); // create omap buffer object from the gbm buffer dma handle
void *omap_bo_userspace = omap_bo_map(omap_bo_); // map the buffer

// 2. Attach the dma buffer to EGL/GL for usage by the GPU

EGLDisplay display = ...; // we must have a valid EGLDisplay, in my case configured by the pipeline

// EGL_LINUX_DMA_BUF_EXT - does not seem to work for anything besides NV12 buffers, which can't be used for rendering?

EGLint attrib_list = EGL_NONE; // should be none when using a pixmap
PFNEGLCREATEIMAGEKHRPROC eglCreateImageKHR = (PFNEGLCREATEIMAGEKHRPROC)eglGetProcAddress("eglCreateImageKHR"); // resolve address of EGL function
EGLImageKHR imgEGL = eglCreateImageKHR(display, EGL_NO_CONTEXT, EGL_NATIVE_PIXMAP_KHR, bo, &attrib_list); // attach the dma buffer to EGLImage

// We can now use the standard OpenGL API to attach the buffer for rendering

// Create a texture and framebuffer, we can use for rendering
GLuint framebufferEGL, textureEGL;
glGenFramebuffers(1, &framebufferEGL);
glGenTextures(1, &textureEGL);
     
// Attach the EGLImage to the Texture   
glBindTexture(GL_TEXTURE_2D, textureEGL);
PFNGLEGLIMAGETARGETTEXTURE2DOESPROC glEGLImageTargetTexture2DOES = (PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)eglGetProcAddress("glEGLImageTargetTexture2DOES");
glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, imgEGL);
glBindTexture(GL_TEXTURE_2D, 0);
        
// Attach the texture to the framebuffer
glBindFramebuffer(GL_FRAMEBUFFER, framebufferEGL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureEGL, 0);
// glBindFramebuffer(GL_FRAMEBUFFER, 0);

// GL commands will now render to the DMA buffer object
// We can access the buffer from the CPU using the mapped userspace pointer
// NOTE: GPU/CPU synchronization is needed (glFlush, glFinish, eglClientWaitSyncNV/eglClientWaitSyncKHR)
// NOTE: If the GBM buffer object is cached, you need to invalidate the cache!

Processors

Processors forum

Linux/TDA2PXEVM: Offscreen GPU rendering to userspace mapped buffer