Linux/TDA2PXEVM: [OpenGl] ARGB Texture support on VisionSDK

Hi Anand:

  Here is the sample code that to show the jpg with ARGB, but no output on screen.

 By using the sgxfrmcopy pluggin.

/*
Copyright (c) [2012 - 2017] Texas Instruments Incorporated

All rights reserved not granted herein.

Limited License.

 Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive
 license under copyrights and patents it now or hereafter owns or controls to
 make,  have made, use, import, offer to sell and sell ("Utilize") this software
 subject to the terms herein.  With respect to the foregoing patent license,
 such license is granted  solely to the extent that any such patent is necessary
 to Utilize the software alone.  The patent license shall not apply to any
 combinations which include this software, other than combinations with devices
 manufactured by or for TI ("TI Devices").  No hardware patent is licensed
 hereunder.

 Redistributions must preserve existing copyright notices and reproduce this
 license (including the above copyright notice and the disclaimer and
 (if applicable) source code license limitations below) in the documentation
 and/or other materials provided with the distribution

 Redistribution and use in binary form, without modification, are permitted
 provided that the following conditions are met:

 * No reverse engineering, decompilation, or disassembly of this software
   is permitted with respect to any software provided in binary form.

 * Any redistribution and use are licensed by TI for use only with TI Devices.

 * Nothing shall obligate TI to provide you with source code for the software
   licensed and provided to you in object code.

 If software source code is provided to you, modification and redistribution of
 the source code are permitted provided that the following conditions are met:

 * Any redistribution and use of the source code, including any resulting
   derivative works, are licensed by TI for use only with TI Devices.

 * Any redistribution and use of any object code compiled from the source code
   and any resulting derivative works, are licensed by TI for use only with TI
   Devices.

 Neither the name of Texas Instruments Incorporated nor the names of its
 suppliers may be used to endorse or promote products derived from this software
 without specific prior written permission.

 DISCLAIMER.

 THIS SOFTWARE IS PROVIDED BY TI AND TI�S LICENSORS "AS IS" AND ANY EXPRESS OR
 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
 IN NO EVENT SHALL TI AND TI�S LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT,
 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
 OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include "sgxRenderAVM.h"
#include "avm_gpu.h"
#include "summ_widget.h"
#include "surround_widget.h"




#include <assert.h>

#include <memory>




#define BOTTOM_LEFT_W
#define BOTTOM_LEFT_H  0



#define SURR_POSITION_X 0
#define SURR_POSITION_Y 0+BOTTOM_LEFT_H
#define SURR_WIDTH 790
#define SURR_HEIGHT 720

#define SUMM_POSITION_X SURR_WIDTH
#define SUMM_POSITION_Y 0+BOTTOM_LEFT_H
#define SUMM_WIDTH 490
#define SUMM_HEIGHT 720



#include <assimp/scene.h>
#include <assimp/postprocess.h>
#include <assimp/Importer.hpp>
#include <IL/il.h>


std::unique_ptr<TAVMServiceServer> AVMServiceServer;

std::unique_ptr<TAVMViewControl> AVMViewControl;


std::unique_ptr<SummWidget> SumAVM;
std::unique_ptr<SurroundWidget> SurrAVM;

namespace {
    
    namespace {
   #define eglCheckGlError(x) \
        { \
        GLint error; \   
           for (error = glGetError(); error; error = glGetError()) { \
            fprintf(stderr, "func:%s line %d GL: after %s() glError (0x%x)\n", __FUNCTION__,__LINE__,x, error); \
           } \
        } 
    
    }


    void Dump2File(const std::string& file, uint8_t* data, uint32_t size)
    {
    
      FILE* pFile;
      pFile = fopen(file.c_str(), "w");
      if (pFile != NULL) {
        fwrite(data, 1, size, pFile);
        fclose(pFile);
      }
    }

unsigned char ClipValue(unsigned char x, unsigned char min_val, unsigned char max_val) {
    if (x > max_val) {
        return max_val;
    } else if (x < min_val) {
        return min_val;
    } else {
        return x;
    }
}


    

}
static const GLfloat gSgxRender1x1_triangleVertices_fullscreen[] = {
        -1.0f, 1.0f, 0.0f,
        -1.0f, -1.0f, 0.0f,
        1.0f, -1.0f, 0.0f,
        1.0f, 1.0f, 0.0f
 };


GLuint *textureIds ;


#define ENABLE_TEST
int SgxRenderAVM_setup(SgxRenderAVM_Obj *pObj)
{
#ifdef ENABLE_TEST
    SgxRender1x1_setup(&pObj->render1x1Obj);
int numTextures = 1;

   // init il, multi init is ok
   ilInit();
   
   /* create and fill array with DevIL texture ids */
   ILuint* imageIds = new ILuint[numTextures];
   ilGenImages(numTextures, imageIds); 

   /* create and fill array with GL texture ids */
   textureIds = new GLuint[numTextures];
   glGenTextures(numTextures, textureIds); /* Texture name generation */
       eglCheckGlError("glGenTextures");
   printf("numTextures %d \n",numTextures);


   /* get iterator */


       ilBindImage(imageIds[0]); /* Binding of DevIL image name */
       ilEnable(IL_ORIGIN_SET);
       ilOriginFunc(IL_ORIGIN_LOWER_LEFT); 
       ILboolean success = ilLoadImage((ILstring)"/opt/vision_sdk/avm_qt_app_res/cs11/LK.JPG");
       if (success) 
       {
       printf("load jpg ok \n");
           /* Convert image to RGBA */
           ilConvertImage(IL_RGBA, IL_UNSIGNED_BYTE); 
             printf("convert ok \n");
           /* Create and load textures to OpenGL */
           glBindTexture(GL_TEXTURE_EXTERNAL_OES, textureIds[0]); 
            eglCheckGlError("glBindTexture");
           glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, ilGetInteger(IL_IMAGE_WIDTH),
               ilGetInteger(IL_IMAGE_HEIGHT), 0, GL_RGBA, GL_UNSIGNED_BYTE,
               ilGetData());
           printf("glTexImage2D ok \n");


          #ifdef YUV_ENABLE
           std::vector<uint8_t>yuv(0,ilGetInteger(IL_IMAGE_WIDTH)*ilGetInteger(IL_IMAGE_HEIGHT)*2);
             RGB2YUV420P(ilGetData(),ilGetInteger(IL_IMAGE_WIDTH),ilGetInteger(IL_IMAGE_HEIGHT),yuv.data());
             glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, ilGetInteger(IL_IMAGE_WIDTH),
                 ilGetInteger(IL_IMAGE_HEIGHT), 0, GL_LUMINANCE, GL_UNSIGNED_BYTE,
                 yuv.data());
             #endif
          //Dump2File("LK.JPG.RGBA",ilGetData(),ilGetInteger(IL_IMAGE_WIDTH)*ilGetInteger(IL_IMAGE_HEIGHT)*4);
           printf("JPEG header %x %x %x %x \n",ilGetData()[0],ilGetData()[1],ilGetData()[2],ilGetData()[3]);

           
           eglCheckGlError("glTexImage2D");
           glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_NEAREST); 
           glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

           

             eglCheckGlError("glTexImage2D");
             printf("load ok \n");
         }else
          {

                printf("failed to load jpg file to texture \n");


          }




  return 0;
}





void SgxRenderAVM_renderFrame(SgxRenderAVM_Obj *pObj, System_EglWindowObj *pEglWindowObj, GLuint texYuv[], UInt16 numTex )
{
    UTILS_assert(numTex==4);
    printf(" textureIds[0] %d \n", textureIds[0]);
     SgxRender1x1_renderFrame1x1(&pObj->render1x1Obj, pEglWindowObj, gSgxRender1x1_triangleVertices_fullscreen,  textureIds[0]);

    
}

(if replace the textureids[0] with texYuv[0] then the camera image can come up on screen.)

Hi,

Yes, the TI link is also relevant here. As I mentioned earlier, RTT (Render To Texture) is a standard technique long in practice for Graphics. I did not mention the TI link in earlier posts because the reference to PBuffers is sub-optimal for newer generation GPUs.

I assume the question is answered. We will mark the thread closed. Please open a new thread or repost if you have any related questions.

Regards,

Anand

HI：

  Reopen for further discussion.

  we finished the render to texture code and run on visionSDK by sgxFrmcpy. please check the code in attachment.

0020.sgxRenderAVM.cpp8358.TRenderToTexture.cpp

The problem right now, the render to texture can not be bind.

func:SgxRender1x1_renderFrame1x1 line 235 GL: after glBindTexture() glError (0x502)

and the source code is in sgxRender1x1.c

void SgxRender1x1_renderFrame1x1(SgxRender1x1_Obj *pObj, System_EglWindowObj *pEglWindowObj, const GLfloat *vertices, GLuint texYuv)
{
    glVertexAttribPointer(pObj->vPositionHandle, 3, GL_FLOAT, GL_FALSE, 0, vertices);
    eglCheckGlError("glVertexAttribPointer");
    glEnableVertexAttribArray(pObj->vPositionHandle);
    eglCheckGlError("glEnableVertexAttribArray");

    glVertexAttribPointer(pObj->vTexHandle, 2, GL_FLOAT, GL_FALSE, 0, gSgxRender1x1_texCoords);
    eglCheckGlError("glVertexAttribPointer");
    glEnableVertexAttribArray(pObj->vTexHandle);
    eglCheckGlError("glEnableVertexAttribArray");

    glUniform1i(pObj->yuvTexSamplerHandle, 0);
    eglCheckGlError("glUniform1i");
    glBindTexture(GL_TEXTURE_EXTERNAL_OES, texYuv);
    eglCheckGlError("glBindTexture");

    glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
    eglCheckGlError("glDrawArrays");
}

Hi,

As mentioned in earlier post:

If you face any issues when you start using RTT, please provide a standalone self contained OpenGLES application like the following:
git.ti.com/.../kmscube

You can run the kmscube application on either PSDKLA (with omapdrm) or VisionSDK (with virtualdrm).

Regards,
Anand

Hi Anand：
It will take much time to move to that repo and it can't prove anything.
we do not have the experience with virtualdrm.
And also consider the difference between kmscube and visionSDK sgxFrmCpy that:
1. sgxFrmCpy use GL_TEXTURE_EXTERNAL_OES
2. the shader program use "samplerExternalOES"

we can't see any connection of the kmscube with this issue.

and We're wondering does visionSDK have any sample to display picture by OpenGL?

Hi,

The main intent is to create an OpenGLES application that can be built separately without VisionSDK. We need to have a sample application that can reproduce the problem on TI EVM.

This problem is specific to Graphics. For that, VisionSDK components are unnecessary. kmscube is an example OpenGLES application that can be built standalone. We would like you to give us an example application that is similar to kmscube that can reproduce the problem.

Regards,
Anand

Hi  Anand：

   please share the document how to cross build the kmscue to run on tda2px.

Now we configure like:

./autogen.sh  CC=/mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ --with-sysroot=/mnt/workshop/GK-APA/apa-tda2/dist/apa/targetfs/ --host --host-alias=arm

but got the error as:

checking if /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ supports -fno-rtti -fno-exceptions... yes
checking for /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ option to produce PIC... -fPIC -DPIC
checking if /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ PIC flag -fPIC -DPIC works... yes
checking if /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ static flag -static works... yes
checking if /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ supports -c -o file.o... yes
checking if /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ supports -c -o file.o... (cached) yes
checking whether the /mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ linker (/mnt/workshop/GK-APA/apa-tda2/base/ti_vision_sdk_v3.4/ti_components/os_tools/linux/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/bin/ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... no
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... no
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for --host-alias=arm-pkg-config... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for DRM... yes
checking for GBM... no
configure: error: Package requirements (gbm) were not met:

No package 'gbm' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables GBM_CFLAGS
and GBM_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
configure: WARNING: cache variable ac_cv_host contains a newline

Thanks

Hi,
I used this steps to build kmscube with Vision-SDK's targetfs. Can you please try this and let me know.

git clone git://git.ti.com/glsdk/kmscube.git
cd kmscube

export PSDKLA=/home/PROCESSOR_SDK_VISION_03_04_00_00/targetfs
export PATH=/home/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin:$PATH
export PKG_CONFIG_PATH=$PSDKLA/usr/lib/pkgconfig
export DRM_CFLAGS="-I$PSDKLA/usr/include -I$PSDKLA/usr/include/libdrm/ -I$PSDKLA/usr/include/omap"
export DRM_LIBS="-L$PSDKLA/usr/lib -L$PSDKLA/lib -lattr -ldrm -ldrm_omap"
export GBM_CFLAGS=-I$PSDKLA/usr/include/gbm
export GBM_LIBS="-L$PSDKLA/usr/lib -lgbm"
export LIBUDEV_CFLAGS=-I$PSDKLA/usr/include
export LIBUDEV_LIBS="-L$PSDKLA/usr/lib -ludev"

./autogen.sh --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf --prefix=$INSTALL_PATH --with-sysroot=$PSDKLA

cp targetfs/lib/libudev* targetfs/usr/lib/

make

Please check if you can build kmscube with this setting.

TI has not tested kmscube with vDRM since this is a KMS application and only wayland client applications like simple-egl have been tested.
TI will make changes if anything is required to run this on VISION_SDK setup and update you.

Thanks
Ramprasad

Hi:

 Add comments:

 1. This issues related with  which we want to attach the car picture(JPG) to 3D car model.

2. we'are working on VisionSDK with sgxfrmcpy plugin for rendering.

Ramprasad said:

How are you planning to decode jpeg to YUV.? IVAHD can decode and output NV12 which can be attached as a texture. This has been demonstrated in display-kmscube example of omapdrmtest

 Yes, it's the another way to attach as a texture, we already have the software code that convert JPG to ARGB and ARGB to YUV

Ramprasad said:

Are you able to build kmscube with VisionSDK?

 we'are working on it.

Will update ASAP

Thanks.