Sliced-mode JPEG Encoding on DM355

Gennadiy Kiryukhin

I need to implement a slice-mode JPEG encoding on DM355. The input data frame 2592x1944 is in Bayer format. IPIPE will be used to generate YUV slices from the Bayer data. Later each slice will be fed into the encoder.

My problem is that I cannot figure out the correct way of using slice-mode. Example in the JPEG encoder user guide (in the imgenc\docs\ folder of the codec) is very bad. It is unclear how the input data buffer is specified for the first call to process function.

I tried to compress an image but I the first call to process returns an error.

The entire image corresponds to 39366 MCUs, which corresponds to 128 pixels (256 bytes of YUV) per MCU. In my code, I have defined the slice size to be (2592/16 * 2) * 8 = 2592 MCUs which if I understand correctly corresponds to 128 lines.

dynamicparams.size = sizeof(dynamicparams);

dynamicparams.numAU = XDM_DEFAULT; // number of Access units to encode.

// (if want to feed the data in chunks to save buffer space)

dynamicparams.inputChromaFormat = XDM_YUV_422ILE; // format of the input data. see enum XDM_ChromaFormat for all formats

dynamicparams.inputHeight = height;

dynamicparams.inputWidth = width;

dynamicparams.captureWidth = 0; // set to zero to use image width. don't know what it's for.

dynamicparams.generateHeader = XDM_ENCODE_AU; // don't knwo what it's for but this is what dvtb used.

dynamicparams.qValue = Q; // compress JPEG qith quality = 75%

memset(&status, 0, sizeof(status));

status.size = sizeof(status); // define the size of status structure

res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE params set OK\n");

res = IMGENC1_control(ieh, XDM_GETSTATUS, &dynamicparams, &status);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE get status OK\n");

printf("Total numAU=%d\n", (int)status.totalAU);

totalAUslice = status.totalAU;

numAUslice = width/16*2 * 8; // multiple value corresponds to 16 lines, we process x8 of that , i.e. 128 lines

dynamicparams.size = sizeof(dynamicparams);

dynamicparams.numAU = numAUslice;

res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE params set OK to slice mode\n");

Then the slice is encoded using the following code:

int res;

IJPEGENC_InArgs inargs;

IJPEGENC_OutArgs outargs;

if (totalAUslice - currAUslice < numAUslice * 2)

{

if (totalAUslice - currAUslice != numAUslice)

{

dynamicparams.size = sizeof(dynamicparams);

numAUslice = totalAUslice - currAUslice;

dynamicparams.numAU = numAUslice;

res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE params set OK to slice mode\n");

}

inargs.sliceNum = -1;

}

else

inargs.sliceNum = sliceNumber;

inargs.imgencInArgs.size = sizeof(inargs);

outargs.imgencOutArgs.size = sizeof(outargs);

outargs.imgencOutArgs.bytesGenerated = 0;

inbufdesc.numBufs = 1;

inbufdesc.descs[0].buf = ptr; // pointer to 2592 x 128 x 2 of YUV data

inbufdesc.descs[0].bufSize = jpegEncWidth*jpegEncSliceHeight*2; // i.e., 2592*128*2

outbufdesc.numBufs = 1;

outbufdesc.descs[0].buf = tmpbuf; // temp buffer

outbufdesc.descs[0].bufSize = jpegEncWidth*jpegEncSliceHeight*2; // i.e., 2592*128*2

res = IMGENC1_process(ieh, &inbufdesc, &outbufdesc, (IIMGENC1_InArgs*)&inargs, (IIMGENC1_OutArgs*)&outargs) ;

if (res != IMGENC1_EOK)

{

printf("Error encoding image\n");

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

currAUslice += numAUslice;

sliceNumber++;

IMGENC1_process return an error.

What could be the problem?

Is there an example or maybe an explanation of the slice-mode method?

over 14 years ago

0 Akshay Panday over 14 years ago

Expert 1640 points

Gennadiy,

What is the version of the JPEG Encoder you are using? The last release has a better explanation of slice based encoding.

Also, I see that you are doing only one process call. The correct way to do slice based encode, is to do one process call which will encode the header and a series of process calls in a loop to encode the data. In dvtb, this is carried out as follows:

imgencDynamicParams.generateHeader = XDM_GENERATE_HEADER; //will encode only header

IMGENC1_control(SETPARAMS)

IMGENC1_process() //slice #1 - header

IMGENC1_control(GETSTATUS)

imgencDynamicParams.generateHeader = JPEGENC_TI_ENCODE_AU_NOHEADER; //will encode only AUs

numAU = ienc1Status.numAU; // the GETSTATUS call will update this value of numAU. In case we give an arbitrary value, it will extended till end of the line and returned in this parameter. This is the value that should be used for calculating buffer offsets, etc.

IMGENC1_control(SETPARAMS)

while(imgencOutArgs.currentAU < imgencStatus.totalAU)

{

IMGENC1_process()

ienc1InArgs.sliceNum++;

/*update pointers*/

if(!(imgencOutArgs.currentAU < (imgencStatus.totalAU - numAU)))

{ //last slice

ienc1InArgs.sliceNum = -1;

imgencDynamicParams.numAU = imgencStatus.totalAU - imgencOutArgs.currentAU; //readjust numAU for last slice

}

Regards,

Akshay

0 Gennadiy Kiryukhin over 14 years ago in reply to Akshay Panday

Expert 2800 points

I have defined three functions. StartJPEG(), SliceJPEG(), and CloseJPEG(). Firs function prepares JPEG encoder for "slicing". The first part (yellow, see above) of my code is in that function. The second function is called from a loop and is used to compress slices generated by the IPIPE. This function includes the second part (light green, see above) of my code.

My JPEG encoder is from SDK 3.01. JPEG Encoder document is from May 2010 (literature number TBD).

I do seem to understand the concept of slice-based JPEG encoding. I just don't understand the specifics of what is needed.

Why is there a ringbuffer in the slice mode code sample? How come it is not used anywhere else in the code? The document says that slice-mode and ring-buffer are both automatically enabled. When? All the time? How do I disable the ring buffer? How come first slice (#0) process does not have any input data buffers specified? How much data should be available for each process() call?

0 Gennadiy Kiryukhin over 14 years ago in reply to Akshay Panday

Expert 2800 points

Akshay,

You were saying that first call to process should encode the headers only. That means that no data buffers need be provided for the process() function, right? If that is correct what values for the parameters to the process function should I use?

Thank you.

0 Gennadiy Kiryukhin over 14 years ago in reply to Gennadiy Kiryukhin

Expert 2800 points

I have solved the error problem (encoder-and-friends were closed/deleted during JPEG encoder preparation because open("filename") failed)...

However, I am still trying to figure out how to properly define parameters for control() and process() calls in slice-mode encoding.

Update:

Now, the first process() call locks (never returns).

Update, Part 2:

Well, I changed something in the code and the process no longer locks but returns with OK. However, the program "Segmentation-faults" somewhere between printf("some text"); return 0; and printf("") in the caller function. Long story short, something gets really messed up. I think it is because the encoder is trying to process the entire frame instead of a small slice thus overrunning the buffer and corrupting something in the memory.

Here is how I tell the encoder that I an going to use slice-mode encoding:

printf("Total numAU=%d\n", (int)status.totalAU);

totalAUslice = status.totalAU;

numAUslice = width/16*2 * 8; // multiple value corresponds to 16 lines, we process x8 that , i.e. 128 lines

dynamicparams.size = sizeof(dynamicparams);

dynamicparams.numAU = numAUslice;

printf("numAUslice = %d\n", numAUslice);

memset(&jstatus, 0, sizeof(jstatus));

jstatus.imgencStatus.size = sizeof(jstatus);

res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, (IMGENC1_Status*)&jstatus);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)jstatus.imgencStatus.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE params set OK to slice mode, numAU=%d\n", jstatus.numAU);

I get the following printout:

Total numAU=39366

numAUslice = 2592

CE params set OK to slice mode, numAU=0

For some reason the function does not take the request to use slice-mode. Otherwise, the numAU in the status would be something other than 0.

In the previous setparam call I define my dynamicparams as follows:

dynamicparams.size = sizeof(dynamicparams);
dynamicparams.numAU = XDM_DEFAULT; // number of Access units to encode.
dynamicparams.inputChromaFormat = XDM_YUV_422ILE; // format of the input data. see enum XDM_ChromaFormat for all formats
dynamicparams.inputHeight = height;
dynamicparams.inputWidth = width;
dynamicparams.captureWidth = 0; // set to zero to use image width. don't know what it's for.
dynamicparams.generateHeader = XDM_GENERATE_HEADER;
dynamicparams.qValue = Q; // compress JPEG qith quality = 75%

I would expect the content of the structure remain the unchanged so that the next call (listed above) would reuse all the values but numAU.

If it helps, later in the code I request the required buffer sizes and get the following:

Input buffers needed: 1

Input buffer 0 size: 10077696

Output buffers needed: 1

Output buffer 0 size: 10077696

0 Akshay Panday over 14 years ago in reply to Gennadiy Kiryukhin

Expert 1640 points

Gennadiy,

So for the 1st (header) process call, your dynamicparams.numAU is XDM_DEFAULT?

You should set it to 'numAUslice' at the first SETPARAMS call itself.

-Akshay

0 Gennadiy Kiryukhin over 14 years ago in reply to Akshay Panday

Expert 2800 points

Before the first process call, numAU is set to numAUslice. After parameter is set, I read the numAU from the status and it is still 0. Here is the whole function that initializes the encoder.

Engine_Handle ceh;
Engine_Error cerr;
IMGENC1_Handle ieh;
IIMGENC1_Params params;
IIMGENC1_DynamicParams dynamicparams;
IMGENC1_Status status;
IJPEGENC_Status jstatus;
FILE * jpegFile;
XDM1_BufDesc inbufdesc;
XDM1_BufDesc outbufdesc;
int totalAUslice;
int numAUslice;
int currAUslice;
int sliceNumber;
int jpegEncWidth;
int jpegEncHeight;
int jpegEncSliceHeight;

int StartJPEG(int width, int height, int Q, int bufsize, char * filename)
{
    int res, i;

    params.size = sizeof(params);

    // init Code Engine (run-time variables)
    CERuntime_init();

    // open codec engine
    ceh = Engine_open("encode", NULL, &cerr);
    if (!ceh)
    {
        switch(cerr)
        {
            case Engine_EOK: printf("Open success\n"); break;
            case Engine_EEXIST: printf("Open name does not exist\n"); break;
            case Engine_ENOMEM: printf("Open can't allocate memory\n"); break;
            case Engine_EDSPLOAD: printf("Open can't load the DSP\n"); break;
            case Engine_ENOCOMM: printf("Open can't create a comm connection to DSP\n"); break;
            case Engine_ENOSERVER: printf("Open can't locate the server on the DSP\n"); break;
            case Engine_ECOMALLOC: printf("Open can't allocate communication buffer\n"); break;
            default: printf("Open error undefined\n");
        }
        CERuntime_exit();
        return -1;
    }

    params.size = sizeof(params);
    params.maxHeight = height;
    params.maxWidth = width;
    params.maxScans = XDM_DEFAULT;
    params.dataEndianness = XDM_BYTE;
    params.forceChromaFormat = XDM_DEFAULT;
    // create JPEG encoder instance
    ieh = IMGENC1_create(ceh, "jpegenc", &params);
    if (!ieh)
    {
        printf("Failed to create jpegenc instance\n");
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }

    dynamicparams.size = sizeof(dynamicparams);
    dynamicparams.numAU = XDM_DEFAULT;     // number of Access units to encode.
                        // (if want to feed the data in chunks to save buffer space)
    dynamicparams.inputChromaFormat = XDM_YUV_422ILE;    // format of the input data. see enum XDM_ChromaFormat for all formats
    dynamicparams.inputHeight = height;
    dynamicparams.inputWidth = width;
    dynamicparams.captureWidth = 0; // set to zero to use image width. don't know what it's for.
    dynamicparams.generateHeader = XDM_GENERATE_HEADER; //ENCODE_AU; // don't knwo what it's for but this is what dvtb used.
    dynamicparams.qValue = Q; // compress JPEG qith quality = 75%

    memset(&status, 0, sizeof(status));
    status.size = sizeof(status); // define the size of status structure

    res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);
    if (res != IMGENC1_EOK)
    {
        printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);
        IMGENC1_delete(ieh);
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }
    printf("CE params set OK\n");

    res = IMGENC1_control(ieh, XDM_GETSTATUS, &dynamicparams, &status);
    if (res != IMGENC1_EOK)
    {
        printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);
        IMGENC1_delete(ieh);
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }
    printf("CE get status OK\n");
    printf("Total numAU=%d\n", (int)status.totalAU);

    totalAUslice = status.totalAU;
    numAUslice = width/16*2 * 8; // multiple value corresponds to 16 lines, we process x8 of that , i.e. 128 lines
    dynamicparams.size = sizeof(dynamicparams);
    dynamicparams.numAU = numAUslice;
    printf("numAUslice = %d\n", numAUslice);

    memset(&jstatus, 0, sizeof(jstatus));
    jstatus.imgencStatus.size = sizeof(jstatus);

    res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, (IMGENC1_Status*)&jstatus);
    if (res != IMGENC1_EOK)
    {
        printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)jstatus.imgencStatus.extendedError);
        IMGENC1_delete(ieh);
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }
    printf("CE params set OK to slice mode, numAU=%d\n", jstatus.numAU);

    memset(&status, 0, sizeof(status));
    status.size = sizeof(status); // define the size of status structure
    res = IMGENC1_control(ieh, XDM_GETBUFINFO, &dynamicparams, &status);
    if (res != IMGENC1_EOK)
    {
        printf("Error (%d), Extended Error (%d) in image Encoder Control: getbufinfo\n", res, (int)status.extendedError);
        IMGENC1_delete(ieh);
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }

    printf("Input buffers needed: %d\n",(int)status.bufInfo.minNumInBufs);
    for(i=0;i<status.bufInfo.minNumInBufs;i++)
        printf("Input buffer %d size: %d\n",i, (int)status.bufInfo.minInBufSize[i]);

    printf("Output buffers needed: %d\n",(int)status.bufInfo.minNumOutBufs);
    for(i=0;i<status.bufInfo.minNumOutBufs;i++)
        printf("Output buffer %d size: %d\n",i, (int)status.bufInfo.minOutBufSize[i]);

    sliceNumber = 0;

    jpegEncSliceHeight = 128;

    currAUslice = 0;

    jpegFile = fopen(filename,"w");
    if (!jpegFile)
    {
        IMGENC1_delete(ieh);
        Engine_close(ceh);
        CERuntime_exit();
        return -1;
    }

    return 0;
}

0 Gennadiy Kiryukhin over 14 years ago in reply to Gennadiy Kiryukhin

Expert 2800 points

I got it working.

In the code above I have added one call to process() with input buffer set to NULL and output to a temp buffer.

memset(&jpegInargs, 0, sizeof(jpegInargs));

memset(&jpegOutargs, 0, sizeof(jpegOutargs));

jpegInargs.imgencInArgs.size = sizeof(jpegInargs);

jpegInargs.sliceNum = 0;

jpegOutargs.imgencOutArgs.size = sizeof(jpegOutargs);

jpegOutargs.imgencOutArgs.bytesGenerated = 0;

inbufdesc.numBufs = 1;

inbufdesc.descs[0].buf = NULL; // no input buffer (generating header only)

inbufdesc.descs[0].bufSize = 0;

outbufdesc.numBufs = 1;

outbufdesc.descs[0].buf = tmpbuf;

outbufdesc.descs[0].bufSize = jpegEncWidth*jpegEncSliceHeight*2;

res = IMGENC1_process(ieh, &inbufdesc, &outbufdesc, (IIMGENC1_InArgs*)&jpegInargs, (IIMGENC1_OutArgs*)&jpegOutargs) ;

This call generats 613 bytes of header data that gets saved to the file.

fwrite(tmpbuf, jpegOutargs.imgencOutArgs.bytesGenerated, 1, jpegFile);

After that I change the parameter to create AU only.

dynamicparams.generateHeader = ENCODE_AU;
res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);

To compress the image, I call process() for each slice (2592x128). the output data is saved into a file:

int SliceJPEG(void * ptr, void * tmpbuf)

{

int res;

if (totalAUslice - currAUslice < numAUslice)

{

dynamicparams.size = sizeof(dynamicparams);

numAUslice = totalAUslice - currAUslice;

dynamicparams.numAU = numAUslice;

res = IMGENC1_control(ieh, XDM_SETPARAMS, &dynamicparams, &status);

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)status.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("CE params set OK to slice mode\n");

jpegInargs.sliceNum = -1;

}

else

jpegInargs.sliceNum = sliceNumber;

jpegOutargs.imgencOutArgs.bytesGenerated = 0;

inbufdesc.numBufs = 1;

inbufdesc.descs[0].buf = ptr;

inbufdesc.descs[0].bufSize = jpegEncWidth*jpegEncSliceHeight*2;

outbufdesc.numBufs = 1;

outbufdesc.descs[0].buf = tmpbuf;

outbufdesc.descs[0].bufSize = jpegEncWidth*jpegEncSliceHeight*2;

// printf("Calling process\n");

res = IMGENC1_process(ieh, &inbufdesc, &outbufdesc, (IIMGENC1_InArgs*)&jpegInargs, (IIMGENC1_OutArgs*)&jpegOutargs) ;

if (res != IMGENC1_EOK)

{

printf("Error (%d), Extended Error (%d) in image Encoder Control:setparams\n", res, (int)jpegOutargs.imgencOutArgs.extendedError);

IMGENC1_delete(ieh);

Engine_close(ceh);

CERuntime_exit();

return -1;

}

printf("Encoded slice %d: %d bytes\n", sliceNumber, (int)jpegOutargs.imgencOutArgs.bytesGenerated);

currAUslice += numAUslice;

sliceNumber++;

fwrite(tmpbuf, jpegOutargs.imgencOutArgs.bytesGenerated, 1, jpegFile);

return currAUslice*128/jpegEncWidth;

}

0 Gennadiy Kiryukhin over 14 years ago in reply to Gennadiy Kiryukhin

Expert 2800 points

Apparently, JPEG encoder runs without any errors. However, the output jpeg file cannot be viewed or opened in a photo editor.

I dumped YUV buffer to a file. The data looks correct. It is in U Y V Y format, i.e. first byte is U, second is Y, third is V, fourth is Y. Is that a correct format for JPEG encoder? My input data format is defined as YUV_422ILE.

0 Akshay Panday over 14 years ago in reply to Gennadiy Kiryukhin

Expert 1640 points

Gennadiy,

Yes, that is how UYVY (422ILE) is.

Can you send me your input and output file so that I can verify it here?

-Akshay

0 Gennadiy Kiryukhin over 14 years ago in reply to Akshay Panday

Expert 2800 points

I have attached the YUV data and JPEG output from that data.

4331.yuv-2-jpeg.zip

0 Gennadiy Kiryukhin over 14 years ago in reply to Gennadiy Kiryukhin

Expert 2800 points

Apparently, calling _control() with SETPARAMS does not set the status structure. I was wrong to assume that. Once I made a separate call with GETSTATUS, numAU in extended structure now matched the requested value. So, the encoder does indeed gets set to slice-mode encoding. The image is still unreadable though.

0 Gennadiy Kiryukhin over 14 years ago in reply to Gennadiy Kiryukhin

Expert 2800 points

When encoding the slices, I had to change the encoding mode to JPEGENC_TI_ENCODE_AU_NOHEADER (I had XDM_ENCODE_AU, which is what the example was using).

Second, when encoding slices, I had to start from 0 not 1 (even though I called process once already to generate header information).

After applying those changes, my code started working and I was able to slice-encode a picture.

Here it is:

Thin horizontal lines are because when IPIPEing slicing I was not using overlaps. To fix that I would have to process few extra lines at the end of each slice and JPEG-encode only the desired number.

Not sure why I am getting black border along the bottom and right edges, but I think the border is present in Bayer data too. This may be just due to image sensor initialization.

It is nice, that I can use 1.2Megs (or as little as 324K if using the smallest slice allowed) of buffers instead of 20Meg to compress 5M picture (2592x1944).

0 Akshay Panday over 14 years ago in reply to Gennadiy Kiryukhin

Expert 1640 points

Gennadiy,

Glad you could get it working.

It is indeed a handy feature, but then, we need to achieve a balance between a smaller memory footprint and the overhead that slice mode imposes.

-Akshay

0 Gennadiy Kiryukhin over 14 years ago in reply to Akshay Panday

Expert 2800 points

Akshay,

You are right about the balance. Personally I don't think there is a need to use the minimum slice size as the memory saving becomes insignificant compared to the rest of the system. However, when compressing a picture, often there is no strict deadline (compared to video mode where you need to keep up with the incoming frames). At the same time, a system can be build on 64M of RAM instead of 128M. This can be a big $ saving especially if mobile DDR is used. Also IPIPE and Encoder can run in separate threads with ping-ponging buffers in between. This should improve the throughput.

Thank you for your help.

Processors

Processors forum

Sliced-mode JPEG Encoding on DM355