Resize On Dm365

Joni,

They way I see it, we have your classic case of flexible but complex API vs. simple but less flexible API.  Normally, driver APIs are designed to be more flexible but a little more complex to use; you could write your user application right on top of these driver APIs if you wish, and we did write our demos this way before DMAI existed.  DMAI is simply a layer between the driver API and the application which tries to hide some of the complexity of driver APIs and provide a simpler API for applications to use and hiding some of the complex choices.

We see similar scenerious all over the industry; windows OS is an abstration of the hardware, MS Word abstracts things even further and for a very specific purpose (word processing) when in fact, the hardware is capable of much more (fexibility sacrificed).  Choices need to be made to simplify user interface at the sacrifice of fexibility of what the hardware can do because it would be too difficult for all computer users to learn how to program to the hardware directly (e.g. no os, no word processor...).

Fortunately, embedded systems are simpler than desktop PCs and you do have the source code for the Linux kernel (drivers), DMAI and the demo applications.  The closer you stay to the upper layers of software, the more you will be able to leverage these layers to make your life easier.  If DMAI does not meet your goal but is close, I would probrably try to change it to support your required scenerious. If DMAI is way off in meeting your goals, then I may consider talking to the driver layer directly (what DMAI does for you) and bypassing DMAI altogether.  I doubt you will need to modify the drivers, but you do have the source code if you need to do that as well. 

Hi

I went into the code (Capture.c in of DMAI) and there is a flag "captureChainMode" only for DM365.

Is it recommended to set it to false and to do the resizing and the chroma conversion in my application?

Is there others considerations in this case?

Thanks

Joni

Hey Joni,
                 There are two options available to you - the resizer hardware supports outputting two different resolution images from a single source, and it can be configured in either 'chain' mode (where the source comes directly from the VPFE raw stream) or in single-shot-mode, where the source comes from SDRAM. The current driver doesn't support dual resizer outputs in chain mode i think (i've had to add support for this) but if you do the DDR mode, all of this is entirely possible. You can then setup the resizer to output two different outputs, and then trigger the resizer each and every time with your video input in ddr memory, and two video output locations in memory as well.

Jerry

Hi Joni,

I confirm that you can use the resizer multiple times using DMAI,  but you must disable chaining in DMAI's source code, as already suggested. Also you have to configure IPIPE in One-shoot mode (kernel boot parameter). Then you can use DMAI's resize API to do what need. I didn't know that the resize can do multiple jobs while chained...

The DaVinciLSP_02_10_00_14.tar.gz has resizer A&B support already, what you need is a patch to DMAI.

Put the following code into Resize.c, and recompiled DMAI

When initialization,  call Resizer_B_config() after Resizer_continous_config() was called

Then call Capture_get(), get Resizer_A output followed by Resizer_B output

/******************************************************************************
 * Initialize & configure resizer B in on-the-fly (continous) mode
 ******************************************************************************/
int Resizer_B_config(int rsz_fd, unsigned int width, unsigned int height)
{
    unsigned int user_mode;
    struct rsz_channel_config rsz_chan_config;
    struct rsz_continuous_config rsz_cont_config;

    user_mode = IMP_MODE_CONTINUOUS;
    if(rsz_fd <= 0) {
        Dmai_err0("Cannot use resize device \n");
        return NULL;
    }
    
    bzero(&rsz_cont_config, sizeof(struct rsz_continuous_config));
    rsz_chan_config.oper_mode = user_mode;
    rsz_chan_config.chain = 1;
    rsz_chan_config.len = sizeof(struct rsz_continuous_config);
    rsz_chan_config.config = &rsz_cont_config;

    if (ioctl(rsz_fd, RSZ_G_CONFIG, &rsz_chan_config) < 0) {
        Dmai_err1("Error in getting channel configuration from resizer (%s)\n",
            strerror(errno));
        return Dmai_EFAIL;
    }
       
    /* we can ignore the input spec since we are chaining. So only
       set output specs */

   
    rsz_cont_config.output2.width = width;
    rsz_cont_config.output2.height= height;
    rsz_cont_config.output2.pix_fmt = IPIPE_YUV420SP;
    rsz_cont_config.output2.enable = 1;
    rsz_chan_config.len = sizeof(struct rsz_continuous_config);
    rsz_chan_config.config = &rsz_cont_config;
    if (ioctl(rsz_fd, RSZ_S_CONFIG, &rsz_chan_config) < 0) {
        Dmai_err1("Error in setting resizer configuration (%s)\n",
            strerror(errno));
        return Dmai_EFAIL;
    }
    Dmai_dbg0("Resizer initialized\n");
    return Dmai_EOK;
}

The ploblem is, when I use Resizer A&B simultaneously, I get jittering images, anybody can help?

It works for me, but I have to increase size of input buffer for second resizer output. The output is normally stored after first resizer output.

bufSize += 640*480*2; // add maximum size for second buffer

I found also the limitation about max size 640x480 for second rescaller output. E.g. in davinci_vpfe.c - there is 640x480 constant and there are also in other sources.

Thank to Thomas Lee for the post about simple enhancement of DMAI for using second resizer.

hi,I just want to resize the D1 to the cif format. But when I do this ,the error displayed as follows:

root@192.168.1.180:/mnt# ./myencoder 

davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:108

davinci_previewer davinci_previewer.2: ipipe_set_preview_config

davinci_previewer davinci_previewer.2: ipipe_set_preview_config

vpfe ccdc capture vpfe ccdc capture.1: IMP chained 

vpfe ccdc capture vpfe ccdc capture.1: Resizer present

vpfe ccdc capture vpfe ccdc capture.1: hpitch = 720, vpitch = 576, bpp = 1

vpfe ccdc capture vpfe ccdc capture.1: hpitch = 720, vpitch = 576, bpp = 1

Starting ccdc_config_ycbcr...<7>

starting ccdc_reset...<7>

End of ccdc_reset...<7>

Starting ccdc_setwin...<7>ipipe_set_resizer, resizer - A enabled

DavinciDisplay DavinciDisplay.1: Before finishing with S_FMT:

layer.pix_fmt.bytesperline = 736,

 layer.pix_fmt.width = 720, 

 layer.pix_fmt.height = 576, 

 layer.pix_fmt.sizeimage =635904

DavinciDisplay DavinciDisplay.1: pixfmt->width = 720,

 layer->layer_info.config.line_length= 736

davinci_resizer davinci_resizer.2: 

 mode doesn't allow multiple instances

Failed to create hResize: Device or resource busy

Can you give me some ways for this problem?

weidong shi, You probably trying to open resizer more than once.

"mode doesn't allow multiple instances" shown when more instances are going to be opened (/drivers/char/imp_resizer.c)

Please open it at most once.

weidong shi said:

hi,I just want to resize the D1 to the cif format. But when I do this ,the error displayed as follows:

Can you give me some ways for this problem?

Running the encode demo which comes with the DVSDK with the proper options on top of DM365 EVM should do the resizing you need.  No need to modify code or rebuild anything.  (./encode -y 1 -r 352x288 -v test.mpeg4 ); the -y option is your input resolution, the -r defines your output resolution, if there is a difference resizer will be used to resize accordingly, see accompanying encode.txt for more details.

Hi, Thank you for your reply. 

But actually I want to encode the stream to D1 format and meanwhile save it to disk as cif format .

So maybe I need to resize the video stream two times.

What should I do?

Could you give me some help?

How can I get Resizer_A output followed by Resizer_B output?

Could you explain to me clearly?

Thank you sincerely.

Hi Weidong Shi,

DM365 has two resizers RSZA and RSZB. Both of them work simultaneously to give output of different resolutions. LSP drivers for resizer support enabling both RSZA and RSZB.

DVSDK demos already support RSZB output, but if you want to change your code, please use refer to the following example code, where i am enabling RSZB (output2) in CONTINUOUS mode for output resolution of 352x288. The same is applicable for Single Shot mode also.

/* rsz_cont_config.output2.enable = 1;
 rsz_cont_config.output2.width = 352;
 rsz_cont_config.output2.height = 288; 
 rsz_cont_config.output2.pix_fmt = IPIPE_YUV420SP;
// rsz_cont_config.output2.pix_fmt = IPIPE_UYVY; */

Please note that the output of RSZB follows just after RSZA, so address for RSZB is not configured by application but is configured by kernel driver itself.

Regards,

Anshuman

Dear Anshuman:

     Thank you for your support.

     I do this configure in Resizer_continous_config(void) in Resizer.c and rebuild Dmai.

    I suppose that when I call Capture_get(hCapture, &hCapBuf) in my application.

   Do you mean that the RSZA output and RSZB output are both in hCapBuf ?

   How can I get the second buffer output.

 Regards,

 Weidong.Shi

Dear Weidong,

Yes, both resizer output are in the same buffer. RSZA output is followed by RSZB. You can get RSZB ouput by adding (RSZA_LineOffset*RSZA_Height*2) to get RSZB output when the output format is UYVY.

Regards,

Anshuman

Yes, you would have to do a proper allocation of memory buffers. Also, please make sure you allocate big enough buffers so that the start address for Y and UV output data for each resizer are 32 byte aligned.

If you refer to the DMAI apps, there are specific functions like calcLineLength, calcDimenstions etc. available to calculate the size.

Regards,

Anshuman

Dear Anshuman:

          I am really sorry to interrupt you for some problems.

        I do this as you describe but still have some problems.

              int hbufsize;

          hbufsize = 736*576*3/2 + 368*288*3/2;

            hCapBuf = Buffer_create(hbufsize, &bAttrs);

        /* Above is  a initialization  for process*/

        /*The Main loop shows as follows*/

        while(1){

            if (Capture_get(hCapture, &hCapBuf) < 0) {

            printf("Failed to get capture buffer\n");

            exit(-1);

                }

        /* Get a buffer from the display device */

           if (Display_get(hDisplay, &hDisBuf) < 0) {

            printf("Failed to get display buffer\n");

            exit(-1);

}

           printf("hCapBuf:\n");

        Buffer_print(hCapBuf);

Please refer to the documentation of Capture Driver in the link below:

http://software-dl.ti.com/dsps/dsps_registered_sw/sdo_sb/targetcontent/psp/mv_lsp_2_10/02_10_00_14/exports/docs/DaVinciLinux_VPFECaptureDriverUsersGuide_SPRUGP6.pdf

http://software-dl.ti.com/dsps/dsps_registered_sw/sdo_sb/targetcontent/psp/mv_lsp_2_10/02_10_00_14/exports/lsp_psp_02_10_00_14_release_notes.html

If you see in the document, sec 2.2.5.18 describes how the output buffers are available from RSZ-A and RSZ-B.

Even the message "vpfe ccdc capture vpfe ccdc capture.1: Second output present" in your console confirms that RSZ-B is enabled and data is generated.

So this seems to be application related issue, where the buffers are handled. Which DMAi application are you using and modified? Please re-check the flow of your application.

Regards,

Anshuman

Dear juraj:

         I am really confused about how to get buffer after as Thomas Lee described to change code about resize.c.

        I call the function as follows :

     Capture_get(hCapture, &hCapBuf) 

    I suppose the first output and the second resizer output are both in hCapBuf.

  But it actually exist the first output resizer.

  How should I do?

 Could you spare a little time to help me?

Regards

Weidong.Shi 

Weidong.Shi, You are right.

Both images (from first and the second resizer) are in one hCapBuf.

There is first image (736*576) and the second one (368*288). The buffer size must be allocated for these two images before capture process is started - as You wrote 10-12-2009 4:40 AM.

In capture.c - You can store the image into file to verify, how it is stored using call:

SaveRawImage
(
                (unsigned char*)Buffer_getUserPtr(hCapBuf),
                Buffer_getSize(hCapBuf)
);

void SaveRawImage(unsigned char *buf, unsigned int size)
{
    FILE *fp;
    char filename[20];
    fp = fopen("picture.raw","wb");
    fwrite(buf,1,size,fp);
    fclose(fp);
}

When Your color space is ColorSpace_YUV420PSEMI, You should see

Note: picture is scaled down. You can use irfanview - raw view 8 bit to see the same picture.

But most the important ,How do you allocate your buffer size?

I only get the first buffer in  hCapBuf  . I write the your code to my program. But the file only exist the first buffer.

regards

Weidong.Shi

For Weidong.Shi:

I use the encode demo from the dvsdk_demos_2_10_00_17.

In the capture.c there is calculating of bufSize. I added the following line before the bufSize is used for BufTab_create:

bufSize += 640*480*2; // this is maximum size for resizer B

Juraj

Hello all,

The logic that i and Juraj had suggested is the right way to use two resizer output. Now, the question is that what exactly has to be changed in DMAI demo to provide more memory to capture driver. For this i need to get clear answers on the following questions:

1. Which DMAI demo application are you using? we will look into that code accordingly. For different demos, we use either MMAP buffers or USERPTR buffers.

2. What are the code changes done in the regular DMAI demo? This will help us to analyze the changes you are trying to do.

Regards,

anshuman

Hi anshuman,

 When I use  interleave input, such as BT656, I enabled chain mode, and it seems that the resizer  use ONLY ONE FIELD image as input, how can I  use both of  top and bottom field?

Hi, I suppose it can use your arm cpu to resize the second format.

But the defect have too low efficiency .

The code is as follows.  Maybe still have some problems.

/*Resize D1 Buffer_Handle to cif Buffer_Handle*/

int resizer(Buffer_Handle *hCapptr,Buffer_Handle *resizeptr)

{

        Int32 num;

        Int8 *cptr,*rptr;

        int i,j;

        cptr = Buffer_getUserPtr(*hCapptr);

        num = Buffer_getSize(*hCapptr);

        rptr = Buffer_getUserPtr(*resizeptr);

        for(i=0;i<576*3/2;i=i+2)

        {

          for(j=0;j<720;j=j+2)

        {

        *(cptr++) = *(rptr);

        rptr += 2;

        }

        }

        Buffer_setNumBytesUsed(*resizeptr,152064);

        return 0;

}

(1)the DVSDK version is dvsdk_dm365_setuplinux_2_10_01_18.bin，DMAI version is dmai_1_21_00_10；

(2)we use 720x576 to capture video ,no 720P,so framecopy=true,if we set framecopy = false force ,we also only can get Buffer1, the video in buffer2 is mixed and disorderly(we call Resizer_B_config() after Resizer_continous_config() was called）(3)int resizer(Buffer_Handle *hCapptr,Buffer_Handle *resizeptr),we have try this some days ago ,the CPU is to 70%, and cannot achieve 25 frame per second!

It's work perfect for me.

I do it use arm cpu and it can achieve 25 frames

Hi All,

Trying to get this open issue to a closure, we tried to make changes to DVSDK 2.10.01.18 codebase itself and get RSZB output also working. The modified files are attached with the post. Actually, the changes were done on following two vectors:

1. To setup RSZB configuration in dmai/linux/dm365/resize.c

2. Calculation of ouffset for RSZB in dvsdk_demos_2_10_00_1x/dm365/encode/capture.c

We verified using the following command line option on demo.

root# ./encode -v out.264

The connected input is D1from a DVD player into DM365 EVM.

The code will dump the output of Resizer B into a file named "interface.yuv".

Please refer to the following line in capture.c, where we calculate the offset

   fwrite(Buffer_getUserPtr(hCapBuf)+5299200,126720,1,outFile); //Buffer pointer+ D1 size(736*480*1.5) , dumping the size of cif(352 x 240 x 1.5)

Regards,

Anshuman

Thank you for your support.

Thank you very much!

Why the offset is 5299200?

I think it should be 736*480*1.5=529920 not 5299200.

Is it true?

weidong shi said:

I think it should be 736*480*1.5=529920 not 5299200.

Is it true?

Yes, you are right. It should be 529920.

Sorry for the typo error. We had verified code with offset = 529920

Regards,

Anshuman

I covered your code to my program. And change  rsz_cont_config.output2.enable = 1;

        rsz_cont_config.output2.width = 352;

        rsz_cont_config.output2.height = 288;

Meanwhile I change it in capture.c as follows:

    fwrite(Buffer_getUserPtr(hCapBuf)+635904,152064,1,outfile); //635904=736*520*1.5     152064=352*288*1.5

After run it in DM365 EVM, the outfile size is 151552 not 152064?

What have happened?

Hi,dehui

Have you finished your code as Anshuman Saxena posted code?

I do resize as follows.

        memcpy(Buffer_getUserPtr(hCapBuf)+635904,Buffer_getUserPtr(resizeBuf),152064);

My fault.  

Hi weidong shi:

i have get two pictures 720x576 and 360x288 and also decode the pictures streaming to H.264 both.

I not use the framecopy_execute and use memcpy to copy buffer,but before you do this ,you have to do this :

dim.x=0;dim.y=0;dim.width=360;dim.height=288;

BufferGfx_setDime......(hCifBuff,&dim);

memcpy(......)

if you need help can send email to lin_dehui@163.com .

Thanks  Anshuman Saxena、Juraj very much! thank you very much!

Dear pavel:

         I just finished encode two resolutions use rsz_A and rsz_B.

        I suggest you can use arm cpu to resize the third captured frame but it will make your code low efficiency.

Regards

Weidong.Shi

One thing I fail to understand is why the opportunity to use the resizer in one-shoot mode has been neglected. Also DMAI decided to allow only for continuous mode on DM365 and I had to tinker it to revert to one-shoot mode. In this mode you have to reconfigure the resizer every time instead of once for all, but the encoder is basically disconnected from video capture. You can use it when you want, as you want. Do you want to resize the input video into 3 different formats? Then simply configure for res 1, resize, configure for res 2, resize, configure for res 3, resize. It might be slightly less efficient than direct resizing, but it is a lot more flexible.

Yes. I agree to you. I think the biggest advantage is that you can't do some configures when encode one format.

I also confused why resizer don't support in one-shoot mode.

Thank you very much for Weidong.Shi and Marco Braga

I want to use resizer by myself, and I try that:
(1),In DMAI, I set captureChainMode is FALSE in Capture.c
(2),In LSP, I set oper_mode is IMP_MODE_SINGLE_SHOT in dm365_ipipe.c

But when i use the DMAI API(Capture_create) to create capture, there is a error:
Pixel format 0x3231564e not supported. Received 0x59565955

And i find the problem is that:
I set the Pixel is V4L2_PIX_FMT_NV12(ColorSpace is ColorSpace_YUV420PSEMI), but when the Capture_create in DMAI calls the iotcl VIDIOC_TRY_FMT,
the Pixel becomes to V4L2_PIX_FMT_UYVY(ColorSpace is ColorSpace_UYVY)

If i change the ColorSpace from ColorSpace_YUV420PSEMI to ColorSpace_UYVY, all is OK. But i want to use ColorSpace_YUV420PSEMI. Why the driver changes my Pixel?

There is an other problem: what is the correct size should i set?
(1),In the resize.c of DMAI, i set the RSZB for cif resize
 rsz_cont_config.output2.width = 352;
    rsz_cont_config.output2.height = 288;  
   
(2),what is the correct offset?
 to save the first YUV image(D1, PAL), i use:
 fwrite(Buffer_getUserPtr(hCapBuf),(736*576*3)/2,1,outFileD1);
 
 to save the second YUV image(CIF), i use:
 fwrite(Buffer_getUserPtr(hCapBuf)+(736*576*3)/2,(368*288*3)/2,1,outFileCIF);
 
Is there any wrong for my settting?
 
Thanks

Pavel 

Hi Anshuman,

This is regarding the patch that you shard for one capture and two resizes in DM365.

Apart from replacing the resize.c and capture.c files, is there any other change that is required to be done?

On running the encode application, I could get only one output file (out.264). 

But based on your code changes, I think one more output file "interface.yuv" should have been generated. right?

Regards,

Vinod

Dear Vinod,

It depends on how you are using the resizers. Have you enabled resizer output1 and output2 both? If yes, then the output of Resizer B should also be available in the DDR. You can change your application to take the data and encode it or store it to a yuv file.

Regards,

Anshuman

Hi,

When I enbale output2 of the resizer (as described in this thread), I do get extra data in the capture buffer. It follows output1, but it does not appear to be a well formed image. I suspect that the resizer is not configured correctly, but I don't see any difference between what I am running and what has been described here. Has anyone encountered similar issues?

Reagrds

Bernard

Yes. Data from resizer follows after the output1 capture. They may look incorrect if You expect the same resolution as output1.

Try to display them in viewer, with less number of pixels per line (=width). I used the Irfanview to see if data are stored in right place and size.

Unfortunately I am not running on HW that can display the image directly. I could store it, but I have not tried that yet. However, I don't think my issue has to do with mismatch in resolution because:

1) I have tested with both output  running at the same resolution (352x240) and seen the same issue

2) I just manage to get this working with 2 different resolution (702x480 and 352x240) when I set the color space to UYVY (4:2:2) rather than YUV420PSEMI (4:2:0) with an mpeg4 encoder.

Unfortunately I cannot use UYVY because I need to use the h264 encoder.

There is a comment in dm365_ipipe.h that implies only UYVY is supported for output 2.

struct rsz_output_spec {

...   

/* output pixel format. Has to be UYVY */
    enum ipipe_pix_formats pix_fmt;
...

Is this comment true? Because your example seemed to make use of YUV420PSEMI.

I use YUV420PSEMI (4:2:0) with H264. It works perfect.

E.g. 1280x720 - output 1, 320x240 - output 2. Both outputs are in same color space. The picture data are stored in buffer in this order:

1280x720xY (=YYYYYYYYYYYYYYY...)

(1280/2)x(720/2)x(U+V) (=UVUVUVUVUVUV.........)

320x240xY (=YYYYYYYYYYYYYYY...)

(320/2)x(240/2)x(U+V) (=UVUVUVUVUVUV.........)

Hi Juraj,

I am trying to configure the DM365 resizer to give 4:2:0SP output in single shot mode. But I am not getting the proper output. How ever if I configure the resizer output to UYVY I am getting the proper output.

Is it not possible to configure the resizer in singlr shot mode to output yuv :42:0SP?

Please help.

Regards

Raghavendra