This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5728: [H264enc using IVA] How to pass the SEI data to iva codec from linux ?

Part Number: AM5728

Tool/software: Linux

Hi,


I would like to do H264 encoding include SEI data using AM572x evm board.

As written in H264_Encoder_HDVICP2_UserGuide.pdf(Appendix A), gst-ducati-plugin was changed as follow.

diff --git a/src/gstducatih264enc.c b/src/gstducatih264enc.c
index 5c5f234..6382989 100644
--- a/src/gstducatih264enc.c
+++ b/src/gstducatih264enc.c
@@ -572,6 +572,20 @@ gst_ducati_h264enc_configure (GstDucatiVidEnc * videnc)
params->entropyCodingMode = self->entropy_coding_mode;
videnc->params->maxInterFrameInterval = inter_interval;

+ /* Enable SEI */
+ params->videnc2Params.metadataType[0] = IH264_SEI_USER_DATA_UNREGISTERED;
+ params->videnc2Params.metadataType[1] = IVIDEO_METADATAPLANE_NONE;
+ params->videnc2Params.metadataType[2] = IVIDEO_METADATAPLANE_NONE;
+
+ params->nalUnitControlParams.naluControlPreset =
+ IH264_NALU_CONTROL_USERDEFINED;
+
+ IH264ENC_SET_NALU(params->nalUnitControlParams.naluPresentMaskIDRPicture, SPS);
+ IH264ENC_SET_NALU(params->nalUnitControlParams.naluPresentMaskIDRPicture, PPS);
+ IH264ENC_SET_NALU(params->nalUnitControlParams.naluPresentMaskIDRPicture, USER_DATA_UNREGD_SEI);
+ GST_DEBUG_OBJECT(self, "H264 Set SEI Params: 0x%x",
+ params->nalUnitControlParams.naluPresentMaskIDRPicture);
+
/* Dynamic params */
dynParams = (IH264ENC_DynamicParams *) videnc->dynParams;
dynParams->rateControlParams.rateControlParamsPreset =
diff --git a/src/gstducatividenc.c b/src/gstducatividenc.c
index d939905..c85acd9 100644
--- a/src/gstducatividenc.c
+++ b/src/gstducatividenc.c
@@ -27,6 +27,7 @@
#include "gstducati.h"
#include "gstducatividenc.h"
#include "gstducatibufferpriv.h"
+#include "gstducatih264enc.h"

#include <string.h>

@@ -375,7 +376,6 @@ gst_ducati_videnc_configure (GstDucatiVidEnc * self)
return FALSE;
}
}
-
err = VIDENC2_control (self->codec,
XDM_SETPARAMS, self->dynParams, self->status);
if (err) {
@@ -569,6 +569,9 @@ gst_ducati_videnc_allocate_params_default (GstDucatiVidEnc * self,
memset (self->outArgs, 0, outargs_sz);
self->outArgs->size = outargs_sz;

+ self->sei_msg = dce_alloc (sizeof(SEI_TIMESTAMP_t));
+ memset (self->sei_msg, 0, sizeof(SEI_TIMESTAMP_t));
+
GST_INFO_OBJECT (self, "started");

return TRUE;
@@ -612,6 +615,11 @@ gst_ducati_videnc_free_params (GstDucatiVidEnc * self)
self->outBufs = NULL;
}

+ if (self->sei_msg) {
+ dce_free (self->sei_msg);
+ self->sei_msg = NULL;
+ }
+
if (self->codec) {
VIDENC2_delete (self->codec);
self->codec = NULL;
@@ -824,6 +832,19 @@ have_inbuf:
self->outBufs->numBufs = 1;
self->outBufs->descs[0].buf = (XDAS_Int8 *) dmabuf_fd_out;

+ /* add SEI data */
+ if(self->params->metadataType[0] == IH264_SEI_USER_DATA_UNREGISTERED){
+ memset (self->sei_msg, 0, sizeof(SEI_TIMESTAMP_t));
+ self->sei_msg->size = sizeof(GTimeVal);
+ GST_TIME_TO_TIMEVAL(ts, self->sei_msg->ts);
+ self->inBufs->numMetaPlanes = 1;
+ self->inBufs->metadataPlaneDesc[0].buf = (XDAS_Int8*)self->sei_msg;
+ self->inBufs->metadataPlaneDesc[0].bufSize.bytes = -1;
+ GST_DEBUG_OBJECT (self, "SEI Buffer size = %d", self->sei_msg->size);
+ GST_DEBUG_OBJECT (self, "SEI ts = %04d.%06d",
+ self->sei_msg->ts.tv_sec, self->sei_msg->ts.tv_usec);
+ }
+
GstStructure *conf;
guint size;
conf = gst_buffer_pool_get_config (self->output_pool);
diff --git a/src/gstducatividenc.h b/src/gstducatividenc.h
index 5d68016..2b48d52 100644
--- a/src/gstducatividenc.h
+++ b/src/gstducatividenc.h
@@ -42,9 +42,17 @@
#define GST_DUCATIVIDENC_GET_CLASS(obj) \
(G_TYPE_INSTANCE_GET_CLASS((obj), GST_TYPE_DUCATIVIDENC, GstDucatiVidEncClass))

+#define IDX_SEI_METADATA 0
+
typedef struct _GstDucatiVidEnc GstDucatiVidEnc;
typedef struct _GstDucatiVidEncClass GstDucatiVidEncClass;

+typedef struct {
+ XDAS_Int32 size;
+ GTimeVal ts;
+}SEI_TIMESTAMP_t;
+//__attribute__((__packed__)) SEI_TIMESTAMP_t;
+
typedef struct {
gint x;
gint y;
@@ -82,6 +90,8 @@ struct _GstDucatiVidEnc
guint intra_interval;

const char *error_strings[32];
+
+ SEI_TIMESTAMP_t *sei_msg;
};

I could include SEI in encoded H264 file. But, the data was broken.

I think the cause is that passed sei_msg address is virtual address ,not physical address.

Is there a right way to pass SEI data to IVA from Linux?

Regards,

Reona Shiode

  • Hello,

    Please refer gstducatimpeg2dec.c file.
    The mpeg2 decoder has example of how METADATA is handled.

    Best Regards,
    Margarita
  • Thank you for fast reply, Margarita.

    I changed the code as follow. But, SEI data was still broken.

    ( payload size of encoded sei data was not 8, and payload is not zero) 

    in gst_ducati_videnc_allocate_params_default()

      self->bo_sei = omap_bo_new (self->device, sizeof(SEI_TIMESTAMP_t),
    			      OMAP_BO_WC);
      self->sei_fd = omap_bo_dmabuf (self->bo_sei);
      self->psei = (SEI_TIMESTAMP_t*)omap_bo_map(self->bo_sei);
    
      self->inBufs->numMetaPlanes = 1;
      self->inBufs->metadataPlaneDesc[0].buf =
          (XDAS_Int8 *)(self->sei_fd);
      self->inBufs->metadataPlaneDesc[0].bufSize.bytes = -1;
      memset(self->psei, 0, sizeof(SEI_TIMESTAMP_t));
      self->psei->size = 8;
    

    Is it wrong that fd is passed to metadataPlaneDesc[].buf ?

    Regards,

    Reona Shiode

  • Hello,

    We will check this.
    What you mean SEI is broken? Is not inserted properly?
    We will get back to you when we have something. Meanwhile please refer the encoder user guide :
    /ipumm/extrel/ti/ivahd_codecs/packages/ti/sdo/codecs/h264enc/docs

    BR
    Margarita
  • Thank you for checking, Margarita.

     Broken means that the payload of encoded SEI data  is not match the data I defined in ducatih264enc.

    But, it was solved.

    I had to change like this.

          self->inBufs->numPlanes = 4; // default numPlanes + nummetaPlanes + 1
          self->inBufs->planeDesc[2].buf = (XDAS_Int8 *) self->sei_fd;
          self->inBufs->planeDesc[2].memType = XDM_MEMTYPE_RAW;
          self->inBufs->planeDesc[2].bufSize.bytes = -1;
          
          self->inBufs->numMetaPlanes = 1;
          self->inBufs->metadataPlaneDesc[0].buf =
             (XDAS_Int8 *)(self->sei_fd);
          self->inBufs->metadataPlaneDesc[0].bufSize.bytes = -1;

    libdce translates pointers for remote core, but numMetaPlanes is not considered.

    So, MetaPlanesDesc.buf was not translated correctly.

    I think this is the reason why the SEI data was broken.

    I think it is better considering MetaPlanes in libdce.

    Is there a plan that changing to consider MataPlanes in libdce?

  • Hello,

    Great I am glad that this issue is solved.

    Reona Shiode said:
    Is there a plan that changing to consider MataPlanes in libdce?

    I must check. 

    BR
    Margarita

  • Hello,

    Now metaplanes will not considered in libdce as this is a custom usecase.

    Hope this helps.

    Best Regards,
    Margarita
  • Margarita,

    I understand.

    Thank for very much for your support.

    Best Regards,
    Reona