AM62A7: Is Gstreamer tiovxmultiscaler support dynamic adjust?

Hi Jason,

We'll try something for this and get back to you next week.

Regards,

Jianzhong

Hi Jason,

Can you try and use gst-validate and create a scenario to change the parameters of tiovxmultiscalar that we have tried on Encoder element to change the bitrate and key frame request.

 SK-AM62A-LP: H264 Encoder - I-Frames on Demand and Dynamic Bitrate Control 

Best Regards,

Suren

Hi Suren

The encoder can indeed be dynamically modified, which we have experimented with and already implemented. However, we encountered issues when attempting to modify the ROI parameters of the TIOVX MultiScaler.

During my testing, I had to stop and restart the pipeline for the modified ROI parameters to take effect, which resulted in visible screen freezes and flickering issues.

BR,

Jason

Hi Jason,

Can you provide us the sample example application that you are trying to test, so we can reproduce the issue and analyze.

Best Regards,

Suren

Hi Suren

I've written a test demo where the issue can be reproduced using videotestsrc. The program flow creates a simple pipeline of videotestsrc->multiscaler->kmssink. After the pipeline starts playing, it attempts to modify ROI parameters after 3 seconds while printing "No Stop pipeline". On the machine, we can observe that the ROI changes don't take effect immediately. After waiting another 3 seconds, when the pipeline is stopped and restarted, the ROI modifications become immediately effective.

// Perhaps due to my network issud, I cannot put my code in code blocks and can only paste it directly here

#include <stdio.h>
#include <gst/gst.h>

int gstTestRoi(int argc, char *argv[])
{
g_setenv("GST_DEBUG_DUMP_DOT_DIR", "/tmp/", TRUE);

GstElement *pipeline;
GstElement *videoSrc, *capsSrc;
GstElement *msc, *capsFilterMsc0;
GstElement *kmssink;

GstPad *mscSrcPad0;
GstPad *capsFilterMsc0Sink;

/* Initialize GStreamer */
gst_init(&argc, &argv);

/* Create Main Loop */
GMainLoop *loop = g_main_loop_new(NULL, FALSE);

/* Create the elements */
videoSrc = gst_element_factory_make("videotestsrc", "videotestsrc");
capsSrc = gst_element_factory_make("capsfilter", "capsSrc");
msc = gst_element_factory_make("tiovxmultiscaler", "msc");
capsFilterMsc0 = gst_element_factory_make("capsfilter", "capsFilterMsc0");
kmssink = gst_element_factory_make("kmssink", "kmssink");

/* Create the empty pipeline */
pipeline = gst_pipeline_new("test-pipeline");

if (!pipeline || !videoSrc || !capsSrc || !msc || !capsFilterMsc0 || !kmssink)
{
g_printerr("Not all elements could be created.\n");
return -1;
}

// set property
// ================ CAMERA
g_object_set(videoSrc, "pattern", 0, NULL);
g_object_set(videoSrc, "is-live", TRUE, NULL);
GstCaps *capsCam0;
// config v4l2 src
capsCam0 = gst_caps_new_simple("video/x-raw",
"width", G_TYPE_INT, 1920,
"height", G_TYPE_INT, 1080,
"framerate", GST_TYPE_FRACTION, 60, 1,
"format", G_TYPE_STRING, "NV12",
"colorimetry", G_TYPE_STRING, "bt709",
"interlace-mode", G_TYPE_STRING, "progressive",
NULL);
g_object_set(G_OBJECT(capsSrc), "caps", capsCam0, NULL);
gst_caps_unref(capsCam0);

// ================ MSC
g_object_set(msc, "target", 0, NULL);
GstCaps *capsMsc0;
capsMsc0 = gst_caps_new_simple("video/x-raw",
"width", G_TYPE_INT, 1280,
"height", G_TYPE_INT, 720,
"format", G_TYPE_STRING, "NV12",
NULL);
g_object_set(G_OBJECT(capsFilterMsc0), "caps", capsMsc0, NULL);
gst_caps_unref(capsMsc0);

// ================ KMSSINK
g_object_set(G_OBJECT(kmssink), "driver-name", "tidss", NULL);
g_object_set(G_OBJECT(kmssink), "sync", FALSE, NULL);
g_object_set(G_OBJECT(kmssink), "async", TRUE, NULL);
g_object_set(G_OBJECT(kmssink), "skip-vsync", TRUE, NULL);

/* Link all elements that can be automatically linked because they have "Always" pads */
gst_bin_add_many(GST_BIN(pipeline), videoSrc, capsSrc, msc, capsFilterMsc0, kmssink, NULL);
if (gst_element_link_many(videoSrc, capsSrc, msc, NULL) != TRUE ||
gst_element_link_many(capsFilterMsc0, kmssink, NULL) != TRUE)
{
g_printerr("Elements could not be linked.\n");
gst_object_unref(pipeline);
return -1;
}

/* Manually link the "Request" pads */
// GstPad *mscSrcPad0;
// GstPad *capsFilterMsc0Sink;
mscSrcPad0 = gst_element_request_pad_simple(msc, "src_%u");

capsFilterMsc0Sink = gst_element_get_static_pad(capsFilterMsc0, "sink");

if (!mscSrcPad0 || !capsFilterMsc0Sink)
{
g_printerr("Not all pad could be created.\n");
return -1;
}

g_object_set(mscSrcPad0, "roi-startx", 0, NULL);
g_object_set(mscSrcPad0, "roi-starty", 0, NULL);
g_object_set(mscSrcPad0, "roi-width", 1280, NULL);
g_object_set(mscSrcPad0, "roi-height", 720, NULL);

if (gst_pad_link(mscSrcPad0, capsFilterMsc0Sink) != GST_PAD_LINK_OK)
{
g_printerr("Could not be linked.\n");
gst_object_unref(pipeline);
return -1;
}

// gst_object_unref(mscSrcPad0);
gst_object_unref(capsFilterMsc0Sink);

/* Start playing the pipeline */
gst_element_set_state(pipeline, GST_STATE_PLAYING);
g_usleep(3 * 1000 * 1000);
// not work
g_message("No Stop pipeline");
g_object_set(mscSrcPad0, "roi-startx", 500, NULL);
g_object_set(mscSrcPad0, "roi-starty", 300, NULL);
g_object_set(mscSrcPad0, "roi-width", 1280, NULL);
g_object_set(mscSrcPad0, "roi-height", 720, NULL);
// GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");

// stop and play, roi change work
g_usleep(3 * 1000 * 1000);
g_message("Stop pipeline");
gst_element_set_state(pipeline, GST_STATE_NULL);
g_usleep(1 * 1000 * 1000);
gst_element_set_state(pipeline, GST_STATE_PLAYING);
g_main_loop_run(loop);

gst_object_unref(pipeline);
return 0;
}

int main(int argc, char *argv[])
{
gstTestRoi(argc, argv);
return 0;
}

BR,

Jason

Hi Suren

Is there any update about this issue?

BR

Jason

Hi Jason,

Will try and run some experiments this week and report back. Apologies for the delay.

Have you had any breakthrough? Also, I suggested to use a valve element to see if you can toggle between the two multiscaler's output in the other thread

Best Regards,

Suren