Hello,
In our model, we are doing the following:
[ capture with VPSS v4l2 VIN module ] -> [ use OMX VFPC-DEI to scale ] -> [ use OMX encoder to encode ]
We have decided to use the VFPC-DEI component as scaler after the discussion with TI guys:
https://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/469287
I am now developing and testing the app and I am having a peculiar problem between VPSS v4l2 VIN capture and OMX VFPC-DEI when trying to scale:
DEI component does not return with EmptyBufferDone callback when capturing from one of the inputs on our card and when telling DEI to scale down.
Here is more details:
- my capture card has 2 inputs; data on one of them is captured through VIN output port 0 and the other one is through VIN output port 1
- initially, data is captured as YUV422 from the one input on the card and as RGB24 from the other, but in both cases the VIN capture module is "told" to output NV12 on both of its output ports and pass it to DEI component with EmptyThisBuffer
- when capturing through VIN out port 1 (video originally in RGB24), the capture and scaling by DEI works well
- when capturing through VIN out port 0 (video originally in YUV422), data is captured but scaling by DEI does not work (as mentioned above, DEI does not return with EmptyBufferDone callback). What is interesting is that for this case, it will all work if data is captured and passed to DEI and DEI is told to not scale but just pass the data 1-to-1 to its output buffers 1:1 (e.g. 1920x1080 -> 1920x1080). DEI will then return with EmptyBufferDone and of course with 2x FillBufferDone on its outputs...
- note that my previous tests with just DEI component and no capture through VIN were successful. They were simple tests where DEI was just processing "dummy" data from its input to output in various combinations of: no scaling, scaling down, scaling up. That's why I think that somehow the the VIN v4l2 capture component is involved in the problem.
My suspicions head towards one of the following (could you please confirm if they can be the case here):
1) DEI is tied to only one of the VPSS VIN output ports when scaling and will not cooperate with the other one?
2) I am initialising DEI component first and then VPSS VIN is initialised. Is it possible that for the failure case I am overwriting settings in the scaler? Is the inline scaler in VIN the same hardware as the scaler used by DEI? This was my first thought since DEI is first initialised to scale and then VIN is always initialised to capture 1-to-1 (not scale). However, I did a test in which VIN component is initialised first and DEI is initialised after that and the behaviour did not change, that is I was still getting the same failure as described above...
3) Would it matter here that the data on VIN input is in different formats for the success and the failure case (RGB24 vs. YUV422)? I do not understand why that would be relevant since in all cases VIN capture is told to colour-convert the data to NV12 and then pass it to DEI...
I will be grateful for your suggestions / help with this problem.
Also, please see below printouts from the app and from the VIN v4l2 driver which may put more light on this case. The print outs and for the initial loop for 4 allocated buffers.
1) Working case: capture through VIN output port 1 and pass to DEI, source: 1920x1080, DEI told to scale down to 1280x720
APP:
v4l2_capture_loop: V4L2-1-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(1), latest_dq_buf_index(0), pBuffer(0x40783100)
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(0), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-1-0: memcpy 3110400 bytes from pBuffer[0](0x40783100) to DEI's(0x135338) input pBuffer and call ETB on buffer[0](0x41210900):
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(0), nTimeStamp(0)(0x0), nFlags(0x0)
DeiCbEmptyBufferDone: client(0), handle(1355f0), count(0), pBuffer(0x41210900)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(16), nInputPortIndex(0), nSize(80), nAllocLen(1843712), nFilledLen(1843200), pBuffer(0x41def000), latest_dq_buf_index(0)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(17), nInputPortIndex(0), nSize(80), nAllocLen(1382912), nFilledLen(1382400), pBuffer(0x424f7f00), latest_dq_buf_index(0)
v4l2_capture_loop: V4L2-1-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(2), latest_dq_buf_index(1), pBuffer(0x408d4b00)
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(1), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-1-0: memcpy 3110400 bytes from pBuffer[1](0x408d4b00) to DEI's(0x135338) input pBuffer and call ETB on buffer[1](0x41508100):
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(1), nTimeStamp(0)(0x0), nFlags(0x0)
DeiCbEmptyBufferDone: client(0), handle(1355f0), count(1), pBuffer(0x41508100)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(16), nInputPortIndex(0), nSize(80), nAllocLen(1843712), nFilledLen(1843200), pBuffer(0x41fb1200), latest_dq_buf_index(1)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(17), nInputPortIndex(0), nSize(80), nAllocLen(1382912), nFilledLen(1382400), pBuffer(0x42649900), latest_dq_buf_index(1)
v4l2_capture_loop: V4L2-1-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(3), latest_dq_buf_index(2), pBuffer(0x40a26500)
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(2), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-1-0: memcpy 3110400 bytes from pBuffer[2](0x40a26500) to DEI's(0x135338) input pBuffer and call ETB on buffer[2](0x417ff900):
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(2), nTimeStamp(0)(0x0), nFlags(0x0)
DeiCbEmptyBufferDone: client(0), handle(1355f0), count(2), pBuffer(0x417ff900)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(16), nInputPortIndex(0), nSize(80), nAllocLen(1843712), nFilledLen(1843200), pBuffer(0x42173400), latest_dq_buf_index(2)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(17), nInputPortIndex(0), nSize(80), nAllocLen(1382912), nFilledLen(1382400), pBuffer(0x4279b300), latest_dq_buf_index(2)
v4l2_capture_loop: V4L2-1-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(4), latest_dq_buf_index(3), pBuffer(0x40b77f00)
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(3), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-1-0: memcpy 3110400 bytes from pBuffer[3](0x40b77f00) to DEI's(0x135338) input pBuffer and call ETB on buffer[3](0x41af7100):
nOutputPortIndex(1), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(3), nTimeStamp(0)(0x0), nFlags(0x0)
DeiCbEmptyBufferDone: client(0), handle(1355f0), count(3), pBuffer(0x41af7100)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(16), nInputPortIndex(0), nSize(80), nAllocLen(1843712), nFilledLen(1843200), pBuffer(0x42335600), latest_dq_buf_index(3)
DeiCbFillBufferDone: DEI-0 nOutputPortIndex(17), nInputPortIndex(0), nSize(80), nAllocLen(1382912), nFilledLen(1382400), pBuffer(0x428ecd00), latest_dq_buf_index(3)
DRIVER (vin and vpss):
ti81xxvin ti81xxvin: ti81xxvin_check_format: START
ti81xxvin ti81xxvin: ti81xxvin_check_format: pf(0x3231564e) cs(0x3) fld(0x1) w(1920) h(1080)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check pixel format: 0x3231564e
ti81xxvin ti81xxvin: ti81xxvin_check_format(2) - after update: check pixel format: 0x3231564e
ti81xxvin ti81xxvin: ti81xxvin_check_format: inst(0xd0230000)->win.w(0xd0230280) l(0) w(0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check width: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: using width: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check height: 1080 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: using height: 1080 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: set hpitch: bpl(1920)
ti81xxvin ti81xxvin: ti81xxvin_check_format: switch pixelformat(0x3231564e)
ti81xxvin ti81xxvin: ti81xxvin_check_format: YUYV(0x56595559) NV12(0x3231564e)
ti81xxvin ti81xxvin: ti81xxvin_check_format: NV16(0x3631564e) RGB24(0x33424752)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check hpitch: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: set numlines: sizeimage(3110400) / hpitch(1920) = 1620)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check numlines: 1620 (min: 1620)
ti81xxvin ti81xxvin: ti81xxvin_check_format(1): Check for 8 byte alignment: hpitch(1920)
ti81xxvin ti81xxvin: ti81xxvin_check_format: END - ret: 0 - pf(0x3231564e), width(1920) height(1080), bytesperline(1920), sizeimage(3110400)
VPSS_CAPTURE: capture_create(2.1): ccparams->videoCaptureMode(6)(0x6), ccparams->videoIfMode(2)(0x2), ccparams->inDataFormat(4103)(0x1007)
ccparams->numCh(1), ccparams->numStream(1)
ccparams->vipParserInstConfig(0xbfbaf774), ccparams->vipParserPortConfig(0xbfbaf77c), ccparams->cscConfig(0x (null)), ccparams->inScanFormat(1)
ccparams->outStreamInfo[0].dataFormat(7)(0x7)
ccparams->outStreamInfo[0].memType(0), ccparams->outStreamInfo[0].pitch[0](1920), outStreamInfo[0].maxOutWidth(0), outStreamInfo[0].maxOutHeight(0), outStreamInfo[0].scEnable(0)
outStreamInfo[0].subFrameModeEnable(0), outStreamInfo[0].numLinesInSubFrame(0)
2) Not-working case: capture through VIN output port 0 and pass to DEI, source: 1920x1080, DEI told to scale down to 1280x720
APP:
v4l2_capture_loop: V4L2-0-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(1), latest_dq_buf_index(0), pBuffer(0x4063a100)
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(0), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0: memcpy 3110400 bytes from pBuffer[0](0x4063a100) to DEI's(0x135338) input pBuffer and call ETB on buffer[0](0x410c7900):
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(0), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(2), latest_dq_buf_index(1), pBuffer(0x4078bb00)
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(1), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0: memcpy 3110400 bytes from pBuffer[1](0x4078bb00) to DEI's(0x135338) input pBuffer and call ETB on buffer[1](0x413bf100):
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(1), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(3), latest_dq_buf_index(2), pBuffer(0x408dd500)
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(2), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0: memcpy 3110400 bytes from pBuffer[2](0x408dd500) to DEI's(0x135338) input pBuffer and call ETB on buffer[2](0x416b6900):
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(2), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0, client(0): DQed: outPortParams(0x135270), chain_handle(0x1355f0), chain_omx_comp(0x135338), queued(4) dequeued(4), latest_dq_buf_index(3), pBuffer(0x40a2ef00)
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(3), nTimeStamp(0)(0x0), nFlags(0x0)
v4l2_capture_loop: V4L2-0-0: memcpy 3110400 bytes from pBuffer[3](0x40a2ef00) to DEI's(0x135338) input pBuffer and call ETB on buffer[3](0x419ae100):
nOutputPortIndex(0), nInputPortIndex(0), nSize(80), nAllocLen(3110912), nFilledLen(3110400), nOffset(0), nTickCount(3), nTimeStamp(0)(0x0), nFlags(0x0)
DRIVER (vin and vpss):
ti81xxvin ti81xxvin: ti81xxvin_check_format: START
ti81xxvin ti81xxvin: ti81xxvin_check_format: pf(0x3231564e) cs(0x3) fld(0x1) w(1920) h(1080)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check pixel format: 0x3231564e
ti81xxvin ti81xxvin: ti81xxvin_check_format(2) - after update: check pixel format: 0x3231564e
ti81xxvin ti81xxvin: ti81xxvin_check_format: inst(0xd0393400)->win.w(0xd0393680) l(0) w(0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check width: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: using width: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check height: 1080 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: using height: 1080 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: set hpitch: bpl(1920)
ti81xxvin ti81xxvin: ti81xxvin_check_format: switch pixelformat(0x3231564e)
ti81xxvin ti81xxvin: ti81xxvin_check_format: YUYV(0x56595559) NV12(0x3231564e)
ti81xxvin ti81xxvin: ti81xxvin_check_format: NV16(0x3631564e) RGB24(0x33424752)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check hpitch: 1920 (min: 0)
ti81xxvin ti81xxvin: ti81xxvin_check_format: set numlines: sizeimage(3110400) / hpitch(1920) = 1620)
ti81xxvin ti81xxvin: ti81xxvin_check_format: check numlines: 1620 (min: 1620)
ti81xxvin ti81xxvin: ti81xxvin_check_format(1): Check for 8 byte alignment: hpitch(1920)
ti81xxvin ti81xxvin: ti81xxvin_check_format: END - ret: 0 - pf(0x3231564e), width(1920) height(1080), bytesperline(1920), sizeimage(3110400)
VPSS_CAPTURE: capture_create(2.1): ccparams->videoCaptureMode(0)(0x0), ccparams->videoIfMode(1)(0x1), ccparams->inDataFormat(6)(0x6)
ccparams->numCh(1), ccparams->numStream(1)
ccparams->vipParserInstConfig(0xbfb0720c), ccparams->vipParserPortConfig(0xbfb07214), ccparams->cscConfig(0x (null)), ccparams->inScanFormat(1)
ccparams->outStreamInfo[0].dataFormat(7)(0x7)
ccparams->outStreamInfo[0].memType(0), ccparams->outStreamInfo[0].pitch[0](1920), outStreamInfo[0].maxOutWidth(0), outStreamInfo[0].maxOutHeight(0), outStreamInfo[0].scEnable(0)
outStreamInfo[0].subFrameModeEnable(0), outStreamInfo[0].numLinesInSubFrame(0)
Thanks,
Przemek