TDA4VM: Mosaic node input image format

Hi Raju,

This is strange, U8 data type is supported in the Mosaic node. Are you doing any upscaling or different output format? Because in this case, output format must also be U8. MSC does not support format conversion. 

Regards,

Brijesh

Hi sir,

Brijesh Jadav said:

Are you doing any upscaling or different output format?

No.

We have changed the custom kernel to give RGB output, Is display_m2m useful for conversion from RGB to NV12? If it is, can you please tell me the way to use it for my requirement?

Regards,

Seetharama Raju.

Hi,

Yes, display_m2m can be used to convert from RGB to NV12. You can just create display node with input and output images and start using it in your graph. 

Regards,

Brijesh

Hi sir,

I think our new pipeline is going to be File->scaler->pre-proc->custom node->Display_m2m->mosaic->display(DSS). I have created the display_m2m node and used it in the graph. I ran the application, no errors came. I am keeping below both output of custom node(RGB) and its equivalent m2m output(NV12) visualised on the display. I can't understand how come they both be different? can you please guide me on this? 

Note 1: I have used below parameters for m2m node.

displayObj->disp_m2m_params.instId = 0;//TIVX_KERNEL_DISPLAY_BUFFER_COPY_MODE;
displayObj->disp_m2m_params.numPipe= 1; /* pipe ID = 2 */
displayObj->disp_m2m_params.pipeId[0U]= 3;
displayObj->disp_m2m_params.overlayId=3;

Note 2: I have used another API vxColorConvertNode to convert from RGB to NV12, it also gave the similar visualization like the above right image.

Regards,

Seetharama Raju.

Hi,

It does not look like due to display m2m node, are you reading the images correctly? Are there any issues in reading the image, like have you taken care of pitch/line offset ?

Regards,

Brijesh

Hi sir,

There are no issues regrading the image reading. Is there anything that I need to set in source file of display_m2m for the format conversion?

Thanks,

Seetharama Raju.

Hi,

No, Display m2m node just works based on input and output image's format. There is no additional configuration required. Can you please save the images after display m2m node? Also how do you confirm that input images is correct? 

Regards,

Brijesh

Hi sir,

The input given to the display_m2m is not proper. We have a custom node before m2m node. The output of the custom node is RGB interleaved and it is dumped inside the custom node and found to be valid. Display_m2m takes this as input. In app_run_graph_for_one_frame_pipeline function, I have used API tivx_utils_save_vximage_to_bmpfile to save the input image of display_m2m. The image dumped inside the custom node and input image to display_m2m are supposed to be same but found different. Can you please guide me on this?

Thanks,

Seetharama Raju.

Hi,

Where is this custom node running? Have you properly taken care of mapping and unmapping the images in the custom node? 

Regards,

Brijesh

Hi sir,

The custom node is running on A72 core. This node takes a tensor( pre-proc output i.e RGB planar) and a LUT as inputs and gives RGB interleaved output. 

I have attached here the map and unmap snippets below.

void *input_tensor_target_ptr;
void *output_image_target_ptr;
void *lut_reversal_target_ptr;

map-->

input_tensor_target_ptr = tivxMemShared2TargetPtr(&input_tensor_desc->mem_ptr);
tivxCheckStatus(&status, tivxMemBufferMap(input_tensor_target_ptr,
input_tensor_desc->mem_size, (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_READ_ONLY));

printf("input_tensor_desc->mem_size is %d \n",input_tensor_desc->mem_size);

output_image_target_ptr = tivxMemShared2TargetPtr(&output_image_desc->mem_ptr[0]);
tivxCheckStatus(&status, tivxMemBufferMap(output_image_target_ptr,
output_image_desc->mem_size[0], (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_WRITE_ONLY));

printf("output_image_desc->mem_size[0] is %d \n",output_image_desc->mem_size[0]);

lut_reversal_target_ptr = tivxMemShared2TargetPtr(&lut_reversal_desc->mem_ptr[0]);
tivxCheckStatus(&status, tivxMemBufferMap(lut_reversal_target_ptr,
lut_reversal_desc->mem_size[0], (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_READ_ONLY));

printf("lut_reversal_desc->mem_size[0] is %d \n",lut_reversal_desc->mem_size[0]);

unmap--->

tivxCheckStatus(&status, tivxMemBufferUnmap(input_tensor_target_ptr,
input_tensor_desc->mem_size, (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_READ_ONLY));

tivxCheckStatus(&status, tivxMemBufferUnmap(output_image_target_ptr,
output_image_desc->mem_size[0], (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_WRITE_ONLY));

tivxCheckStatus(&status, tivxMemBufferUnmap(lut_reversal_target_ptr,
lut_reversal_desc->mem_size[0], (vx_enum)VX_MEMORY_TYPE_HOST,
(vx_enum)VX_READ_ONLY));

Note:

1)I have printed the size of the buffers for the three objects.

input_tensor_desc->mem_size is 1179648
output_image_desc->mem_size[0] is 570112
lut_reversal_desc->mem_size[0] is 1572864

2)input_tensor_desc->mem_size is 1179648  (input is 768x512  1 byte : RGB planar, so 768x512x3 )

 lut_reversal_desc->mem_size[0] is 1572864 (input 4 byte:  LUT is 768x512, so 768x512x4)

But I dont know why it didn't come as expected for output_image_desc. It is supposed to be 720x262x3=565920 but it came as 570112.

Thanks,

Seetharama Raju.

Hi,

But aren't they three input buffers for RGB planar mode? Also why is there only 1byte for input? I am really doubtful that this node is running fine. 

Regards,

Brijesh

Hi sir,

RGB planar output of preproc node of sdk is given to the custom node. We already know that most of the demo apps TIDL node input takes the object array of tensors, the similar way the custom node is generated for input as tensor. The output of this node is of VX_DF_IMAGE_RGB type. 

Brijesh Jadav said:

Also why is there only 1byte for input?

I mean to say size of data type i.e image pixel is 1 byte.

Thanks,

Seetharama Raju

Hi,

I meant to say how is input size matching? Even if you have planar input, the size should be exactly matching with the output, because even though output is interleaved, it requires almost same number of bytes.. 

Regards,

Brijesh

Hi sir,

 We are resizing in the custom node. Input size is 768x512 planar RGB and output requirement is 720x262 interleaved RGB.

Thanks,

Seetharama Raju.

I would suggest checking the output of this node. It doesn't not look like an issue in display_m2m node. 

Rgds,

Brijesh

Hi sir,

The above issue is due to the consideration of width instead of stride while reading the output image of our custom node. The issue is resolved. The application is running now. But after running the app for 2 or 3 minutes, the following log is getting appeared on the console and the app stops working. I didn't find any references in E2E. Can you please guide me on this?

Enter Choice: [MCU2_0] 57.855464 s: VX_ZONE_WARNING:[tivxCaptureSetTimeout:774] CAPTURE: WARNING: Error frame not provided using tivxCaptureRegisterErrorFrame, defaulting to waiting forever !!!
[ 113.106118] vx_app_arm_remo invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[ 113.118270] CPU: 1 PID: 1159 Comm: vx_app_arm_remo Tainted: G O 5.10.100-g7a7a3af903 #1
[ 113.127553] Hardware name: Texas Instruments K3 J721E SoC (DT)
[ 113.133368] Call trace:
[ 113.135814] dump_backtrace+0x0/0x1a0
[ 113.139463] show_stack+0x18/0x68
[ 113.142768] dump_stack+0xd0/0x12c
[ 113.146156] dump_header+0x48/0x19c
[ 113.149636] oom_kill_process+0x200/0x208
[ 113.153630] out_of_memory+0xe4/0x320
[ 113.157280] __alloc_pages_slowpath.constprop.0+0x958/0x9f8
[ 113.162835] __alloc_pages_nodemask+0x1ec/0x240
[ 113.167351] pagecache_get_page+0x140/0x2f8
[ 113.171518] filemap_fault+0x5fc/0x9d0
[ 113.175254] ext4_filemap_fault+0x34/0x900
[ 113.179337] __do_fault+0x3c/0x178
[ 113.182727] handle_mm_fault+0xa1c/0xf00
[ 113.186637] do_page_fault+0x130/0x3a8
[ 113.190373] do_translation_fault+0x58/0x68
[ 113.194544] do_mem_abort+0x40/0xa0
[ 113.198020] el0_ia+0x64/0xb8
[ 113.200974] el0_sync_handler+0x98/0xb8
[ 113.204796] el0_sync+0x180/0x1c0
[ 113.208360] Mem-Info:
[ 113.210681] active_anon:222 inactive_anon:31088 isolated_anon:0
[ 113.210681] active_file:33 inactive_file:0 isolated_file:0
[ 113.210681] unevictable:0 dirty:0 writeback:0
[ 113.210681] slab_reclaimable:262 slab_unreclaimable:738
[ 113.210681] mapped:40 shmem:456 pagetables:165 bounce:0
[ 113.210681] free:1891 free_pcp:0 free_cma:0
[ 113.241416] Node 0 active_anon:14208kB inactive_anon:1989632kB active_file:640kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2560kB dirty:0kB writeback:0kB shmems
[ 113.269455] DMA free:30336kB min:23872kB low:29824kB high:35776kB reserved_highatomic:0KB active_anon:0kB inactive_anon:430720kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB preB
[ 113.302271] lowmem_reserve[]: 0 0 111 111
[ 113.312672] Normal free:89216kB min:90624kB low:113280kB high:135936kB reserved_highatomic:0KB active_anon:14208kB inactive_anon:1559936kB active_file:1920kB inactive_file:320kB unevictable:0kB writeB
[ 113.353878] lowmem_reserve[]: 0 0 0 0
[ 113.362260] DMA: 0*64kB 1*128kB (M) 0*256kB 1*512kB (M) 1*1024kB (M) 0*2048kB 1*4096kB (M) 1*8192kB (M) 1*16384kB (M) 0*32768kB 0*65536kB 0*131072kB 0*262144kB 0*524288kB = 30336kB
[ 113.380813] Normal: 43*64kB (M) 5*128kB (M) 0*256kB 1*512kB (M) 1*1024kB (M) 0*2048kB 1*4096kB (M) 0*8192kB 1*16384kB (M) 0*32768kB 1*65536kB (M) 0*131072kB 0*262144kB 0*524288kB = 90944kB
[ 113.410964] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB
[ 113.423345] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=524288kB
[ 113.434862] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 113.443284] 475 total pagecache pages
[ 113.450763] 0 pages in swap cache
[ 113.454064] Swap cache stats: add 0, delete 0, find 0/0
[ 113.463037] Free swap = 0kB
[ 113.465905] Total swap = 0kB
[ 113.468775] 65536 pages RAM
[ 113.471561] 0 pages HighMem/MovableOnly
[ 113.479204] 29526 pages reserved
[ 113.482422] 0 pages cma reserved
[ 113.485636] 0 pages hwpoisoned
[ 113.497203] Tasks state (memory values in pages):
[ 113.506270] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 113.519191] [ 167] 999 167 69 13 327680 0 0 rpcbind
[ 113.531158] [ 168] 0 168 318 59 393216 0 -250 systemd-journal
[ 113.548903] [ 185] 0 185 248 51 327680 0 -1000 systemd-udevd
[ 113.560532] [ 206] 992 206 1284 24 393216 0 0 systemd-timesyn
[ 113.574273] [ 878] 0 878 2369 11 327680 0 0 rngd
[ 113.590941] [ 885] 0 885 44 9 327680 0 0 atd
[ 113.613784] [ 886] 0 886 50 7 327680 0 0 crond
[ 113.631243] [ 887] 997 887 81 18 393216 0 -900 dbus-daemon
[ 113.658569] [ 892] 0 892 1239 9 458752 0 0 irqbalance
[ 113.675716] [ 893] 0 893 144 28 393216 0 0 ofonod
[ 113.703677] [ 900] 0 900 44 8 327680 0 0 tee-supplicant
[ 113.724338] [ 917] 994 917 136 27 327680 0 0 systemd-network
[ 113.748748] [ 939] 0 939 126 25 327680 0 0 systemd-logind
[ 113.775018] [ 948] 0 948 53 7 393216 0 0 telnetd
[ 113.800594] [ 950] 0 950 118 19 393216 0 0 systemd-network
[ 113.825639] [ 952] 993 952 129 27 393216 0 0 systemd-resolve
[ 113.846700] [ 955] 996 955 87 19 393216 0 0 avahi-daemon
[ 113.856458] [ 957] 998 957 63 12 327680 0 0 rpc.statd
[ 113.875720] [ 958] 0 958 201 72 327680 0 0 snmpd
[ 113.892504] [ 961] 0 961 40 7 327680 0 0 agetty
[ 113.907163] [ 962] 0 962 94 32 393216 0 0 login
[ 113.930336] [ 966] 0 966 40 7 393216 0 0 agetty
[ 113.938596] [ 968] 996 968 87 19 393216 0 0 avahi-daemon
[ 113.947370] [ 1144] 0 1144 150 32 327680 0 0 systemd
[ 113.970403] [ 1145] 0 1145 192 57 327680 0 0 (sd-pam)
[ 113.993144] [ 1150] 0 1150 80 31 393216 0 0 sh
[ 114.021256] [ 1158] 0 1158 2560 68 327680 0 0 vx_app_arm_remo
[ 114.037155] [ 1162] 0 1162 39872 30126 655360 0 0 vx_app_btc_cam_
[ 114.061946] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-0.slice/session-c1.scope,task=vx_app_btc_cam_,pid=1162,uid=0
[ 114.085196] Out of memory: Killed process 1162 (vx_app_btc_cam_) total-vm:2551808kB, anon-rss:1928000kB, file-rss:0kB, shmem-rss:64kB, UID:0 pgtables:640kB oom_score_adj:0
[ 114.147185] oom_reaper: reaped process 1162 (vx_app_btc_cam_), now anon-rss:0kB, file-rss:0kB, shmem-rss:128kB
Killed
 

Thanks,

Seetharama Raju.

Seems to cause crash on Linux. You will need to analyze your application.

Regards,

Brijesh