TDA4VM: TIDL uses all of TIVX_MEM_INTERNAL_L3 heap

FredC_LT

Part Number: TDA4VM

Hi,

I developed a C7X kernel that needs a bit of memory from TIVX_MEM_INTERNAL_L3. It runs fine in standalone, but tivxMemAlloc() fails when my kernel runs alongside TIDL. I added a print of tivxMemStats() in my kernel after failure and there are 0 bytes available in C7X L3!

[C7x_1 ] 1233482.518632 s: VX_ZONE_ERROR:[Allocate:132] Couldn't allocate m_myVariable for 7552 bytes!
[C7x_1 ] 1233482.518661 s: VX_ZONE_LOG:[Allocate:137] l3_stats.mem_size = 8159232, l3_stats.free_size = 0

After digging a bit, I found that TIVX_MEM_INTERNAL_L3 is configured as APP_MEM_HEAP_FLAGS_TYPE_LINEAR_ALLOCATE in app_init.c, thus a quick tivxMemFree(nullptr, 0, TIVX_MEM_INTERNAL_L3) at beginning and end of my kernel should fix my conflict with TIDL.

However, I also found this comment in ti-processor-sdk-rtos-j721e-evm-08_06_00_12/tiovx/source/platform/vision_sdk/bios/tivx_mem.c

case (vx_enum)TIVX_MEM_INTERNAL_L3:

/* In case of EVE, L3 memory correspond to one of the OCMC memory.

* We use hardcoded addresses because on EVE, only TI-DL use case needs allocation

* in OCMC and the TI_DL implementation assumes that the entire OCMC memory is available for its consumption.

* The only issue that can arise from using hard-coded address is in case On-the-fly capture from VIP is enabled

* because it also uses OCMC_1. However there is no use-case up-to-date that combines both TI-DL and OTF capture.

So the question arises: Is it safe to use L3 while TIDL is running? Does it reinitialize L3 contents every frame?

Thanks,

Fred

over 2 years ago

0 Brijesh Jadav over 2 years ago

TI__Guru**** 479675 points

Hi FredC_LT,

Yes, the L3 memory is currently completely allocated to TIDL. If you require this in your kernel, can we please split the memory statically between TIDL and your kernel? I think we can set the size of hte L3 memory that TIDL can use and then remaining can be used by your kernel.

Regards,

Brijesh

0 FredC_LT over 2 years ago in reply to Brijesh Jadav

Intellectual 796 points

Hi Brijesh,

Can it not be shared completely, i.e. reset before and after each algo pass with tivxMemFree(0, nullptr, TIVX_MEM_INTERNAL_L3)? Or is TIDL putting some critical data in L3 at init time and assumes it never changes?

+1 Nikhil Dasan over 2 years ago in reply to FredC_LT

TI__Guru* 85456 points

Hi Fred,

Typically, all the models that is available in the SDK, has a parameter msmcSizeKB, where the size of L3 would be mentioned to the model before importing it.
In the SDK, msmcSizeKB is 7.78MB and the rest of the MSMC is used for ATF and DMSC.

Here, most of the MSMC is provided for the model and if you see the TIDL node, tivxMemFree() to L3 is called only during the create function of the node and not during every process function.

Hence, it is not advised to call this while the tidl node is running (i.e. between activate and deactivate from TIDL perspective)

So as Brijesh suggested, you would have to split the MSMC such that less msmcSizeKB is given for the model and rest is used by your kernel.

Please note that if you decrease the msmc size to the model, this could affect the model performance as more layers would go to DDR as the size is less.

Regards,

Nikhil

0 FredC_LT over 2 years ago in reply to Nikhil Dasan

Intellectual 796 points

Thank you for the explanation.

By extension, that means that if I have two identical models of TIDL running at the same time, then the second one should be slower since the first one reserved all L3 first?

0 Nikhil Dasan over 2 years ago in reply to FredC_LT

TI__Guru* 85456 points

Hi Fred,

I thought that you were running c7x kernel and one tidl model together..

Is this the case or are both tidl model that are accessing the msmc?

Regards,

Nikhil

0 FredC_LT over 2 years ago in reply to Nikhil Dasan

Intellectual 796 points

It's one after the other.

Init TIDL model 1
Init TIDL model 2
Create custom C7X kernel
Run TIDL model 1
Run custom C7X kernel
Run TIDL model 2
Go back to 4.

+1 Nikhil Dasan over 2 years ago in reply to FredC_LT

TI__Guru* 85456 points

Hi Fred,

In case of run 2 tidl models, the models are usually imported with the info that around 7.78 MB Msmc is provided to them.

So there is a way for both models to run and use this whole MSMC region.

Internal to TIDL, as both models won't run in parallel as they run on the same C7 core, it would 2 tasks with either same or different priority on the same Core.

The preemption is taken care inside TIDL, i.e., when the second model preempts the first, then the MSMC data of the first model is backed up to the DDR and the second model starts using the MSMC region.

But in case of C7x kernel and TIDL model, this cannot happen and MSMC region has to split among these 2 in order for them to use.

Please note that if you decrease the msmc size to the model, this could affect the model performance as more layers would go to DDR as the size is less.

Regards,

Nikhil

0 FredC_LT over 2 years ago in reply to Nikhil Dasan

Intellectual 796 points

Thank you, that clarifies a lot.

0 Nikhil Dasan over 2 years ago in reply to FredC_LT

TI__Guru* 85456 points

Thank you. Closing this thread.

Processors

Processors forum

TDA4VM: TIDL uses all of TIVX_MEM_INTERNAL_L3 heap