IWR1843BOOST: Fourth software start causes crash

Ansis Sprugevics

Part Number: IWR1843BOOST

Hello,

I am using Industrial Toolbox 4.8.0. Why not 4.9.0? Because the Traffic Monitoring MSS program crashes with -30109 when running sensorStart and I wasn't able to figure out the issue.

When loading the configuration taken from the lab 3D chirp config the radar starts fine and starts sending data. However, exactly on the 4th startup it crashes with "error {id:0x110000, args:[0x801d574, 0x1388]}".

I've determined from the stack trace that this happens because HeapMem_allocUnprotected() Can't free up the necessary memory.

Here is a snippet of the stack trace:

I couldn't figure out how to export the complete stack trace. If you need the full trace I can try to figure something out.

Is this an issue with the garbage collection? Is the radar not freeing up necessary memory so it eventually gets clogged up?

Best regards,
Ansis

over 3 years ago

0 Alec May over 3 years ago

TI__Genius 17265 points

Hi Ansis,

Would you mind attaching the full debug output log from ccs? This will help me to follow what is happening across all your starts and stops. Like you suggested, I would expect this is occurring due to a memory leak somewhere in the application.

Best Regards,
Alec

0 Ansis Sprugevics over 3 years ago in reply to Alec May

Prodigy 30 points

I couldn't find how to export the stack trace and I also can't find how to export a debug log (or even view it).

I'm assuming you don't mean the console, right?

[C674X_0] Debug: DPM Module Sync is done
[Cortex_R4_0] **********************************************
Debug: Launching the MMW Demo on MSS
**********************************************
Debug: Launched the Initialization Task
Debug: mmWave Control Initialization was successful
Debug: mmWave Control Synchronization was successful
Debug: CLI is operational
Debug: Sending rlRfSetLdoBypassConfig with 0 0 0
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
   System Heap(TCMB)        98304        31360        66944         3072
                  L3       786176       491520       294656
      localRam(TCMB)         4096          512         3584
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
     System Heap(L2)        32768        16008        16760            0
                  L3       786176        32768       753408
        localRam(L2)        50176        23848        26328
        localRam(L1)        16384         5872        10512
Starting Sensor (issuing MMWave_start)
[C674X_0] Data Path Stopped (last frame processing done)
DSS Task Stack Usage (Note: Task Stack Usage) ==========
            initTask         4096          716         3380
           Task Name         Size         Used         Free
          ObjDet DPM         4096         2272         1824
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048          748         1300
[Cortex_R4_0] Data Path Stopped (last frame processing done)
============================================
MSS Task Stack Usage (Note: Task Stack Usage) ==========
           Task Name         Size         Used         Free
                Init         2048         1744          304
      Mmwave Control         3072         2368          704
          ObjDet DPM         8192         6128         2064
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048         1048         1000
Sensor has been stopped: startCount: 1 stopCount 1
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
   System Heap(TCMB)        98304        71856        26448         3072
                  L3       786176       491520       294656
      localRam(TCMB)         4096          512         3584
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
     System Heap(L2)        32768        16008        16760            0
                  L3       294656        32768       261888
        localRam(L2)        50176        23848        26328
        localRam(L1)        16384         5872        10512
Starting Sensor (issuing MMWave_start)
[C674X_0] Data Path Stopped (last frame processing done)
DSS Task Stack Usage (Note: Task Stack Usage) ==========
            initTask         4096          716         3380
           Task Name         Size         Used         Free
          ObjDet DPM         4096         2272         1824
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048          748         1300
[Cortex_R4_0] Data Path Stopped (last frame processing done)
============================================
MSS Task Stack Usage (Note: Task Stack Usage) ==========
           Task Name         Size         Used         Free
                Init         2048         1744          304
      Mmwave Control         3072         2368          704
          ObjDet DPM         8192         6128         2064
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048         1080          968
Sensor has been stopped: startCount: 2 stopCount 2
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
   System Heap(TCMB)        98304        83136        15168         3072
                  L3       786176       491520       294656
      localRam(TCMB)         4096          512         3584
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
     System Heap(L2)        32768        16008        16760            0
                  L3       294656        32768       261888
        localRam(L2)        50176        23848        26328
        localRam(L1)        16384         5872        10512
Starting Sensor (issuing MMWave_start)
[C674X_0] Data Path Stopped (last frame processing done)
DSS Task Stack Usage (Note: Task Stack Usage) ==========
            initTask         4096          716         3380
           Task Name         Size         Used         Free
          ObjDet DPM         4096         2272         1824
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048          748         1300
[Cortex_R4_0] Data Path Stopped (last frame processing done)
============================================
MSS Task Stack Usage (Note: Task Stack Usage) ==========
           Task Name         Size         Used         Free
                Init         2048         1744          304
      Mmwave Control         3072         2368          704
          ObjDet DPM         8192         6128         2064
HWI Stack (same as System Stack) Usage ============
                             Size         Used         Free
                             2048         1080          968
Sensor has been stopped: startCount: 3 stopCount 3
============ Heap Memory Stats ============
                             Size         Used         Free      DPCUsed
   System Heap(TCMB)        98304        94784         3520         3072
                  L3       786176       491520       294656
      localRam(TCMB)         4096          512         3584
{module#40}: line 221: error {id:0x110000, args:[0x801d574, 0x1388]}
xdc.runtime.Error.raise: terminating execution

0 Alec May over 3 years ago in reply to Ansis Sprugevics

TI__Genius 17265 points

Hi,

This is exactly what I was looking for. Doing a quick diff, it seems like the "System Heap(TCMB)" is leaking. Just to confirm, have you made any modifications to this demo, or is it the same as from the industrial toolbox?

Best Regards,
Alec

0 Ansis Sprugevics over 3 years ago in reply to Alec May

Prodigy 30 points

Hi,

The demo code is untouched. The commands themselves are also from the demo chirp configs. The only difference is that I am not using the MATLAB visualizer, but rather a Python script which sends out the CLI commands.

Best, Ansis

0 Alec May over 3 years ago in reply to Ansis Sprugevics

TI__Genius 17265 points

Hi Ansis,

After looking a bit further, if you take a look at <INDUSTRIAL_TOOLBOX_LOC\labs\common\src\dpu\trackerproc\src\trackerproc_3d.c
DPU_TrackerProc_deinit() function, the memory that is allocated during DPU_TrackerProc_init is deallocated, but the memory allocated during DPU_TrackerProc_config is NOT deallocated, which is likely causing the leak. We will look into this further and get a solution in a future version of the Industrial Toolbox.

In the meantime, you have two possible routes to resolve this so that you may unblock your project. The first is to add the necessary calls to MemoryP_ctrlFree() in the DPU_TrackerProc_deinit function to ensure that all memory allocated during DPU_TrackerProc_config is cleaned up. The second option is to restart the device instead of doing sensorStart/sensorStop calls. This will take slightly longer, and will consume slightly more power, but this should be acceptable if you are only evaluating the device at this time.

If you would like information on how to do a soft reset of the device, take a look at the following thread:
https://e2e.ti.com/support/sensors-group/sensors/f/sensors-forum/1034315/iwr6843isk-reset-from-usb-uart/3824060#3824060

Best Regards,
Alec

0 Ansis Sprugevics over 3 years ago in reply to Alec May

Prodigy 30 points

Hi,

I am attempting to fix the memory leak by introducing some MemoryP_ctrlFree() calls, but there's additional issues.

I noticed that the MSS project uses trackerproc_overhead, not trackerproc. I'm not sure why that is, but I added my fixes to both of them. However, I think it's either not compiling or something's wrong, because nothing has changed. I would like to debug the file, but the debug configuration can't find the file for some reason. It keeps suggesting debug symbols from some C:/Nitin/etc, which seems like the file path of some TI developer's computer, not mine.

I am very confused about this CCS behaviour. How do I force it to recompile the libraries and lookup the source only in my folders, not some other computer?

0 Ansis Sprugevics over 3 years ago in reply to Ansis Sprugevics

Prodigy 30 points

Nevermind, I figured out that you need to rebuild trackerproc by calling the .bat file included in the folder. My bad.

0 Ansis Sprugevics over 3 years ago in reply to Ansis Sprugevics

Prodigy 30 points

So far I discovered that:

objdetrangehwa.c DPC_ObjectDetection_deinit() does NOT call DPU_TrackerProc_deinit()
DPC_ObjectDetection_deinit() does not get called on sensorStop

Any idea why?

0 Alec May over 3 years ago in reply to Ansis Sprugevics

TI__Genius 17265 points

Hi,

I believe this is just an oversight. While most demos do support start->stop->start, they are not the typical use case so may not be tested as extensively, since the code is for evaluation and not intended to be used for customer production.

Best Regards,
Alec

Sensors

Sensors forum

IWR1843BOOST: Fourth software start causes crash