CCS/IWR1443BOOST: IWR1443: Porting Gesture Swipe Demo to SDK2.0

ZIHENG LI

Part Number: IWR1443BOOST
Other Parts Discussed in Thread: MMWAVE-SDK

Tool/software: Code Composer Studio

Hey,

I'm trying to port the gesture_swipe demo given in mmWave industrial toolbox 3.3 to the newest SDK. I am aware that the current version of the gesture demo supports SDK1.2 and I'm trying to make it run on SDK2.1.

Right now I'm trying to understand the process with which the gesture is interpreted. Here's what I found, please correct me if I'm wrong:

In comparing the main_swipe.c in the gesture demo and main.c of the newest visualizer demo, apart from the difference due to the fact they implement different versions of SDK, the only difference is that in 'MmwDemo_dataPathTask,' the gesture demo has these additional lines:

		//Gesture_findNumDetections(dataPathObj);
        //numDetectedObjects=Gesture_findNumDetections(4000,dataPathObj );
        //counterstart = Pmu_getCount(0);
        Gesture_findNumDetections(dataPathObj->numRangeBins, dataPathObj->numDopplerBins, dataPathObj->rangeDopplerLogMagMatrix,8000 ,gestureMetrics,maxIndices);

        //counterend = Pmu_getCount(0);

        counterstart = Pmu_getCount(0);
        angleIdx=Gesture_angleEst(dataPathObj,maxIndices);
        counterend = Pmu_getCount(0);
...

The above code calls the procedure in "gesture_swipe.c".

There's also a file called "handCraftedClassifier.c" where it predicts detected gestures.

Those are my initial views of the code. If it is correct, I think my question is: how does the gesture detection pipeline work and how does it identify different gestures? Getting on with this lab has been hard because of the lack of comments in the code.

I would much appreciate it if someone helps me understand this lab.

Many thanks in advance.

Ziheng

over 6 years ago

0 Raghunandan Kamath over 6 years ago

TI__Mastermind 22155 points

Hi Ziheng,

We will have our expert reply to your query on the gesture code. Thanks for your patience.

-Raghu

0 ZIHENG LI over 6 years ago in reply to Raghunandan Kamath

Prodigy 250 points

Thank you Raghu,

I'll be expecting answers then.

-Ziheng

0 Akash Gondalia over 6 years ago

TI__Guru 50240 points

Hi Ziheng,

Unfortunately the Gesture swipe demo is not yet compatible with mmWave-SDK 2.1.

You shouldn't need to change any of the gesture specific code in order to port it over. There is a separate thread with requirements for migration, e2e.ti.com/.../752805

Let me know if you have more questions.

Cheers,
Akash

0 ZIHENG LI over 6 years ago in reply to Akash Gondalia

Prodigy 250 points

Thank you, Akash,

I'm trying to port the gesture demo but encountered some problem. The program runs into an exception in Gesture_angleEst call in MmwDemo_dataPathTask. Here is the modified MmwDemo_dataPathTask and the whole ccs project is attached:

gesture_swipe_port.zip

void MmwDemo_dataPathTask(UArg arg0, UArg arg1)
{
    MmwDemo_DataPathObj *dataPathObj = &gMmwMCB.dataPathObj;
    //uint16_t numDetectedObjects;
    uint16_t i;//,*predetMatrix,*ptrRadarCube;
    short angleIdx;
    uint16_t numDetectedObjects;
    uint32_t startTime, transmitOutStartTime;
    for(i=0;i<lengesture;i++) {
        wt_rangearr[i] = 0;
        wt_dopplerarr[i] = 0;
        range_disparr[i] = 0;
        vel_disparr[i] = 0;
        anglearr[i] = 0;
        instenergyarr[i] = 0;
        numdetectptsarr[i] = 0;
    }
    uint32_t txOrder[SYS_COMMON_NUM_TX_ANTENNAS] = {0,2,1};
    while(1)
    {
        Semaphore_pend(dataPathObj->frameStart_semHandle, BIOS_WAIT_FOREVER);

        Load_update();
        dataPathObj->timingInfo.interFrameCPULoad=Load_getCPULoad();

        MmwDemo_dataPathWait1D(dataPathObj);
        /* 1st Dimension FFT done! */

        Load_update();
        dataPathObj->timingInfo.activeFrameCPULoad=Load_getCPULoad();

        startTime = Pmu_getCount(0);

        if(dataPathObj->cliCfg->calibDcRangeSigCfg.enabled)
        {
             if (dataPathObj->cliCfg->calibDcRangeSigCfg.numAvgChirps <  dataPathObj->numDopplerBins)
             {
                 dataPathObj->cliCfg->calibDcRangeSigCfg.enabled = 0;
                 dataPathObj->dcRangeForcedDisableCntr++;
             }
             else
             {
                 MmwDemo_dcRangeSignatureCompensation(dataPathObj);
             }
        }

        /* Clutter removal */
        if (dataPathObj->cliCfg->clutterRemovalCfg.enabled)
        {
            int32_t rngIdx, antIdx, dopIdx;
            cmplx16ImRe_t *fftOut1D = (cmplx16ImRe_t *) dataPathObj->radarCube;
            cmplx32ImRe_t meanVal;

            for (rngIdx = 0; rngIdx < dataPathObj->numRangeBins; rngIdx++)
            {
                for (antIdx = 0; antIdx < dataPathObj->numVirtualAntennas; antIdx++)
                {
                    meanVal.real = 0;
                    meanVal.imag = 0;
                    for (dopIdx = 0; dopIdx < dataPathObj->numDopplerBins; dopIdx++)
                    {
                        meanVal.real += fftOut1D[rngIdx*(dataPathObj->numDopplerBins*dataPathObj->numVirtualAntennas) +
                                                 antIdx + dopIdx*(dataPathObj->numVirtualAntennas)].real;
                        meanVal.imag += fftOut1D[rngIdx*(dataPathObj->numDopplerBins*dataPathObj->numVirtualAntennas) +
                                                 antIdx + dopIdx*(dataPathObj->numVirtualAntennas)].imag;
                    }
                    meanVal.real = meanVal.real/dataPathObj->numDopplerBins;
                    meanVal.imag = meanVal.imag/dataPathObj->numDopplerBins;
                    for (dopIdx = 0; dopIdx < dataPathObj->numDopplerBins; dopIdx++)
                    {
                        fftOut1D[rngIdx*(dataPathObj->numDopplerBins*dataPathObj->numVirtualAntennas) +
                                 antIdx + dopIdx*(dataPathObj->numVirtualAntennas)].real -= meanVal.real;
                        fftOut1D[rngIdx*(dataPathObj->numDopplerBins*dataPathObj->numVirtualAntennas) +
                                 antIdx + dopIdx*(dataPathObj->numVirtualAntennas)].imag -= meanVal.imag;
                    }
                }
            }
        }

        MmwDemo_process2D(dataPathObj);
        /* 2nd Dimension FFT done! */

        /* Procedure for range bias measurement and Rx channels gain/phase offset measurement */
        if(dataPathObj->cliCommonCfg->measureRxChanCfg.enabled)
        {
            MmwDemo_rangeBiasRxChPhaseMeasure(
                    dataPathObj->cliCommonCfg->measureRxChanCfg.targetDistance,
                    dataPathObj->rangeResolution,
                    dataPathObj->cliCommonCfg->measureRxChanCfg.searchWinSize,
                    dataPathObj->rangeDopplerLogMagMatrix,
                    dataPathObj->numDopplerBins,
                    dataPathObj->numVirtualAntennas,
                    dataPathObj->numVirtualAntennas * dataPathObj->numDopplerBins,
                    dataPathObj->radarCube,
                    dataPathObj->numRxAntennas,
                    dataPathObj->numTxAntennas,
                    txOrder,
                    &dataPathObj->cliCommonCfg->compRxChanCfg);

        }

        MmwDemo_processCfar(dataPathObj, &numDetectedObjects);
        /* CFAR done! */

        /* Postprocessing/angle estimation */
        dataPathObj->numHwaCfarDetections = numDetectedObjects;
        MmwDemo_processAngle(dataPathObj);

        /* Calculate noise floor. This is for EVM diagnostics only. Assumes a stationary scene */
        dataPathObj->noiseEnergy = calcNoiseFloor (dataPathObj->radarCube, dataPathObj->numDopplerBins,
                dataPathObj->numRangeBins, dataPathObj->numVirtualAntennas);

        transmitOutStartTime = Pmu_getCount(0);


        /* Sending range bias and Rx channel phase offset measurements to MSS and from there to CLI */
         if(dataPathObj->cliCommonCfg->measureRxChanCfg.enabled)
         {
             MmwDemo_measurementResultOutput (dataPathObj);
         }

        /* Send user data over LVDS */
        /*Unverified code. Conflicts with data path processing and should not be used.*/
        if(dataPathObj->cliCfg->lvdsStreamCfg.isSwEnabled == 1)
        {
            MmwDemo_transferLVDSUserData(dataPathObj);
        }

//        MmwDemo_transmitProcessedOutput(gMmwMCB.loggingUartHandle,
//                                        dataPathObj);

        dataPathObj->timingInfo.transmitOutputCycles = Pmu_getCount(0) - transmitOutStartTime;

        /* Processing cycles for 2D, CFAR, Azimuth/Elevation
           processing excluding sending out data */
        dataPathObj->timingInfo.interFrameProcessingEndTime = Pmu_getCount(0);
        dataPathObj->timingInfo.interFrameProcCycles = dataPathObj->timingInfo.interFrameProcessingEndTime - startTime -
            dataPathObj->timingInfo.transmitOutputCycles;

        dataPathObj->interFrameProcToken--;





        /******************************************************************************** Added gesture code*/

        //Gesture_findNumDetections(dataPathObj);
        //numDetectedObjects=Gesture_findNumDetections(4000,dataPathObj );
        //counterstart = Pmu_getCount(0);
        Gesture_findNumDetections(dataPathObj->numRangeBins, dataPathObj->numDopplerBins, dataPathObj->rangeDopplerLogMagMatrix,8000 ,gestureMetrics,maxIndices);

        //counterend = Pmu_getCount(0);

        counterstart = Pmu_getCount(0);
        angleIdx=Gesture_angleEst(dataPathObj,maxIndices);
        counterend = Pmu_getCount(0);
        counter_delta=counterend-counterstart;
         gestureMetrics[5]= (float)angleIdx;
         gestureMetrics[6]=maxIndices[0];
         gestureMetrics[7]=maxIndices[1];
         gestureMetrics[8]=pktCounter++;
         //counterstart = Pmu_getCount(0);
         for(i=0;i<(lengesture-1);i++) {
             wt_rangearr[i] = wt_rangearr[i+1];
             wt_dopplerarr[i] = wt_dopplerarr[i+1];
             range_disparr[i] = range_disparr[i+1];
             vel_disparr[i] = vel_disparr[i+1];
             instenergyarr[i] = instenergyarr[i+1];
             numdetectptsarr[i] = numdetectptsarr[i+1];
             anglearr[i] = anglearr[i+1];
         }
         wt_rangearr[lengesture-1] = gestureMetrics[3]*range_res;
         wt_dopplerarr[lengesture-1] = gestureMetrics[2]*vel_res;
         range_disparr[lengesture-1] = gestureMetrics[9]*range_res;
         vel_disparr[lengesture-1] = gestureMetrics[10]*vel_res;
         instenergyarr[lengesture-1] = gestureMetrics[4];
         numdetectptsarr[lengesture-1] = gestureMetrics[1];
         anglearr[lengesture-1] = (180.0*asin((gestureMetrics[5]/32.0)))/PI;

         //counterend = Pmu_getCount(0);


         /*Ziheng(Leo) 3/35/19
          * Observation: it seems like the code is waiting for 20 'for prediction' before making an
          * actual prediction.
          */
         if(prediction_result!=1 || counter_forprediction>20) {

         prediction_result = Handmadeclassifier_prediction(wt_rangearr,wt_dopplerarr,range_disparr,vel_disparr,instenergyarr,numdetectptsarr,anglearr);

         gestureMetrics[11] = prediction_result;
         counter_forprediction = 0;
         }
         else {
             counter_forprediction++;
             gestureMetrics[11] = 0;
         }

         //gestureMetrics[11] = prediction_result;

         /**********fill in the zero-doppler range******************/
         /*predetMatrix=dataPathObj->rangeDopplerLogMagMatrix;
         for(i=0;i<7;i++)
         {
            gestureMetrics[i+5]=*predetMatrix;
            predetMatrix+=dataPathObj->numDopplerBins;
        }*/
        /*
        ptrRadarCube= (uint16_t*)(dataPathObj->radarCube);
         for(i=0;i<8;i++) //8 range bins : first antenna : real only
         {
            gestureMetrics[i+5]=*ptrRadarCube++;
            ptrRadarCube+=(dataPathObj->numDopplerBins)*(dataPathObj->numVirtualAntennas)*2;
        }*/


        GestureDemo_transmitProcessedOutput(gMmwMCB.loggingUartHandle,

                                        gestureMetrics,48);
        /******************************************************************************** End of Added gesture code*/



        




        if(dataPathObj->datapathStopped == true)
        {
            MmwDemo_notifyDathPathStop();
        }
        else
        {
            /* Prepare for next frame */
            MmwDemo_config1D_HWA(dataPathObj);
            MmwDemo_dataPathTrigger1D(dataPathObj);
        }
    }
}

I put the gesture code after all the processing seems to be finished. But it always goes down near the azimuth virtual antennas in the 'Gesture_angleEst'. See picture below, the program runs into a binary file and then to exception handler after that:

And here's the stack trace and the console output:

Console:

[Cortex_R4_0] Debug: Launched the Initialization Task
Exception occurred in ThreadType_Task.
Task handle: 0x8004270.
Task stack base: 0x80042c0.
Task stack size: 0xc00.
R0 = 0x00000008  R8  = 0x08008430
R1 = 0x52030000  R9  = 0x08008000
R2 = 0x511fffc0  R10 = 0x08008030
R3 = 0x00000008  R11 = 0x9e84fbbe
R4 = 0x08008630  R12 = 0x00000010
R5 = 0x00000010  SP(R13) = 0x08004e38
R6 = 0x0800800c  LR(R14) = 0x000026a1
R7 = 0x00000040  PC(R15) = 0x00016274
PSR = 0x080c019f
DFSR = 0x0000000d  IFSR = 0x00000000
DFAR = 0x511fffc0  IFAR = 0x00000000
{module#40}: line 205: error {id:0x180000, args:[0x16274, 0x26a1]}
xdc.runtime.Error.raise: terminating execution

I'm using the Visualizer Web app to set up the chirp profile (instead of the Gesture Gui because it does not seem to work with ES3.0's visualizer demo), which could be a possible cause of the problem?

Could you help me with this and please let me know if you need additional information?

Thanks!

Best,

Ziheng

0 ZIHENG LI over 6 years ago in reply to Akash Gondalia

Prodigy 250 points

Hey Akash,

Could you help me with this bug?

Thanks!

Ziheng

0 Akash Gondalia over 6 years ago in reply to ZIHENG LI

TI__Guru 50240 points

Hi Ziheng,

As per my previous post there shouldn't be any difference in the Gesture specific part of the code. The changes are related to both initialization and architecture. I'd recommend looking at either the 14xx SDK demo or the 14xx Vital Signs Demo because both of these are for ES3.0 silicon on the mmWave-SDK2.1.

Much of this is discussed in this thread on migration, e2e.ti.com/.../752805

There is also a migration guide in the mmWave-SDK2.1. release notes.

Cheers,
Akash

Sensors

Sensors forum

CCS/IWR1443BOOST: IWR1443: Porting Gesture Swipe Demo to SDK2.0