This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS/CC1310: Radio locks up after electrostatic discharge test and cannot be recovered by watchdog reset.

Part Number: CC1310

Tool/software: TI-RTOS

I have a product based on the CC1310 / TI-RTOS.

It has an external antenna with a SMA connector.

During EMC compliance testing the SMA connector (with antenna fitted) was subject to electrostatic discharge.  The CC1310's radio locked up.  The watchdog was triggered and rebooted the CC1310.

My problem is that the reboot does not always succeed.  About 1/2 the time the CC1310 totally locks up, not even the main or 32.768 KHz oscillators show any activity when in this state.  A power cycle clears the fault.

I really don't want to have to use an external watchdog chip, when the CC1310 already has a  watchdog that should be capable of resetting itself.

I have pasted some code below to help explain the problem.  It is based on the rfPacketRx example.

The example listens for packets to arrive with a 500 ms window set by using "RF_cmdPropRxAdv.endTrigger.triggerType = TRIG_REL_START;"

Prior to calling "RF_runCmd" I turn on a LED and then clear it when "RF_runCmd" completes.

When the eletrostatic discharge occurs (you will need to apply the discharge a few times), the LED stays illuminated indicating that the "RF_runCmd" has not returned.

After 10 seconds the watchdog's callback function is executed. In the callback I turn on another LED to show it has been triggered and then I call "RF_flushCmd", which appears to block (the next line of code turns off the first LED, but it stays lit so my assumption is that the "RF_flushCmd" blocks.  At this point, the external crystal oscillators are still running.

After another 10 seconds the watchdog performs a device reset, however this is not always successful and the CC1310 does not reboot and its crystal oscillators stop running.

I hope you can tell me how to recover the CC1310 from this situation as the EMC directive requires the device to self-recover without manual intervention.

Regards,

Tony Bland.

/***** Includes *****/
#include <stdlib.h>
#include <xdc/std.h>
#include <xdc/cfg/global.h>
#include <xdc/runtime/System.h>

#include <ti/sysbios/BIOS.h>
#include <ti/sysbios/knl/Task.h>

/* Drivers */
#include <ti/drivers/rf/RF.h>
#include <ti/drivers/PIN.h>

#include <ti/drivers/Watchdog.h>
#include <ti/posix/ccs/unistd.h>


#ifdef DEVICE_FAMILY
    #undef DEVICE_FAMILY_PATH
    #define DEVICE_FAMILY_PATH(x) <ti/devices/DEVICE_FAMILY/x>
    #include DEVICE_FAMILY_PATH(driverlib/rf_prop_mailbox.h)
#else
    #error "You must define DEVICE_FAMILY at the project level as one of cc26x0, cc26x0r2, cc13x0, etc."
#endif

/* Board Header files */
#include "Board.h"

#include "RFQueue.h"
#include "smartrf_settings/smartrf_settings.h"

#include <stdlib.h>

/* Pin driver handle */
static PIN_Handle ledPinHandle;
static PIN_State ledPinState;

/*
 * Application LED pin configuration table:
 *   - All LEDs board LEDs are off.
 */
#define FIRSTLED IOID_1
#define SECONDLED IOID_2
#define LED_ON 0
#define LED_OFF 1
#define START_STATE PIN_GPIO_HIGH
//#define FIRSTLED CC1310_LAUNCHXL_PIN_RLED
//#define SECONDLED CC1310_LAUNCHXL_PIN_GLED
//#define LED_ON CC1310_LAUNCHXL_GPIO_LED_ON
//#define LED_OFF CC1310_LAUNCHXL_GPIO_LED_OFF
//#define START_STATE PIN_GPIO_LOW


PIN_Config pinTable[] =
{
    FIRSTLED | PIN_GPIO_OUTPUT_EN | START_STATE | PIN_PUSHPULL | PIN_DRVSTR_MAX,
    SECONDLED | PIN_GPIO_OUTPUT_EN | START_STATE | PIN_PUSHPULL | PIN_DRVSTR_MAX,
    PIN_TERMINATE
};


/***** Defines *****/
#define RX_TASK_STACK_SIZE 1024
#define RX_TASK_PRIORITY   2

/* Packet RX Configuration */
#define DATA_ENTRY_HEADER_SIZE 8  /* Constant header size of a Generic Data Entry */
#define MAX_LENGTH             128 /* Max length byte the radio will accept */
#define NUM_DATA_ENTRIES       2  /* NOTE: Only two data entries supported at the moment */
#define NUM_APPENDED_BYTES     8  /* The Data Entries data field will contain:
                                   * 1 Header byte (RF_cmdPropRx.rxConf.bIncludeHdr = 0x1)
                                   * Max 30 payload bytes
                                   * 1 status byte (RF_cmdPropRx.rxConf.bAppendStatus = 0x1) */

#define HEADER_SIZE 4
#define CRC_SIZE 2


/***** Prototypes *****/
static void rxTaskFunction(UArg arg0, UArg arg1);
static void callback(RF_Handle h, RF_CmdHandle ch, RF_EventMask e);

/***** Variable declarations *****/
static Task_Params rxTaskParams;
Task_Struct rxTask;    /* not static so you can see in ROV */
static uint8_t rxTaskStack[RX_TASK_STACK_SIZE];

static RF_Object rfObject;
static RF_Handle rfHandle;

// watchdog
Watchdog_Handle watchdogHandle;
Watchdog_Params params;


// function body just to satisfy a link error from my RTOS build, where this function is declared in ccfg.c
void ClearWatchdog()
{

}


// set the LED after closing
void watchdogCallback(uintptr_t unused)
{
    PIN_setOutputValue(ledPinHandle, SECONDLED,LED_ON);

    if (rfHandle != NULL)
    {
        // abort any pending command
        RF_flushCmd(rfHandle, RF_CMDHANDLE_FLUSH_ALL, 0);
        // turn off the first LED
        PIN_setOutputValue(ledPinHandle, FIRSTLED,LED_OFF);
        // try and power down the radio
        RF_yield(rfHandle);
        // this call never returns when the radio locks up after electrostatic discharge event.
        RF_close(rfHandle);
        rfHandle = NULL;
    }
}


static rfc_propRxOutput_t rxStatistics_prop; // Output structure for CMD_PROP_RX


/* Buffer which contains all Data Entries for receiving data.
 * Pragmas are needed to make sure this buffer is 4 byte aligned (requirement from the RF Core) */
#if defined(__TI_COMPILER_VERSION__)
    #pragma DATA_ALIGN (rxDataEntryBuffer, 4);
        static uint8_t rxDataEntryBuffer[RF_QUEUE_DATA_ENTRY_BUFFER_SIZE(NUM_DATA_ENTRIES,
                                                                 MAX_LENGTH,
                                                                 NUM_APPENDED_BYTES)];
#elif defined(__IAR_SYSTEMS_ICC__)
    #pragma data_alignment = 4
        static uint8_t rxDataEntryBuffer[RF_QUEUE_DATA_ENTRY_BUFFER_SIZE(NUM_DATA_ENTRIES,
                                                                 MAX_LENGTH,
                                                                 NUM_APPENDED_BYTES)];
#elif defined(__GNUC__)
        static uint8_t rxDataEntryBuffer [RF_QUEUE_DATA_ENTRY_BUFFER_SIZE(NUM_DATA_ENTRIES,
            MAX_LENGTH, NUM_APPENDED_BYTES)] __attribute__ ((aligned (4)));
#else
    #error This compiler is not supported.
#endif

/* Receive dataQueue for RF Core to fill in data */
static dataQueue_t dataQueue;
static rfc_dataEntryGeneral_t* currentDataEntry;
static uint8_t packetLength;
static uint8_t* packetDataPointer;

static PIN_Handle pinHandle;

static uint8_t packet[MAX_LENGTH + NUM_APPENDED_BYTES - 1]; /* The length byte is stored in a separate variable */

int8_t min_rssi;

/***** Function definitions *****/
void RxTask_init(PIN_Handle ledPinHandle) {
    pinHandle = ledPinHandle;

    Task_Params_init(&rxTaskParams);
    rxTaskParams.stackSize = RX_TASK_STACK_SIZE;
    rxTaskParams.priority = RX_TASK_PRIORITY;
    rxTaskParams.stack = &rxTaskStack;
    rxTaskParams.arg0 = (UInt)1000000;

    Task_construct(&rxTask, rxTaskFunction, &rxTaskParams, NULL);
}

static void rxTaskFunction(UArg arg0, UArg arg1)
{
    RF_Params rfParams;
    RF_Params_init(&rfParams);

    if( RFQueue_defineQueue(&dataQueue,
                            rxDataEntryBuffer,
                            sizeof(rxDataEntryBuffer),
                            NUM_DATA_ENTRIES,
                            MAX_LENGTH + NUM_APPENDED_BYTES))
    {
        /* Failed to allocate space for all data entries */
        while(1);
    }

    /* Modify CMD_PROP_RX command for application needs */
    RF_cmdPropRx.pQueue = &dataQueue;           /* Set the Data Entity queue for received data */
    RF_cmdPropRx.pOutput = (uint8_t*)&rxStatistics_prop;
    RF_cmdPropRx.rxConf.bAutoFlushIgnored = 0;  /* Don't Discard ignored packets from Rx queue */
    RF_cmdPropRx.rxConf.bAutoFlushCrcErr = 0;   /* Don't Discard packets with CRC error from Rx queue */
    RF_cmdPropRx.maxPktLen = MAX_LENGTH;        /* Implement packet length filtering to avoid PROP_ERROR_RXBUF */
    RF_cmdPropRx.pktConf.bRepeatOk = 0;         // exit on OK
    RF_cmdPropRx.pktConf.bRepeatNok = 0;        // exit on NOT OK


    // try advanced RX
    RF_cmdPropRxAdv.pQueue = &dataQueue;
    // output info for RSSI
    RF_cmdPropRxAdv.pOutput = (uint8_t*)&rxStatistics_prop;

    // trigger now
    RF_cmdPropRxAdv.startTrigger.triggerType = TRIG_NOW;
    RF_cmdPropRxAdv.startTrigger.pastTrig = 1;
    RF_cmdPropRxAdv.startTime = 0;
    // end trigger
    RF_cmdPropRxAdv.endTrigger.triggerType = TRIG_REL_START;
    RF_cmdPropRxAdv.endTime =  500 *   1000 *4; // 500 ms to recieve

    RF_cmdPropRxAdv.condition.rule = COND_NEVER;


    // turn off CRC and status and add RSSI
    // then decode the CRC manually
    // length field should terminate the transmission early.


    Watchdog_init();
    // defaults are good
    Watchdog_Params_init(&params);
    params.resetMode = Watchdog_RESET_ON;
    params.callbackFxn = (Watchdog_Callback)watchdogCallback;
    watchdogHandle = Watchdog_open(Board_WATCHDOG0, &params);
    Watchdog_setReload(watchdogHandle, Watchdog_convertMsToTicks(watchdogHandle,10000));



    /* Request access to the radio */
    rfHandle = RF_open(&rfObject, &RF_prop, (RF_RadioSetup*)&RF_cmdPropRadioDivSetup, &rfParams);

    /* Set the frequency */
    RF_postCmd(rfHandle, (RF_Op*)&RF_cmdFs, RF_PriorityNormal, NULL, 0);


    while(1)
    {
        // show we are waiting for a packet
        PIN_setOutputValue(pinHandle, FIRSTLED,LED_ON);
        // block waiting for data
        RF_runCmd(rfHandle, (RF_Op*)&RF_cmdPropRxAdv, RF_PriorityNormal, &callback, IRQ_RX_ENTRY_DONE);
        // clear the watchdog
        Watchdog_clear(watchdogHandle);
        // show we have finished waiting
        PIN_setOutputValue(pinHandle, FIRSTLED,LED_OFF);
        // sleep for 500 ms
        usleep(500000);
    }
}

// dummy function call
unsigned short CalcRxCheckSum(unsigned char *ptrStart,unsigned char len)
{
    unsigned short start = 0xffff;
    return start;
}



void callback(RF_Handle h, RF_CmdHandle ch, RF_EventMask e)
{
    if (e & RF_EventRxEntryDone)
    {

        /* Get current unhandled data entry */
        currentDataEntry = RFQueue_getDataEntry();

        // Handle the packet data, located at &currentDataEntry->data:
        packetDataPointer = (uint8_t*)(&currentDataEntry->data);
        uint8_t maxpacketLength      = *(uint8_t*)(&currentDataEntry->length );
        // packet length is the first byte as the 4 byte header sequence is reversed by the packet engine
        packetLength  = *packetDataPointer;
        if ((packetLength > HEADER_SIZE+CRC_SIZE) && (packetLength < maxpacketLength))
        {
            int8_t rssi=0;

            /* Copy the payload + the status byte to the packet variable */
            memcpy(packet, packetDataPointer, (packetLength + 1 + NUM_APPENDED_BYTES));
            //memcpy(packet, packetDataPointer, maxpacketLength);

            unsigned short calcCheckSum = CalcRxCheckSum(packet,packetLength-CRC_SIZE);

            unsigned short headerCheckSum = *((unsigned short *)(packet+packetLength-CRC_SIZE));

            if (headerCheckSum == calcCheckSum)
            {
                rssi = rxStatistics_prop.lastRssi;

                if (rssi <min_rssi)
                {
                    min_rssi = rssi;
                }

                /* Toggle pin to indicate RX */
               // PIN_setOutputValue(pinHandle, Board_PIN_LED2,!PIN_getOutputValue(Board_PIN_LED2));
            }


        }


        RFQueue_nextEntry();
    }
}

/*
 *  ======== main ========
 */
int main(void)
{
    /* Call driver init functions. */
    Board_initGeneral();

    /* Open LED pins */
    ledPinHandle = PIN_open(&ledPinState, pinTable);
    if(!ledPinHandle)
    {
        System_abort("Error initializing board LED pins\n");
    }

    min_rssi = 127;

    /* Initialize task */
    RxTask_init(ledPinHandle);

    /* Start BIOS */
    BIOS_start();

    return (0);
}




 

  • Hello Antony Bland,

    The REASON for this behavior is:

    Is that RF_yield or close APIs should not be called from a callback and only from a task context. We cannot guarantee that the behavior of these will do what it is expected to do from a callback. The solution is to not call the these APIs from a callback context and let the MCU reset, but since the watchdog only does a warm reset this will also cause the MCU to fall into an unknown state.

    Here the Description of the Watchdog:

    The watchdog causes a warm reset in the system. This warm reset can be blocked by ICEPick, which is useful for debugging. When ICEPick is asserted, the warm reset is blocked from the rest of the system; however, watchdog itself is reset. During normal operation the Warm Reset Converted to System Reset feature must be enabled to ensure that the WDT performs a full system reset.   Because warm reset does not reset the analog parts of the device, such as the radio, doing a warm reset will put the device in a partly unknown state. It is therefore strongly recommended to enable Warm Reset Converted to System Reset feature

    SOLUTION:

    Warm reset can be programmed with the PRCM:WARMRESET.WR_TO_PINRESET register to result in a

    system reset when any warm reset source is triggered (TI strongly recommends enabling the Warm Reset Converted to System Reset feature)

  • Thank you for the fast response, however I have "Boot.trimDevice = true;" set in my .cfg file. This causes SetupTrimDevice function to be called which in turn, calls "HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 1;".

    I removed the calls to the RF_XXXX functions from the watchdog callback.
    To be on the safe side I have explicitly called HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 1 in my application, but the behaviour is the same, with the CC1310 locking up into an unrecoverable state after the watchdog reset.

    However if I clear the bit by calling HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 0 the cc1310 does reboot eventually.
    The behaviour is: The first watchdog callback sets the 'SECONDLED' and the watchdog reset clears it 10 seconds later, however the 'FRSTLED' is not cleared. After another 10 seconds the 'SECONDLED'is set again to indicate another watchdog callback and after another 10 seconds both LEDS clear and the receive sequence restarts.
    It seems like it takes 2 watchdog callbacks and 2 watchdog resets, but the device does eventually reboot.
    This behaviour would seem to contradict the documentation in the technical reference manual and code called by the SetupTrimDevice function.
    Is this the behaviour that you would expect?

    Regards,

    Tony Bland
  • Thank you for the fast response, however I have "Boot.trimDevice = true;" set in my .cfg file. This causes SetupTrimDevice function to be called which in turn, calls "HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 1;".

    I removed the calls to the RF_XXXX functions from the watchdog callback.
    To be on the safe side I have explicitly called HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 1 in my application, but the behaviour is the same, with the CC1310 locking up into an unrecoverable state after the watchdog reset.

    However if I clear the bit by calling HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 0 the cc1310 does reboot eventually.
    The behaviour is: The first watchdog callback sets the 'SECONDLED' and the watchdog reset clears it 10 seconds later, however the 'FRSTLED' is not cleared. After another 10 seconds the 'SECONDLED'is set again to indicate another watchdog callback and after another 10 seconds both LEDS clear and the receive sequence restarts.
    It seems like it takes 2 watchdog callbacks and 2 watchdog resets, but the device does eventually reboot.
    This behaviour would seem to contradict the documentation in the technical reference manual and code called by the SetupTrimDevice function.
    Is this the behaviour that you would expect?

    Regards,

    Tony Bland
  • Hello Tony,

    It looks like the Watchdog id behaving as described on page 1257 of the technical reference manual (www.ti.com/.../swcu117h.pdf).

    "The WDT can be configured to generate an interrupt to the controller on its first time-out, and to generate
    a reset signal on its second time-out. Once the WDT has been configured, the lock register can be written
    to prevent the timer configuration from being inadvertently altered."

    Regards,
    AB
  • Dear AB,
    Thank you for your reply. I understand the two phase nature of the Watchdog, this is expected behaviour.
    My problem is that TI-RTOS calls "HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 1;".
    in SetupTrimDevice and with this setting configured by TI-RTOS and in accordance with the technical reference manual, the watchdog is STILL NOT ABLE to reboot the CC1310 into a working state.
    Only by using the setting HWREGBITW( PRCM_BASE + PRCM_O_WARMRESET, PRCM_WARMRESET_WR_TO_PINRESET_BITN ) = 0 that the technical reference manual explicitly recommends against using and which also contradicts the code in TI-RTOS is there any hope of the CC1310 resetting to a known state.

    I want to know how to RELIABLY reset the CC1310 and all of its subsystems, when using TI-RTOS.
    Using an electrostatic discharge, I can lock up the CC1310's radio. I need to be able to recover from this and so does anyone hoping to CE certify the CC1310. Please can you tell me how to reliably recover from this lockup.

    Regards,

    Tony.
  • Hello,

    Another option is to reset manually inside the watchdog callback, this will ensure that the MCU resets and not have to worry about the watchdog doing it for you.

    use the following call:

    SysCtrlSystemReset()

    Regards,
    AB
  • Hi AB,
    I have tried calling SysCtrlSystemReset() in the watchdog callback. It does not recover the CC1310 to a working state after the radio has locked up.
    Regards,
    Tony
  • Hi AB,

    Have you any more information on this issue?

    Regards,

    Tony.

  • Hi AB,

    Have you any more information on this issue. I am worried that I have to use a 2 phase warm boot process to recover the CC1310 that contradicts all the TI recommendations. Following the TI recommendations leads to a totally locked up CC1310.
    Have you been able to reproduce the behaviour that I reported?

    Regards,

    Tony.