This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CC2652R7: ZED Randomly leaves network

Part Number: CC2652R7
Other Parts Discussed in Thread: Z-STACK,

Hello,

I am facing a quite troublesome problem with my Z-Stack based ZED : it randomly leaves the Zigbee Network after few days.

After looking for similar threads, I found several ones, but none with clear answers on the issue. The most relatable one being CC2530: ZED leaving network randomly - Zigbee & Thread forum - Zigbee & Thread - TI E2E support forums.

Basic Product Information

  • ZED is a custom product based on the SimpleLinkTm cc13xx_cc26xx SDK 6.40.00.13 and its ZED DoorLock Example (for LP_CC2652R7).
  • ZC is also a CC2652R7, straight from the SimpleLinkTm cc13xx_cc26xx SDK 5.40.00.40 and its ZNP Example (and Z2M as controller).
  • ZED Poll Period is 6000 ms.

Initial State

  • Several identical ZEDs (4 or 5) are paired successfully with the ZC.
  • All ZEDs interact properly with the ZC.
  • No ZR, all ZEDs are directly routed to the ZC.
  • Range is not an issue.
  • There is no other device paired with the ZC.

This situation will last 2 or 3 days, where :

  • ZEDs will properly report some attributes.
  • ZC can interact with ZEDs on demand.

Problem Symptoms

After these few days, some of the ZEDs, not all of them, will cease to communicate with the ZC : 

  • ZC see these ZEDs as Non-Responsive, still in network.
  • ZC Logs do not show any Leave Network messages.
  • ZC interactions to these ZEDs do not succeed. Interactions with the other ZEDs do succeed.
  • These ZEDs will cease to report their attributes. The others still report them properly.
  • These ZEDs are still alive and active, and react to manual interactions.

With some code instrumentation, i could extract some Z-Stack states for these ZEDs  : 

  • zstack_DevState -> zstack_DevState_HOLD
  • bdbCommissioningMode -> BDB_COMMISSIONING_NWK_STEERING
  • bdbCommissioningStatus -> BDB_COMMISSIONING_NO_NETWORK

Which I assume being an Out Of Network state.

Here's my question : What could possibly produce such behavior ?

Most solutions or hints I could find imply a ZR, a ZED Timeout, a great amount of devices or chips from previous generation, but none apply here.

More Informations :

  • What about sniffing the RF ? Sadly, this is a quite complex procedure to implement in my environment, especially if it must last several days.
  • Why trying with several ZEDs at once ? Due to the randomness of the issue, I have to use several to ensure its reproduction.
  • ZED Z-Stack Configuration : 

zstack.deviceType = "zed";
zstack.pm.pollPeriod = 6000;
zstack.rf.txPower = "5";
zstack.rf.primaryChannels = [11,25];
zstack.network.endDeviceTimeout = 14;
zstack.advanced.routing.routeExpiryTime = 255;

Other parameters are presumably default.

  • ZED Symbols/Macros : 

ZCL_REPORT_CONFIGURING_DEVICE
ZCL_REPORT_DESTINATION_DEVICE
ZCL_READ
ZCL_DISCOVER
ZCL_WRITE
ZCL_BASIC
ZCL_IDENTIFY
ZCL_DOORLOCK
ZCL_SCENES
ZCL_GROUPS
BDB_REPORTING
TIMAC_ROM_PATCH
CUI_DISABLE
MAX_STATUS_LINES=10
ZSTACK_SECURITY
FREQ_2_4G
OSAL_PORT2TIRTOS
OSAL_PORT2TIRTOS_OSALMAP
ZDO_API_BASIC
TC_LINKKEY_JOIN
NV_RESTORE
NV_INIT
FEATURE_NON_BEACON_MODE
ZCL_STANDALONE
MAX_DEVICE_TABLE_ENTRIES=50
DEVICE_FAMILY=cc26x0
DeviceFamily_CC26X2X7
TIMAC_ROM_IMAGE_BUILD
NVOCMP_NVPAGES=2
TIRTOS7_SUPPORT
HEAPMGR_CONFIG=0x80
HEAPMGR_SIZE=0x00

Thanks for your time. Any lead or suggestion would be greatly appreciated !

  • Hi Elarif,

    Does bdb_isDeviceNonFactoryNew return bdbAttributes.bdbNodeIsOnANetwork as TRUE?  If not then the NV memory has been reset to a factory new setting.

    • zstack_DevState -> zstack_DevState_HOLD
    • bdbCommissioningMode -> BDB_COMMISSIONING_NWK_STEERING
    • bdbCommissioningStatus -> BDB_COMMISSIONING_NO_NETWORK

    Based on these values, it is likely that the bdb_nwkDiscoveryAttempt -> bdb_reportCommissioningState BDB_COMMISSIONING_STATE_JOINING case had failed.  The following code is from the SimpleLink v7.10 SDK and may vary for your version (there are very few changes from v6.40 so I doubt this makes a difference):

          case BDB_COMMISSIONING_STATE_JOINING:
            if(ZG_DEVICE_JOINING_TYPE)
            {
              //Prepare for the next state or commissioning mode to be excecuted
              OsalPortTimers_startTimer(bdb_TaskID,BDB_CHANGE_COMMISSIONING_STATE,50);
    
              if(didSuccess)
              {
                //Next state is TC link key exchange
                bdbCommissioningProcedureState.bdbCommissioningState = BDB_COMMISSIONING_STATE_TC_LINK_KEY_EXCHANGE;
                //Free the list of nwk discovered
                while(pBDBListNwk)
                {
                  bdb_nwkDescFree(pBDBListNwk);
                }
    
                //No notification in this step
                return;
              }
              else
              {
                uint8_t temp = FALSE;
    
                bdbAttributes.bdbCommissioningStatus = BDB_COMMISSIONING_NO_NETWORK;
                bdbCommissioningModeMsg.bdbCommissioningMode = BDB_COMMISSIONING_NWK_STEERING;
                bdbCommissioningProcedureState.bdbCommissioningState = BDB_COMMISSIONING_STATE_START_RESUME;
                bdbAttributes.bdbCommissioningMode &= ~BDB_COMMISSIONING_MODE_NWK_STEERING;
    
                //Turn off the radio
                ZMacSetReq(ZMacRxOnIdle, &temp);
                //Set the device to FN, to start as new for subsequent attempts
                bdb_setFN();
                NLME_ResetRequest();
                ZDApp_ChangeState( DEV_HOLD );
    
                //Free the list of nwk discovered
                while(pBDBListNwk)
                {
                  bdb_nwkDescFree(pBDBListNwk);
                }
              }
            }
          break;

    You can also further debug the device directly to determine when bdbCommissioningStatus is set to BDB_COMMISSIONING_NO_NETWORK and what the call stacks and network status are at this point.  Are you able to replicate this behavior with a default project?  If not then please further clarify your application changes.  The ZED DoorLock example at least does not report attributes, and the changes implemented could be causing a memory leak which eventually corrupts the NV memory, or causes an unexpected factory reset.  If you know that the device has reset to factory new then you should further debug to try and break at bdb_setFN to determine why this is taking place.  Here are some other threads which you may consider:

    https://e2e.ti.com/f/1/t/1274985 
    https://e2e.ti.com/f/1/t/1288546 

    Regards,
    Ryan

  • Hi Ryan,

    Thank you for that quick answer and these suggestions.

    The v7.10 code you provided is indeed identical to the v6.40 .

    My next actions wille be to Implement more instrumentation in my code to identify the complete state of the device when the problem occurs.

    Plus, after reading the threads you mentioned :

    • Add some stack modifications on my ZED regarding that DEV_END_DEVICE_UNAUTH behavior
    • Upgrade my ZC firmware.

    Based on the results, I may start looking for NV Corruption and/or try to program my ZED with a clean example after that.

      

    I'll get back to you with my test results (few days are expected...) !

  • One more thought: if the ZED are battery powered, please make sure that the voltage level is not dropping below 2 V as this could cause flagrant NV memory writes.  See Non-Volatile Memory Low Voltage Detection for more details.  Otherwise, if the ZED are mains powered and power consumption is not an issue then please consider using a ZR project instead, at least as a test to determine if behavior is similar.

    Regards,
    Ryan