Compiler/CC2538: 某一ZED一直入不了某ZC

Can you attach your sniffer log instead of screenshot?

Hi,Mike & Ryan,

As far as I know,this issue has a long history( at least 2+ years on CC2530/CC2538 MCU platform,even if current release Z-stack 3.0.2 version) but not be resolved by TI LPRF team.

You may be has been reference "Know Issue"  https://processors.wiki.ti.com/index.php/Zigbee_Known_Issues_and_Proposed_Fixes ,but does not help.

You may be defined the macro TP2_LEGACY_ZC to compatibility older Z-stack devices(For example 1.2.2a and so on),but does not help too.

You may be increace the data buffer as following segment in znp.cfg config file,but does not help.

-DNWK_MAX_DATABUFS_WAITING=10
-DNWK_MAX_DATABUFS_SCHEDULED=8
-DNWK_MAX_DATABUFS_CONFIRMED=8
-DNWK_MAX_DATABUFS_TOTAL=18

In short,brief of issue other device(ZR or ZED) could not join ZC(Coordinator) frequently whenever introduced by middle ZR,if your device direct join network without middle ZR,all work well.

By the way,I have been capture the ubiqua logs in attachment files,this may be helpful.

network key
EF:02:8E:81:65:80:2B:20:1D:38:89:F2:A6:65:3B:0B

direct join ZC successful.zip

join ZC introduce with ZR failure.zip

Hi miffy,

Here is a relevant post by Dhanraj which was not resolved: https://e2e.ti.com/support/wireless-connectivity/zigbee-and-thread/f/158/t/838237 

It is curious that the ZC acknowledges the Update Device commands but does not send a new Link Status message after 15 seconds, is the ZC capable of any network communication with the existing ZR device?  Are you using a ZC project or a ZNP configured as a Zigbee coordinator?  Can you please debug your ZC for ZDSecMgrUpdateDeviceInd -> ZDSecMgrDeviceJoin -> and determine what step fails before reaching APSME_TransportKeyReq?

Regards,
Ryan

Please don’t close this thread,I will attemp to debug this issue with your advices,may be difficult and need long time hardworking.

Hi Ryan,

With many times test(at least 30+ times),it may be not issue in ZDSecMgrUpdateDeviceInd and ZDSecMgrDeviceJoin,there are a very strange behaviour,if join ZC introduce with middle ZR,Update Device,Tunnel Transport Key and final Transport Key by middle ZR work well,I can confirm the device join ZC successful at the same time,but ZC don't send out Active Endpoint Request and Model Identifier and so on,this result caused ZC does not identifier join device type,if user manual trigger attribute report on join device,the ZC send out Active Endpoint Request immediately.

I think there are bugs in Z-Stack 3.0.2 private source code.

Attachments is results of test.

direct join successful.zip

introduce with ZC join success but not exchange Request Active endpoint.zip

Join device introduce middle ZR,if only present Update Device send to ZC by middle ZR,and not have Tunnel Transport Key next time issue from ZC, in In fact ZC receive Update Device and issue a Acknowledgement but not post to upper layer callback ZDSecMgrUpdateDeviceInd some reasons(may be bugs).I made UART debug output in ZDSecMgrUpdateDeviceInd and not have any output at the same time.

ZC receive Update Device in fact but not post to upper layer.zip

For easy viewing, I post shortscreen previous mentioned cubx(Only Update Device):

The attachment below is the log.

3763direct-join-fail.zip

Hello,miffy:

Thank you for your answer.I did refer to the link you said before, and it was indeed useless,the link just say that problem was caused by NV write block,change the 700 ms to 3500 ms, indeed unseless.

I did an experiment last Friday afternoon and set that number of macros(ZDSECMGR_TC_DEVICE_MAX). When it was set to 10, the eleventh device could not be accessed after ten devices were successfully connected to the network. It was also stuck in ZED requesting TCLK from ZC, and ZC kept No response, and finally leave request. When set to 40, 40 devices were successfully connected to the network, and the 41st (can be called ZED41) could not be connected. The phenomenon is the same as before. I deleted a device that was successfully connected to the network through the command, and then connected ZED41 to this ZC, It is still the same phenomenon, and the device that was just deleted can be successfully reconnected to the network.

Therefore, I speculate that TI's internal code logic does not handle the operation of deleting the device. It declares a fixed-size device management array (probably the address manager) according to the number of macros. Every time a device is successfully connected to the network, this array adds a record to the array. When a new device is added when it is full, ZC It will judge and compare the size records of the array at this time, and it will not respond to the TCLK request of the new device after the array space is full. This is normal. The abnormal thing is that when a device is deleted, the new device is not allowed to join, it seems Only the devices that successfully join for the first time are allowed (until they are full).

Hello,Ryan,

I copied the content of reply Miffy to you and want to confirm whether it is as I guess.

I did an experiment last Friday afternoon and set that number of macros(ZDSECMGR_TC_DEVICE_MAX). When it was set to 10, the eleventh device could not be accessed after ten devices were successfully connected to the network. It was also stuck in ZED requesting TCLK from ZC, and ZC kept No response, and finally leave request. When set to 40, 40 devices were successfully connected to the network, and the 41st (can be called ZED41) could not be connected. The phenomenon is the same as before. I deleted a device that was successfully connected to the network through the command, and then connected ZED41 to this ZC, It is still the same phenomenon, and the device that was just deleted can be successfully reconnected to the network.

Therefore, I speculate that TI's internal code logic does not handle the operation of deleting the device. It declares a fixed-size device management array (probably the address manager) according to the number of macros. Every time a device is successfully connected to the network, this array adds a record to the array. When a new device is added when it is full, ZC It will judge and compare the size records of the array at this time, and it will not respond to the TCLK request of the new device after the array space is full. This is normal. The abnormal thing is that when a device is deleted, the new device is not allowed to join, it seems Only the devices that successfully join for the first time are allowed (until they are full).

When you delete a device by sending leave request, do you see the device responding leave response? If not, the device won’t be removed from association list. Please make sure your coordinator receives leave response and turn off your removing device immediately. Then, try to join a new device and see if it can join.

Hi,YiKai

You reminded me.It's strange that the ubiqua didn't catch anything when I deleted the device,I use MT_NlmeLeaveRequest()--some code I add

static void MT_NlmeLeaveRequest(uint8 *pBuf)
{
  NLME_LeaveReq_t req;
  uint8 retValue = ZFailure;
  uint8 index, cmdId, len;
  uint8 found;

  /* parse header */
  len =  pBuf[MT_RPC_POS_LEN];
  cmdId = pBuf[MT_RPC_POS_CMD1];
  pBuf += MT_RPC_FRAME_HDR_SZ;

  /* If extAddr is all zeros, it means null pointer */
  for(index=0;((index < Z_EXTADDR_LEN) && (pBuf[index] == 0));index++);

  if (index == Z_EXTADDR_LEN)
  {
    req.extAddr = NULL;
  }
  else
  {
    req.extAddr = pBuf;
  }

  /* Increment the pointer */
  pBuf += Z_EXTADDR_LEN;
  if ( len > Z_EXTADDR_LEN )
  {
    req.removeChildren = *pBuf++;
    req.rejoin         = *pBuf++;
  }
  else
  {
    req.removeChildren = FALSE;
    req.rejoin         = FALSE;
  }
  req.silent         = FALSE;

  retValue = NLME_LeaveReq(&req);

  /* NEW ADD :free TCLK info to NV tclk sec when delete a device */
  if( ( ZG_BUILD_COORDINATOR_TYPE ) && ( ZG_DEVICE_COORDINATOR_TYPE ))
  {
    uint16 tempIndex;
    APSME_TCLKDevEntry_t TCLKDevEntry;
    uint8 found;
    //Reset the frame counter associated to this device  TCLinkKeyFrmCntr
    tempIndex = APSME_SearchTCLinkKeyEntry(req.extAddr, &found, &TCLKDevEntry);
    
    if(found)
    {
      uint16 i;
      
      i = tempIndex - ZCD_NV_TCLK_TABLE_START;
      
      TCLinkKeyFrmCntr[i].txFrmCntr = 0;
      TCLinkKeyFrmCntr[i].rxFrmCntr = 0;
      TCLinkKeyFrmCntr[i].pendingFlag = FALSE;
      
      if(TCLKDevEntry.keyAttributes == ZG_PROVISIONAL_KEY)
      {
        APSME_EraseICEntry(&TCLKDevEntry.SeedShift_IcIndex);
      }

      TCLKDevEntry.keyAttributes = ZG_DEFAULT_KEY;
      osal_memset(&TCLKDevEntry,0,sizeof(APSME_TCLKDevEntry_t));
      osal_nv_write( ( tempIndex), 0, sizeof(APSME_TCLKDevEntry_t), &TCLKDevEntry );
      retValue = ZSuccess;
    }
  }
  else
  {
    retValue = ZDSecMgrDeviceRemoveByExtAddr( req.extAddr );
  }

  /* Build and send back the response */
  MT_BuildAndSendZToolResponse(((uint8)MT_RPC_CMD_SRSP | (uint8)MT_RPC_SYS_NWK), cmdId, 1, &retValue);
}

It's useless.

I use MT_ZdoMgmtLeaveRequest()--some code I add

static void MT_ZdoMgmtLeaveRequest(uint8 *pBuf)
{
uint8 cmdId;
uint8 retValue;
zAddrType_t destAddr;
uint8 *pIEEEAddr;
uint8 removeChildren;
uint8 rejoin;
uint8 extAddr[Z_EXTADDR_LEN];
uint8 found;

/* parse header */
cmdId = pBuf[MT_RPC_POS_CMD1];
pBuf += MT_RPC_FRAME_HDR_SZ;

/* Destination Address */
destAddr.addrMode = Addr16Bit;
destAddr.addr.shortAddr = osal_build_uint16( pBuf );
pBuf += 2;

/* IEEE address */
pIEEEAddr = pBuf;
pBuf += Z_EXTADDR_LEN;

/* Rejoin if bit0 is set */
rejoin = ( *pBuf & 0x01 ) ? TRUE : FALSE;

/* Remove Children if bit1 is set */
removeChildren = ( *pBuf & 0x02 ) ? TRUE : FALSE;

retValue = (byte)ZDP_MgmtLeaveReq( &destAddr, pIEEEAddr, removeChildren, rejoin, 0);

/* NEW ADD :free TCLK info to NV tclk sec when delete a device */
if( ( ZG_BUILD_COORDINATOR_TYPE ) && ( ZG_DEVICE_COORDINATOR_TYPE ))
{
uint16 tempIndex;
APSME_TCLKDevEntry_t TCLKDevEntry;
uint8 found;
//Reset the frame counter associated to this device TCLinkKeyFrmCntr
tempIndex = APSME_SearchTCLinkKeyEntry(pIEEEAddr, &found, &TCLKDevEntry);

if(found)
{
uint16 i;

i = tempIndex - ZCD_NV_TCLK_TABLE_START;

TCLinkKeyFrmCntr[i].txFrmCntr = 0;
TCLinkKeyFrmCntr[i].rxFrmCntr = 0;
TCLinkKeyFrmCntr[i].pendingFlag = FALSE;

if(TCLKDevEntry.keyAttributes == ZG_PROVISIONAL_KEY)
{
APSME_EraseICEntry(&TCLKDevEntry.SeedShift_IcIndex);
}

TCLKDevEntry.keyAttributes = ZG_DEFAULT_KEY;
osal_memset(&TCLKDevEntry,0,sizeof(APSME_TCLKDevEntry_t));
osal_nv_write( ( tempIndex), 0, sizeof(APSME_TCLKDevEntry_t), &TCLKDevEntry );
retValue = ZSuccess;
}
}
else
{
retValue = ZDSecMgrDeviceRemoveByExtAddr( pIEEEAddr );
}
#if 0
uint16_t index = APSME_SearchTCLinkKeyEntry(pIEEEAddr, &found, NULL);
if(found == TRUE)
{
APSME_TCLKDevEntry_t TCLKDevEntry;
osal_memset(&TCLKDevEntry, 0x00, sizeof(APSME_TCLKDevEntry_t));
TCLKDevEntry.keyAttributes = ZG_DEFAULT_KEY;

osal_nv_write(ZCD_NV_TCLK_TABLE_START + index, 0, sizeof(APSME_TCLKDevEntry_t), &TCLKDevEntry);
TCLinkKeyFrmCntr[index].txFrmCntr = 0;
TCLinkKeyFrmCntr[index].rxFrmCntr = 0;
}
else
{
ZDSecMgrDevice_t MgrDevice;
MgrDevice.nwkAddr = destAddr.addr.shortAddr;
MgrDevice.extAddr = pIEEEAddr;
MgrDevice.parentAddr = 0x0000;
//MgrDevice.secure = 0;
//MgrDevice.devStatus = 0;
ZDSecMgrDeviceRemove(&MgrDevice);
}
#endif
MT_BuildAndSendZToolResponse(((uint8)MT_RPC_CMD_SRSP | (uint8)MT_RPC_SYS_ZDO), cmdId, 1, &retValue);
}

It's useless ,too.I also can control the device .

So I confused,did I use the wrong function ?

You should use MT_ZdoMgmtLeaveRequest with correct destAddr and IEEEAddr and set rejoin flag to false. Make sure see leave request is sent and there is leave response from your leaving device.

Thanks for sharing back your codes.

Mike&Hu,

I'm glad to know that you resolved the issue, if using the MT then I would also suggest you look at MT_ZdoSecDeviceRemove and MT_ZdoRemoveLinkKey.

miffy,

If the answer that Mike&Hu found does not also resolve your issue then can you please explain how another user could replicate such behavior?

Regards,
Ryan

Hi Ryan, According to my more than 3+ years of development experience in TI Z-Stack 3.0.x platform, there are such problems join ZC difficultly introdue middle ZR.I don't think the environment or hardware caused issue.

Everyone could replicate such issue easily,one ZC,and two or more ZR has been joined network(one by one join),and one ZR will for join test.every hardware should apply on CC253x platform with Z-Stack 3.0.2 newest software version patched acoording Known Issues wiki hyperlink.

First steps,ZC should formed a network in clean channel,and one or more ZR join the network,finally join ZR introduce middle ZR,if ZR join ZC directly,you should leave manual and continue join again, such repeated,you can observer the OTA packet used ubiqua tools,you may be found the issue.

Hi miffy,

Thank you for providing more information.  I will attempt to further evaluate this behavior once I can access the correct hardware.  Meanwhile, does any particular setup or scenario produce the behavior?  Like do the routers join at a similar time, is the ZC permit join enabled, have any devices been reset, etc?

Regards,
Ryan

Hi, Ryan

   All take part in device(ZR and ZC) not have power cycle never,and you could reference Mac Seq. at the same time.You should define the macro TP2_LEGACY_ZC on all device projects to disable "Key Exchange Steps",these are beneficial for test.

After my combing,there are three scenario Issue:

1. Whenever ZR steering join network,only observer Beacon Request/Response and not have another steps(Transport Key,Update Device etc.),join failed finally.

2. Whenever ZR steering join network send out Beacon Request,middle ZR reply Beacon Response immediately,and send out Update Deivce to ZC and receive ZC acknowledgement,but procedure does not hit ZDSecMgrUpdateDeviceInd,so that not have another steps next(Transport Key,Device Announce etc.),join failed finally.

3. Whenever ZR steering join network send out Beacon Request,middle ZR reply Beacon Response,and send out Update Deivce to ZC,ZC reply Tunnel Transport Key to middle ZR,middle ZR reply Transport Key to join device,Node Descriptor Request/Response exchange work well,but not have next steps(Acitve Endpoint Request/Response and Simple Descriptor Resquest/Response,Read Attributes and so on),so that ZC could not identifier joined device type.

It seems like above mentioned issue only occur join centralized network introduce middle ZR,if device join network directlly,all very well.