• State Resolved
  • Locked Locked
  • Replies 6 replies
  • Answers 1 answer
  • Subscribers 56 subscribers
  • Views 620 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CC2651P3: ZC stops responding to Rejoin Request after running for a while

Andy Pozsgay
Intellectual 260 points

Part Number: CC2651P3
Other Parts Discussed in Thread: CC2538, Z-STACK

We are running the following:

Zigbee Coordinator on CC2651P3, stack simplelink_cc13xx_cc26xx_sdk_6_41_00_17
Zigbee Sleepy End Device on CC2538, stack Z-STACK-3.0.1

The ZED is joined directly to the ZC and functioning normally. After some time, the ZED is restarted. When it boots it sends Rejoin Request followed by 4 Data Request. It expects Rejoin Response or it cannot continue.

If the ZC has been running for a while, eventually it will stop responding to Rejoin Request. It does ACK the Data Request, so that would indicate that it still recognizes the ZED as its child. Once the ZC is in this state, the only way to recover is to reboot the ZC. After reboot, the ZC will resume responding to Rejoin Request.

I am attaching a sniffer trace showing this behavior. At the start of the trace, the ZED has not yet joined the network.

Packet #4: ZED initiates the join process, which succeeds

Packet #1138: ZED is rebooted and sends Rejoin Request. ZC responds with Rejoin Response and operation resumes normally.

Packet #1507: ZED is rebooted and sends Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1532: ZED makes its 2nd attempt with Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1537: ZED is rebooted and sends Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1552: ZED makes its 2nd attempt with Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1568: ZED is rebooted and sends Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1582: ZED makes its 2nd attempt with Rejoin Request. ZC does not respond to any of the 4 Data Request.

Packet #1595: ZC is rebooted and sends Parent Annce.

Packet #1604: ZED is rebooted and sends Rejoin Request. ZC responds with Rejoin Response and operation resumes normally.

Please advise.

Thanks,
Andy

  • 0
    TI__Guru**** 223727 points

    Hi Andy,

    Thank you for providing sniffer logs and a detailed description in advance.  Please confirm that you are using a default ZC project from the v6.41 SDK and the steps necessary to replicate this behavior.  For example, can a ZED switch cause the ZC light to eventually fail by periodically rebooting (for example, once every minute)?  Are any messages being queued on the ZC while the ZED is inactive?  Have you tried to debug the ZC to determine the application state (via call stack) during this failure?

    Regards,
    Ryan

  • 0 in reply to Ryan Brown1
    Intellectual 260 points

    > Please confirm that you are using a default ZC project from the v6.41 SDK and the steps necessary to replicate this behavior.

    The project is based on zc_sampleapp_LP_CC2651P3_tirtos7, and customized for our hardware.

    Steps to reproduce (these are all shown in the sniffer log):

    1. Open the network using bdb_StartCommissioning(BDB_COMMISSIONING_MODE_NWK_STEERING);
    2. Initiate a join on the ZED.
    3. Ensure the ZC application is communicating normally with the ZED (bi-directional AF traffic).
    4. Wait a while.
    5. Verify AF data is passed to the ZC application when sent by the ZED.
    6. Reboot the ZED, and check to see if the ZC responds to Rejoin Request.
    7. If the ZC responds to Rejoin Request, repeat steps 4-6.

    > For example, can a ZED switch cause the ZC light to eventually fail by periodically rebooting (for example, once every minute)? 

    Once I found the problem, I was able to reproduce it in 2 of 2 attempts. I did not setup a test where the ZED reboots once per minute. I joined the ZED and then went off to work on something else for a while (maybe an hour or so). Then I came back and rebooted the ZED. Within a couple of reboot attempts I was able to reproduce.

    > Are any messages being queued on the ZC while the ZED is inactive?

    No, I am not sending any AF data to the ZED during this time, so nothing should be queued.

    > Have you tried to debug the ZC to determine the application state (via call stack) during this failure?

    I have not done this. However, before I reboot the ZED I verify that when the ZED sends AF data, those packets are passed to the application. So, the application is functionally normally at least in the ZED->ZC direction. I have not been verifying the ZC->ZED direction, so it is possible that the problem is the ZC->ZED direction is broken completely (not just for Rejoin Response).

    Thanks,
    Andy

  • 0 in reply to Andy Pozsgay
    TI__Guru**** 223727 points

    Do you have a list of stack definitions which have been altered?  Given that the ZC may not be providing any responses, I'm concerned about memory leakage from the custom sample application.  Are you able to enable default ZCL responses from the ZED, or send out a Temperature Measurement from the ZC as with packet 340 and further explore whether a memory error is returned (ZMemError, ZBufferFull, etc)? You could enable HEAPMGR_METRICS and follow the Heap Allocation and Management guide to further debug.  I'm not confident that this can be replicated with the default sample application.

    Regards,
    Ryan

  • 0 in reply to Ryan Brown1
    Intellectual 260 points

    Hi Ryan,

    I was able to track down this issue. It is a nwk frame counter problem on the ZED side. The Z-STACK-3.0.1 has code to add 1250 to the value of the frame counter at bootup. But it seems there is some kind of bug there, and sometimes it doesn't do the addition. I will need to track that down.

    Regards,
    Andy

  • 0 in reply to Andy Pozsgay
    TI__Guru**** 223727 points

    Hi Andy, 

    Thanks for responding with updates!  There is a known Z-Stack 3.0.X issue involving frame counter incrementing after a device reset.  The workaround is provided in number 24 of the Z-Stack 3.0.2 Known Issues and Fixes E2E page.

    Regards,
    Ryan