This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CC2640: Something causing battery powered device to halt and draw 4mA+ when connected, how do I figure out what it could be?

Part Number: CC2640
Other Parts Discussed in Thread: CC2650

A project I'm working on seems to have developed a problem where after a period of time of being connected, usually 1-2h, the device will stop responding and seems to be frozen. 

On investigating the problem further it appears that when this happens, simply removing and replacing the CR2016 coin cell battery will allow the device to function again. Looking at the power consumption when after failure occurs shows the current draw jump up to 4mA and remain there until power cycled.

No failures happen if the device is left unconnected and not advertising. I'm currently testing if a device will fail when advertising but not connected.

It appears that a previous revision of the firmware does not suffer the same problem. The only difference is that the earlier version does not require the central to take its connection parameters so will use the device's preferred params, and a clock to measure the battery level runs more frequently (1h vs 8h). 

Currently we're using custom hardware, but I have not been able to reproduce the problem on a CC2650 launchpad. I've connected up out own hardware and run it in debug mode but I so far cannot replicate the issue like that either. What is additionally strange is that the devices that fail consistently when connected to android function fine for prolonged periods on iOS.

Does anyone have any idea what the cause of the issue could be? Or even any tips on how to go about debugging this?

Thanks
Craig

  • If you use power supply(not coin cell battery), will you be able to reproduce this issue?
  • Hi Christin,

    When I try to power the devices using pins from the LaunchPad I see a lot of strange behaviour, such as central's unwillingness to connect at all.
    The device appears outwardly to be operating corrently, however advertising and connectivity seems disfunctional. I have no idea what might be able to cause such behaviour.

    I did packet sniff of the device when powered by battery to see what happened there when the device failed, but it didn't turn up much. I've attached it anyway.

    FailureEventLogs.psd

  • Could someone please get back to me on this and try to help me out, it's really urgent.

    Thanks,
    Craig

  • An update on the problem:

    We've noted that it might (not at all sure) be a problem relating to connection interval and sleep clock accuracy. Though problems we had with this in the past on iOS only resulted in a disconnect, not a complete halting and unresponsiveness of the device requiring power cycling. Could a problem with SCA cause the application to become unresponsive? if so, how/why?

    Right now I am testing a change of the sleep clock accuracy using the HCI_EXT_SetSCACmd() as had previously been suggested by JXS to resolve the iOS problem. I'll let you know how that goes,

    Thanks,
    Craig
  • Did you manage to test the device only in adverting state and see if it halts?
    If you change a new battery, and then start the connection again, will it halt after 2 hrs or will it stay connected longer?

    The reason for asking those questions is that a weak coin cell battery(which has high internal resistance and does not handle current pulses well) degrades a lot quicker than it should be (when you remove the circuitry, then battery has time to recover, so the measure voltage could still be 3V, but the charges are depleted)

    If you have a oscilloscope, you can put a small resistor in series with your custom design and then set a trigger at falling to be 1.8V on VDDS. See if the VDDS actually dropped down below 1.8V before your device failed.

    It could be that once your VDDS dropped below 1.8V and then CC26xx is not operating anymore, then the current draw from the circuitry will be 0, and then battery will have time to recover. Once it recovers to 3V, CC26xx will power up, however, the power up sequence draws current again(but there is no charges available in the battery), and battery can't handle this, so it drops below 1.8V. Then the device will stuck in a reboot circle, which will consume current in mA scale.
  • Hi Christin,

    I tested the device in an advertising state and it seems like it does not halt at all.

    It just seems strange to me that we have really struggled to reproduce this issue on iOS if it is a battery issue, but I will test with more known to fail devices there.

    I'll test the new battery approach next time we have a failure, and I'll set up the scope for a test too.

    Thanks for your help so far,
    Craig
  • Hi Christin,

    Finally managed to capture this on the scope after a long wait. Unfortunately my scope isn't good enough for me to be able to see what happened before/after this, but the trigger was set at 1.8V as you suggested, so this was the first dip that low.

    http://imgur.com/LpadMlD

    As you can see from the picture, not only did it hit 1.8v, it tanked right past it. Seemed to recover okay though, by the time I came back to find this, the device had become unresponsive as before. It took somewhere between 8 and 12 hours I think. The voltage a the start of the test was sitting around 2.8V after recovering from a previous failure.

    In a previous incident where I'd messed up setting the trigger and missed capturing the actual dip, I noticed on my return to a failed unit that the batt voltage had actually dropped way down and was hovering around 1.8V. It did recover quickly after taking out and putting back in. Does this add some weight to your theory?

    Any other thoughts on what could be causing this?

    Again, we've seen no failure on a device connected to iOS for 5+ days. This device had previously failed on android, so we reset it, and connected it using the same battery as before too.


  • Please can someone catch up on this thread, it's been a number of days now since we've heard from anyone at TI. The problem is extremely urgent

    Some additional information:

    Managed to finally get a launchpad in debug mode to fall into the same error. after almost 30 hours. When I paused the debug it seems it had been sent to AssertHandler() and fallen into this case:

        case HAL_ASSERT_CAUSE_ICALL_ABORT:
          Display_print0(dispHandle, 0, 0, "***ERROR***");
          Display_print0(dispHandle, 2, 0, ">> ICALL ABORT!");
          HAL_ASSERT_SPINLOCK;
          break;

    I have no idea where this was called from, however.  There's not a whole lot about it in the documentation either.

    How would I determine what caused this iCall abort assertion? 

    Obviously the scope trace above is still concerning, not sure how it links to this iCall abort assertion. Still waiting on someone to weigh in on that too.

  • Can someone within TI please assist on this issue as it is urgent. 

    We have production halted for a client with the issue under investigation.

    The key answers we are still looking for are:

    1. The failure seen on Launchpad wrt. cause of the iCall abort assertion? 2. Your inputs on the scope trace we posted last week. 

    Kind regards,

  • What you are seeing now fits what I mentioned in the previous comment. The next step I would recommend you to do for debugging is adding GPIO toggling to see which part of your code make device drains that much current. On top of that, you can add a bigger cap to the VDDS trace and check using another battery(change to another brand).

    What brand of battery are you using?
  • When this happened are you running on battery and use attach to running target? Or just had it connected to debugger and running all the time?
    If you have it running with debugger attach all the time, can you implement exception handle?
  • Hi Christin,

    We are using a Sony cr2016 battery. We have also done the test with a Duracell battery. The issue seems to have been software related wrt. a callback. Currently under test for confirmation. 

    To rule out coin battery issues, is there a test process you can refer us to. Although we use Sony there is reports some Sony marked batteries are not original. We want to ensure this will not be a source of problem.

    Thanks.

  • Unfortunately we don't have any way for testing battery. You can contact battery maker to see how they test the performance for their battery under current pulses.

    You can ask Sony if they can provide you information like Pulse Discharge Characteristics (2nd page of the datasheet from energizer )
    data.energizer.com/.../cr2032.pdf

    On top of that, you can add a SW timer in main.c. Whenever device restarts due to BOD, then you can have the SW timer to run for 5s to let the battery recovers, and then start execute your tasks when the timer expires
  • Hi Christin,

    I think we maybe have resolved the issue. There was a stack API call done in a clock callback to update one of the characteristic values. We've changed that now to queue an update to the value in the task instead and it seems a lot more stable.
    The error we were getting was a result of an icall_abort() function being called, which invoked the AssertHandler and put us into a case statement where HAL_SPINLOCK was called.

    We're not totally sure that the battery isn't still causing issues though. Certainly there is some funny thing power related going on on the board, and I'd appreciate if you could chime in on that...

    Basically I can program the launchpad fine and that will perform as expected. If I solder a ribbon cable to the custom hardware we can program that fine from the launchpad's XDS110 too.
    However, problems arise when I try to power the custom board from the launchpad. The lights will flash and it appears to be advertising (I can see the device on my phone), but it refuses to connect (Error 133: GATT ERROR) . We do not ever see this behaviour when the device is powered from battery, so it's very strange.

    Managed to get a sniffer trace of the problem:
    unable_to_connect_when_powered.psd


    I'm assuming the issue is also linked to the fact that I cannot program the devices using an XDS100C programmer and our programming rig:
    e2e.ti.com/.../585269

    Any ideas?