• State
  • Locked Locked
  • Replies 2 replies
  • Subscribers 51 subscribers
  • Views 986 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS/CC2640: Device crash on massive notifications

Harald Ilg
Genius 3060 points

Part Number: CC2640

Tool/software: TI-RTOS

Hy,

I have had similar issues in the past, now it unfortunately comes up in series. We have about 5000parts in the field, up to now, there are 5 occurrences of the following:

CC2640 based, battery powered, peripheral devices, some I2C devices attached, crash in different scenarios: They either reboot, get stuck (no advertising, no reaction) or get in a state, where they still advertise but are not connectable. My strong guess is, that this happens during mass data transfer from device to scanner: My device collects data which is transferred to the master by notifications. In my test scenario I try to transfer 4kByte by sending 205 notifications. I always call 6 times GATT_Notification in a for loop, than set another semaphore to send the next 6 packages.

When the devices are in the erroneous state, I observe different phenomens in debug session:

1) Software hangs in MB_SendCommand in mb_PATCH.c

  // check for any unexpected Mailbox error
  MB_ASSERT( (MB_CMDSTA_REG & 0xFF) == CMDSTA_DONE );
MB_CMDSTA_REG contains 0x87

2) endless loop in osal_bm_free

In function osal_bm_free from osal_bufmgr.c I see an endless loop at 

while ( loop_ptr != NULL )

because loop_ptr and loop_ptr->next_ptr; show to the same address.

There is one pecularity: In RTOS Object View (ROV) I see "stackPeak" message "Overrun!". But actually, I always see this, even in well working systems.

Does anyone can give me some hints on where to start to track down this issue?

Regards

Harald

  • 0
    Genius 3060 points

    There are more phenomens I observe:

    3) device just resets

    4) device advertises and runs OK, connection is possible, but no data transfer works (read / write characteristics)

    Fullscreen btoollog.txt Download 
    [1] : <Tx> - 09:55:03.664
-Type           : 0x01 (Command)
-OpCode         : 0xFE09 (GAP_EstablishLinkRequest)
-Data Length    : 0x09 (9) byte(s)
 HighDutyCycle  : 0x00 (0) (Disable)
 WhiteList      : 0x00 (0) (Disable)
 AddrTypePeer   : 0x00 (0) (Public)
 PeerAddr       : A0:E6:F8:EB:D0:B7
Dump(Tx):
0000:01 09 FE 09 00 00 00 B7 D0 EB F8 E6 A0          .............
--------------------------------------------------------------------
[2] : <Rx> - 09:55:03.695
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x06 (6) bytes(s)
 Event          : 0x067F (1663) (GAP_HCI_ExtentionCommandStatus)
 Status         : 0x00 (0) (Success)
 OpCode         : 0xFE09 (GAP_EstablishLinkRequest)
 DataLength     : 0x00 (0)
Dump(Rx):
0000:04 FF 06 7F 06 00 09 FE 00                      .........
--------------------------------------------------------------------
[3] : <Info> - 09:55:05.463
Device Connected
Handle = 0x0000
Addr Type = 0x00 (Public)
BDAddr = A0:E6:F8:EB:D0:B7
--------------------------------------------------------------------
[4] : <Error> - 09:55:05.465
Could Not Convert All The Data In The Following Message
(Message Is Missing Data Bytes To Process)
--------------------------------------------------------------------
[5] : <Rx> - 09:55:05.435
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x13 (19) bytes(s)
 Event          : 0x0605 (1541) (GAP_EstablishLink)
 Status         : 0x00 (0) (Success)
 DevAddrType    : 0x00 (0) (Public)
 DevAddr        : A0:E6:F8:EB:D0:B7
 ConnHandle     : 0x0000 (0)
 ConnRole       : 0x50 (80) ()
 ConnInterval   : 0x0000 (0)
 ConnLatency    : 0xD000 (53248)
 ConnTimeout    : 0x0007 (7)
Dump(Rx):
0000:04 FF 13 05 06 00 00 B7 D0 EB F8 E6 A0 00 00 50 ...............P
0010:00 00 00 D0 07 00                               ......
--------------------------------------------------------------------
[6] : <Tx> - 09:55:09.329
-Type           : 0x01 (Command)
-OpCode         : 0xFD90 (GATT_DiscAllPrimaryServices)
-Data Length    : 0x02 (2) byte(s)
 ConnHandle     : 0x0000 (0)
Dump(Tx):
0000:01 90 FD 02 00 00                               ......
--------------------------------------------------------------------
[7] : <Rx> - 09:55:09.362
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x06 (6) bytes(s)
 Event          : 0x067F (1663) (GAP_HCI_ExtentionCommandStatus)
 Status         : 0x00 (0) (Success)
 OpCode         : 0xFD90 (GATT_DiscAllPrimaryServices)
 DataLength     : 0x00 (0)
Dump(Rx):
0000:04 FF 06 7F 06 00 90 FD 00                      .........
--------------------------------------------------------------------
[8] : <Rx> - 09:55:39.365
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x06 (6) bytes(s)
 Event          : 0x0511 (1297) (ATT_ReadByGrpTypeRsp)
 Status         : 0x17 (23) (Timed Out Performing Function)
 ConnHandle     : 0x0000 (0)
 PduLen         : 0x00 (0)
Dump(Rx):
0000:04 FF 06 11 05 17 00 00 00                      .........
--------------------------------------------------------------------
[9] : <Tx> - 09:55:39.365
-Type           : 0x01 (Command)
-OpCode         : 0xFD84 (GATT_DiscAllCharDescs)
-Data Length    : 0x06 (6) byte(s)
 ConnHandle     : 0x0000 (0)
 StartHandle    : 0x0001 (1)
 EndHandle      : 0xFFFF (65535)
Dump(Tx):
0000:01 84 FD 06 00 00 01 00 FF FF                   ..........
--------------------------------------------------------------------
[10] : <Rx> - 09:55:39.419
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x06 (6) bytes(s)
 Event          : 0x067F (1663) (GAP_HCI_ExtentionCommandStatus)
 Status         : 0x17 (23) (Timed Out Performing Function)
 OpCode         : 0xFD84 (GATT_DiscAllCharDescs)
 DataLength     : 0x00 (0)
Dump(Rx):
0000:04 FF 06 7F 06 17 84 FD 00                      .........
--------------------------------------------------------------------
[11] : <Info> - 09:56:02.761
Device Disconnected
Handle = 0x0000
Addr Type = 0x00 (Public)
BDAddr = A0:E6:F8:EB:D0:B7
--------------------------------------------------------------------
[12] : <Rx> - 09:56:02.761
-Type           : 0x04 (Event)
-EventCode      : 0x00FF (Event)
-Data Length    : 0x06 (6) bytes(s)
 Event          : 0x0606 (1542) (GAP_TerminateLink)
 Status         : 0x00 (0) (Success)
 ConnHandle     : 0x0000 (0)
 Reason         : 0x13 (19) (Peer Requested)
Dump(Rx):
0000:04 FF 06 06 06 00 00 00 13                      .........
--------------------------------------------------------------------

  • 0 in reply to Harald Ilg
    TI__Guru** 105180 points
    Hi Harald,

    From the TRM SWCU117, Radio chapter, Command Status error 0x87 equates to "ParError" with a description:

    "There were errors in the command parameters that are parsed on
    submission. For radio operation commands, errors in parameters
    parsed after start of the command are signaled by the command
    ending, and an error is indicated in the status field of that command
    structure."

    I've seen similar radio errors that were traced to memory corruption, usually in the heap / system RAM. These radio faults are usually a manifestation of an earlier memory corruption, not the root cause. For example, an application that writes to a pointer that was already freed in the heap could corrupt RAM that is subsequently fed into the RF core. The fact that you see this and some odd behavior during high throughput may be a clue to look into the heap and see if there is any corruption with your memory usage. We do not see this when we run our Throughput demos, so I don't see it originating in the stack.

    Best wishes