• State
  • Locked Locked
  • Replies 9 replies
  • Subscribers 51 subscribers
  • Views 832 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CC2564MODN: STM32L4 randomly HardFaults when executing Bluetopia scheduler

user5141793
Prodigy 90 points
Part Number: CC2564MODN


Hi,

We are having a hard time debugging a hardfault error with CC2564MODN and a STM32L476RG that runs BTPS 4.0.2.2. The CC2564MODN and STM32L476RG communicate via UART.

Details:

  • Device Chipset: 4.1
  • BTPS Version : 4.0.2.2
  • Project Type : 6
  • FW Version : 7.35

We have a EMC qualification software that runs on a PC with BLED112 dongle. The dongle connects to the CC2564 via BLE and sends serial commands (based on SPPLE demo) to create traffic and CC2564 sends notifications on another standard service as fast as possible (16ms). After approx. 1h of testing in these conditions, the STM32L476RG falls into hardfault and the call stack indicates it was running in a Bluetopia callback which we don't have the source to. We see the hardfault more frequently with high BT traffic and BT traffic from both sides.

The call stack indicates it happened in HCITR_COMDataCallback_UART and, according to the assembly and the registers' values, it happens just before HCILL_ReceiveBytes is called. STM32's register R6 always contains a RAM address (0x20001800) that is outside values mapped in the map file (outside STM32's stack and heap and outside Bluetopia's heap) while R4 and R5 always points to the same addresses in Bluetopia's heap. It is very difficult to extract any information, since we do not know what HCITR_COMDataCallback_UART and HCILL_ReceiveBytes do and it happens randomly (not with any specific BLE command).

 

As suggested in other posts, we increased the heap size (Bluetopia's heap = BTPS_MEMORY_BUFFER_SIZE) and the bug still happened. I tried to reduce the heap size just to see if it could make it unstable faster, but it did not.

We increased the STM32's stack size to the maximum stack size defined in BTPS kernel (in our case, the stack size is increased from 0x800 to 0x8000 = BTPS_CONFIGURATION_PAN_DISPATCH_THREAD_STACK_SIZE). The hardfault still happened, same place.

How big should the Stack and Heap be? The hardfault seems to be caused by a wrong RAM address during HCITR_COMDataCallback_UART and just before HCILL_ReceiveBytes... What is done in Bluetopia between those two?

Thank you

  • 0
    TI__Mastermind 27080 points

    Are you using in non-Os or RTOS environment? Did, you also print fault status register values?

    Thanks

  • 0 in reply to Hari Nagalla
    Prodigy 90 points

    We are using non-Os environment with our own scheduler that calls BTPS_ProcessScheduler first at every loop (and it is the only place we call it).

    This is the fault report I get:

    It's a bus fault and the faulty address is 0x20018000, which is actually the end of RAM... RAM is mapped from 0x20000000 to 0x2000F290 in the map file. Bluetopia's heap is mapped from 0x20003608 to 0x20008608 (we already increased it too be sure). The call stack indicates it was running in HCITR_COMDataCallback_UART.

    Thank you for your fast reply!

  • 0 in reply to user5141793
    TI__Mastermind 27080 points

    Can, you try disabling HCILL and sleep modes for a test? These needs to be changed in the BTS file, convert to *.h file and rebuild the application..

    Thanks

  • 0 in reply to Hari Nagalla
    Prodigy 90 points

    We already disabled the sleep in the .bts file (there was a note in http://processors.wiki.ti.com/index.php/CC256x_Bluetooth_Hardware_Evaluation_Tool). I am not sure where I should disable HCILL in the .bts... Do you mean disable HCILL sleep using "VS_Enable_Sleep_Mode(BluetoothStackID, FALSE)" ?

    On a side note, I saw that there was a new Service Pack for the CC256x (1.7) since december. We used SP 1.6 with sleep disabled. I am testing with 1.7 and sleep disabled just to see.

    Thank you

  • 0 in reply to user5141793
    Prodigy 90 points

    Hi,

    I can confirm the hardfault still happens with SP 1.7 and sleep disabled in the .BTS file, with VS_Enable_Sleep_Mode(BluetoothStackID, FALSE) or VS_Enable_Sleep_Mode(BluetoothStackID, TRUE).

    I still see the R3 register containing the start address of HCILL_ReceiveBytes() and the call stack indicating it was in HCITR_COMDataCallback_UART()...

    Is there anything else that could cause Bluetopia to call an invalid address? This hardfault happens faster if the BLE traffic is increased (if we send commands to the SPPLE and if the CC256 notifies faster)...

    Thank you

  • 0 in reply to user5141793
    Prodigy 90 points

    I still have this issue. Any help would be appreciated...

    I tried to increase stack and heap sizes, update the service pack to the latest version 1.7 and the sleep is disabled in the BTS file. I still get the hardfault.

    Just to add more information:

    I see that the caller code of the hardfault is in HCITR_COMDataCallback_UART() at <thumb code address for HCITR_COMDataCallback_UART> + 0x4B.

    Example

    In the map file:

    Extract from Image Symbol Table

    Symbol Name Value Ov Type Size Object (Section)
    i.HCITR_COMDataCallback_UART 0x0803ac0c Section 0 HCIComm.o(i.HCITR_COMDataCallback_UART)
    HCITR_COMDataCallback_UART 0x0803ac0d Thumb Code 470 HCIComm.o(i.HCITR_COMDataCallback_UART)


     
        

    For this map file, the hardfault caller code is at address 0x0803ac58.

    If I put a breakpoint on 0x0803ac56 (can only put breakpoint on 32-bit aligned mem addresses), I see that the code usually gets executed without any problem. It is only after a couple of hours with high BLE traffic (both directions) that I get a "random" hardfault.

    Is it possible to know what is done at this particular address in HCITR_COMDataCallback_UART? Maybe that would help to find the cause!

    Thank you!

  • 0 in reply to user5141793
    Prodigy 90 points

    Hi, 

    After further monitoring, I see one memory allocation of 249 bytes just before the hardfault (everytime). That is a pretty big allocation compared to the usual 6 to 16 bytes we see during normal operation.

    The allocation is done in HCITR_COMDataCallback_UART, not in our source code. The Bluetopia's memory buffer is not even half-full when the hardfault trigs and we do not see memory leak before that (bytes used and count of the number of alloc vs free are steady). 

    I checked with a BLE sniffer, just to see if it could be a huge or ill formed BLE packet, but nothing. The notifications just stops on the hardfault, but the connection is still holding with empty packets at fixed connection interval.

    I suspect there might be an overflow in the allocation size in HCITR_COMDataCallback_UART... And the allocated 249 bytes would be used later in HCITR_COMDataCallback_UART (at function start address + 0x4B, to be precise), triggering a hardfault. We use "Bluetopia_16_M4.fp_HW_FP.lib" with Keil (rvmdk). 

    What can we do to fix this?

    Thank you

  • 0 in reply to user5141793
    TI__Mastermind 27080 points

    I wonder, if any illegal HCI packets from UART is causing the issue.. I suggest the following:

    a) Try, with out DMA from UART, if being used.

    b) See, if can sniff the UART lines between the host and controller.

    put a break point, if the allocation size is 249 (which you mentioned is consistent before all crashes) and see, if there were any UART, DAM errors..

    Thanks

  • 0 in reply to Hari Nagalla
    TI__Mastermind 27080 points

    I wonder, if any illegal HCI packets from UART is causing the issue.. I suggest the following:

    a) Try, with out DMA from UART, if being used.

    b) See, if can sniff the UART lines between the host and controller.

    put a break point, if the allocation size is 249 (which you mentioned is consistent before all crashes) and see, if there were any UART, DAM errors..

    Thanks