This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CC3200: OTA update simulatiously for multiple cc3200 WIFI devices

Part Number: CC3200
Other Parts Discussed in Thread: UNIFLASH,

Hello All,

We have successfully implemented over the air update for our devices and deployed at customer places & its works well in the beginning.

As More number of devices deployed at customer places, Whenever new firmware updates released. 10% of devices are Stopped working.

When Debugged we found that Flash got corrupted and not accessible.

This faulty devices working only after formatting & programming firmware through Uniflash.

This is a critical issue, as Every time we need to interact with our remote customer to manually load the firmware using Uniflash and to train them for loading firmware into CC3200. This is time consuming

The same firmware is loaded into all the devices with the same OTA_SERVER_APP_TOKEN.

Please, help us to resolve the issue by allowing all the cc3200 devices to get the OTA updates simultaneously or by some mechanism so that only one CC3200 WIFI devices should get access to the Dropbox API. and dropbox API access should be assigned to other CC3200 device only after completion of first CC3200 OTA update process.

The OTA update code is as below.

// Initialize OTA
pvOtaApp =
sl_extLib_OtaInit(RUN_MODE_NONE_OS|RUN_MODE_BLOCKING,0);
strcpy((char *)ucVendorStr,OTA_VENDOR_STRING);
OTAServerInfoSet(&pvOtaApp,(char *)ucVendorStr);

//
// Check if this image is booted in test mode
//
sl_extLib_OtaGet(pvOtaApp,EXTLIB_OTA_GET_OPT_IS_PENDING_COMMIT,
&OptionLen,&OptionVal);
UART_PRINT("EXTLIB_OTA_GET_OPT_IS_PENDING_COMMIT? %d \n\r",OptionVal);
if(OptionVal == true)
{
UART_PRINT("OTA:PENDING COMIT & WLAN OK ==> PERFORM COMIT \n\r");

SetCommitInt = OTA_ACTION_IMAGE_COMMITED;
sl_extLib_OtaSet(pvOtaApp, EXTLIB_OTA_SET_OPT_IMAGE_COMMIT,
sizeof(int), (_u8 *)&SetCommitInt);
}
else
{
UART_PRINT("Starting OTA \n\r");
lRetVal = 0;
while(!lRetVal)
{
lRetVal = sl_extLib_OtaRun(pvOtaApp);
}

UART_PRINT("OTA run = %d\n\r",lRetVal);
if(lRetVal < 0)
{
UART_PRINT("OTA: Error with OTA server\n\r");
}
else if(lRetVal == RUN_STAT_NO_UPDATES)
{
UART_PRINT("OTA: RUN_STAT_NO_UPDATES\n\r");
}
else if ((lRetVal & RUN_STAT_DOWNLOAD_DONE))
{
//
// Set OTA File for testing
//
lRetVal = sl_extLib_OtaSet(pvOtaApp, EXTLIB_OTA_SET_OPT_IMAGE_TEST,
sizeof(int), (_u8 *)&SetCommitInt);
UART_PRINT("OTA: NEW IMAGE DOWNLOAD COMPLETE\n\r");
UART_PRINT("Rebooting...\n\r");
RebootMCU();
}

Thanks and Regards,

Rohit

  • Hi,

    Can you share the log of the OTA update?

    Can you see the files list after the error occurs?

    Does the devices do anything else in parallel to the OTA?

    br,

    Kobi

  • Hello Kobi Leibovitch,

              Thank you for your response. The comments for your questions is as below.         

    Can you share the log of the OTA update?

    >> We had disabled the log. We will enable the log, reproduce the issue and share the logs

    Can you see the files list after the error occurs?

    >> once this error occurs, flash getting corrupted , not listing any files in CCS uniflash.

    Does the devices do anything else in parallel to the OTA?

    >> No, Device checking only OTA update.

    I am feeling, there is no issue in implementation.

    We are not facing any issue with single cc3200 device, if checked OTA update for multiple times.

    If there are more cc3200 devices checking OTA update and getting updated at the same time.

    At one of our customer place 40 cc3200 wifi devices, started OTA update at the same time. The new firmware in 36 wifi devices got  updated successfully, but failed in 4 devices, in these 4 devices external flash was got corrupted.

    We are using dopbox api, with single token for OTA update, and same firmware is loaded into all the cc3200 devices.

    Is there any provision to get dropbox api access to only a single device for firmware OTA update at a time. The other gateway should get access to dropbox API only after completion of first connected cc3200 device. This way the issue might get resolved.

    Please, provide your inputs to resolve the issue.

    Thanks and Regards,

    Bhairu

  • I don't think there is an option to limit the download to a single device OTA at a time.

    Maybe it is a question for Dropbox. Anyway, I'm not sure how this can impact the content of the OTA (as there are separate sockets to read the content and i believe the Dropbox servers must be able to handle this). Even with this, it is not clear how the flash can be corrupted by a wrong content (?), especially if you use the full OTA implementation (including the flc library). What exactly are you updating? (SP? MCU Image?)

    Hopefully, we'll get something from the logs.

    br,

    Kobi

  • Hello Kobi,

    Thank you for your inputs.

    Please, find the comments as below.

     it is not clear how the flash can be corrupted by a wrong content (?), especially if you use the full OTA implementation (including the flc library).

    >> The issue may not be due to wrong content, We will share the logs to understand this issue.

    What exactly are you updating? (SP? MCU Image?)

    >> Updating MCU image.

    We are using single firmware for all cc3200 devices with a common token for drop box api access for OTA update, is it fine?

    Thanks and Regards,

    Rohit

  • yes, your update method should be fine.

  • Hello Kobi,

    Thank you for your reply.

    Whenever the FLASH getting corrupted, we are getting file system not formatted error while List file system in uniflash.

    The console logs of uniflash is as below.

    [12:26:54] Operation ListFileSystem returned.
    [12:26:58] Begin ListFileSystem operation.
    [12:26:59] INFO: > Executing Operation: Connect
    [12:27:01] INFO: setting break signal
    [12:27:01] INFO: connection succeeded
    [12:27:01] INFO: getting storage list
    [12:27:01] INFO: > Executing Operation: Init
    [12:27:01] INFO: reading version info
    [12:27:01] INFO: DEVICE CC3200 ES1.33
    [12:27:01] INFO: reading version info
    [12:27:02] INFO: reading version info
    [12:27:04] INFO: > Executing Operation: ListFileSystem
    [12:27:04] INFO: extracting file system information...
    [12:27:04] INFO: Serial Flash block size: 4096 bytes
    [12:27:04] INFO: Serial Flash capacity: 256 blocks

    [12:27:04] INFO: file system is not formatted
    [12:27:04] INFO: > Executing Operation: Disconnect
    [12:27:04] Operation ListFileSystem returned.

    Please, suggest your inputs to address the above issue.

    Thanks and Regards,

    Rohit

  • I need to see the OTA log before the flash gets corrupted.

    If you can capture the NWP log during the "problematic" OTA session (see /cfs-file/__key/communityserver-discussions-components-files/968/CC3100-_2600_-CC3200-Capture-NWP-Logs-_2D00_-Texas-Instruments-Wiki.pdf) - it may be even more helpful. 

    br,

    Kobi

  • Hello Kobi,

    I  have enabled the NWP logs, as per your suggestion and capture the logs.

    The issue is reproduced and logs are captured as below.

    REXTLIB_OTA_GET_OPT_IS_PENDING_COMMIT? 0
    Starting OTA
    OTA run = -7
    OTA: Error with OTA server
    Version: Client LIB 1.4.0, Common LIB 1.5.0.

    C: Alloc for 1 2002bc84
    C: Alloc for 1 2002bc84
    C: Alloc for 1 2002bc84

    >> After this device reseted and files got corrupted

    >> List file system captured using Uniflash tools as below.

    Uniflash file system..
    [18:07:23] Begin ListFileSystem operation.
    [18:07:24] INFO: > Executing Operation: Connect
    [18:07:26] INFO: setting break signal
    [18:07:27] INFO: detecting FTDI for device reset
    [18:07:28] INFO: connection succeeded
    [18:07:28] INFO: getting storage list
    [18:07:28] INFO: > Executing Operation: Init
    [18:07:28] INFO: reading version info
    [18:07:28] INFO: DEVICE CC3200 ES1.33
    [18:07:29] INFO: reading version info
    [18:07:30] INFO: reading version info
    [18:07:31] INFO: > Executing Operation: ListFileSystem
    [18:07:31] INFO: extracting file system information...
    [18:07:32] INFO: Serial Flash block size: 4096 bytes
    [18:07:32] INFO: Serial Flash capacity: 256 blocks

    [18:07:32] INFO: file system is not formatted
    [18:07:32] INFO: > Executing Operation: Disconnect
    [18:07:32] Operation ListFileSystem returned.

    The above issue seen whenever, there is a a negative values returned from below function.

    lRetVal = sl_extLib_OtaRun(pvOtaApp);

    for.e.g. - OTA run = -7

    Please, help us to fix this issue, as this is critical and issues seen at customer places very often for every new releases, and most of the support engineer time is getting utilized for this issue only.

    Thanks and Regards,

    Rohit

  • Hello Rohit,

    Kobi is out of office and will return September 6.

    Thank you,
    Jacob

  • Hello Kobi,

    I have enabled the logs and reproduced the issue. Th ecaptured logs is as below.

    sl_extLib_OtaInit: OTA lib version = 1.00
    EXTLIB_OTA_GET_OPT_IS_PENDING_COMMIT? 0

    Starting OTA

    sl_extLib_OtaRun: call OtaClient_ConnectServer OTA server=api.dropbox.com
    OtaClient_ConnectServer: http_connect_server api.dropbox.com
    sl_extLib_OtaRun: OtaClient_UpdateCheck, vendorStr=FW_0012
    CdnDropbox_SendReqDir: uri=/2/files/list_folder
    metadata file=/FW_0012f80_sys_mcuimgA.bin, size=111728
    sl_extLib_OtaRun: OtaClient_UpdateCheck, numUpdates=1
    sl_extLib_OtaRun: OtaClient_GetNextUpdate: file=/FW_0012/f80_sys_mcuimgA.bin, size=111728
    OtaClient_ResourceMetadata: file flags=80,metadata flags=80
    CdnDropbox_SendReqFileUrl: uri=/2/files/get_temporary_link
    sl_extLib_OtaRun: ResourceMetadata CDN file URL = content.dropboxapi.com/.../A3FnaAXyHiA-y887E8f16AmNIRpKifJUls6DKWV1heVzfxAmnjqzpR6scHJ4srnInvQPt4LDbYpPnJI2DFt5rgsrvEtrZJh63ReToC56XZpMPxepqWMU10PJKQ8ojBvC5r0SU2uCbGRNcUoiVYfyyfcb8UBiBJzPMfQ1LxfWp1fAG2LWvHUtsrKRMktdREw8KpIZPbEqTpfFjc496EzusAN1IRZDi9Dhpfkufyh5qOScxwb6X7XfOYrHSpreE05ooL0pEZ7kH2xSeMeD7kMbEMvOED4T2Vxmo5jJCGfCJ6nOIjvb4CIvf0uqdbXJe-DNSvByBX2HTIjCQnpmeUF_YzkLUMSOGLZ9x_2IErfm09P6tga
    _ReadFileHeaders: domain=content.dropboxapi.com, file=/apitl/1/A3FnaAXyHiA-y887E8f16AmNIRpKifJUls6DKWV1heVzfxAmnjqzpR6scHJ4srnInvQPt4LDbYpPnJI2DFt5rgsrvEtrZJh63ReToC56XZpMPxepqWMU10PJKQ8ojBvC5r0SU2uCbGRNcUoiVYfyyfcb8UBiBJzPMfQ1LxfWp1fAG2LWvHUtsrKRMktdREw8KpIZPbEqTpfFjc496EzusAN1IRZDi9Dhpfkufyh5qOScxwb6X7XfOYrHSpreE05ooL0pEZ7kH2xSeMeD7kMbEMvOED4T2Vxmo5jJCGfCJ6nOIjvb4CIvf0uqdbXJe-DNSvByBX2HTIjCQnpmeUF_YzkLUMSOGLZ9x_2IErfm09P6tga
    _ReadFileHeaders: skip http headers
    CdnClient_Run: Create/Open for write file /sys/mcuimgA.bin
    OTA run = -7

    OTA: Error with OTA server
    _OpenStorageFile: error in pOpenFile, status=-3
    CdnClient_Run ERROR: pCdnClient->pOpenFileCB
    sl_extLib_OtaRun ERROR: Failed on CdnClient_Run

    Version: Client LIB 1.4.0, Common LIB 1.5.0.

    C: Alloc for 1 2002bc84

    --------------------

    After above scenario when gateway resets, The file system gets corrupted and it won't give access to file and shows "file system is not formatted" error when checked in uniflash.

    please, provide your input to resolve this issue.

    Thanks and Regards,

    Rohit

  • This is strange.

    First, the order of the errors messages doesn't make sense (i would expect "OTA run = -7" to come after "_OpenStorageFile: error in pOpenFile, status=-3").

    Second, i would expect another message "sl_extlib_FlcOpenFile: MCU image name converted to /sys/mcuimg1.bin" that is missing here (please verify that sl_extlib_FlcOpenFile is being called).

    and third, it seems that that the file system was already in a bad state when the mcu image was written.

    Can you set a breakpoint in sl_extlib_FlcOpenFile and check what gets wrong? (basically, when "/sys/mcuimgA.bin" is being opened, the "A" is being changed to either "2" or "3"). 

    Br,

    Kobi