Other Parts Discussed in Thread: CC3220SF
We are using the CC3220 chipset as part of our product which is deployed by tens of thousands units in Latin America , in the past 2 weeks we are suffering from severe issue related to FOTA process
We are using TI SDK DropBox integration to manage our FOTA Process , this used to work very good up until 2 weeks ago
Suddenly we are unable to complete the process of the FOTA which means we cant support our customers and update the sensors anymore
After long debug process we understood the root cause of the issue , its related to the fact the DropBox decided to block the CERTIFICATE used by default by TI SDK
We tried to approach DropBox with no success and we need support here , no need to mention that if that is the case we are not the only customer that will suffer from it , but every customer using the CC3220 and DropBox method will be blocked
I am also adding some technical information we collected
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hi,
Here is a summary of the problem:
- CoreTigo is deploying WIFI based IOT sensors throughout south America
- The connectivity technology used in the sensors is TI CC3220SF asic , using the “simplelink” SDK OTA library
- The chosen cloud solution for FW upgrade is dropbox
- The sensors are querying CoreTigo’s dropbox account for checking new FW version and upgrade
- For the last year or so the FW upgrade through the dropbox account was working well
- In the last week we are experiencing failures – sensors are failing to load new FW exists in the dropbox folder:
- The start of the process looks ok – the dropbox token looks OK and the folder is accessed successfully, from the sensor logs:
|
OtaConfig: call OTA_set EXTLIB_OTA_SET_OPT_SERVER_INFO, ServerName=api.dropboxapi.com
OtaConfig: call OTA_set EXTLIB_OTA_SET_OPT_VENDOR_ID, VendorDir=RabbitMQ_Migration_Phase2
|
- The new FW file is seen properly by the sensor, the file name can be seen clearly, from the sensor logs:
|
OtaRunStep: CurrentVersion=00000000000000, NewVersion=20210118231832, Start download ...
|
- However, after download start there seem to be a connectivity issue in the socket level, from the sensor logs:
|
SL_SOCKET_ASYNC_EVENT socket event 2, do restart
OtaRunStep: WARNING Ota_run, Status=20006, continue for next OTA retry
|
- The API and content servers use certificates signed by different top level Digicert CAs. We have the high assurance CA certificate used by the API servers, while the content servers use the DigiCert Global Root CA.
Since these are self-signed root certificates there is no chain between them. As a result, the TLS on CC is unable to authenticate the certificate - There is some change in dropbox certificate, so the IOT devices certification process fails: "Root CA configured is wrong - please install DigiCert Global Root CA" (It appears the new certificates on the content servers were issues towards the end of the 2020).
Things we checked from dropbox perspective:
- Checked 2 different dropbox accounts and few different folders - Still Failed
- Upgraded the dropbox account to “professional” – Worked once and then Fails every time since then
- Simulated FW download by python like the sensor does - with the script it works
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The API and content servers use certificates signed by different top level Digicert CAs.
We have the high assurance CA certificate used by the API servers, while the content servers use the DigiCert Global Root CA.
Since these are self-signed root certificates there is no chain between them. As a result the TLS on CC is unable to authenticate the certificate.
It appears the new certificates on the content servers were issues towards the end of the 2020.
List of DigiCert root certificates for reference:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
One of the findings : which may solve the issue inTI SDK but will not help us now because it means we have to recall all the sensors
We succeed to perform FOTA from our dropbox server.
We first reproduced the problem that FOTA failed because the change in certificates structure in dropbox.
We removed the ROOT_CA check in TI’s SDK (I attach the updated files – they are in C:\ti\simplelink_cc32xx_sdk_3_30_01_02\source\ti\net\ota)
We also enabled debug information in otauser.h.
Than we make ota.a and rebuild CC3220 bin file.
We build the UCF – we resized the transdutor.bin MAX file size to 100000.
Than we ran and the FOTA succeeded (attached the log)
This solution will disregard any future changes in certificate structure in dropbox.
