This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
Hi,
Today I saw the OTA update fail with an error message of -6. We don't have any changes in the firmware. As per logs below, It is not able to receive the CDN URL. Is it because of the Dropbox change? Could you please help us fix it ASAP?
Starting OTA
_ReadStatFile: ERROR in sl_FsOpen, status=-11
sl_extLib_OtaRun: call OtaClient_ConnectServer OTA server=api.dropbox.com
OtaClient_ConnectServer: http_connect_server api.dropbox.com
0 OTA run = 0
sl_extLib_OtaRun: OtaClient_UpdateCheck, vendorStr=Vid01_Pid00_Ver0302100000
OtaClient_UpdateCheck: call http_build_request /1/metadata/auto/
CdnDropbox_SendReqDir: uri=/2/files/list_folder
metadata file=/Vid01_Pid00_Ver0302100000/f80_sys_mcuimgA.bin, size=142888
sl_extLib_OtaRun: OtaClient_UpdateCheck, numUpdates=1
0 OTA run = 0
sl_extLib_OtaRun: OtaClient_GetNextUpdate: file=/Vid01_Pid00_Ver0302100000/f80_sys_mcuimgA.bin, size=142888
OtaClient_ResourceMetadata: call http_build_request /1/media/auto
OtaClient_ResourceMetadata: file flags=80,metadata flags=80
CdnDropbox_SendReqFileUrl: uri=/2/files/get_temporary_link
0 OTA run = 0
sl_extLib_OtaRun: ResourceMetadata CDN file URL = f=
CdnClient_ConnectByUrl: ERROR, http_extract_domain_by_url, status=-1
sl_extLib_OtaRun ERROR: Failed on CdnClient_ConnectByUrl
0 OTA run = -6
OTA run = -6
OTA: Error with OTA server
Regards,
Sundar
Hi Sundar,
We are familiar with this issue but still don't understand what was changed in Dropbox to cause this.
I'll update here when we find a fix.
Note that in general we don't recommend using Dropbox for production since they are constantly changing their servers interfaces which causes issues to IOT device. It is recommended to use a more IOT focused cloud vendor.
Hi Kobi,
Since we have installed the IOT devices in different locations, we cannot change the Dropbox.
Any update on this issue?
Regards,
Sundar
Hi Kobi,
I have created the ticket in the Drobox forum and they said no changes from their side.
www.dropboxforum.com/.../757521
Regards,
Sundar
Hi Kobi,
We have 100k devices installed in the client location. We need to update the firmware over OTA. If you are able to point out the exact issue with the Dropbox API, we can reach out to the Dropbox team and ask them to fix it.
Regards,
Sundar
I understand your situation but the problem is with the connection to Dropbox (when started to check this we couldn't find any problem with the API - actually it passed in some cases). We are facing the same issue with CC3220//35 and the debugging is in progress. We will update once we find anything.
When communicating to Dropbox you should only present the HTTP Request (can be printed in sl_Send) and Response (in sl_Recv) - they are not familiar with our HTTP Client messages and error codes.
You can print the message in (or before) calling sl_Send or you can change the target to a Dump HTTP server (google for it, e.g. see https://httpdump.app/) and check the log there.
Hi Kobi,
I was not able to print the logs. Any update on your side?
Regards,
Sundar
Hi Kobi,
We have 90k devices installed all over the US. It is very difficult to ship the devices with the new firmware.
Please let us know if there is any update on this issue?
Regards,
Sundar
It looks like following the first request+response (list folder) the Dropbox server sends a TLS alert from some reason (downgrading the connection to unsecure).
I don't understand the reason (maybe the Dropbox contact can explain this).
If you delay the 2nd request (get-temporary-link) by few seconds (e.g "sleep(5)") the TLS connection will be reestablished and OTA will work.
Hi Kobi,
Thank you so much. I will pass this to drobox.
I'll get back to you when I have their reply.
Regards,
Sundar
Hi Kobi,
They want to reproduce the issue from their end. They are asking for network logs. Do you have anything to share with them?
Regards,
Sundar
This is what I have (the data is encrypted under the TLS):
The Alert is in frame 55151
Hi Kobi,
Thank you so much for sharing the screenshot.
Now they are asking for the dump file to inspect the request/response. Could you please export the PCAP file and share with me?
Regards,
Sundar
Hi Kobi,
Thank you for the logs. Below is the Dropbox team answer.
I had the team look at this, but unfortunately, this dump appears to only contain the WiFi frames. Can you instead share a dump with the actual data (TCP + TLS frames) so we can inspect that particular alert, like shown in the earlier screenshot?
Regards,
Sundar
no. we can't. I was using an air sniffer log where the TLS content is encrypted.
On the application side you can only publish the socket payload (i.e. if you print the data passed to sl_Send - it will include the HTTP content) but not the entire TCP/IP packets (i.e. it will be text file with http text and not a pcap file). I think one of our customers already provided this to them.
I'm quite sure they can capture the connection log on their server.
Hi Kobi,
Dropbox is now able to recreate the issue. The last thing they are asking for is an unencrypted dump file. They provide the steps below.
since it would be helpful to the team, if you could get and share an unencrypted dump (that is, without the WiFi encryption). The screenshot you shared on 4/24/2024 seems to be of the format we would want to see, but that's not what was shared in the dump you shared the next day (as that one included WiFi encryption). I understand you or your vendor may not want to do so because that would involve effectively sharing the WiFi password, so that could instead be saved and shared as a separate dump, e.g., using the "Export Packet Dissections" "As Plain Text" option, so as not to share the WiFi password.
Could you please share the dump file.
Regards,
Sundar
Hi Kobi,
I have created the packet capture and send it to them. I am still waiting for dropbox response.
Regards,
Sundar
Hi Kobi,
Dropbox says they fixed "new line" issue. Could you please check if that issue resolves our OTA issue without code changes on the device.
Regards,
Sundar
I'll be able to check next week. Is it regarding a "new line" within the HTTP headers or at the end of the payload?
Have you checked that you can now complete the update?
Hi Kobi,
I don't know where he wants to add a line.
They have not updated the changes in production yet. Anyway, I tested today, and it is not working.
Below is their answer.
"Would you be able to modify a device you have access to in order to have it not send the additional new line as a test? If you can verify that fixes the issue, I can ask the team to look into changing the server to allow that (though please note I can't make any promises on that)"
Please let us know if this fix our issue.
Regards,
Sundar
so need to better understand where they expect the extra new line.
again, i'll be able to check this only next week (but you can try yourself as the SDK contains all the sources and maefiles.
Tried to add new line "\n\r" at the end of the payload and still get the same failure,
I need to understand what exactly needs to be fixed.
Note - if i set a break point before the "get_temporary_link" ( or sleep for 4 seconds) - this OTA process will work (the request will get accepted).
Hi Kobi,
Please refer the below link.
Dropbox answer below:
To clarify, we have not made any changes on the Dropbox servers to address this. Здравко helpfully identified a potential cause of the problem, which is that your devices may be sending an unexpected new line at the end of the request body payload. In order to validate if that is the cause of the issue, and if addressing that would resolve the issue for you, I asked you to change one of your devices to make it not send that new line and see if it then works. If so, that would validate that as the cause of the issue so that I could then request that the team address that server-side
If you remove the extra line, will it resolve the issue.
Regards,
Sundar
Ok. I was able to verify that removing the new line solves the problem for CC3220/35
(in CC3220/35 it is done by commenting out line 2267 in httpclient.c
//ret = sprsend(cli, 0, "\r\n");".
Then you can rebuilt the httpclient library or copy the httpclient.c to the application and build it as an applications).
However such change can cause problems to access to pother HTTP server - so it needs to be used carefully.
I may try with CC3200 later but you can try commenting out line 73 of OtaHttp.c:
_i32 http_build_request(_u8 *pHttpReq, _u8 *method, _u8 *host_name, _u8 *req_uri_prefix, _u8 *req_uri, _u8 *hdr_name, _u8 *hdr_val) { /* start with method GET/POST/PUT */ strcpy((char *)pHttpReq, (const char *)method); /* fill uri_req_prefix, metadata/media */ if (req_uri_prefix && strlen((const char *)req_uri_prefix)) { strcat((char *)pHttpReq, (const char *)req_uri_prefix); } /* fill request URI */ if (req_uri && strlen((const char *)req_uri)) { strcat((char *)pHttpReq, (const char *)req_uri); } /* fill domain */ strcat((char *)pHttpReq, " HTTP/1.1\r\nhost: "); strcat((char *)pHttpReq, (const char *)host_name); strcat((char *)pHttpReq, "\r\n"); /* fill access_token */ if (hdr_name && strlen((const char *)hdr_name)) { strcat((char *)pHttpReq, (const char *)hdr_name); strcat((char *)pHttpReq, (const char *)hdr_val); strcat((char *)pHttpReq, "\r\n"); } //strcat((char *)pHttpReq, "\r\n"); /**** THIS SHOULD BE COMMENTED OUT ****/ return OTA_STATUS_OK; }
The file part of the OTA library - so you'll need to rebuild the library and then the application.
Hi Kobi,
Thank you so much for checking. I will let them know about it.
Regards,
Sundar
Kobi,
I tried removing empty line like you suggested for CC3200 but I don't find it resolving the issue. Keep us updated if you could get it resolved.
are you sure that the instance of the ota library you built (ota.a) is linked with the application?
I currently don't have a cc3200 setup and was doing all the tests with CC3235 (i assumed it is the same issue since it started at the same time).
I'll try to reproduce but it will take some time.