Linux Automotive SDK : Connection time out - git.ti.com

Subramanian Dhandapani

Hello,

When we are doing a yocto based build setup, to create an image using Linux Automotive Processor SDK to create arago image. When triggered bitbake, getting fetch failures on various repositories on 'git.ti.com' due to git fetch timeout errors.

The timeout happens sporadically, pointing some network issues on TI servers, as we have tested it various network connections ( with proxy, without proxy), to verify the problems on our servers/ISPs

for eg; the following recipes failed, between multiple runs ....

meta-psdkla/recipes-bsp/rpmsg-char-helper/rpmsg-char-helper.bb

meta-psdkla/recipes-benchmark/cockpit-fw/cockpit-bootstrap-fw_git.bb

meta-ti/recipes-bsp/ti-display-sharing-fw/ti-display-sharing-fw_git.bb

meta-ti/recipes-bsp/cadence-mhdp-fw/cadence-mhdp-fw_git.bb

meta-arago/meta-arago-distro/recipes-bsp/ti-sci-fw/ti-sci-fw_2019.08.bb

Eg; of output what we got between two clone tries on one of the repo, we see errors between tries with in a minute apart, having different results ...

~/TI/git/test$ time git clone git://git.ti.com/glsdk/rpmsg-char-helper.git
Cloning into 'rpmsg-char-helper'...
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 27 (delta 12), reused 0 (delta 0)
Receiving objects: 100% (27/27), 13.79 KiB | 0 bytes/s, done.
Resolving deltas: 100% (12/12), done.
Checking connectivity... done.

real 0m14.931s
user 0m0.004s
sys 0m0.012s

a Minute Later :

the same call .failed ....

~/TI/git/test$ time git clone git://git.ti.com/glsdk/rpmsg-char-helper.git
Cloning into 'rpmsg-char-helper'...
fatal: unable to connect to git.ti.com:
git.ti.com[0: 198.47.28.207]: errno=Connection refused

real 1m7.320s
user 0m0.000s
sys 0m0.004s

Regards,

Subramanian

over 5 years ago

0 Piyali Goswami over 5 years ago

TI__Mastermind 30245 points

Subramanian

One of our experts have tried few of these mentioned repos and he is not facing issues. I will bring your observations to his notice.

Thanks and Regards

Piyali

0 Karthik Ramanan over 5 years ago in reply to Piyali Goswami

TI__Guru** 113810 points

Subramanian,

I've tried the exact same sequence multiple times and I don't see a problem. I will likely trigger a script to monitor this situation and let you know the result.

Please let me know if this problem persists, we will likely need to check some additional things on the server side.

Regards

Karthik

0 Karthik Ramanan over 5 years ago in reply to Karthik Ramanan

TI__Guru** 113810 points

Subramanian,

I have run the script to clone the git://git.ti.com/glsdk/rpmsg-char-helper.git and I have not seen any failure in a 90 minute test.

Please try this again and provide feedback. I'm marking this issue as resolved.

Regards

Karthik

0 Subramanian Dhandapani over 5 years ago in reply to Karthik Ramanan

Prodigy 20 points

Hi Karthik,

We are still having issues.

On two different systems we are getting two different ip-address for git.ti.com, namely 192.47.28.207 and 192.47.28.208

On both the systems, they failed between various repo's residing on the system ( it is sporadic).

Both of our systems are connected to two different ISPs, so the problem is clearly not on our servers.

Could you please check again?

More over we see this issue when we try to clone using the git protocol, with http or https, it is working more stable.

Regards,

Subramanian

0 Karthik Ramanan over 5 years ago in reply to Subramanian Dhandapani

TI__Guru** 113810 points

Hi Subramanian,

I will try to investigate this further with our infrastructure team. Please continue using the https/http to enable you to proceed further.

Regards

Karthik

0 Karthik Ramanan over 5 years ago in reply to Karthik Ramanan

TI__Guru** 113810 points

Hi Subramanian,

So far, we are pretty sure "errno=Connection refused" is more related to firewall than to IP-Address changes.

In order to make log file analysis easier, we would appreciate to get some time stamps of the error in order to find corresponding log entries.

Could you please reproduce the error with a `date` command upfront, i. e.:

> date ; git clone ....

Appreciate your help here.

Regards

Karthik

0 Karthik Ramanan over 5 years ago in reply to Karthik Ramanan

TI__Guru** 113810 points

Hi Subramanian,

We are still unable to recreate the problem.

Did you get a chance to reproduce the error and provide the additional requested details?

Regards

Karthik

0 Brad Griffis over 5 years ago in reply to Karthik Ramanan

TI__Guru*** 125430 points

Karthik,

FYI, I've had recent customer reports stating the same issue. Furthermore, I just ran a test of my own using a Linux PC on my home network (i.e. to avoid any potential issues with corporate firewalls, etc.). I see similar behavior. Here's a test I just ran:

brad@brad-XPS-15-9560:~/git-trees$ date; git clone git://git.ti.com/graphics/omap5-sgx-ddk-um-linux.git
Mon Dec 16 13:47:04 CST 2019
Cloning into 'omap5-sgx-ddk-um-linux'...
fatal: unable to connect to git.ti.com:
git.ti.com[0: 198.47.28.207]: errno=Connection timed out

Best regards,
Brad

0 Karthik Ramanan over 5 years ago in reply to Brad Griffis

TI__Guru** 113810 points

Hi Brad,

Thanks for the details. We are still in the process of tracking this down.

Regards

Karthik

0 Karthik Ramanan over 5 years ago in reply to Karthik Ramanan

TI__Guru** 113810 points

Hi Brad, Subramanian,

I just received an update from the IT department that we have made a few changes concerning the timeout settings and a few more changes are due to be done this weekend.

If you can please try cloning again and report back whether you can see improvements or degradation, we will take further actions.

Regards

Karthik

0 Brad Griffis over 5 years ago in reply to Karthik Ramanan

TI__Guru*** 125430 points

Karthik,

I'm not seeing an issue any more. I'm checking with other customers that reported issues to see if their issues are resolved too.

Thanks,
Brad

Processors

Processors forum

Linux Automotive SDK : Connection time out - git.ti.com