flaky 'Remote end closed connection' in docker build

[ryanramage]

flaky 'Remote end closed connection' in docker build

[harsha-y]

Both these mirrors seem to be consistently throwing errors today -
[IP: 5.153.231.35 80]
[IP: 128.31.0.66 80]

[sublimino]

Same errors for me when building Docker containers.

Dockerfile inherits from latest debian:jessie image (sadly, but necessarily, it installs a lot of packages. Failures are non-deterministic).

Errors:

...
Get:52 http://httpredir.debian.org/debian/ jessie/main fontconfig-config all 2.11.0-6.3 [273 kB]
Err http://httpredir.debian.org/debian/ jessie/main libqt4-dbus amd64 4:4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1
  Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:53 http://httpredir.debian.org/debian/ jessie/main libfontconfig1 amd64 2.11.0-6.3 [329 kB]
...
Get:351 http://httpredir.debian.org/debian/ jessie/main sqlite3 amd64 3.8.7.1-1+deb8u1 [101 kB]
Fetched 159 MB in 1min 1s (2605 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/m/mesa/libglapi-mesa_10.3.2-1+deb8u1_amd64.deb  Error reading from server. R
emote end closed connection [IP: 5.153.231.35 80]

E: Failed to fetch http://httpredir.debian.org/debian/pool/main/q/qt4-x11/libqt4-dbus_4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1_amd64.deb  Error re
ading from server. Remote end closed connection [IP: 5.153.231.35 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I've previously had errors building this same Dockerfile with other IPs:

E: Failed to fetch http://httpredir.debian.org/debian/pool/main/q/qt4-x11/qdbus_4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1_amd64.deb  Error reading 
from server. Remote end closed connection [IP: 128.31.0.66 80]
...
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/t/texlive-base/texlive-base_2014.20141024-2_all.deb  Error reading from serv
er. Remote end closed connection [IP: 5.153.231.35 80]
...
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/g/gsfonts/gsfonts_8.11+urwcyr1.0.7~pre44-4.2_all.deb  Error reading from ser
ver. Remote end closed connection [IP: 128.31.0.66 80]

Switching to a localised mirror in the Dockerfile corrects this error (I used the domain http://httpredir.debian.org/debian/ redirected me to)

FROM debian:jessie

ENV HOME /home/mnemosyne

RUN (\
    export DEBIAN_FRONTEND=noninteractive; \
    sed --in-place 's/httpredir.debian.org/mirror.sov.uk.goscomb.net/' /etc/apt/sources.list && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
      <pkg list>

[MichalKononenko]

Same issue over here. The 5.153.231.35 mirror is acting up.

[joenap]

It seems that #72 #74 are related to this, which I believe is caused by the underlying mirrors not having the specific packages. My team has experienced this for months, but we usually rebuild until we get a good mirror.

I propose retrying individual packages from other "random" mirrors (or whatever the method that determines the mirror in the first place), until the missing packages are successfully found and downloaded. It can fail after some number of attempts, or a timeout.

That may be tricky depending on how and where the mirror list is available in the code, and how it interfaces with apt, I'm sure. If I find time I may poke around at it. Otherwise, I would love to hear from the authors about the feasibility.

Cheers

[mattpickle]

Same issue with [IP: 128.31.0.66 80]

[kralos]

Update: I switched my apt mirror to one near my ci server and it hasn't failed since. I think this is just an issue with the httpredir.debian.org mirror. I suggest if you aren't publishing your docker image to do the same.

[ryanramage]

This issue seems to be related to issue #74

[emirotin]

Hit this one today

E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libx/libxau/libxau-dev_1.0.8-1_armhf.deb  Error reading from server. Remote end closed connection [IP: 128.31.0.66 80]

[dnorwood]

Hi Harsha,

What are the errors you are receiving?

Best regards,

Donald Norwood
-Debian Mirrors Team

On 06/13/2016 03:51 PM, Harsha Yalamanchili wrote:

Both these mirrors seem to be consistently throwing errors today -
[IP: 5.153.231.35 80]
[IP: 128.31.0.66 80]

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#75 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAePXItM4ERv70tOsk4UeZ86OBWyR0OZks5qLbSygaJpZM4IhZFu.

[jonahoffline]

Experiencing the same issues with mirror 5.153.231.35

[inakrin]

All the day today issues with Remote end closed connection [IP: 5.153.231.35 80]
Able to build containers after 3-9 runs.

[Demuxx]

Same issue with [IP: 128.31.0.66 80] and [IP: 5.153.231.35 80] for xz-utils amd64 5.1.1alpha+20120614-2+b3 and libssl-dev amd64 1.0.1t-1+deb8u3 and libevent-openssl-2.0-5 amd64 2.0.21-stable-2.

[kralos]

I'm getting this same issue on my own project (nothing to do with this repo). From the many builds I've investigated I'm beginning to get the feeling it's related to concurrent connections from the same build machine. If I have a test run by itself it seems to always work, but if I push more than 1 branch at a time they seem to break.

[anisse]

Seeing this for 2001:41c8:1000:21::21:35 as well.