Comments (9)
On the log-courier side I get this when it tries to connect. I've restarted both the log-courier forwarder and the logstash receiver.
Connected to xx.xx.xx.xx
2014/07/30 08:53:42.707864 Transport error, will try again: write tcp xx.xx.xx.xx:pp: connection reset by peer
from log-courier.
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a firewall or something.
Jason
from log-courier.
It worked fine for about 6 hours and now won¹t restart without the error
Richard Williams
Director of Product Development
http://elabsolutions.com/
610 Airport Road€ Suite 200 € Huntsville, AL 35802
256.337.9952 (Direct) €270.747.9952 (Fax)
http://www.linkedin.com/pub/elab-solutions/84/194/723
https://twitter.com/eLabSolutions
https://www.facebook.com/pages/eLab-Solutions/156689697863760
[email protected] | www.elabsolutions
http://elabsolutions.com/ | Map
<https://www.google.com/maps/preview/place/610+Airport+Rd+SW,+Huntsville,+AL
+35802/@34.691661,-86.571944,17z/data=!3m1!4b1!4m2!3m1!1s0x8862132fc72c7b07:
0xb1748584e03640e3>
This communication may contain information or materials that are subject to
confidentiality, privilege or intellectual property restrictions. Please
refer to our legal notices http://elabsolutions.com/legalnotices for more
detail.
From: driskell [email protected]
Reply-To: driskell/log-courier
<reply+i-39088478-0a9f0e3848253bbca9801abed1af6fccdb021408-8291659@reply.git
hub.com>
Date: Wednesday, July 30, 2014 at 12:16 PM
To: driskell/log-courier [email protected]
Cc: Richard Williams [email protected]
Subject: Re: [log-courier] Protocol error on connection (#28)
Hi @rwilliamselab https://github.com/rwilliamselab
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs
and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a
firewall or something.
Jason
‹
Reply to this email directly or view it on GitHub
#28 (comment) .
from log-courier.
Have the same thing occurring on other instances too on different machines
Richard Williams
Director of Product Development
http://elabsolutions.com/
610 Airport Road€ Suite 200 € Huntsville, AL 35802
256.337.9952 (Direct) €270.747.9952 (Fax)
http://www.linkedin.com/pub/elab-solutions/84/194/723
https://twitter.com/eLabSolutions
https://www.facebook.com/pages/eLab-Solutions/156689697863760
[email protected] | www.elabsolutions
http://elabsolutions.com/ | Map
<https://www.google.com/maps/preview/place/610+Airport+Rd+SW,+Huntsville,+AL
+35802/@34.691661,-86.571944,17z/data=!3m1!4b1!4m2!3m1!1s0x8862132fc72c7b07:
0xb1748584e03640e3>
This communication may contain information or materials that are subject to
confidentiality, privilege or intellectual property restrictions. Please
refer to our legal notices http://elabsolutions.com/legalnotices for more
detail.
From: driskell [email protected]
Reply-To: driskell/log-courier
<reply+i-39088478-0a9f0e3848253bbca9801abed1af6fccdb021408-8291659@reply.git
hub.com>
Date: Wednesday, July 30, 2014 at 12:16 PM
To: driskell/log-courier [email protected]
Cc: Richard Williams [email protected]
Subject: Re: [log-courier] Protocol error on connection (#28)
Hi @rwilliamselab https://github.com/rwilliamselab
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs
and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a
firewall or something.
Jason
‹
Reply to this email directly or view it on GitHub
#28 (comment) .
from log-courier.
Maybe this will help. We tried using both logstash-forwarder and logstash
with lumberjack output and began to get client shutdowns much like with
log-courier. However the reported errors on the logstash server were:
Zlib::DataError: invalid distance code and logstash Zlib::DataError:
invalid block type that causes the logstash service to stop.
Richard Williams
Director of Product Development
http://elabsolutions.com/
610 Airport Road€ Suite 200 € Huntsville, AL 35802
256.337.9952 (Direct) €270.747.9952 (Fax)
http://www.linkedin.com/pub/elab-solutions/84/194/723
https://twitter.com/eLabSolutions
https://www.facebook.com/pages/eLab-Solutions/156689697863760
[email protected] | www.elabsolutions
http://elabsolutions.com/ | Map
<https://www.google.com/maps/preview/place/610+Airport+Rd+SW,+Huntsville,+AL
+35802/@34.691661,-86.571944,17z/data=!3m1!4b1!4m2!3m1!1s0x8862132fc72c7b07:
0xb1748584e03640e3>
This communication may contain information or materials that are subject to
confidentiality, privilege or intellectual property restrictions. Please
refer to our legal notices http://elabsolutions.com/legalnotices for more
detail.
From: driskell [email protected]
Reply-To: driskell/log-courier
<reply+i-39088478-0a9f0e3848253bbca9801abed1af6fccdb021408-8291659@reply.git
hub.com>
Date: Wednesday, July 30, 2014 at 12:16 PM
To: driskell/log-courier [email protected]
Cc: Richard Williams [email protected]
Subject: Re: [log-courier] Protocol error on connection (#28)
Hi @rwilliamselab https://github.com/rwilliamselab
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs
and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a
firewall or something.
Jason
‹
Reply to this email directly or view it on GitHub
#28 (comment) .
from log-courier.
One last clue
I wondered how you kept a marker on the current files you were shipping and
found a .log-courier file which contains offsets in our logs. If I remove
the file the service starts up again with no problem. Here is the content of
the placeholder
{"/opt/jboss/domain/servers/server-four/log/server.log":{"inode":133638,"dev
ice":64768,"source":"/opt/jboss/domain/servers/server-four/log/server.log","
offset":87849449}
,"/opt/node/quikshell/rotating.log":{"inode":666812,"device":64768,"source":
"/opt/node/quikshell/rotating.log","offset":2716}}
Thanks
Richard
Richard Williams
Director of Product Development
http://elabsolutions.com/
610 Airport Road€ Suite 200 € Huntsville, AL 35802
256.337.9952 (Direct) €270.747.9952 (Fax)
http://www.linkedin.com/pub/elab-solutions/84/194/723
https://twitter.com/eLabSolutions
https://www.facebook.com/pages/eLab-Solutions/156689697863760
[email protected] | www.elabsolutions
http://elabsolutions.com/ | Map
<https://www.google.com/maps/preview/place/610+Airport+Rd+SW,+Huntsville,+AL
+35802/@34.691661,-86.571944,17z/data=!3m1!4b1!4m2!3m1!1s0x8862132fc72c7b07:
0xb1748584e03640e3>
This communication may contain information or materials that are subject to
confidentiality, privilege or intellectual property restrictions. Please
refer to our legal notices http://elabsolutions.com/legalnotices for more
detail.
From: driskell [email protected]
Reply-To: driskell/log-courier
<reply+i-39088478-0a9f0e3848253bbca9801abed1af6fccdb021408-8291659@reply.git
hub.com>
Date: Wednesday, July 30, 2014 at 12:16 PM
To: driskell/log-courier [email protected]
Cc: Richard Williams [email protected]
Subject: Re: [log-courier] Protocol error on connection (#28)
Hi @rwilliamselab https://github.com/rwilliamselab
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs
and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a
firewall or something.
Jason
‹
Reply to this email directly or view it on GitHub
#28 (comment) .
from log-courier.
Hey
Your offset marker was very helpful in nailing down the offender. It looks
like our developers have blasted a single line in the log that contains an
encoded pdf image. The problem is apparently the size of the line. Is there
anyway to prefilter the lines to eliminate the overrun caused by bad log
files? Apparently all of the logstash cllients suffer from the problem.
Thanks
Richard
Richard Williams
Director of Product Development
http://elabsolutions.com/
610 Airport Road€ Suite 200 € Huntsville, AL 35802
256.337.9952 (Direct) €270.747.9952 (Fax)
http://www.linkedin.com/pub/elab-solutions/84/194/723
https://twitter.com/eLabSolutions
https://www.facebook.com/pages/eLab-Solutions/156689697863760
[email protected] | www.elabsolutions
http://elabsolutions.com/ | Map
<https://www.google.com/maps/preview/place/610+Airport+Rd+SW,+Huntsville,+AL
+35802/@34.691661,-86.571944,17z/data=!3m1!4b1!4m2!3m1!1s0x8862132fc72c7b07:
0xb1748584e03640e3>
This communication may contain information or materials that are subject to
confidentiality, privilege or intellectual property restrictions. Please
refer to our legal notices http://elabsolutions.com/legalnotices for more
detail.
From: driskell [email protected]
Reply-To: driskell/log-courier
<reply+i-39088478-0a9f0e3848253bbca9801abed1af6fccdb021408-8291659@reply.git
hub.com>
Date: Wednesday, July 30, 2014 at 12:16 PM
To: driskell/log-courier [email protected]
Cc: Richard Williams [email protected]
Subject: Re: [log-courier] Protocol error on connection (#28)
Hi @rwilliamselab https://github.com/rwilliamselab
Does this occur constantly or after a period of time?
When you restart the client does it occur immediately or does it send logs
and then do it after a bit?
I'll see about getting more logging on ProtocolError.
At first glance it looks like something is killing the connection. Maybe a
firewall or something.
Jason
‹
Reply to this email directly or view it on GitHub
#28 (comment) .
from log-courier.
@rwilliamselab Nice find! Thanks!
This is something that we should try to handle as gracefully as possible. I will check the code soon. I know there are 1MB hard coded limits in places but they may not be interacting nicely and causing this issue as opposed to failing gracefully to the next new line.
Jason
from log-courier.
I'm going to close this in favour of #40 as the error handling has changed slightly so it now reports "Protocol error - packet too large" to show the problem more explicitly.
I'll be working on the changes required for this next.
from log-courier.
Related Issues (20)
- Duplicates observed when log-courier configuration file is overwritten HOT 3
- Undocumented behavior for paths matched by multiple fileglobs HOT 1
- log-courier can't seem to handle `~` for home in certain contexts. HOT 2
- error in logstash 7 HOT 4
- log-courier and logstash > 7.4.0? HOT 10
- Throughput question HOT 15
- does the includes configuration still work correctly? HOT 3
- Hold time setting not closing files properly
- syslog "progname" uses a full path rather than a basename HOT 1
- Information about payloads / different debug level ? HOT 3
- error while compiling code in the command line using MK in windows for Z wave HOT 1
- version.rb missing from the plugin package
- log-courier admin socket stuck HOT 2
- lc-admin does not default to tcp transport HOT 2
- order of files in lc-admin UI
- Logstash configuration auto reload is blocked by log-courier input HOT 2
- PPA packaging is using deprecated compatibility level 9 HOT 1
- 2.10.0 packages missing from PPA (Ubuntu)
- Receiver reload can cause panic
- TCP streaming receiver aborts if too many events received
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from log-courier.