Retry connection

[jmxtrans]

Retry connection

[cyrille-leclerc]

Hi @trikosuave, I am surprised because jmxtrans-agent is supposed to handle changes of domain-name to ip-address mappings.

Can you share with us more details?

• Which OutputWriter are you using? (see GraphitePlainTextTcpOutputWriter#ensureGraphiteConnection() or StatsDOutputWriter#flush() for example)
• Do you have a stacktrace?

I don't mind forking and adding this in myself, just wanted to make sure there wasn't a flag or anything I could set.

It should have been implemented, if there is a bug, I hope to be able to fix the existing code instead of rewriting from scratch.

[cyrille-leclerc]

Hello @trikosuave did you see my notes 15 days ago?

[bmmchugh]

I've submitted a pull request #62 for this issue.

[flisky]

Hi @cyrille-leclerc ,

Here is a stacktrace. This problem happened to me.

futex(0x7f2f942b5b48, FUTEX_WAKE_PRIVATE, 1) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 216571698}) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 216688028}) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 216765398}) = 0
gettimeofday({1451983697, 242573}, NULL) = 0
gettimeofday({1451983697, 242719}, NULL) = 0
gettimeofday({1451983697, 242819}, NULL) = 0
gettimeofday({1451983697, 242891}, NULL) = 0
gettimeofday({1451983697, 242995}, NULL) = 0
gettimeofday({1451983697, 243100}, NULL) = 0
sendto(30, "servers.jx-lj-zookeeper02.zookeeper.0.quorumSize 3 1451983697\nservers.jx-lj-zookeeper02.zookeeper.follower.requestLatency 1 1451983697\nservers.jx-lj-zookeeper02.zookeeper.follower.connection 2 1451983697\nservers.jx-lj-zookeeper02.zookeeper.follower.outstanding 0 1451983697\nservers.jx-lj-zookeeper02.zookeeper.follower.node 762 1451983697\nservers.jx-lj-zookeeper02.zookeeper.follower.watch 3 1451983697\n", 403, 0, NULL, 0) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=2140, si_uid=99} ---
rt_sigreturn()                          = -1 EPIPE (Broken pipe)
gettimeofday({1451983697, 243558}, NULL) = 0
gettimeofday({1451983697, 243773}, NULL) = 0
gettimeofday({1451983697, 243865}, NULL) = 0
write(1, "2016-01-05 16:48:17.243 WARNING [jmxtrans-agent-1] org.jmxtrans.agent.JmxTransExporter - Ignore exception flushing metrics ", 123) = 123
write(1, "\n", 1)                       = 1
gettimeofday({1451983697, 245395}, NULL) = 0
write(1, "java.net.SocketException: Broken pipe", 37) = 37
write(1, "\n", 1)                       = 1
write(1, "\tat java.net.SocketOutputStream.socketWrite0(Native Method)", 59) = 59
write(1, "\n", 1)                       = 1
write(1, "\tat java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)", 72) = 72
write(1, "\n", 1)                       = 1
write(1, "\tat java.net.SocketOutputStream.write(SocketOutputStream.java:159)", 66) = 66
write(1, "\n", 1)                       = 1
write(1, "\tat sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)", 63) = 63
write(1, "\n", 1)                       = 1
write(1, "\tat sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)", 68) = 68
write(1, "\n", 1)                       = 1
write(1, "\tat sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)", 62) = 62
write(1, "\n", 1)                       = 1
write(1, "\tat sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)", 58) = 58
write(1, "\n", 1)                       = 1
write(1, "\tat java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)", 65) = 65
write(1, "\n", 1)                       = 1
write(1, "\tat org.jmxtrans.agent.GraphitePlainTextTcpOutputWriter.postCollect(GraphitePlainTextTcpOutputWriter.java:173)", 110) = 110
write(1, "\n", 1)                       = 1
write(1, "\tat org.jmxtrans.agent.OutputWriterCircuitBreakerDecorator.postCollect(OutputWriterCircuitBreakerDecorator.java:124)", 116) = 116
write(1, "\n", 1)                       = 1
write(1, "\tat org.jmxtrans.agent.JmxTransExporter.collectAndExport(JmxTransExporter.java:161)", 83) = 83
write(1, "\n", 1)                       = 1
write(1, "\tat org.jmxtrans.agent.JmxTransExporter$2.run(JmxTransExporter.java:116)", 72) = 72
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)", 75) = 75
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)", 68) = 68
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)", 121) = 121
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)", 114) = 114
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)", 83) = 83
write(1, "\n", 1)                       = 1
write(1, "\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)", 83) = 83
write(1, "\n", 1)                       = 1
write(1, "\tat java.lang.Thread.run(Thread.java:745)", 41) = 41
write(1, "\n", 1)                       = 1
clock_gettime(CLOCK_MONOTONIC, {8554718, 235569757}) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 235647017}) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 235699673}) = 0
clock_gettime(CLOCK_MONOTONIC, {8554718, 235751311}) = 0
futex(0x7f2f942b5b74, FUTEX_WAIT_BITSET_PRIVATE, 1, {8554748, 235674051}, ffffffff

[cyrille-leclerc]

Thanks @flisky!

@trikosuave I am closing this ticket as #62 has been merged.