write_points method fails if called at exactly 5 second intervals

[influxdata]

write_points method fails if called at exactly 5 second intervals

[aviau]

Thank you for the detailed bug report! I wonder what we can do about this. Do you have any ideas? Our use of requests seems pretty standard. We might have to send this to the requests team. Or should we create a request session object for every request?

[pddenhar]

I was able to reproduce the issue using the requests library alone, so I've reported the bug to them.

As for using the library for the time being, it's hard to say. The behavior seems to get more likely to happen proportional to the latency of the internet connection requests are being made on (more latency == more time for the FIN from the server to be in flight but not received). Unfortunately for me, my use of InfluxDB-python is all on devices with extremely high latency internet connections, sometimes up to seconds. This seems to leave a pretty large window where an HTTP request can be sent out while a FIN is in flight from the server, leading to this BadStatusLine error.

For now, it might be best to not use a Session and simply make requests directly using requests.post or requests.get? I honestly can't say for sure that they don't use connection pooling (I'm not experienced with the library) but if so I think it would solve this problem for now.

A side benefit I've been thinking about is that it also might serve to reduce strain on the Influx server, because every single client would not be keeping TCP connections open at all times, but rather opening and closing them upon calls to write_points.

[aviau]

The session object does not keep a connection open.

From the requests docs (here):

The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance.

... or does it?

keep-alive is 100% automatic within a session! Any requests that you make within a session will automatically reuse the appropriate connection!

Would you like to try it out? See this branch.

[languitar]

I am also experiencing this issue. Is there any easy workaround?

[aviau]

@languitar

Can you try this branch and give us an update please?

[languitar]

@ReAzem I can try, but I have no clue how to reliably reproduce the case that I observe. So it might work out or not...

[kmacrow]

I'm also having this problem. @languitar tried that branch, no improvement.

[aviau]

@kmacrow I have pushed a new commit. Can you try again?

This is unrelated to the influxdb-python as this is a requests issue. A fix would be to use an interval longer than 5 seconds.

[nekto0n]

We had somehow similar issues with requests, keep-alive and broken connections. We had http client, REST server and IPVS balancer in between:

client <-> balancer <-> server

After connection being quiet for a while balancer simply forgets about established TCP connection and subsequent request after a long pause results in RST being received on client side. It is the same as in-flight-FIN race being discussed here.
requests does it's best to check if TCP connection is still alive - it calls poll() with 0 timeout to check connection status upon taking connection from pool, but if client hasn't yet received FIN/RST from server it rightfully assumes that connection can be used and issues HTTP request.

This problem could be worked around by enabling SO_KEEPALIVE on socket, but:

• Race condition would still be possible
• There's no interface to set in requests

Our solution was to setup HTTPAdapter in requests so that connection pool size is less than retries count. This works like this, suppose we have connection pool size = 2 and retries count = 3. We take first connection from pool and request fails with socket error, we take second connection from pool and it fails too, so we create new connection and it works (or doesn't, but that's another case).

I took a look at your implementation and as you don't use retries, so these errors are expected and should be handled explicitly.

[jove1]

I wonder if #124 would have any effect?

[aviau]

these errors are expected and should be handled explicitly.

I was hoping for another solution, but you are right. I'll push a patch that tries to send the packet for up to three times.

edit: I have begun working on a fix here 4f17ba3 . I'll write tests before I merge in master.

[nekto0n]

@jove1 This will put connection back to the pool, but won't close it - so I doubt it will help in this case. (If I'm correct about how requests work)