Hung connections!

[darkk]

Hung connections!

[darkk]

Can you post redsocks.conf too?

[semigodking]

This is the correct one.

cat /etc/config/redsocks.conf
base {
log_debug = on;
log_info = off;
daemon = on;
redirector= iptables;
}

redsocks {
local_ip = 192.168.10.2;
local_port = 1081;
ip = 127.0.0.1;
//listenq = 256;
//port = 1080;
//type = socks5;
port = 8081;
type = direct;
//type = http-connect;
//login = gaeproxy;
//password = gaeproxy;
}
redsocks {
local_ip = 192.168.10.2;
local_port = 1082;
ip = 192.168.10.216;
type = socks5;
port = 9050;
//type = http-connect;
//login = gaeproxy;
//password = gaeproxy;
}

Note: I extended this tool with a new direct method. So that, no proxy is required.

The implementation is simple. Basically, do nothing in relay but invoking start relay on read/write callback. It has a customized connect_relay method to connect to destination directly. I will post my code changes here when i got access to my code.

[darkk]

Ok

[192.168.10.101:53661->74.125.71.120:80]: client: 41 (-/W) SHUT_RD, relay: 42 (-/-) SHUT_WR, age: 19581 sec, idle: 19578 sec.

Reads like that:

1. client has sent EOF and redsocks finished reading from client (client: SHUT_RD)
2. client has sent EOF and redsocks relayed it to the server (relay: SHUT_WR)
3. redsocks still has something to send to the client (client: -/W)
4. MAYBE server has something to sent to redoscks, but redsocks does not wait for it (as soon as it wait for client to consume data)

So it looks like quite valid situation. Moreover, age: 19581 is more than default net.ipv4.tcp_keepalive_time = 7200, so remote end is still alive. Maybe, it's actually dead, but bug prevents redsocks from detecting it.

Maybe that's bug in redsocks but there is another possible reason: descriptor leak in your browser (I've seen in in firefox long-long-ago). Open connection leaks to subprocess (e.g. PDF reader) and is stuck there for quite a long time. See http://bugs.debian.org/410671 for details

Is 192.168.10.101 linux-based? Can you run sudo netstat -tanp and/or sudo lsof there to verify that there is no leak?

And regarding your note: what is the reason to implement direct in redsocks? As far as I see, you can use iptables DNAT for that and it'll be more lightweight than userspace daemon.

[semigodking]

192.168.10.101 is for a Windows and it was shutdown when I noticed this issue.
For implementation of 'direct', it is to bypass limitation of my ISP. My ISP blocks HTTP connections to some web sites if the connection is shared with a router. It is very interesting that another router behind the 1st level router is not limited. So, to bypass such limitation, I tran-proxied all HTTP/HTTPS traffic through the 2nd level router.
Such behavior of ISP in our country is common.

I plan to add some code in redsocks_shutdown() to detect such case and drop the clients.
The criteria can be something like relay->enabled == 0.
Will let you know the results.

[darkk]

How long was 192.168.10.101 shut down? (to check if keepalive worked or not).

I think, the better way is to use TCP_KEEP* options and to detect connection death.

Am I right, that some websites are blocked when you connect like that:
[client] -> [router] -> [isp]
and they are reachable when you connect like that:
[client] --redsocks--> [router1-with-redsocks] --> [router2] --> [isp]
?

I would recommend to play with iptables -j TTL instead of using redsocks in this case.

Another option to do quick-check is to run ubuntu live CD on client with the 1st topology and to check if websites are reachable or not.

[semigodking]

Almost right. But, some differences.
[client] -> [router1] -> [isp] works only if [client] does PPPoE directly. Mac clone does not make sense.
[client] -> [router2 - w/o redsocks] -> [router1] -> [isp] does not work for client, but work for router2 when MAC clone enabled on router1 (clone the mac of router2]
[client] -> [router2 - w/ redsocks] -> [router1] -> [isp] works

iptables -j TTL and iptables -j IPID are already applied in router2 as well as MAC clone in router1.
router1 is too limited to do any additional work/verification on it.

[darkk]

Ok, I see.
Have you applied -j TCPMSS --clamp-mss-to-pmtu on router2 ? It may be another reason of broken connection. And it's especially true if your connection is broken only to some sites in [client] -> [router2 - w/o redsocks] -> [router1] -> [isp] topology.

[darkk]

BTW: keepalive is not a silver bullet: http://lkml.indiana.edu/hypermail/linux/kernel/0508.2/0757.html
Subject: 2.6.12.5 bug? per-socket TCP keepalive settings

[semigodking]

Here is code for implementation of 'direct' method.

void redsocks_direct_connect_relay(redsocks_client *client);
static void direct_relay_init(redsocks_client *client)
{
client->state = 0;
}

static void direct_instance_fini(redsocks_instance *instance)
{
}
static void direct_read_cb(struct bufferevent *buffev, void *_arg)
{
redsocks_client *client = _arg;
redsocks_touch_client(client);
if (client->state == 0)
{
client->state = 1;
redsocks_start_relay(client);
}
}
static void direct_write_cb(struct bufferevent *buffev, void *_arg)
{
redsocks_client *client = _arg;
redsocks_touch_client(client);
if (client->state == 0)
{
client->state = 1;
redsocks_start_relay(client);
}
}
relay_subsys direct_connect_subsys =
{
.name = "direct",
.payload_len = 0,
.instance_payload_len = 0,
.readcb = direct_read_cb,
.writecb = direct_write_cb,
.init = direct_relay_init,
.instance_fini = direct_instance_fini,
.connect_relay = redsocks_direct_connect_relay,
};

void redsocks_direct_connect_relay(redsocks_client *client)
{
client->relay = red_connect_relay(&client->destaddr,
redsocks_relay_connected, redsocks_event_error, client);
if (!client->relay) {
redsocks_log_errno(client, LOG_ERR, "red_connect_relay");
redsocks_drop_client(client);
}
}

[darkk]

Have you verified if TCPMSS helps ?

[semigodking]

No. But I will understand this option and try it later.

Here is how I understand the hung connections:

1. client connection established (R/W enabled) and relay connection is being setup (W enabled).
2. peer of client disconnects before relay connection is established. And EOF reaches client and client shutdown RD and relays the EOF to relay (relay shutdown WR).
3. Since the connection of relay is not established before EOF is relayed, the event of relay now is -/-. Because RD is not enabled on relay yet.
4. Now the connection of client is half closed. we need an event to close this connection and drop clients. The only possible event is timeout. But, I am not sure if libevent can get such events in this situation.

[semigodking]

Patch below works fine for me.

diff --git a/redsocks.c b/redsocks.c
index ba5eab2..fff89d3 100644
--- a/redsocks.c
+++ b/redsocks.c
@@ -395,6 +436,11 @@ static void redsocks_shutdown(redsocks_client *client, struct bufferevent *buffe
redsocks_log_error(client, LOG_DEBUG, "both client and server disconnected");
redsocks_drop_client(client);
}

• else
• {

•    if (how == SHUT_WR && buffev == client->relay && client->relay->enabled == 0)
  

•               redsocks_drop_client(client);
  

• }
  }

// I assume that -1 is invalid errno value