negasus/haproxy-spoe-go

Haproxy SPOE (Stream Processing Offload Engine) Agent Golang implementation

https://negasus.dev

haproxy-spoe-go's Issues

Broken pipe on unix socket error handle worker: error send AgentDisconnect frame

When listening on a unix socket, I get this error a few seconds after each event is processed:

2020/12/14 18:37:58 error handle worker: error send AgentDisconnect frame: write unix /var/run/gohaproxy/agent.sock->@: write: broken pipe

Example code https://github.com/alechenninger/go-haproxy-agent/blob/cd1d9681bcfe818f134dc1bc2d2ba1ec4e5983f2/main.go#L43

I've tried both listneing on "unix" and "unix packet"

Haproxy configs here https://github.com/alechenninger/go-haproxy-agent/tree/cd1d9681bcfe818f134dc1bc2d2ba1ec4e5983f2/conf

No ip_score returned

Hello,

we're facing something weird, here is the haproxy part:
...
http-request capture var(sess.myiprep.ip_score) len 3
http-response set-header X-IP-Score "%[var(sess.myiprep.ip_score)]"
...

in the Go file, "ip_score" is correct (verified with a dedicated log file) but it seems not properly sent back to Haproxy by "req.Actions.SetVar(action.ScopeSession, "ip_score", ipScore)"

here is the iprep.conf:
[ip-reputation]
spoe-agent iprep-agent
messages check-client-ip
option var-prefix myiprep
option continue-on-error
timeout hello 2s
timeout idle 2m
timeout processing 10ms
use-backend agents
no log
spoe-message check-client-ip
args ip=src
event on-client-session if ! { src -f /etc/haproxy/whitelist.lst }

any idea ?

bye Fred

Wrong encoding of bool value

Encoder doesn't set boolean data type in the buffer when the value being encoded is false:

haproxy-spoe-go/typeddata/typeddata.go

Lines 51 to 57 in 9a8c9fb

	case bool:
	var b byte = 0x11
	if !v {
	b = 0x10
	}
	buf = append(buf, b)
	return buf, 1, nil

False value should be 0x01, not 0x10 as it's set currently.

Agent graceful shutdown

Hey all 👋🏻

Thank you for the amazing package made starting crowdsec-spoa a breeze, however, I was surprised we dont have a way to gracefully shutdown the server. If your open to it I could have a go?

I believe it should be something like a root context on the agent that is checked by the for loop on serve, however, because we dont keep track of any connections I believe we should also note if there is a response pending 🤷🏻 let me know how you feel about it 👍🏻

Problem acquiring messages with binary fields

Hi,

Thanks for the work! Very useful :)

I'm trying to port the libmodsecurity SPOA to Golang using your library. It happens that when trying to acquire a message with a binary field (as req.hdr_bin) I get a panic, as following:

2019/11/13 21:22:57 listen 3000
panic: runtime error: index out of range [0] with length 0

goroutine 34 [running]:
github.com/negasus/haproxy-spoe-go/message.(*Messages).Decode(0xc00000e060, 0xc00001e0b2, 0x0, 0x1, 0x1, 0x53)
	/media/rkatz/data/rkatz/go/codes/haproxy-spoe-go/message/decode.go:22 +0x339
github.com/negasus/haproxy-spoe-go/frame.(*Frame).Read(0xc0000e8080, 0x562880, 0xc000072180, 0xc00010f000, 0xc00010c000)
	/media/rkatz/data/rkatz/go/codes/haproxy-spoe-go/frame/read.go:61 +0x3c4
github.com/negasus/haproxy-spoe-go/worker.(*worker).run(0xc0000e41e0, 0x0, 0x0)
	/media/rkatz/data/rkatz/go/codes/haproxy-spoe-go/worker/worker.go:53 +0x12c
github.com/negasus/haproxy-spoe-go/worker.Handle(0x5648e0, 0xc00010c000, 0x5461c0)
	/media/rkatz/data/rkatz/go/codes/haproxy-spoe-go/worker/worker.go:24 +0x66
created by github.com/negasus/haproxy-spoe-go/agent.(*Agent).Serve
	/media/rkatz/data/rkatz/go/codes/haproxy-spoe-go/agent/agent.go:31 +0xe7

I've been trying to dig into the code, but I'm not having success to dig what is going on. Is there any way to acquire binary fields?

This errors happens earlier, with no code, just with

mes, err := req.Messages.GetByName(messageName)
	if err != nil {
		log.Printf("message %s not found: %v", messageName, err)
		return
	}

Is there anything I'm missing?

Thank you!!

Memory leak?

Hi. We are using coraza-spoa which is using this library. Notices that this service is gradually increasing it's memory usage. This service is now enabled on a site that has a lot of incoming requests to process.

Took out a pprof file and a graph of the memory consumption.

Together with this memory usage we notice this:
level=warning msg="spoe: error handling connection: disconnect error: I/O error"

Not sure if this is connected to haproxy-spoe-go or something else but that causes a lot of the requests to not be processed correctly (Failed to parse request body).

Any clue on how we can troubleshoot this further?

No release with fixes already merged

Hi,

is there any chance for a release (v1.0.1?) with current fixes?
There are some differences between 1.0.0 and master v1.0.0...master

ack frame write failed

Hey under heavy load on the server seems after sometime to spoa server cannot write to the connection as haproxy has closed it? Do you have any pointers or information on where I can look to debug this?

spoa-1      | time="2024-05-15T18:24:46Z" level=error msg="ack frame write failed: cannot write frame to connection: write tcp 10.5.5.254:9000->10.5.5.4:38230: use of closed network connection" worker=spoa1

Thank you for spending time on writing this package 👍🏻

Wrong encoding of binary data

The code encoding []byte into binary typed data doesn't add its length which is required by the specification:

haproxy-spoe-go/typeddata/typeddata.go

Lines 112 to 116 in 9a8c9fb

	case []byte:
	buf = append(buf, TypeBinary)
	buf = append(buf, v...)
	return buf, len(v) + 1, nil
	}

However the decoding part processes binary data correctly.

Bug with 64-bit varint encodings

The buffer created here

buf = append(buf, TypeInt64)
b := make([]byte, 8)
i := varint.PutUvarint(b, uint64(v))

Isn't big enough for 64-bit integers (which can take up to 10 bytes when encoded as varints).