knight42/krelay

A better alternative to `kubectl port-forward` that can forward TCP or UDP traffic to IP/Host which is accessible inside the cluster.

krelay

krelay is a drop-in replacement for kubectl port-forward with some enhanced features.

Table of Contents

• Highlights
• Demo
• Installation
• Usage
• Flags
• How It Works

✨Highlights

• Supports UDP port forwarding
• Supports simultaneous forwarding of data to multiple targets.
• Forwarding data to the given IP or hostname that is accessible within the kubernetes cluster
  • You could forward a local port to a port in the Service or a workload like Deployment or StatefulSet, and the forwarding session will not be interfered even if you perform rolling updates.
  • The hostname is resolved inside the cluster, so you don't need to change your local nameserver or modify the /etc/hosts.

Demo

Forwarding UDP port

Forwarding traffic to a Service

Note

The forwarding session is not affected after rolling update.

Forwarding traffic to a IP or hostname

Forwarding traffic to multiple targets

$ cat > targets.txt <<EOF
# Each line in the file represents a target, the syntax is the same as the command line.
# Empty line or line starts with '#' or '//' will be ignored.

# namespace of the object can be specified by the -n flag
-n kube-system svc/kube-dns 10053:53@udp

# The default namespace is used if no namespace is specified
svc/nginx 8080:80

host/redis.cn-north-1.cache.amazonaws.com 6379
EOF

$ kubectl relay -f targets.txt

Installation

Distribution	Command / Link
Krew	kubectl krew install relay
Homebrew	brew install knight42/tap/krelay
Pre-built binaries for macOS, Linux	GitHub releases

Note

If you only have limited access to the cluster, please make sure the permissions specified in rbac.yaml is granted:

wget https://raw.githubusercontent.com/knight42/krelay/main/manifests/rbac.yaml
# Edit rbac.yaml to update the user name
vim rbac.yaml
kubectl create -f rbac.yaml

Build from source

git clone https://github.com/knight42/krelay
cd krelay
make krelay
cp krelay "$GOPATH/bin/kubectl-relay"
kubectl relay -V

Usage

# Listen on port 8080 locally, forwarding data to the port named "http" in the service
kubectl relay service/my-service 8080:http

# Listen on a random port locally, forwarding udp packets to port 53 in a pod selected by the deployment
kubectl relay -n kube-system deploy/kube-dns :53@udp

# Listen on port 5353 on all addresses, forwarding data to port 53 in the pod
kubectl relay --address 0.0.0.0 pod/my-pod 5353:53

# Listen on port 6379 locally, forwarding data to "redis.cn-north-1.cache.amazonaws.com:6379" from the cluster
kubectl relay host/redis.cn-north-1.cache.amazonaws.com 6379

# Listen on port 5000 and 6000 locally, forwarding data to "1.2.3.4:5000" and "1.2.3.4:6000" from the cluster
kubectl relay ip/1.2.3.4 5000@tcp 6000@udp

# Create the agent in the kube-public namespace, and forward local port 5000 to "1.2.3.4:5000"
kubectl relay --server.namespace kube-public ip/1.2.3.4 5000

Flags

flag	default	description
--address	127.0.0.1	Address to listen on. Only accepts IP addresses as a value.
-f/--file	N/A	Forward traffic to the targets specified in the given file.
--server.image	ghcr.io/knight42/krelay-server:v0.0.1	The krelay-server image to use.
--server.namespace	default	The namespace in which krelay-server is located.

How It Works

krelay will install an agent(named krelay-server) to the kubernetes cluster, and the agent will forward the traffic to the target ip/hostname.

If the target is an object in the cluster, like Deployment, StatefulSet, krelay will automatically select a pod it managed like kubectl port-forward does. After that krelay will tell the destination IP(i.e. the pod's IP) and the destination port to the agent by sending a special Header first, and then the data will be forwarded to the agent and sent to the target address.

Specifically, if the target is a Service, krelay will try to determine the destination address automatically:

• If the Service has a clusterIP, then the clusterIP is used as the destination IP.
• If the type of Service is ExternalName, then the external name is used as the destination address.
• If none of the above scenario is met, then krelay will choose a pod selected by this Service.

The Header looks like this:

	Version	Header Length	Request ID	Protocol	Destination Port	Address Type	Address
Byte Count	1	2	5	1	2	1	Variable

• Version: This field is preserved for future extension, and it is not in-use now.
• Header Length: The total length of the Header in bytes.
• Request ID: The ID of the request.
• Protocol: The protocol of the request, 0 stands for TCP and 1 stands for UDP.
• Destination Port: The destination port of the request.
• Address Type: The type of the destination address, 0 stands for IP and 1 stands for hostname.
• Address: The destination address of the request:
  • 4 bytes for IPv4 address
  • 16 bytes for IPv6 address
  • Variable bytes for hostname

krelay's Issues

Make docker image multi-arch with support both for amd64 and arm64

Lost connection to krelay-server pod

Hello, whether using port-forward or your krew plugin, I get a similar problem, only it seems to be slightly worse (more frequent) with port-forward.

There appears to be no logging from your pod component that would shed light on this.

The pod container is publishing data very quickly on the TCP socket initiated by my local client. I wonder if some buffer somewhere is getting overwhelmed?

Command was kubectl relay --kubeconfig=... --address 0.0.0.0 pod/$POD 23456:23456

For testing try sending multiple gigabytes as quickly as you can from pod back to client.
EDIT: and try sending a stop signal to the client (ctrl+z on max/linux) so that it isn't draining the socket... problem appears faster

I0810 18:07:48.590131 73054 tcp.go:22] "Handling tcp connection" reqID="70e80503-0e95-48e9-a466-2624ce13ddb8" dstAddr="10.244.9.32:23456" localAddr="127.0.0.1:23456" clientAddr="127.0.0.1:57880"
I0810 23:07:48.677016 1 main.go:59] "Start proxy tcp request" reqID="70e80503-0e95-48e9-a466-2624ce13ddb8" dstAddr="10.244.9.32:23456"
I0810 23:20:28.096485 1 tcp.go:48] "Client close connection" reqID="70e80503-0e95-48e9-a466-2624ce13ddb8"
I0810 23:20:28.096895 1 tcp.go:66] "ProxyTCP exit" reqID="70e80503-0e95-48e9-a466-2624ce13ddb8"
I0810 18:20:48.518316 73054 main.go:182] "Lost connection to krelay-server pod"
I0810 18:20:48.518368 73054 utils.go:96] "Removing krelay-server pod" pod="krelay-server-8h7mk"

always using default namespace?

First of all: Thanks for the awesome plugin, this is exactly what I need.

Unfortunately I am a bit stuck, because a `kubectl relay' seems to always create a deployment in the default namespace, for which I don't have access to. Is there a way to specify the target namespace somehow (-n is ignored)?

I0413 08:27:51.904065    1474 main.go:142] "Check if krelay-server exists"
Error: ensure krelay-server: get krelay-server: deployments.apps "krelay-server" is forbidden: User "u-fnkzi4b3ce" cannot get resource "deployments" in API group "apps" in the namespace "default"

Command was:

kubectl relay -n mynamespace host/my.host.com 6333

Thanks!

Windows build

Hi,

Is it possible to add support for Windows build?

Thank you!

FR: add --detach flag

I like this plugin, but would be nice to have a --detach / -d flag that would detach / go background once the (initial) port-forward channel has been established. My current workaround is launching it with &, followed by a sleep 5 before I let my script continue with commands trying to connect to the service to avoid connection refused..

In general, this is OK, while usually the connection becomes ready much before the 5s wait.. But sometimes, when the system is slow, it can take longer, so it would be great to get this feature instead & let it go to background once it's ready (with some configurable timeout).

--server.namespace fails unless default namespace is used

If I don't execute kubectl config set-context --current --namespace="$ENVIRONMENT" before creating a relay, an error occurs. However, when the default namespace is set, the process works correctly.

The relay command is kubectl relay --server.namespace "$ENVIRONMENT" "service/$service" "$ports" --address 0.0.0.0

The error is Error: ensure krelay-server: create krelay-server pod: pods is forbidden: User "foobar" cannot create resource "pods" in API group "" in the namespace "default"

[Feature Request] Deployment clean-up

Hello,

It would be nice if there was an extra flag, e.g. --rm, to delete the deployment when the process exits.

Thank you for creating this plugin.

Migrate from SPDY to websocket

kubernetes/enhancements#4006 SPDY has been deprecated and upstream is migrating to websocket

Switch to SOCKS5

To enable better error handling, I plan to migrate from custom protocol to websocket.

• Enable websocket in server
• Enable websocket in client
• Deprecate custom protocol

Clean up orphan pods (self destruct idle pods)

Hi, we are using krelay to forward ip address which is accessible to the pod only, and forward to our laptop, it works perfectly, however, when the season forkubectl krelay is not cleaned up properly, for example close the laptop to put it to sleep for a few minutes.. then reopen the laptop, the network will be reconnected, we just find krelay-server is still running and we need manually clean them up, just curious if there's any plan to support to self destruct krelay-server after idle for some time, thanks.

feature request: Forwarding to multiple endpoints in one session

We have a use case where developers are testing their apps locally, the apps connect to multiple endpoints both on the cluster and external to the cluster.
Currently we have to launch multiple krelay sessions to accomplish this, it seems a bit resource heavy to do this.
Ideally we'll like something like this to be supported:
kubectl relay ip/10.14.204.99 55432:5432 ip/10.14.204.100 45432:5432

Where we can specify one session and one krelay pod on the cluster to relay to multiple different endpoints.
Is this something that can be supported?

Feature request: new pod select for rolled deployments

When using the deployment as the target, the agent doesn't see that the pod was deleted and the next connection to the port-forwarded port still tries to forward to the old pod. Once "kubectl relay" is restarted this is fixed, but I'd really like not to do that.

Introduce krelayrc which saves user preferences

Unpredictable behaviour when not specifying a namespace?

I believe that kubectl-relay uses --all-namespaces (when no --namespace flag has been provided) to look up the service to connect to, instead of looking in the context's configured namespace.

I have a service mysvc of type ClusterIP, in a namespace called staging. The current namespace (default) doesn't have a service with that name: kubectl get svc -o name mysvc shows Error from server (NotFound): services "mysvc" not found, whereas kubectl get svc -o name mysvc -n staging shows service/mysvc. However I can execute kubectl relay svc/mysvc 8080 and I can connect to the forwarded port.

This is annoying because it means there is no guarantee that the relay will connect to the service(s) in the current namespace when there are services with the same name in other namespaces. The workaround is to explicitly specify the --namespace, but this is unexpected behaviour. I have bumped into exactly this situation when I discovered this anomaly, where I was expecting krelay to forward to one service but it was forwarding to a service in a different namespace.

Replace `klog` with built-in `slog`