kubenetworks / kubevpn Goto Github PK
View Code? Open in Web Editor NEWKubeVPN offers a Cloud Native Dev Environment that connects to kubernetes cluster network.
Home Page: https://www.kubevpn.cn
License: MIT License
KubeVPN offers a Cloud Native Dev Environment that connects to kubernetes cluster network.
Home Page: https://www.kubevpn.cn
License: MIT License
given the following kubeconfig
apiVersion: v1
kind: Config
clusters:
- cluster:
server: http://localhost:8001
name: localhost
- cluster:
server: http://localhost:8002
name: localhost2
contexts:
- name: localhost
context:
cluster: localhost
- name: localhost2
context:
cluster: localhost2
current-context: localhost
without --context
, it will use default context (http://localhost:8001)
$ kubectl kubevpn connect
Error: rpc error: code = Unknown desc = failed to get configmap kubevpn-traffic-manager, err: Get "http://localhost:8001/api/v1/namespaces/default/configmaps/kubevpn-traffic-manager": dial tcp 127.0.0.1:8001: connect: connection refused
but with --context localhost2
(http://localhost:8002), it still try to connect to cluster defined in default context
$ kubectl kubevpn --context localhost2 connect
Error: rpc error: code = Unknown desc = failed to get configmap kubevpn-traffic-manager, err: Get "http://localhost:8001/api/v1/namespaces/default/configmaps/kubevpn-traffic-manager": dial tcp 127.0.0.1:8001: connect: connection refused
=======Following is my error logs=========
$ kubevpn connect
get cidr from cluster info...
get cidr from cluster info ok
get cidr from cni...
get cidr from svc...
get cidr from svc ok
traffic manager not exist, try to create it...
pod kubevpn-traffic-manager-f89667975-d4qwg is Pending
Type Reason Message
pod kubevpn-traffic-manager-f89667975-d4qwg is Running
Container Reason Message
control-plane ContainerRunning
vpn ContainerRunning
webhook ContainerRunning
update ref count successfully
error upgrading connection: unable to upgrade connection: <html><body><h1>Resource not found</h1></body></html>
error upgrading connection: unable to upgrade connection: <html><body><h1>Resource not found</h1></body></html>
prepare to exit, cleaning up
update ref count successfully
clean up successful
======= Env Info =========
I run kubevpn proxy command kubevpn proxy deploymeny/xx
,
PS D:\kubevpn\bin> kubevpn proxy deployment/test-deploy --headers name=jack
connectting to cluster
start to connect
got cidr from cache
get cidr successfully
update ref count successfully
traffic manager already exist, reuse it
port forward ready
add ipv6 addr to tun err:Element not found.
create tun device error: Element not found.
create tun listener error: Element not found.
parse route error: Element not found.
start local tun service failed: Element not found.
do connect error: Element not found.
prepare to exit, cleaning up
update ref count successfully
clean up successfully
Error: rpc error: code = Unknown desc = Element not found.
error occurs in tun_windows.go:createTun func: ifName.AddIPAddress(prefix)
func createTun(cfg Config) (conn net.Conn, itf *net.Interface, err error) {
if cfg.Addr == "" && cfg.Addr6 == "" {
err = fmt.Errorf("ipv4 address and ipv6 address can not be empty at same time")
return
}
interfaceName := "kubevpn"
if len(cfg.Name) != 0 {
interfaceName = cfg.Name
}
tunDevice, err := wireguardtun.CreateTUN(interfaceName, cfg.MTU)
if err != nil {
err = fmt.Errorf("failed to create TUN device: %w", err)
return
}
ifName := winipcfg.LUID(tunDevice.(*wireguardtun.NativeTun).LUID())
var ipv4, ipv6 net.IP
if cfg.Addr != "" {
if ipv4, _, err = net.ParseCIDR(cfg.Addr); err != nil {
return
}
var prefix netip.Prefix
if prefix, err = netip.ParsePrefix(cfg.Addr); err != nil {
return
}
if err = ifName.AddIPAddress(prefix); err != nil {
return
}
}
if cfg.Addr6 != "" {
if ipv6, _, err = net.ParseCIDR(cfg.Addr6); err != nil {
return
}
var prefix netip.Prefix
if prefix, err = netip.ParsePrefix(cfg.Addr6); err != nil {
return
}
if err = ifName.AddIPAddress(prefix); err != nil {
log.Infof("add ipv6 addr to tun err:%v", err)
return
}
}
i used nocalhost's proxy mode before, and nocalhost does not support ipv6, but it works for me.
if i remove the setting env code:
func (c *ConnectOptions) startLocalTunServe(ctx context.Context, forwardAddress string, lite bool) (err error) {
// todo figure it out why
if util.IsWindows() {
c.localTunIPv4.Mask = net.CIDRMask(0, 32)
}
var list = sets.New[string]()
if !lite {
list.Insert(config.CIDR.String())
}
for _, ipNet := range c.cidrs {
list.Insert(ipNet.String())
}
// add extra-cidr
for _, s := range c.ExtraCIDR {
_, _, err = net.ParseCIDR(s)
if err != nil {
return fmt.Errorf("invalid extra-cidr %s, err: %v", s, err)
}
list.Insert(s)
}
// if err = os.Setenv(config.EnvInboundPodTunIPv6, c.localTunIPv6.String()); err != nil {
// return err
// }
createTun will work successfully.
does anyone know how to solve this problem?
如题,kubevpn客户端默认添加到本地的是K8S集群内部各服务的IP路由。
请问,有没有额外的配置项可以把其它IP范围也添加上?
例如说独立在K8S集群之外的数据库IP?
Hello Team,
Please check on the below issue, it says timeout and cannot connect to the local services.
Forwarding from 0.0.0.0:10800 -> 10800
2022/02/10 10:47:18 connect.go:165: port forward ready
2022/02/10 10:47:18 connect.go:186: your ip is 223.254.254.117
2022/02/10 10:47:18 connect.go:190: tunnel connected
Handling connection for 10800
2022/02/10 10:47:21 connect.go:202: dns service ok
2022/02/10 10:47:23 dnsserver.go:49: read udp 10.61.64.0:53651->10.61.89.222:53: i/o timeout
2022/02/10 10:47:23 dnsserver.go:49: read udp 10.61.64.0:61890->10.61.89.222:53: i/o timeout
2022/02/10 10:47:23 dnsserver.go:49: read udp 10.61.64.0:49355->10.61.89.222:53: i/o timeout
2022/02/10 10:47:23 dnsserver.go:49: read udp 10.61.64.0:58423->10.61.89.222:53: i/o timeout
2022/02/10 10:47:24 dnsserver.go:49: read udp 10.61.64.0:58097->10.61.89.222:53: i/o timeout
2022/02/10 10:47:24 dnsserver.go:49: read udp 10.61.64.0:60749->10.61.89.222:53: i/o timeout
2022/02/10 10:47:26 dnsserver.go:49: read udp 10.61.64.0:58175->10.61.89.222:53: i/o timeout
2022/02/10 10:47:26 dnsserver.go:49: read udp 10.61.64.0:57496->10.61.89.222:53: i/o timeout
Look this:
pods is forbidden: User "system:serviceaccount:NS:NAME" cannot list resource "pods" in API group "" at the cluster scope
If this serviceaccount only in ${NS} and has ${NS} permission:
rules:
- apiGroups:
- ""
resources:
- pods
- services
- namespaces
verbs:
- get
- list
- watch
- update
- create
- apiGroups:
- ""
resources:
- configmaps
- serviceaccounts
- secrets
- pods/portforward
verbs:
- create
- get
- list
- watch
- update
- patch
- delete
- apiGroups:
- apps
resources:
- deployments
- statefulsets
verbs:
- get
- list
- watch
- create
- update
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
- rolebindings
verbs:
- get
- list
- watch
- create
- update
Now i create a clusterrole below:
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- list
- watch
- get
- create
- delete
- patch
and create rolebinding to the serviceaccount${NAME} under ${NS}.
Then exec kubevpn connect
, it still output: pods is forbidden: User "system:serviceaccount:minum:minum-dev" cannot list resource "pods" in API group "" at the cluster scope
Port 6060 is a well-known port. It is used for golang pprof. So kubevpn's port will be conflict with other golang programs. It is better to use ports that are rarely used.
When running kubevpn upgrade
, after downloading the new kubevpn binary to /tmp, the upgrade fails with the message
FATA[0001] rename /usr/bin/kubevpn /tmp/344018565: invalid cross-device link
WARN[0001] exit status 1
This is likely due to my /tmp directory on a different mount point/file system than /usr. On my system, /tmp
uses the tmpfs
file system and /usr/bin
is on a different file system.
The API call invoked to rename the kubevpn
binary is likely the wrong one to handle cross-file system file moves.
Hello.
I have setup KubeVPN for my cluster but when use kubectl connect to cluster with kubevpn plugin it show error can not setup ipv6. For my company policy, we must disable IPv6 on our computers and not enable it.
Is there any way we can setup a tunnel with kubevpn without enabling IPv6 on my PC?
Thank you.
执行命令kubevpn proxy deployment/xx-test -n xx-dev --image registry.cn-hangzhou.aliyuncs.com/xx/kubevpn:v2.2.17
,可以正常使用本地机器代理节点流量。但执行命令kubevpn proxy deployment/xx-test -n xx-dev --image registry.cn-hangzhou.aliyuncs.com/xx/kubevpn:v2.2.17 --headers user=naison
,加不加header都无法正常进行请求。应该如何排查这个问题?
hi, is there LICENSE statement on this repo? thanks.
resolve dns with ingress configuration after connecting kubevpn.
without depending on the global dns configuration of using system.
Hi, over time the kubevpn traffic manager pod increases the CPU usage, I need to restart it in order to lower CPU usage. This happens regardless of using or not the VPN.
Installed as krew plugin case:
It looks like it check whether need to upgrade kubevpn but only check /usr/local/bin (or kubevpn version).
workaround is copy kubectl-kubevpn to /usr/local/bin/kubevpn
kubevpn connect:
pod kubevpn-traffic-manager-847dbd46c-9thpr is Running
Container Reason Message
control-plane CrashLoopBackOff back-off 40s restarting failed container=control-plane pod=kubevpn-traffic-manager-847dbd46c-9thpr_default(dc8489b6-fbfd-415c-85ee-c6acd93f06d3)
vpn ContainerRunning
webhook ContainerRunning
kubectl logs:
kubectl logs kubevpn-traffic-manager-847dbd46c-9thpr -c control-plane
failed to create file watcher, err: too many open files
ulimit on the node:
root@node3:/home/minum# ulimit -a
real-time non-blocking time (microseconds, -R) unlimited
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63555
max locked memory (kbytes, -l) 2048064
max memory size (kbytes, -m) unlimited
open files (-n) 128000
For now, the AddMeshContainer(pkg/mesh/controller.go:30) func add an envoy sidecar with log level of error.
...
spec.Spec.Containers = append(spec.Spec.Containers, v1.Container{
Name: config.ContainerSidecarEnvoyProxy,
Image: config.Image,
Command: []string{
"envoy",
"-l",
"error",
"--base-id",
"1",
"--service-node",
nodeId,
"--service-cluster",
nodeId,
"--config-yaml",
},
Args: []string{
string(envoyConfig),
},
...
could this param be set like config.Image. So user are able to inject this log level when build a kubevpn.exe.
1、和ktconnect区别有哪些
2、proxy和dev模式的区别
3、当采用proxy和dev模式时原有的服务实例为什么会被kill掉
➜ ~ kubevpn connect
INFO[0000] [sudo kubevpn connect]
Password:
2022/02/12 22:05:19 connect.go:334: kubeconfig path: /Users/naison/.kube/config, namespace: default, services: []
2022/02/12 22:05:19 main.go:12: listen tcp 127.0.0.1:6060: bind: address already in use
2022/02/12 22:05:20 remote.go:43: traffic manager already exist, reuse it
2022/02/12 22:05:20 cleaner.go:81: update ref count successfully
Forwarding from 0.0.0.0:10800 -> 10800
2022/02/12 22:05:21 connect.go:172: port forward ready
2022/02/12 22:05:21 connect.go:194: your ip is 223.254.254.152
2022/02/12 22:05:21 connect.go:198: tunnel connected
Handling connection for 10800
2022/02/12 22:05:22 connect.go:212: dns service ok
E0212 23:00:30.725601 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0212 23:44:25.688538 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 05:38:41.357787 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 07:03:25.100489 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 08:00:46.077903 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 08:31:19.048864 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 10:47:16.774766 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 11:03:44.509446 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 11:36:35.102138 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 17:03:14.256355 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 18:30:53.096133 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 19:36:09.137256 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 20:05:43.623330 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
2022/02/13 20:05:46 dnsserver.go:52: read udp 223.254.254.152:61743->172.27.0.3:53: i/o timeout
2022/02/13 20:05:47 dnsserver.go:52: read udp 223.254.254.152:52977->172.27.0.3:53: i/o timeout
Handling connection for 10800
2022/02/13 20:05:49 dnsserver.go:52: read udp 223.254.254.152:62322->172.27.0.3:53: i/o timeout
E0213 22:28:38.162477 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0213 23:13:10.898237 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0214 01:41:03.626462 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0214 07:00:54.685699 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0214 07:32:11.338149 61200 portforward.go:233] lost connection to pod
next loop
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0214 09:38:49.408919 61200 portforward.go:233] lost connection to pod
next loop
2022/02/14 09:38:49 util.go:107: error upgrading connection: error sending request: Post "https://cls-q7562lt8.ccs.tencent-cloud.com/api/v1/namespaces/default/pods/kubevpn.traffic.manager/portforward": dial tcp: lookup cls-q7562lt8.ccs.tencent-cloud.com: no such host
2022/02/14 09:38:49 connect.go:160: port-forward occurs error, err: error upgrading connection: error sending request: Post "https://cls-q7562lt8.ccs.tencent-cloud.com/api/v1/namespaces/default/pods/kubevpn.traffic.manager/portforward": dial tcp: lookup cls-q7562lt8.ccs.tencent-cloud.com: no such host, retrying
2022/02/14 09:38:51 util.go:107: error upgrading connection: error sending request: Post "https://cls-q7562lt8.ccs.tencent-cloud.com/api/v1/namespaces/default/pods/kubevpn.traffic.manager/portforward": dial tcp: lookup cls-q7562lt8.ccs.tencent-cloud.com: no such host
2022/02/14 09:38:51 connect.go:160: port-forward occurs error, err: error upgrading connection: error sending request: Post "https://cls-q7562lt8.ccs.tencent-cloud.com/api/v1/namespaces/default/pods/kubevpn.traffic.manager/portforward": dial tcp: lookup cls-q7562lt8.ccs.tencent-cloud.com: no such host, retrying
Forwarding from 0.0.0.0:10800 -> 10800
Handling connection for 10800
E0214 09:40:13.071024 61200 portforward.go:385] error copying from local connection to remote stream: write tcp 10.94.100.104:55602->139.186.120.240:443: write: can't assign requested address
2022/02/14 09:40:13 dnsserver.go:52: read udp 223.254.254.152:51126->172.27.0.3:53: i/o timeout
2022/02/14 09:40:14 dnsserver.go:52: read udp 223.254.254.152:61839->172.27.0.3:53: i/o timeout
2022/02/14 09:40:16 dnsserver.go:52: read udp 223.254.254.152:58635->172.27.0.3:53: i/o timeout
2022/02/14 09:40:17 dnsserver.go:52: read udp 223.254.254.152:60042->172.27.0.3:53: i/o timeout
2022/02/14 09:40:18 dnsserver.go:52: read udp 223.254.254.152:52012->172.27.0.3:53: i/o timeout
2022/02/14 09:40:18 dnsserver.go:52: read udp 223.254.254.152:50451->172.27.0.3:53: i/o timeout
2022/02/14 09:41:21 dnsserver.go:52: read udp 223.254.254.152:51154->172.27.0.3:53: i/o timeout
2022/02/14 09:41:21 dnsserver.go:52: read udp 223.254.254.152:50329->172.27.0.3:53: i/o timeout
2022/02/14 09:41:22 dnsserver.go:52: read udp 223.254.254.152:61923->172.27.0.3:53: i/o timeout
2022/02/14 09:41:22 dnsserver.go:52: read udp 223.254.254.152:62057->172.27.0.3:53: i/o timeout
2022/02/14 09:41:24 dnsserver.go:52: read udp 223.254.254.152:64283->172.27.0.3:53: i/o timeout
2022/02/14 09:41:24 dnsserver.go:52: read udp 223.254.254.152:51440->172.27.0.3:53: i/o timeout
2022/02/14 09:56:12 dnsserver.go:52: read udp 223.254.254.152:50922->172.27.0.3:53: i/o timeout
2022/02/14 09:56:12 dnsserver.go:52: read udp 223.254.254.152:65462->172.27.0.3:53: i/o timeout
2022/02/14 09:56:13 dnsserver.go:52: read udp 223.254.254.152:52830->172.27.0.3:53: i/o timeout
2022/02/14 09:56:13 dnsserver.go:52: read udp 223.254.254.152:57397->172.27.0.3:53: i/o timeout
2022/02/14 09:56:15 dnsserver.go:52: read udp 223.254.254.152:59620->172.27.0.3:53: i/o timeout
2022/02/14 09:56:15 dnsserver.go:52: read udp 223.254.254.152:54049->172.27.0.3:53: i/o timeout
2022/02/14 09:56:19 dnsserver.go:52: read udp 223.254.254.152:56031->172.27.0.3:53: i/o timeout
2022/02/14 09:56:19 dnsserver.go:52: read udp 223.254.254.152:58310->172.27.0.3:53: i/o timeout
2022/02/14 09:56:27 dnsserver.go:52: read udp 223.254.254.152:62202->172.27.0.3:53: i/o timeout
2022/02/14 09:56:27 dnsserver.go:52: read udp 223.254.254.152:54247->172.27.0.3:53: i/o timeout
2022/02/14 09:56:43 dnsserver.go:52: read udp 223.254.254.152:55924->172.27.0.3:53: i/o timeout
2022/02/14 09:56:43 dnsserver.go:52: read udp 223.254.254.152:59982->172.27.0.3:53: i/o timeout
2022/02/14 09:57:15 dnsserver.go:52: read udp 223.254.254.152:62051->172.27.0.3:53: i/o timeout
2022/02/14 09:57:15 dnsserver.go:52: read udp 223.254.254.152:52193->172.27.0.3:53: i/o timeout
^[2022/02/14 09:58:18 dnsserver.go:52: read udp 223.254.254.152:57344->172.27.0.3:53: i/o timeout
2022/02/14 09:58:18 dnsserver.go:52: read udp 223.254.254.152:62950->172.27.0.3:53: i/o timeout
2022/02/14 09:58:19 dnsserver.go:52: read udp 223.254.254.152:62851->172.27.0.3:53: i/o timeout
2022/02/14 09:58:19 dnsserver.go:52: read udp 223.254.254.152:61293->172.27.0.3:53: i/o timeout
2022/02/14 09:58:21 dnsserver.go:52: read udp 223.254.254.152:55878->172.27.0.3:53: i/o timeout
2022/02/14 09:58:21 dnsserver.go:52: read udp 223.254.254.152:63568->172.27.0.3:53: i/o timeout
2022/02/14 09:58:25 dnsserver.go:52: read udp 223.254.254.152:51031->172.27.0.3:53: i/o timeout
2022/02/14 09:58:25 dnsserver.go:52: read udp 223.254.254.152:62581->172.27.0.3:53: i/o timeout
2022/02/14 09:58:33 dnsserver.go:52: read udp 223.254.254.152:55098->172.27.0.3:53: i/o timeout
2022/02/14 09:58:33 dnsserver.go:52: read udp 223.254.254.152:62414->172.27.0.3:53: i/o timeout
2022/02/14 09:58:49 dnsserver.go:52: read udp 223.254.254.152:61411->172.27.0.3:53: i/o timeout
2022/02/14 09:58:49 dnsserver.go:52: read udp 223.254.254.152:53475->172.27.0.3:53: i/o timeout
2022/02/14 09:59:48 dnsserver.go:52: read udp 223.254.254.152:61811->172.27.0.3:53: i/o timeout
2022/02/14 09:59:48 dnsserver.go:52: read udp 223.254.254.152:58350->172.27.0.3:53: i/o timeout
2022/02/14 09:59:49 dnsserver.go:52: read udp 223.254.254.152:53736->172.27.0.3:53: i/o timeout
2022/02/14 09:59:49 dnsserver.go:52: read udp 223.254.254.152:61616->172.27.0.3:53: i/o timeout
2022/02/14 09:59:51 dnsserver.go:52: read udp 223.254.254.152:52342->172.27.0.3:53: i/o timeout
2022/02/14 09:59:51 dnsserver.go:52: read udp 223.254.254.152:51834->172.27.0.3:53: i/o timeout
2022/02/14 09:59:55 dnsserver.go:52: read udp 223.254.254.152:63901->172.27.0.3:53: i/o timeout
2022/02/14 09:59:55 dnsserver.go:52: read udp 223.254.254.152:53082->172.27.0.3:53: i/o timeout
2022/02/14 10:00:03 dnsserver.go:52: read udp 223.254.254.152:52142->172.27.0.3:53: i/o timeout
2022/02/14 10:00:03 dnsserver.go:52: read udp 223.254.254.152:50053->172.27.0.3:53: i/o timeout
2022/02/14 10:00:19 dnsserver.go:52: read udp 223.254.254.152:51965->172.27.0.3:53: i/o timeout
2022/02/14 10:00:19 dnsserver.go:52: read udp 223.254.254.152:65493->172.27.0.3:53: i/o timeout
2022/02/14 10:05:18 dnsserver.go:52: read udp 223.254.254.152:54433->172.27.0.3:53: i/o timeout
2022/02/14 10:05:18 dnsserver.go:52: read udp 223.254.254.152:56871->172.27.0.3:53: i/o timeout
2022/02/14 10:05:19 dnsserver.go:52: read udp 223.254.254.152:63751->172.27.0.3:53: i/o timeout
2022/02/14 10:05:19 dnsserver.go:52: read udp 223.254.254.152:64957->172.27.0.3:53: i/o timeout
2022/02/14 10:05:21 dnsserver.go:52: read udp 223.254.254.152:52814->172.27.0.3:53: i/o timeout
2022/02/14 10:05:21 dnsserver.go:52: read udp 223.254.254.152:62283->172.27.0.3:53: i/o timeout
2022/02/14 10:05:25 dnsserver.go:52: read udp 223.254.254.152:56852->172.27.0.3:53: i/o timeout
2022/02/14 10:05:25 dnsserver.go:52: read udp 223.254.254.152:56745->172.27.0.3:53: i/o timeout
2022/02/14 10:05:33 dnsserver.go:52: read udp 223.254.254.152:54916->172.27.0.3:53: i/o timeout
2022/02/14 10:05:33 dnsserver.go:52: read udp 223.254.254.152:57536->172.27.0.3:53: i/o timeout
2022/02/14 10:05:49 dnsserver.go:52: read udp 223.254.254.152:61935->172.27.0.3:53: i/o timeout
2022/02/14 10:05:49 dnsserver.go:52: read udp 223.254.254.152:58941->172.27.0.3:53: i/o timeout
^C2022/02/14 10:14:03 cleaner.go:28: prepare to exit, cleaning up
2022/02/14 10:14:03 cleaner.go:81: update ref count successfully
2022/02/14 10:14:03 cleaner.go:42: clean up successful
➜ ~ kubevpn connect
start to connect
got cidr from cache
get cidr successfully
update ref count successfully
traffic manager already exist, reuse it
do connect error: lookup localhost: no such host
prepare to exit, cleaning up
update ref count successfully
no proxy resources found
clean up successfully
Error: rpc error: code = Unknown desc = lookup localhost: no such host
Hello again.
A few days ago, I ran the command 'kubevpn proxy' against two deploymeny in the same namespace, after the command, i found that two pod's tun ip were same(both were 223.254.0.104).
I think this is because of two webhook handler accessing one configmap concurrently.
Is there any optimization for this?
I got docker 137 return code.
Can kubevpn check if docker desktop resource config satisfy remote k8s resource limit, and log WARN while not?
This is my situation:
The k8s service spec in my cluster is like below:
spec:
clusterIP: 12.12.227.122
externalTrafficPolicy: Cluster
ports:
- nodePort: 30234
port: 8004
protocol: TCP
targetPort: 8004
selector:
dev: kk
sessionAffinity: None
type: NodePort
As you can see, spec has no ClusterIPs field. But kubevpn only read ClusterIPs for clusterIp parsing:
func (c *Config) generateHostsEntry(list []v12.Service, hosts []Entry) []Entry {
const ServiceKubernetes = "kubernetes"
var entryList = sets.New[Entry]().Insert(c.Hosts...).Insert(hosts...).UnsortedList()
// get all service ip
for _, item := range list {
if strings.EqualFold(item.Name, ServiceKubernetes) {
continue
}
// only read Spec.ClusterIPs filed
ipList := sets.New[string](item.Spec.ClusterIPs...).Insert(item.Spec.ExternalIPs...).UnsortedList()
domainList := sets.New[string](item.Name).Insert(item.Spec.ExternalName).UnsortedList()
for _, ip := range ipList {
for _, domain := range domainList {
if net.ParseIP(ip) == nil || domain == "" {
continue
}
entryList = append(entryList, Entry{IP: ip, Domain: domain})
}
}
}
So i can't reach k8s svc with service name in my local pc.
Could the svc.Spec.CLusterIP field be considered when generateHostsEntry?
Hi again,
Not sure if this is already covered or partly covered in kubevpn.
We would like to install kubevpn-traffic-manager deployment in a central namespace. In best-case with a helm-chart or similar.
Why?
How?
Maybe a flag in the cli to point to the namespace where the kubevpn-traffic-manager is deployed.
What?
If its possible to do something like this and if you may decide to do it - i can offer at least to build a helmchart.
Hello!
After using this fantastic tool for some time, i have to say the kubevpn proxy
is so convenient. And i encountered few problems when using kubevpn proxy
.
In my k8s cluster, there is a testservice deployment, there shows the containers manifests:
Containers:
hello:
Image: xx
Port: 8003/TCP
Host Port: 0/TCP
hi:
Image: yy
Port: 8004/TCP
Host Port: 0/TCP
For now, after running kubevpn proxy deployment/testservice --headers a=1
.
the envoy config:
- Ports:
- containerPort: 8003
protocol: TCP
- containerPort: 8004
protocol: TCP
Rules:
- Headers:
a: "1"
LocalTunIPv4: 223.254.0.102
LocalTunIPv6: efff:ffff:ffff:ffff:ffff:ffff:ffff:999b
Uid: deployments.apps.testservice
Requesting port 8003 of testservice with headers a=1, the req will be proxied to local port 8003.
Requesting port 8004 of testservice with headers a=1, the req will be proxied to local port 8004.
And it works fine in this scenario.
BUT IN THE BELOW TWO SCENARIO, LOCAL PORT CONFLICTS OCCURS:
scenario 1
:
when i have two service 'testservice1' and 'testservice2', the container manifest:
testservice1:
Containers:
hello:
Image: xx
Port: 8003/TCP
Host Port: 0/TCP
testservice2:
Containers:
hi:
Image: xx
Port: 8003/TCP
Host Port: 0/TCP
And i want to proxy the two service to my local pc, but the two service will both proxied to port 8003 of my local pc, it will not be feasible.
scenario 2
:
when i have a service 'testservice3', the container manifest:
Containers:
hello:
Image: xx
Port: 43029/TCP
Host Port: 0/TCP
and i want to proxy this testservice to my local pc, but the container port 43029 has been used by a import process in my local pc. it will not be feasible neither.
So i wonder if kubevpn proxy
cmd can add an proxy port option for special user, like kubevpn proxy deployment/testservice --headers a=1 --portMap 8003:28003
. And the envoy config can be like this:
ENVOY_CONFIG:
----
- Ports:
- containerPort: 8003
protocol: TCP
- containerPort: 8004
protocol: TCP
Rules:
- Headers:
a: "1"
LocalTunIPv4: 223.254.0.102
LocalTunIPv6: efff:ffff:ffff:ffff:ffff:ffff:ffff:999b
PortMap:
- 8003:28003
Uid: deployments.apps.testservice
So control-plane can create local endpoint 223.254.0.102:28003
. Request with header 'a=1' will be proxied to local pc' 28003 without port conflict.
And containerPort 8004 can not find it's PortMap, it can still be proxied to local pc' 8004.
So this feature will not influence the user who do not need customized local port in proxy scenario.
Newbie here. I currently use telepresence to tunnel traffic between remote k8s cluster and local devcontainer. I have istio service mesh configured and envoy proxy is auto-injected to workload pod. I had to disable both istio sidecar CRD and sidecar injection to make it work with telepresence. I am just curious if kubevpn suffers same challenge. thanks for your time.
$ kubectl kubevpn connect --debug
get cidr from cluster info...
get cidr from cluster info ok
get cidr from cni...
wait pod cni-net-dir-kubevpn to be running timeout, reason , ignore
get cidr from svc...
get cidr from svc ok
traffic manager not exist, try to create it...
Then it just sits there, doing nothing for eternity. Never moves passed this. No pods run. After running kubevpn reset
and starting it again, a pod cni-net-dir-kubevpn
begins to be created, but never finishes. After the timeout message happens, the pod disappears. There is also a deployment called kubevpn-traffic-manager
with 0/0 ready.
Hi
I use kubevpn proxy in my default namespace cannot proxy to other namespaces
If I want proxy other namespace deploy , I need create kubevpn-traffic-manager again in this namespace
Can I use default namespace kubevpn-traffic-manager proxy to any namespace?
This avoids creating too many kubevpn-traffic-manager
Thank you
hello, i found a weird behavior,
once i run export HTTPS_PROXY=...
kubevpn will always use that proxy , even after i run unset HTTPS_PROXY
.
I verified that kubectl itself no longer use that proxy.
is there some kind of cached configuration created by kubevpn?
Error: rpc error: code = Unknown desc = failed to get configmap kubevpn-traffic-manager, err: Get "https://<redacted>/api/v1/namespaces/default/configmaps/kubevpn-traffic-manager": proxyconnect tcp: dial tcp 127.0.0.1:1080: connect: connection refused
Hello Team,
While kubeVPN works well in connecting the services,pods from local laptop using directly POD ID and via Services name using CURL client.
However, I am facing issues, where in reverse proxy mode seems to be not working. It deletes my existing service name automatically.
Steps:
kubevpn connect --namespace=test --workloads=service/myapp
INFO[0000] [sudo kubevpn connect --namespace=test --workloads=service/myapp]
2022/01/20 14:28:32 connect.go:295: kubeconfig path: /Users/myname/.kube/config, namespace: test, serivces: [service/myapp]
2022/01/20 14:28:34 util.go:214: services "myapp" not found
2022/01/20 14:28:38 cleaner.go:74: update ref count successfully
2022/01/20 14:28:40 util.go:214: services "myapp" not found
2022/01/20 14:28:40 connect.go:81: services "myapp" not found
2022/01/20 14:28:40 connect.go:43: services "myapp" not found
WARN[0008] exit status 1
Please suggest.
Thanks,
Mustufa M.
kubevpn changed resolve.conf like below, but it doesn't keep original values.
original value
nameserver 192.168.4.1
after kubevpn connect ssh,
search redis-cluster.svc.cluster.local svc.cluster.local cluster.local us-west-2.compute.internal redis-cluster.svc.cluster.local svc.cluster.local cluster.local us-west-2.compute.internal
nameserver 10.100.0.10
nameserver 192.168.31.14
nameserver 127.0.0.53
options ndots:5 attempts:2 timeout:5
afte kubevpn disconnect
nameserver 127.0.0.53
options edns0 trust-ad
search redis-cluster.svc.cluster.local svc.cluster.local cluster.local us-west-2.compute.internal
as result it lost the dns connectivity.
It looks like even if all ssh configuration correct, it failed with below error.
kubevpn connect --ssh-addr 192.168.1.100:22 --ssh-username root --ssh-keyfile ~/.ssh/ssh.pem --foreground=true
Error: rpc error: code = Unknown desc = not an ip:port
ahd kubevpn logs or debug=true does not generate any logs.
My situation is:
My os is win 10 with powershell of 5.1.19.
When run command:
kubevpn proxy deployment/xx --headers a=1
the process stopped at output, err := cmd.CombinedOutput()
below: (no error and no continue)
func addNicSuffixSearchList(search []string) error {
cmd := exec.Command("PowerShell", []string{
"Set-DnsClientGlobalSetting",
"-SuffixSearchList",
fmt.Sprintf("@(\"%s\", \"%s\", \"%s\")", search[0], search[1], search[2]),
}...)
output, err := cmd.CombinedOutput()
log.Debugln(cmd.Args)
if err != nil {
log.Warnf("error while set dns suffix search list, err: %v, output: %s, command: %v", err, string(output), cmd.Args)
}
return err
}
I took a few hours to find out this issue and then i remove the calling for addNicSuffixSearchList. ALL WORKS FINE.
func (c *Config) SetupDNS() error {
clientConfig := c.Config
tunName := c.TunName
tun, err := net.InterfaceByName(tunName)
if err != nil {
return err
}
luid, err := winipcfg.LUIDFromIndex(uint32(tun.Index))
if err != nil {
return err
}
var servers []netip.Addr
for _, s := range clientConfig.Servers {
var addr netip.Addr
addr, err = netip.ParseAddr(s)
if err != nil {
log.Errorf("parse %s failed: %s", s, err)
return err
}
servers = append(servers, addr)
}
err = luid.SetDNS(windows.AF_INET, servers, clientConfig.Search)
if err != nil {
log.Errorf("set DNS failed: %s", err)
return err
}
//_ = updateNicMetric(tunName)
//_ = addNicSuffixSearchList(clientConfig.Search)
return nil
}
I tried to update my powershell to be latest(7.4.0) and tested again.
kubevpn proxy deployment/xx --headers a=1
the result is:
connectting to cluster
start to connect
got cidr from cache
get cidr successfully
update ref count successfully
traffic manager already exist, reuse it
port forward ready
tunnel connected
adding route...
error while set dns suffix search list, err: exit status 64, output: The argument 'Set-DnsClientGlobalSetting' is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Usage: pwsh[.exe] [-Login] [[-File] <filePath> [args]]
[-Command { - | <script-block> [-args <arg-array>]
| <string> [<CommandParameters>] } ]
[-CommandWithArgs <string> [<CommandParameters>]
[-ConfigurationName <string>] [-ConfigurationFile <filePath>]
[-CustomPipeName <string>] [-EncodedCommand <Base64EncodedCommand>]
[-ExecutionPolicy <ExecutionPolicy>] [-InputFormat {Text | XML}]
[-Interactive] [-MTA] [-NoExit] [-NoLogo] [-NonInteractive] [-NoProfile]
[-NoProfileLoadTime] [-OutputFormat {Text | XML}]
[-SettingsFile <filePath>] [-SSHServerMode] [-STA]
[-Version] [-WindowStyle <style>]
[-WorkingDirectory <directoryPath>]
pwsh[.exe] -h | -Help | -? | /?
PowerShell Online Help https://aka.ms/powershell-docs
All parameters are case-insensitive.
, command: [PowerShell Set-DnsClientGlobalSetting -SuffixSearchList @("default.svc.cluster.local", "svc.cluster.local", "cluster.local")]
dns service ok
start to create remote inbound pod for deployment/test-c3004
patch workload default/deployment/test-c3004 with sidecar
rollout status for deployment/test-c3004
Waiting for deployment "test-c3004" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "test-c3004" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "test-c3004" rollout to finish: 1 old replicas are pending termination...
deployment "test-c3004" successfully rolled out
rollout status for deployment/test-c3004 successfully
create remote inbound pod for deployment/test-c3004 successfully
Though error occurs when addSuffixSearchList, but the process will continue, and the "proxy" still works.
i wonder is there any solution about this issue?
its possible run without sudo or root user ?
kubevpn connect --namespace=staging
INFO[0000] [sudo kubevpn connect --namespace=staging]
2022/01/11 11:30:53 connect.go:40: stat /root/.kube/config: no such file or directory
WARN[0000] exit status 1
for example i have user = user123 with kubeconfig in
/home/user123/.kube/config
running tools is not good as root
this is my situation:
kubevpn proxy deployments/test-c3004 --headers name=kk
.kubevpn leave deployments/test-c3004
.kubevpn proxy deployments/test-c3004 --headers name=kk
.ENVOY_CONFIG:
----
- Ports:
- containerPort: 8003
protocol: TCP
Rules:
- Headers:
name: kk
LocalTunIPv4: 223.254.0.101
LocalTunIPv6: efff:ffff:ffff:ffff:ffff:ffff:ffff:999a
- Headers:
name: kk
LocalTunIPv4: 223.254.0.103
LocalTunIPv6: efff:ffff:ffff:ffff:ffff:ffff:ffff:999c
Uid: deployments.apps.test-c3004
when i request for the pod with headers name:kk
, it's a failure.
And i checked the envoy log, find out that envoy sidecar choose the "old cluster" 223.254.0.101:8003 which is not available.
So i wonder if the "pkg/handler/envoy.go:addEnvoyConfig" func could be optimized. when headers is same, rule shoule be updated instead of append?
I was just trying out kubevpn
for the first time and kubevpn connect
kept hanging just before creating the Deployment
/tmp/kubevpn master !1 ❯ ./bin/kubevpn connect ✘ INT 8 21:29:25
start to connect
get cidr from cluster info...
get cidr from cluster info ok
get cidr from cni...
get cidr from cni ok
get cidr from svc...
get cidr from svc ok
get cidr successfully
traffic manager not exist, try to create it...
label namespace default
create serviceAccount kubevpn-traffic-manager
create roles kubevpn-traffic-manager
create roleBinding kubevpn-traffic-manager
create service kubevpn-traffic-manager
create deployment kubevpn-traffic-manager
I had a look at pkg/handler/remote.go
and saw that the Pod watchstream is started before the Deployment is created
https://github.com/KubeNetworks/kubevpn/blob/99601693d3eb6a773ca870604e0a22940f881e97/pkg/handler/remote.go#L391-L402
My namespace is empty before kubevpn is started, so this command hangs indefinitely. After moving the watch command to after the Deployment is created, everything works fine. Fix is in #94.
I'm not sure how this hasn't happened to anyone else yet though. Is it expected that clientset.CoreV1().Pods(namespace).Watch()
hangs if no matching pod is found? I'm running Kubernetes v1.26.6.
Installation through kubectl krew
is failing due to missing artifact.
└─▪ kubectl krew index add kubevpn https://github.com/KubeNetworks/kubevpn.git
WARNING: You have added a new index from "https://github.com/KubeNetworks/kubevpn.git"
The plugins in this index are not audited for security by the Krew maintainers.
Install them at your own risk.
└─▪ kubectl krew install kubevpn/kubevpn
Updated the local copy of plugin index.
Updated the local copy of plugin index "kubevpn".
Installing plugin: kubevpn
W0717 11:53:39.616542 93538 install.go:164] failed to install plugin "kubevpn": install failed: failed to unpack into staging dir: failed to unpack the plugin archive: failed to obtain plugin archive: failed to download "https://github.com/wencaiwulue/kubevpn/releases/download/v1.1.34/kubevpn_v1.1.34_darwin_amd64.zip", status code 404
failed to install some plugins: [kubevpn]: install failed: failed to unpack into staging dir: failed to unpack the plugin archive: failed to obtain plugin archive: failed to download "https://github.com/wencaiwulue/kubevpn/releases/download/v1.1.34/kubevpn_v1.1.34_darwin_amd64.zip", status code 404
Hello, i have successfully run kubevpn directly from my notebook. Now i tried to run it inside docker but doesn't seems to work.
I have created custom docker image based on bitnami/kubectl
FROM bitnami/kubectl:1.27 AS base
USER 0
RUN apt update -y && apt install -y curl git ca-certificates unzip
WORKDIR /tmp
FROM base AS kubectl-oidc_login
RUN export ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"; \
export VERSION="v1.28.0"; \
curl -sL -o kubelogin.zip "https://github.com/int128/kubelogin/releases/download/${VERSION}/kubelogin_linux_${ARCH}.zip"
RUN unzip kubelogin.zip \
&& mv kubelogin /usr/local/bin/kubectl-oidc_login
FROM base AS kubectl-kubevpn
RUN export ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"; \
export VERSION="v2.2.3"; \
curl -sL -o kubevpn.zip "https://github.com/kubenetworks/kubevpn/releases/download/${VERSION}/kubevpn_${VERSION}_linux_${ARCH}.zip"
RUN unzip kubevpn.zip \
&& mv bin/kubevpn /usr/local/bin/kubectl-kubevpn
COPY --from=kubectl-oidc_login /usr/local/bin/kubectl-oidc_login /usr/local/bin/kubectl-oidc_login
RUN apt-get install -y wget dnsutils vim curl \
net-tools iptables iputils-ping lsof iproute2 tcpdump binutils traceroute conntrack socat iperf3 \
apt-transport-https ca-certificates curl
i run it using docker-compose
x-base-env: &base-env
environment:
KUBECONFIG: ${PWD:-/src}/.kube/kubeconfig.yaml
volumes:
- ${PWD:-.}:${PWD:-/src}
working_dir: ${PWD:-/src}
x-base: &base
<<: *base-env
build: .
services:
start-vpn.sh:
<<: *base
build:
target: kubectl-kubevpn
privileged: true
sysctls:
net.ipv6.conf.all.disable_ipv6: 0
entrypoint: [ ./start-vpn.sh ]
After successful kubevpn connect
I can not resolve any service.
> nslookup fluent-bit
DNS request timed out.
timeout was 2 seconds.
Address: 10.63.225.127
Inside container:
kubectl exec -it kubevpn-traffic-manager-78bdfc9b7-bqxrv -- bash
Defaulted container "vpn" out of: vpn, control-plane, webhook
root@kubevpn-traffic-manager-78bdfc9b7-bqxrv:/app# tcpdump -i any port 53 -nn
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
00:06:31.008866 utun0 In IP 223.254.0.101.62009 > 10.63.225.127.53: 2+ A? fluent-bit. (28)
00:06:31.008903 eth0 Out IP 10.62.128.195.62009 > 10.63.225.127.53: 2+ A? fluent-bit. (28)
00:06:33.024045 utun0 In IP 223.254.0.101.62010 > 10.63.225.127.53: 3+ AAAA? fluent-bit. (28)
00:06:33.024078 eth0 Out IP 10.62.128.195.62010 > 10.63.225.127.53: 3+ AAAA? fluent-bit. (28)
root@kubevpn-traffic-manager-78bdfc9b7-bqxrv:/app# netstat -pantu | grep 53
tcp 0 0 127.0.0.1:33653 0.0.0.0:* LISTEN -
udp6 0 0 :::53 :::* -
OS: Windows 10
I'm getting the following error when trying to use this tool:
Error: no Auth Provider found for name "oidc"
adding the import:
_ "k8s.io/client-go/plugin/pkg/client/auth"
seems to solve the issue.
looks like it validate connection, that's good but if connection is made after portforward, shouldn't it allow insecure? (--insecure-skip-tls-verify only if failed with 127.0.0.1 ?)
https://127.0.0.1:54470/api/v1/namespaces/default/configmaps/kubevpn-traffic-manager": tls: failed to verify certificate
full logs:
wait jump to bastion host...
using temp kubeconfig /var/folders/gb/104yp_bx3nj4p2v4pfrr9h6r0000gq/T/935585872.kubeconfig
Error: rpc error: code = Unknown desc = failed to get configmap kubevpn-traffic-manager, err: Get "https://127.0.0.1:54470/api/v1/namespaces/default/configmaps/kubevpn-traffic-manager": tls: failed to verify certificate: x509: certificate is valid for 10.100.0.1, 192.168.130.132, 10.0.57.113, not 127.0.0.1
Hi all,
first i love your tool! Easy, small and it just works!
Not sure if this is already covered in kubevpn.
We would like to also access resources which are outside of the cluster but part of the same subnet as the nodes.
Why?
We can easily access kube services and pods but we also need sometimes to directly connect to a database which is sadly not in kubernetes.
How?
Maybe a kind of vpn tunnel would be useful for something like this.
What?
To be honest i do not know but i could image to use somehow wireguard, maybe also over a port-forward to build a tunnel to the whole node subnet?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.