changkun/midgard

⛰️ Universal clipboard sharing service (supports macOS/Linux/Windows/iOS)

midgard

English | 中文

midgard is a universal clipboard service, it supports macOS/Linux/Windows/iOS.

Docs

• Installation
• Usage

Contributes

Easiest way to contribute is to provide feedback! I would love to hear what you like and what you think is missing. Issue and PRs are also welcome.

Acknowledgment

The author of this project would like to thank Wen Yang and Quancheng Rao for their inspiring discussion and testing in the early stage of the midgard.

License

Copyright 2020-2021 Changkun Ou. All rights reserved.

midgard's Issues

Add GUI application

There are two options:

• Cross-platform GUI app
• Tray in system menu

Let's see which we will decide.

Support TODO management

I want to have a feature to manage my TODOs.

It should have a similar feature to Things 3, but it should cross platform, and controlled entirely by myself (And I won't have to pay to anybody else).

The management should be simple: pure text based, which is good enough.

Setup issues

Mainly on the server-side, reported by @qcrao :

• Need: git config --global url."[email protected]:".insteadOf "https://github.com/"
• Customized data folder cannot found a template
• hotkey package keeps failing because of XDisplayOpen

workflow failed: command not found: xdpyinfo

We should use an issue to discuss CI problem.

I copied and executed the following code as a .sh file.

midgard/.github/workflows/midgard.yml

Lines 25 to 42 in 8e6acdb

	run: |
	sudo apt update
	sudo apt install -y xvfb libx11-dev gnome-screensaver dbus-x11
	Xvfb :0 -screen 0 1024x768x24 > /dev/null 2>&1 &
	# Wait for Xvfb
	MAX_ATTEMPTS=120 # About 60 seconds
	COUNT=0
	echo -n "Waiting for Xvfb to be ready..."
	while ! xdpyinfo -display "${DISPLAY}" >/dev/null 2>&1; do
	echo -n "."
	sleep 0.50s
	COUNT=$(( COUNT + 1 ))
	if [ "${COUNT}" -ge "${MAX_ATTEMPTS}" ]; then
	echo " Gave up waiting for X server on ${DISPLAY}"
	exit 1
	fi
	done
	echo " Done - Xvfb is ready!"

Then I got the same error as CI。

Waiting for Xvfb to be ready......  Gave up waiting for X server on :0.0

I copied the code of while condition(line 33), and executed it manually.

Then I got this error. I think this is the reason.

$ xdpyinfo -display ":0.0"
zsh: command not found: xdpyinfo

iOS 15 shortcut fail

xclip false read on Linux

Run the following test in internal/clipboard/platform:

func TestLocalClipboardWrite(t *testing.T) {
	platform.Write([]byte("hi"), types.MIMEPlainText)
	buf := platform.Read(types.MIMEImagePNG)
	if buf != nil {
		t.Errorf("write as text but can be captured as image: %s", string(buf))
	}

	buf = nil
	platform.Write([]byte("there"), types.MIMEImagePNG)
	buf = platform.Read(types.MIMEPlainText)
	if buf != nil {
		t.Errorf("write as image but can be captured as text: %s", string(buf))
	}
}

=== RUN   TestLocalClipboardWrite
    clipboard_test.go:35: write as text but can be captured as image: hi
    clipboard_test.go:42: write as image but can be captured as text: there
--- FAIL: TestLocalClipboardWrite (0.01s)
FAIL
exit status 1
FAIL    changkun.de/x/midgard/internal/clipboard/platform       0.008s

Expect a PASS.

Office feature enhancement

• multiple devices support (provide priority?)
• report system version (Ubuntu, macOS or Windows)
• support vacation mode?
• predict returning time?
• mobile devices support

The "ws://" prefix can't be used when server's domain is not "localhost".

midgard/api/daemon/ws.go

Lines 34 to 39 in 8e6acdb

	api := types.EndpointSubscribe
	if strings.Contains(config.Get().Domain, "localhost") || strings.Contains(config.Get().Domain, "0.0.0.0") {
	api = "ws://" + api
	} else {
	api = "wss://" + api
	}

There's no way to use 'ws://' when my server is not listening on port 443.
Have you considered to support this case?

automatic run midgard in background ?

I want to run the midgard daemon in background whenever I restart my machine (all platforms)

on Linux I wasn't able to achieve by running mg on "sudo" and it isn't working. I tried even Docker doesn't work either, it says docker image not found or something

[Desktop Entry]
Name=Midgard
Type=Application
Exec=bash -c 'sleep 60;cd /home/pi/Server7/midgard && sudo mg server'
Terminal=false
Comment=midgard.
Categories=Utility;

On Windows I am unable to achieve this

I used the below commands on admin powershell windows and even on cmd terminal. The mg.exe shows its running on the Task Manager but nothing works. When I kill the process and start using "mg daemon run" then it works fine on the terminal windows.
mg daemon install
mg daemon start

Running mg status shows OK for both server and client for some reason.

What am I doing wrong ?

Less Installation guide

The installation guide on platforms is very confusing, missing and complex.

There are 3 platforms and its kind of jumbled. Which is the server installation and which is the daemon on each platform is kind of confusing. There is no instructions on how to build on macOS or Windows.

I tried to build on macOS and realised I don't have GO installed so I installed it but this is what the issue is for any newcomers. Its really complex to get start using this project. As we have to build it from scratch on each platform.

If there is a premade binary for each platform then that would be easy for anyone to start using without having deep technical skills.

This is a good project, only thing it lacks is ease to start, please address this issue

Install error: -mod=vendor -x -work flag provided but not defined: -mod

I was all excited to find a clipboard tool with both Linux and iOS connections!

But:

ubuntu@arm:~/builds/midgard$ make
go build -o mg -ldflags="-X changkun.de/x/midgard/internal/version.GitVersion=v0.0.9 -X changkun.de/x/midgard/internal/version.BuildTime=2023-07-20T16:52:29-0700" -mod=vendor -x -work
flag provided but not defined: -mod

"The build flags are shared by the build, install, run, and test commands:"
No list item for "-mod"...

Maybe this server is too old:
Ubuntu 14.04.3 LTS (GNU/Linux 3.8.13-bone56 armv7l)

But I just installed go, it wasn't here before, and didn't get errors...

Get:1 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe golang-src armhf 2:1.2.1-2ubuntu1 [4,727 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe golang-go-linux-arm armhf 2:1.2.1-2ubuntu1 [6,235 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe golang-go armhf 2:1.2.1-2ubuntu1 [7,264 kB]

I found this but it is way beyond my go knowledge:
https://stackoverflow.com/questions/75090216/go-build-mod-mod-with-no-go-mod

Is there any way to make this work?

Support SSO

See https://changkun.de/s/login

Add office feature

In changkun.de/s/working, it reports my working status.

However, the implementation is purely basic and does not offer cross-device experiences.
Midgard is a perfect existing system, to have a server, managing all living devices, and their status. Let's add github.com/changkun/office to the midgard, and have a router: /midgard/office to report my working status.

Add Android support

Looking forward to Android.

maybe a concurrency problem?

When following two statements of m.readChs.Store(m.ID, readerCh) run conrrently, will any readerCh be override by the other with the same m.Id? If so the readerCh overrided won't be able to receive message.

// ListDaemons lists all active daemons.
func (m *Daemon) ListDaemons(ctx context.Context, in *proto.ListDaemonsInput) (out *proto.ListDaemonsOutput, err error) {
        //segment1
	readerCh := make(chan *types.WebsocketMessage)
	m.readChs.Store(m.ID, readerCh)
	m.writeCh <- &types.WebsocketMessage{
		Action:  types.ActionListDaemonsRequest,
		UserID:  m.ID,
		Message: "list active daemons",
	}
//.........
}

case <-m.forceUpdate:
	log.Println("received force status updates")
        //segment2
	readerCh := make(chan *types.WebsocketMessage)
	m.readChs.Store(m.ID, readerCh)

Starting Points

Initial needs from @changkun

• sync clipboard between devices (iOS+macOS+Linux)
• code2img, code and images

Initial needs from @qcrao:

• 一键上传剪贴板里的图片到云主机，返回自定域名的 md 格式的图片链接
• 自动调整大小使得整体看起来比较一致（有时我画了一张局部的图，贴上去看起来放的很大） 原因不明，建议自行导出结果再进行上传: mg alloc
• 定时加密/解密上传到 github

Initial needs from @yangwenmai

• searchable code2img history

TODO

• installation script for daemon process
• authenticated gRPC calls: no need because rpc are served from local
• basic auth with maximum failure control
• websocket clipboard push registration/notification
• iOS shortcut for clipboard data fetching
• config initialization, both for client and server (can we use init for daemon/server installation?)
• code2img
• VCS backup
• register keyboard hotkey
• clipboard history
• list all daemons
• code2img can select lines
• code2img timeline
• backup from container
• integrate telegram

Clipboard monitoring may stop working (?)

The problem can occur after a very very long time (probably a year?), not sure why, but here is a stack trace:

  thread #22, name = 'mg', stop reason = signal SIGSTOP
    frame #0: 0x00007f781bc3eaff libc.so.6`__poll + 79
    frame #1: 0x00007f781bb0ac1a libxcb.so.1`___lldb_unnamed_symbol7$$libxcb.so.1 + 154
    frame #2: 0x00007f781bb0c90a libxcb.so.1`xcb_wait_for_event + 74
    frame #3: 0x00007f781bd7cc08 libX11.so.6`_XReadEvents + 376
    frame #4: 0x00007f781bd6b758 libX11.so.6`XNextEvent + 248
    frame #5: 0x0000000000c0a6cf mg`clipboard_read(typ=<unavailable>, buf=0x000000c00026e028) at clipboard_linux.c:199:9
    frame #6: 0x0000000000c0a0af mg`_cgo_bf56599c8496_Cfunc_clipboard_read(v=0x000000c0004c8da8) at cgo-gcc-prolog:53:11
    frame #7: 0x000000000046bf70 mg`runtime.asmcgocall.abi0 at asm_amd64.s:765
    frame #8: 0x000000000040eb09 mg`runtime.notewakeup at lock_futex.go:145
  thread #23, name = 'mg', stop reason = signal SIGSTOP
    frame #0: 0x000000000046e083 mg`runtime.futex.abi0 at sys_linux_amd64.s:520
    frame #1: 0x0000000000434c36 mg`runtime.futexsleep at os_linux.go:44
    frame #2: 0x000000000040ec07 mg`runtime.notesleep at lock_futex.go:160
    frame #3: 0x000000000043e0ca mg`runtime.mPark at proc.go:1441
    frame #4: 0x000000000043f618 mg`runtime.stopm at proc.go:2408
    frame #5: 0x0000000000440ae5 mg`runtime.findrunnable at proc.go:2984
    frame #6: 0x0000000000441a19 mg`runtime.schedule at proc.go:3367
    frame #7: 0x0000000000441f6d mg`runtime.park_m at proc.go:3516
    frame #8: 0x000000000046a043 mg`runtime.mcall at asm_amd64.s:307
  thread #24, name = 'mg', stop reason = signal SIGSTOP
    frame #0: 0x000000000046e083 mg`runtime.futex.abi0 at sys_linux_amd64.s:520
    frame #1: 0x0000000000434c36 mg`runtime.futexsleep at os_linux.go:44
    frame #2: 0x000000000040ec07 mg`runtime.notesleep at lock_futex.go:160
    frame #3: 0x000000000043e0ca mg`runtime.mPark at proc.go:1441
    frame #4: 0x000000000043f618 mg`runtime.stopm at proc.go:2408
    frame #5: 0x0000000000440ae5 mg`runtime.findrunnable at proc.go:2984
    frame #6: 0x0000000000441a19 mg`runtime.schedule at proc.go:3367
    frame #7: 0x0000000000441f6d mg`runtime.park_m at proc.go:3516
    frame #8: 0x000000000046a043 mg`runtime.mcall at asm_amd64.s:307
(lldb) thread select 22
* thread #22, name = 'mg', stop reason = signal SIGSTOP
    frame #0: 0x00007f781bc3eaff libc.so.6`__poll + 79
libc.so.6`__poll:
->  0x7f781bc3eaff <+79>: cmpq   $-0x1000, %rax            ; imm = 0xF000
    0x7f781bc3eb05 <+85>: ja     0x7f781bc3eb32            ; <+130>
    0x7f781bc3eb07 <+87>: movl   %r8d, %edi
    0x7f781bc3eb0a <+90>: movl   %eax, 0x8(%rsp)
(lldb) up
frame #1: 0x00007f781bb0ac1a libxcb.so.1`___lldb_unnamed_symbol7$$libxcb.so.1 + 154
libxcb.so.1`___lldb_unnamed_symbol7$$libxcb.so.1:
->  0x7f781bb0ac1a <+154>: movl   %eax, %r12d
    0x7f781bb0ac1d <+157>: testl  %eax, %eax
    0x7f781bb0ac1f <+159>: js     0x7f781bb0ae20            ; <+672>
    0x7f781bb0ac25 <+165>: movswl 0xc(%rsp), %eax
(lldb) up
frame #2: 0x00007f781bb0c90a libxcb.so.1`xcb_wait_for_event + 74
libxcb.so.1`xcb_wait_for_event:
->  0x7f781bb0c90a <+74>: testl  %eax, %eax
    0x7f781bb0c90c <+76>: je     0x7f781bb0c960            ; <+160>
    0x7f781bb0c90e <+78>: movq   0x10b0(%rbp), %rdi
    0x7f781bb0c915 <+85>: testq  %rdi, %rdi
(lldb) up
frame #3: 0x00007f781bd7cc08 libX11.so.6`_XReadEvents + 376
libX11.so.6`_XReadEvents:
->  0x7f781bd7cc08 <+376>: movq   %rax, %r12
    0x7f781bd7cc0b <+379>: movq   0x950(%rbp), %rax
    0x7f781bd7cc12 <+386>: testq  %rax, %rax
    0x7f781bd7cc15 <+389>: je     0x7f781bd7cc25            ; <+405>
(lldb) up
frame #4: 0x00007f781bd6b758 libX11.so.6`XNextEvent + 248
libX11.so.6`XNextEvent:
->  0x7f781bd6b758 <+248>: movq   0x78(%r12), %rdx
    0x7f781bd6b75d <+253>: jmp    0x7f781bd6b695            ; <+53>
    0x7f781bd6b762:        nopw   %cs:(%rax,%rax)
    0x7f781bd6b76c:        nopl   (%rax)
(lldb) up
frame #5: 0x0000000000c0a6cf mg`clipboard_read(typ=<unavailable>, buf=0x000000c00026e028) at clipboard_linux.c:199:9
   196      XConvertSelection(d, sel, target, prop, w, CurrentTime);
   197      XEvent event;
   198      for (;;) {
-> 199          XNextEvent(d, &event);
   200          if (event.type != SelectionNotify) continue;
   201          break;
   202      }
(lldb) up
frame #6: 0x0000000000c0a0af mg`_cgo_bf56599c8496_Cfunc_clipboard_read(v=0x000000c0004c8da8) at cgo-gcc-prolog:53:11
(lldb) up
frame #7: 0x000000000046bf70 mg`runtime.asmcgocall.abi0 at asm_amd64.s:765
   762          MOVQ    DI, 40(SP)      // save depth in stack (can't just save SP, as stack might be copied during a callback)
   763          MOVQ    BX, DI          // DI = first argument in AMD64 ABI
   764          MOVQ    BX, CX          // CX = first argument in Win64
-> 765          CALL    AX
   766
   767          // Restore registers, g, stack pointer.
   768          get_tls(CX)
(lldb) up
frame #8: 0x000000000040eb09 mg`runtime.notewakeup at lock_futex.go:145
   142                  print("notewakeup - double wakeup (", old, ")\n")
   143                  throw("notewakeup - double wakeup")
   144          }
-> 145          futexwakeup(key32(&n.key), 1)
   146  }
   147
   148  func notesleep(n *note) {

config.yml location

there should be the freedom to pass the location of config.yml as an optional feature

demon start failed in windows platform

windows 平台 daemon 命令运行报错

windows 平台下 mg daemon start 运行报错。
我觉得应该是 mg daemon start 没有调用 s.Run 方法的原因。

复现步骤

1. windows 平台下执行 make 打包应用。
2. 执行 ./mg daemon install 注册服务。
3. 执行 ./mg daemon start 运行服务。
4. 一段时间后控制台报错 midgard: failed to start, err: The service did not respond to the start or control request in a timely fashion.

解释

个人猜测，windows 平台下服务注册与启动流程应该是这样的：

1. Install 方法将 path/to/your/service run 注册到系统服务中。
2. Start 方法通知系统后台执行 path/to/your/service run 命令。
3. 命令被系统调用后。调用 Run 方法通知系统与 Handler 交互。
4. Handler 通过 Execute 方法与系统交互。在其他协程中运行业务逻辑。

而当前的 mg daemon run 命令直接在当前协程运行了业务逻辑，并且没使用 s.Run 方法。
导致 Handler 无法与系统进行交互。最终超时报错。