hack-pad/hackpad

The in-browser IDE for Go

hackpad's Issues

Test cases

This definitely is a fascinating project. The docker ceo apparently said that if WASM existed there would have been no need for docker. He was referring to sand boxing.

Did you hit many bugs when attempting compilation using tinygo. I think they did a release recently or it might still be on a branch . Tinygo is efficient enough that you could run on mobile or have a good chance.

I have a interesting idea. Your sage accounting software uses a bucket file system dB and a git based vcs to presumerable allow sync of the underlying files . Would be interesting test case if these two work running on top of the wasm indexdb / FS. It’s a decent test case I feel.

Would badger dB in theory work ?

Seeing error "package fmt is not in GOROOT (/usr/local/go/src/fmt)" in attempting hello world on hackpad.org

Seeing the following error, see linked screenshot below:

Terminal text here for convenience also:

$ go version
go version go1.16.6 js/wasm
$ go mod init playground
go: creating new go.mod: module playground
go: to add module requirements and sums:
	go mod tidy
$ go mod tidy
$ go build -v .
main.go:4:2: package fmt is not in GOROOT (/usr/local/go/src/fmt)
package playground: cannot find package
exit status 1

Any ideas what I might be doing wrong here? Thanks!

HTTP Server

Hey, did you know that you can intercept requests using service workers (in the web). And in this way make a virtual tcp server which can later be used for http servers

Implement core of Node.js `net` module for servers

Hey, I understand that you are primarily a go programmer but is there any way with which you can implement this for running Nodejs??

Editing the editor (tutorial platform)

Hey,
This is super cool.
I have wanted for quite some time to build tutorials for the Go language using wasm (in browser) - you have probably jump started my work here by a mile.
The first step is to make a super interactive platform and a 'framework' for the tutorials so its fast and easy to add new ones.

Thoughts:

• left hand side panel where the instructions for this step/tutorial are
• pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.
• tutorials can use libraries that already exist (your platform offers this)
• I can use mockers (smocker) to mock out external services in tutorials
• I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

Questions

left hand side panel where the instructions for this step/tutorial are

I looked through the code and see you are designing/rendering the UI in Go which was amazing/surprising. Two questions really - one is do you have a link to tutorial in how you are doing this, and secondly, could this be swapped out for HTML/CSS/JS or React or something easily? My guess is WASM means better to do all in Go, so question one is more the focus - i.e can you point me to anything to learn how to adjust the UI?

pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.

This sounds possible based on your "OS" - Would be cool if tests can run against code?

I can use mockers (smocker) to mock out external services in tutorials
I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

I suspect these last two are closely related and its either "thats easy" or "thats hard" as the OS isn't really "real".....

Anyway would love just to hear your thoughts on this usecase for this platform.
Thanks

caching and etags semantics and perhaps wider reuse patterns

Hey @JohnStarich

Hope all is well.... I am really fascinated with your project and wider implications of this.

I am not sure if this issue / question shoudl be here or in the FS repo but here goes....

One thing i am working on is the relationship between retained mode GUI's and data caches. A retained mode gui can be conceptually designed like a data cache system that we all know and love. A gui has a template that is computed on to produce some markup. In a retained mode gui system you want to know if the template has changed. It woudl be a signal at the control plane level (if you get my drift), so you can do optimistic re rendering of that gui component instance, which is sort of like eager caching. Hope that this is making sense :)

Some background might also help... I know this is pretty out there...
The DOM is a browser is doing the retained mode diffing for you. It knows when you changed the HTML in the inspector, and magically updates the HTML Screen for you. But when you working at the WASM level and rendering with WASM to a Webgl canvas, you need to do this yourself..

So now i get to my question... The FS implementation is super cool because it can work with a WASM, local or remote context, but i was wondering if it supports etags style caching or other meta data semantics to tell the caller that you need to do a cache miss and refetch because your local data is old buddy !!

Golang Http FS has etags: https://go.googlesource.com/go/+/go1.8.3/src/net/http/fs.go; see line 119. So it can easily act as a cache to a DB called over http for example.

The reason i bring this up is because you also want to do the same thing inside the WASM environment for many use cases.
Let say in the WASM environment you have a FS ( that is indexeddb under the hood) that is sourced from S3. When the S3 source changes, i presume there is some sort of etag header to tell you. So you want when in the WASM environment there is a call to the WASM FS, it should realise that the local FS is now "dirty" or "invalidated" and so it should automatically get the data from S3, update the local file, and return that to the caller.

So i am wondering if this is currently supported.

It will be really interesting to see how this also related to the Web Worker and Service Worker stuff too Service Worker.

It will be really interesting to build quasi CDN like architectures where you can have caches at the WASM, local and remote level like a CDN style architecture. An thew also cross mix. Markup templates are dependent on data which can be dependent on the caache and its "turtles all the way down"..... I like the simplicity of this.

You also get into a interesting situation with Design time versus Runtime also.
I use the work real time because the IDE is running in a browser and doing compilation as you change code. At non real time, you working off already compiled golang.
At real time Design time, you want your goimport to be always latest and when someone else PR#s some code that you import you want to know NOW and see it and adapt. you want to get broken by them ...
At Real time Runtime, you want to use a static version, and not break.

The same does for data. YOu have static data ( json, binaries, etc) and you have dynamic data.
At Real time Design time, you want everything to be dynamic. Don't use any caching anywhere.
At real time Runtime, you want everything that is static to never update, and everything that's dynamic to use the cache mechanism.

Demo crashes from high memory usage

The demo eventually crashes at Chromium 89

possible to package hackpad inside astilectron

Compiler as a Service

It would be cool to have a compiler as a service running on a server that the HackPad IDE can interact with.

the reason is because I am interested in making plugins . You write it in golang and compile with Tinygo . The Compiler As A Service ( CAAS ) does that and then you can “ see it “ in the IDE.

I saw the Web Worker Par in HackPadFS and it got me thinking. I presume that’s so that the FS in running isolated in its own isolated area outside of the main GUI and so hence hardening the system from crashes.

so my rational is to do it for all code. The developers code that they are developing in the IDE is compiled and exposed as a Web Worker.

The same can then be done with Wazero so that you can run the wasm on the server also. Why think about this ? Because you end up with reconfigurable ( is that a real word ? ) computing, and so depending on the nature of the code can run it in a Browser or on a Server ( with Wazero ).

The way WASI and WASM exposes the exports is different . But it’s possible to shim that by writing a compile time wrapping that walks the AST and code generates the equivalents needs for it to run in the browser too. If we write the exports out as a JSON file then an IDE can “ discover “ the exports and elevate it in the IDE GUI.

@codefromthecrypt as I figure this is interesting to you .

I probably should have made this a discussion.

My request is to know what are the blockers.
is my codegen concept the way to go for at least the golang community. I read about WIX but the more I delve into it I get the impression that there is no standard yet ( or ever will be ) for WASI and WASM being able to be single sourced and run in both environment easily . Hence sone compile time and runtime wrapper ingredients seems needed.

The isolation aspects of WebWorker and Service Workers can possibly mapped to similar topology primitives on the Server works. A WebWorker is easy because it’s the same as a Fly.io Machine or a Wazero server hosting WASI.
the Service Worker equivalent would be a gateway / Proxy / Caching type of thing similar to Caddy with Caddy Modules but not sure . Service workers have logic , caching and procuring capabilities.

cant run "web-workers-2" branch

Had some time to jump into this rabbit hole again and playing around with File System explorer module, so you get a tree.

On origin/feature/web-workers-2 branch

This is my simple make script that wraps your make script..

"make serve" does not work for me. Any ideas ?

build-init:
	# installs special go.
	cd $(REPO_NAME) && $(MAKE) go

build:
	# builds the many wasm and then the server npm stuff.
	cd $(REPO_NAME) && $(MAKE) build

serve:	
	cd $(REPO_NAME) && $(MAKE) serve

Building the golang IDE gui for Web, Mobile and Desktop.

The current way hackpad is building the GUI using go and wasm is nice.

It just occurred to me that it's possible to do the same using gio. It compiles to Browser ( as wasm ), Desktop and mobile ( as native ).

Web site: https://gioui.org
Code: https://github.com/gioui

Live Example :https://gioui.org/files/wasm/kitchen/index.html

I am positive that most of what the IDE does currently can easily work with hackpad. The port would not be painful.
The examples show more than what your currently doing...

Some examples.

https://github.com/JackMordaunt/kanban

• a typical kanban gui
• desktop and mobile work, but web wont because boltdb is used for the DB.

https://github.com/gioverse/chat

• a typical chat gui.

https://github.com/planetdecred/godcr

• a large gui app for block chain things

---

For DB your indexeddb looks like a good match for Genji.

https://github.com/genjidb/genji
Genji supports Web, Desktop and Mobile. but for Web it uses InMemory

So you IndexedDB would be another driver for Genji. See:
https://github.com/genjidb/genji.js/blob/master/src/bindings/main.go

I used genji for 6 months, and was great. Got it snapshotting to S3 also. Used it for Server, Desktop and Mobile.

Please feel free to ask me if your curious..

Use Web Workers for true concurrency

The biggest performance bottleneck for hackpad right now appears to be true parallelization. The Go CLI performs starts compile and link processes, which could benefit greatly with real parallelization and moving off of the main thread (even though they run in the background).

Going further, I think it could make sense to set up the kernel as a single SharedWorker, with each browser tab representing a new abstract "worker" concept that can run processes. Each tab would start by registering itself as a worker and "requesting" an editor be spawned on itself. Using a single kernel would keep overhead much lower, and enable some interesting inter-tab and project behaviors.

Moving from a single main/background thread to multi-worker is a big undertaking, so this could take a while.

Tinygo

I was playing with the code

the way gs and indexdb is used is really impressive and highly useful

I am thinking that tinygo might be mature enough to deal with this code now .

the main thing was to avoid reflection ( like the standard golang json encoder ). The solution was generally to use flatbuffers.
Flatbuffers is nature now and very useable .

Anyways your project is really awesome . Just wanted to add my thoughts about strategies for using tinygo :)

if you need examples of what I am referring to above just yell

Still a live project?

This project was pretty awesome and just want to find out how its going? Since discovering it I can see more and more cool uses for it! Would love to hear how its going!

made a small contribution to hopefully incentivize the progress on this 😁

The building process seems to be very slow

Hello and thank you for making this!
For some reasons I cannot understand, the build process seems to be very slow. I am waiting for more than 5 minutes and still it didn't finish. I wonder if it is browser related;I am using Firefox un GNU/Linux 89 on a 64x machine.

Update to Go 1.19

What's needed to rebase the go1.16 branch on go1.19?

Race condition in pub sub

https://github.com/JohnStarich/go-wasm/blob/fd0edf41a665e400fef3db6616772439ce4b9738/internal/pubsub/pubsub.go#L40

Go routines A and B call emit(). Both see that visited == false under read lock. A acquires write lock and processes. B acquired right lock and double processes, even though A marked the node as visited. The fix is to check if visited again after write lock is acquired