running on arm

[akkartik]

running on arm

[jackrusher]

RISC-V would be another excellent target. In general, it'd be grand to see something like a nanopass approach that makes it easy to port...

[s-ol]

SubX is completely tied to x86. Since SubX is a very small subset or x86, it would probably be viable to translate SubX (possibly in one of the later stages of the SubX pipeline) to RISC-V or ARM machine code, but I feel like that is somewhat in conflict with the project principles.

I'm not fully up-to-date on what Mu is like at the moment, but even by replacing everything "from Mu on down" things don't work out smoothly; for example Mu is currently tied to the x86 register names. As I said I don't really know enough about it currently to judge that, but there could be other deeper concepts (e.g. variables in memory vs in registers) that don't make sense the same way in a RISC ISA (where there are no addressing modes and load/store are explicit instructions).

Because of this I've started to try to construct a parallel stack as an experiment, trying to port @akkartik's ideas onto the RISC-V ISA. We recently had a conversation in the fediverse about how (predictably) what intention I reconstructed from his writings is different from the goals he is working towards.

I don't necessarily think this is the "right" way to do it, and I'm surely not going to arrive anywhere near Mu at the slow and sporadic pace I'm going, but it's a way for me to combine engaging with both Mu/SubX and RISC-V respectively, in a way that let's me get my hands dirty.

[akkartik]

I would love to have something like Mu on ARM or RISC-V. I've been looking for someone else to drive such a project, because I suffer from RSI, and my hands hurt too much with the typing for a single instruction set[1]. I'd love to support someone by brainstorming, or pairing while they type, or answering questions asynchronously.

@s-ol has it exactly right about SubX. I don't mean to discourage a porting effort, just to inject a note of caution. I don't think SubX is a good enough general-purpose language to merit porting to other instruction sets, and there isn't enough Mu code out there for a port to buy you much leverage.

I think a more interesting approach would be to design a transparent notation for ARM that can be implemented in machine code. In particular, providing good syntax for barrel-shifter operations feels like a really nice self-contained problem. With a little programming experience, you could probably build a prototype akin to mu/linux/bootstrap in 20 hours. Starting with whatever language you prefer.

It's interesting enough that I've spent some time thinking about it, albeit without anything to show for it. The core of SubX is this skeleton:

*(base + index<<scale + displacement)

What does a similar core for ARM look like?

---

@s-ol Don't take my goals too seriously! Mu is an artifact above all for people to use as they think best. I really like this quote:

The future is a disagreement with the past about what is important.

There's a very wide range of directions to take Mu that I would love to contribute significant effort to. As long as someone else does the typing.

---

[1] This may be the single largest problem with the Mu approach: it takes a lot of typing, even with some simple autocomplete and swapping [] with () to reduce Shift chords.

[zcaudate]

would it be worthwhile to target llvm-ir?

then it'll be possible to use the llvm toolchain to generate arm/x86 code.

[akkartik]

I haven't thought about it much, but my immediate reaction is that:

1. LLVM probably doesn't help much with a machine code notation at the SubX level. The source notation is probably lower-level than LLVM IR.
2. On the Mu level you can certainly imagine implementing a compiler frontend that emits LLVM IR. But as @s-ol pointed out, you'd have to want to program with x86 registers regardless of what processor you're on.

Can you elaborate on what features you like about Mu that might make you want to do that? I wonder if you would prefer a subset of the Mu language that doesn't include registers.

[zcaudate]

To be honest, until I saw your project. I didn't know os-less programming was even possible. My experience with low level programming is limited to a couple of MIPS tutorials so I don't really know how programs are designed in assembly. The thing that recently caught my eye have been the progress ARM made by TSMC (in collaboration with Apple, MS, AMD, etc...) and so I've been keeping my eyes open on tech related to hardware/software interfaces

Personally, I've mainly working with transpiling from a clojure syntax to different languages (js,lua,python,c,sql):

So something like:

(defn.lua add [x y] (return (+ x y))

gets transpiled to

function add (x,y)
  return x + y
end

Recently, I was playing around with transpiling to llvm. So

(def.ll ^{:- [:global :i32]}
    variable)
  
  (defn.ll ^{:- [:i32]}
    main
    []
    (let [%0  [:load  :i32 | (:i32* -/variable)]
          %1  [:mul   :i32 %0 2]
          st 
          _   [:store :i32 %1 (:i32* -/variable)]]
      [:ret :i32 %1]))

gets translated to

@variable = global i32 21

define i32 @main() {
    %1 = load i32, i32* @variable  ; load the global variable
    %2 = mul i32 %1, 2
    store i32 %2, i32* @variable   ; store instruction to write to global variable
    ret i32 %2
}

The same technique would work with mu code as well but I wouldn't know WHY I'd want registers or not, only that if the language exists and I feel that there is a need to target a language, then I'd be able to use lisp to target onto it.

So I'm not the right person to ask about features for this. But ideally, I'd want to be able to target ARM as I think it's the medium term future.

[zcaudate]

@akkartik:

So I downloaded qmeu and went though the linux/bootstrap code and the game of life code. Also, I went through your paper in the Portugal conference and did a bit more reading on instruction sets in general.

I have some thoughts (which may be a bit off)

• Re, barrel-shifting ops. Is it enough to use just the raw op codes? http://www.coranac.com/tonc/text/asm.htm#sec-arm, (small section in 23.3.1 on barrel shifting)

• There's actually a lot in common between mu and llvm ir.

• I feel like the mu syntax is pretty much assembly with very minimal block syntax for loops and branching with linking on functions. Of course there's more functionality in the language itself but I'm talking about it purely regarding syntax.

• LLVM IR is more assembly-like than mu so mu can be translated in LLVM IR pretty easily (by eyeballing the blocks and renaming eax -> r0

• LLVM IR has registers https://llvm.org/docs/MIRLangRef.html#mir-registers

• LLVM IR has a type system (that's pretty good)

• There's a lot of advantages in using LLVM IR, code reuse and the ability to target different architectures is probably the most crucial. I'd be more inclined to write LLVM IR than mu But I think even that is risky if there isn't proper benchmarking to show the difference in performance between that and C.

• Personally, unless it was absolutely necessary, I'd still go with Lua on projects needing c-ish performance just because it has good string and json libraries and it's so easy to test and script. For computational heavy stuff, it's trivial to target the gpu. So it's really difficult for me to justify doing anything with languages in mu's type of space without understand the performance benefits.

• For me right now, I would use mu more for education to build an intermediary language that's a bit higher up than assembly but with more control than c. Another advantage of working with the language is that I would be forced to familiarise myself with the underlying architecture (which is a really good thing considering that the world is moving back towards customised hardware). But as of now, I wouldn't be writing code I'd have to maintain in mu.

• Having worked with clojure for a while, I'm used to environments where you connect to a live repl using an editor and eval anything in the editor. I'd like to be able to do that with mu or a lisp derivative of mu (maybe running on qemu). The simplest solution would be some sort of mu interpreter running on qemu and a socket protocol (the simplest one I can think of is the redis protocol which is about 100 lines of c) to send it commands for live eval. I think this would be doable for me (with your help).

---

[akkartik]

Thanks for all that detail! Yeah, this reasoning makes a lot of sense. Performance isn't really Mu's forte, and education is a pretty good niche for Mu.

You're right, lsl/lsr/asr/ror may well be the way to go here. They're not the words I would use, but there's just 4 of them, and there's value in sticking with what others understand. I built SubX because x86 is a complicated mess. Perhaps the lowest level here should just be regular ARM Assembly syntax.

It might actually be interesting to ask "the Mu question" of LLVM IR: what does the simplest possible language translating to LLVM IR look like?

You may be under-estimating the difficulty of getting data out of Mu-on-Qemu. I don't have any device drivers yet for a network. I suppose you only need one, for either serial port or virtio. Perhaps that's the next step here.

[zcaudate]

Hmm. For me, the simplest possible language translating to LLVM IR would be a lispy LLVM IR (like the example I put above) transpiling to IR. Additional language features can be implemented by either adding some keywords to the grammar or by macro transforms (for blocks).

I really think that mu's focus should be on providing a runtime (with a bundled language) to interact with the lower level features of a system - as opposed to purely being a language (there's just too many languages these days). If I can easily interact with the hardware itself without going through LLVM, then it's a big bonus. The trick really is to be able inject the raw code into a runtime environment so yeah... I'd love it if sockets were available (which I thought they were on mu).

I don't think it's really necessary to be a full blown os like https://os.mbed.com/mbed-os/releases/ but to provide a kernel that you can boot up, interact with and then discard. It'd make for a terrific testbed environment for trying out new code.

[zcaudate]

It could be some sort of low level scripting interface for qemu base systems. that would be really cool.

[akkartik]

The first question the simplest possible language has to answer is "why not just program one level below?" Mu has only one reason for existence: it is the simplest language I could come up with that's memory safe. That's why it exposes registers. I wanted a statement of Mu to map to a single instruction of machine code. Here's a summary of Mu's translation table:

http://akkartik.github.io/mu/html/mu_instructions.html

Interacting with hardware might be a second good reason. The problem: I'm not an expert at interacting with hardware 😄 So we'll see how things evolve and how much I'm able to learn.

[zcaudate]

Yep. Got it. I think it's safe to separate out the language (format + packaging) from the runtime. Here are some questions/comments I have for both:

---

Runtime

• How flexible is mu's translator/debugger? It's basically a vm/interpreter right? Would it be possible to inject arbitrary code into it?
• Would it easily to add JIT capabilities to it (say for the x86 instruction set)?
• This is already been LLVM but the advantage is that it could be a much simpler pipeline that you control - https://llvm.org/docs/tutorial/BuildingAJIT1.html
• I'm much more interested in this aspect because the language really is just a minimal translation of the built-in instruction set and could be seen as just data.

---

Language

• I'm not completely sold on the <CODE>/<INSTRUCTION> way of writing subx/subv.
• I think it can be a source of error when the code and instruction are mismatched. Wouldn't having the <INSTRUCTION> be enough?
• the <CODE> bit can be added later during an additional pass if needed and something that can be easily delegated to a compiler.
• the compiler itself should be able to figure out what to do with it (or say that the code is incompatible with the architecture)
• Looking at the transforms presented in the https://git.s-ol.nu/subv/ it'd be trivial to code a similar compiler in clojure (max 200 loc + opcodes table) to do most of the transforms from the ex.subv file to ex.pack. I don't think there's anything fancy in there (which is great).
• I'm thinking of the compiler as a function that takes a data structure and an architecture, emitting the program string. something like

(compile '[:add r0 r1] :arm)
=> source code

(compile  '[:add r0 r1] :x86)
=> <<ERROR, use eax instead or r0>>

Again, I just think there's a whole bunch of overlap between mu and llvm. It would be interesting to figure out how much overlap there is. Here are some docs covering the instruction set. It's pretty dense but I'd like to get your thoughts on which concepts are mappable between them (LLVM must already be doing something like this internally).

https://llvm.org/docs/MIRLangRef.html
https://llvm.org/docs/LangRef.html#instruction-reference

---

[zcaudate]

would something like this be useful: https://fctorial.com/posts/parse_struct.html?

[akkartik]

I don't quite follow how that link is relevant, but perhaps a port to Clojure will help clarify it.

The Mu translator at mu/linux/mu.subx is not an interpreter but rather an almost trivial compiler. All it does is translate Mu from stdin into SubX on stdout. There's no new VM instruction set, it's just all x86.

I am interested in some sort of JIT capability, because it'll be needed to translate and run Mu from within Mu. Right now you have to reboot in between. The major obstacle is Mu's lack of any notion of process.

[akkartik]

LLVM must already be doing something like this internally

That's exactly right. LLVM is already essentially this abstraction:

(compile '[:add r0 r1] :arm)

So depending on your goals, just using it instead of Mu may be an option.

[zcaudate]

@akkartik Yep. That makes sense.

I'm gunna stick to LLVM IR. I think there may be a simplistic way to achieve something resembling live-eval by shelling out to qemu - http://sushihangover.github.io/arm-bare-metal-comparing-llvm-to-arm-gcc/. The OS-less thing is awesome but not crucial for now.

Thanks for the clarification regarding the project and all your inputs. Things make a lot more sense now than before.

[zcaudate]

I don't quite follow how that link is relevant, but perhaps a port to Clojure will help clarify it.

that link was a clojure project - see https://fctorial.com/posts/parse_struct.html#arealworldparsingexample

I thought that maybe someone clojure inclined who might want to build a disassembler for mu programs would find it useful.

[zcaudate]

I am interested in some sort of JIT capability, because it'll be needed to translate and run Mu from within Mu. Right now you have to reboot in between. The major obstacle is Mu's lack of any notion of process.

What you could consider is to gut out the functionality you need in an interpreter project like https://github.com/kanaka/mal/tree/master/impls/cpp and adapt it to mu for the runtime.

[akkartik]

The problem is that Mu doesn't have ELF binaries to disassemble at the top-level.

Another relevant-looking project that just popped up on my radar: https://github.com/oneOS-Project/oneOS. I think Mu will need some of the foundational features this provides before it can provide the sorts of things you're thinking about.

[zcaudate]

wow. that project's impressive. its implemented its own libc and the entire stack.

having some of the lower lever features would be great.

[akkartik]

A disassembler may still be a good idea, even without debug information. I think it may be equally easy to store the sources in the disk image. I'll keep both options in mind. Thanks!

[zcaudate]

Re qemu scripting - I found this: https://github.com/Comsecuris/luaqemu

https://comsecuris.com/blog/posts/luaqemu_bcm_wifi/

[zcaudate]

and this - https://www.unicorn-engine.org/showcase/. it's an entire ecosystem.