techcable/janet-msgpack

A msgpack implementation for Janet

janet-msgpack

An implementation of msgpack for Janet.

Features

• Supports encoding most Janet types (tables, arrays, primitives, etc...)
• Uses ludocdoe/mpack for decode, hand-coded encoding.
• Automated testing with comparison to Python impl

TODO

• American Fuzzy Lop
  • This is difficult to do because we aren't a C program

Credits

I.E: Libraries and copied code

• janet/json by @bakpakin - Code for Janet interop largely copied from here
• ludocode/mpack - Used for msgpack decoding

janet-msgpack's Issues

Allow configuring the type of decoded table keys

Right now the string type is hardcoded to keyword but someone might want them to be a string or a symbol.

Allow customizing UTF8 requirement

Right now we require UTF8 for strings, symbols, and keywords (see here), but not for buffer.

This behavior should be configurable.

jpm test failure on Linux

After a recursive clone of 05372b0, I get the following for jpm test:

$ jpm test
compiling mpack/src/mpack/mpack-reader.c to build/mpack___src___mpack___mpack-reader.o...
generating meta file build/msgpack.meta.janet...
compiling msgpack.c to build/msgpack.o...
compiling mpack/src/mpack/mpack-common.c to build/mpack___src___mpack___mpack-common.o...
compiling mpack/src/mpack/mpack-platform.c to build/mpack___src___mpack___mpack-platform.o...
compiling mpack/src/mpack/mpack-reader.c to build/mpack___src___mpack___mpack-reader.static.o...
compiling msgpack.c to build/msgpack.static.o...
compiling mpack/src/mpack/mpack-common.c to build/mpack___src___mpack___mpack-common.static.o...
linking build/msgpack.so...
compiling mpack/src/mpack/mpack-platform.c to build/mpack___src___mpack___mpack-platform.static.o...
creating static library build/msgpack.a...
running test/test.janet ...
error: cannot open directory data
  in os/dir [src/core/os.c] on line 1882
  in _thunk [test/test.janet] (tailcall) on line 27, column 17
non-zero exit code in test/test.janet: 1
Failing test scripts: 1

After making the following changes, it seemed to work ok:

$ git diff
diff --git a/test/test.janet b/test/test.janet
index dd3761e..41adaf6 100644
--- a/test/test.janet
+++ b/test/test.janet
@@ -1,7 +1,7 @@
 (import msgpack)
 
 
-(def data-dir "data")
+(def data-dir "test/data")
 (defn run-test [data-file]
   (def path (string/join [data-dir data-file] "/"))
   (print "Testing: " path)
@@ -10,7 +10,7 @@
   (file/close f)
   (def actual-data (msgpack/decode data))
   (def proc (def {:out expected-data-out} (os/spawn
-    ["python3" "reference.py" path]
+    ["python3" "test/reference.py" path]
     :exp
     {:out :pipe})))
   (def expected-data-text (:read expected-data-out :all))

Is it possible things are not set up to work with jpm's testing?

Should support extension types

We should support msgpack extension types.

The user will have to specify a custom function to serialize/deserialize it.