rjnw/sham

A DSL for runtime code generation in racket

(rajan-walia)

sham's Issues

libjit?

I found the use of libjit quite. libjit feels pretty much dead development-wise, and since you have bindings for llvm, that would certainly make the use of libjit obsolete. How does libjit interact with llvm in sham and why do you have both?

Installing sham

Hi, I was trying to install sham and it seems that's it's not out of the box. I cloned it and did raco pkg install --auto --link -n sham but there's an rfc dependency. I installed it and ran again but I get all sorts of problems:

➜  sham git:(master) raco pkg install --auto --link -n sham                
Linking current directory as a package
pkg: failure getting info
  error: "get-info: info file has illegal #lang or #reader in private/info.rkt"
pkg: failure getting info
  error: "get-info: info file has illegal #lang or #reader in /home/pmatos/Projects/sham/private/info.rkt"
get-info: info file has illegal #lang or #reader in /home/pmatos/Projects/sham/private/info.rkt
  context...:
   read-extension44
   read
   /home/pmatos/installs/racket-7.1/collects/setup/getinfo.rkt:58:8: temp109
   /home/pmatos/installs/racket-7.1/collects/setup/getinfo.rkt:37:0: get-info/full/ext
   /home/pmatos/installs/racket-7.1/collects/setup/getinfo.rkt:33:0
   /home/pmatos/installs/racket-7.1/collects/racket/contract/private/arrow-val-first.rkt:428:3
   /home/pmatos/installs/racket-7.1/collects/setup/setup-core.rkt:295:2: make-cc*
   /home/pmatos/installs/racket-7.1/collects/setup/setup-core.rkt:333:2: collection-cc!20
   /home/pmatos/installs/racket-7.1/collects/setup/../racket/private/map.rkt:40:19: loop
   /home/pmatos/installs/racket-7.1/collects/setup/setup-core.rkt:526:26
   /home/pmatos/installs/racket-7.1/collects/racket/private/map.rkt:40:19: loop
   [repeats 72 more times]
   /home/pmatos/installs/racket-7.1/collects/racket/list.rkt:586:2: append-map
   /home/pmatos/installs/racket-7.1/collects/setup/setup-core.rkt:509:2: collection-closure
   /home/pmatos/installs/racket-7.1/collects/setup/setup-core.rkt:71:0: setup-core
   /home/pmatos/installs/racket-7.1/collects/setup/setup.rkt:65:3
   ...
raco setup: version: 7.1
raco setup: platform: x86_64-linux [3m]
raco setup: installation name: 7.1
raco setup: variants: 3m
raco setup: main collects: /home/pmatos/installs/racket-7.1/collects
raco setup: collects paths: 
raco setup:   /home/pmatos/.racket/7.1/collects
raco setup:   /home/pmatos/installs/racket-7.1/collects
raco setup: main pkgs: /home/pmatos/installs/racket-7.1/share/pkgs
raco setup: pkgs paths: 
raco setup:   /home/pmatos/installs/racket-7.1/share/pkgs
raco setup:   /home/pmatos//.racket/7.1/pkgs
raco setup: links files: 
raco setup:   /home/pmatos/installs/racket-7.1/share/links.rktd
raco setup:   /home/pmatos//.racket/7.1/links.rktd
raco setup: main docs: /home/pmatos/installs/racket-7.1/doc
raco setup: --- updating info-domain tables ---
raco setup: updating: /home/pmatos/.racket/7.1/share/info-cache.rktd
raco setup: --- pre-installing collections ---
raco setup: --- installing foreign libraries ---
raco setup: --- installing shared files ---
raco setup: --- compiling collections ---
raco setup: --- parallel build using 8 jobs ---
raco setup: 7 making: <pkgs>/sham
raco setup: 7 making: <pkgs>/sham/private
open-input-file: cannot open module file
  module path: /home/pmatos/Projects/sham/private/sham/main.rkt
  path: /home/pmatos/Projects/sham/private/sham/main.rkt
  system error: no such file or directory; rktio_err=3
  compilation context...:
   /home/pmatos/Projects/sham/private/compile.rkt
  context...:
   maybe-raise-missing-module
   default-load-handler
   standard-module-name-resolver
   perform-require!78
   for-loop
   finish
   [repeats 1 more time]
   pass-1-and-2-loop
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:273:0: compile-zo*
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:486:26
   ...
raco setup: 7 making: <pkgs>/sham/private/ast
raco setup: 7 making: <pkgs>/sham/private/compilers
raco setup: 7 making: <pkgs>/sham/private/libjit
raco setup: 7 making: <pkgs>/sham/private/llvm
raco setup: 7 making: <pkgs>/sham/private/llvm/ffi
raco setup: 7 making: <pkgs>/sham/test
test/types.rkt:13:17: compile-module: unbound identifier
  in: compile-module
  compilation context...:
   /home/pmatos/Projects/sham/test/types.rkt
  location...:
   test/types.rkt:13:17
  context...:
   do-raise-syntax-error
   loop
   [repeats 3 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:273:0: compile-zo*
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:486:26
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:478:42
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:548:0: compile-root
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:650:4
   /home/pmatos/installs/racket-7.1/collects/setup/../racket/private/more-scheme.rkt:261:28
   [repeats 1 more time]
test/template.rkt:15:19: compile-module: unbound identifier
  in: compile-module
  compilation context...:
   /home/pmatos/Projects/sham/test/template.rkt
  location...:
   test/template.rkt:15:19
  context...:
   do-raise-syntax-error
   loop
   [repeats 3 more times]
   module-begin-k
   expand-module16
   expand-submodule193
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:273:0: compile-zo*
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:486:26
   ...
test/functions.rkt:18:21: s$:if: unbound identifier
  in: s$:if
  compilation context...:
   /home/pmatos/Projects/sham/test/functions.rkt
  location...:
   test/functions.rkt:18:21
  context...:
   do-raise-syntax-error
   for-loop
   [repeats 2 more times]
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-submodule193
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   ...
test/pointers.rkt:16:75: s$:void: unbound identifier
  in: s$:void
  compilation context...:
   /home/pmatos/Projects/sham/test/pointers.rkt
  location...:
   test/pointers.rkt:16:75
  context...:
   do-raise-syntax-error
   for-loop
   [repeats 2 more times]
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-submodule193
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   ...
test/structs.rkt:17:22: s$:let: unbound identifier
  in: s$:let
  compilation context...:
   /home/pmatos/Projects/sham/test/structs.rkt
  location...:
   test/structs.rkt:17:22
  context...:
   do-raise-syntax-error
   for-loop
   [repeats 3 more times]
   loop
   [repeats 3 more times]
   module-begin-k
   expand-module16
   expand-submodule193
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   ...
standard-module-name-resolver: collection not found
  for module path: disassemble
  collection: "disassemble"
  in collection directories:
   /home/pmatos/.racket/7.1/collects
   /home/pmatos/installs/racket-7.1/collects
   ... [182 additional linked and package directories]
  compilation context...:
   /home/pmatos/Projects/sham/test/vector.rkt
  context...:
   show-collection-err
   standard-module-name-resolver
   perform-require!78
   for-loop
   finish
   [repeats 2 more times]
   pass-1-and-2-loop
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:273:0: compile-zo*
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:486:26
   /home/pmatos/installs/racket-7.1/collects/compiler/private/cm-minimal.rkt:478:42
   ...
test/recursion.rkt:14:21: s$:if: unbound identifier
  in: s$:if
  compilation context...:
   /home/pmatos/Projects/sham/test/recursion.rkt
  location...:
   test/recursion.rkt:14:21
  context...:
   do-raise-syntax-error
   for-loop
   [repeats 2 more times]
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-submodule193
   loop
   [repeats 2 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   ...
raco setup: --- creating launchers ---
raco setup: --- installing man pages ---
raco setup: --- building documentation ---
...
[keeps on going]

How do you install this?

Licensing

I was looking at the code and noticed it was not licensed. I was starting to work on racket-gccjit (https://github.com/LinkiTools/racket-gccjit) when @samth pointed to me that I should check your project and possibly use libgccjit as a backend to it.
As far as I understand it, this is only possibly if you license your project as a GPLv3 project. Fortunately you haven't chosen a license.

Before you make a choice, it would probably be great if you could consider a dual-license agreement that could cater for multiple backends. Thank you.

Docs

Sam mentioned sham here:
https://groups.google.com/d/msg/racket-users/1Ix7Ey3316E/4F5ec8IxAQAJ

Is there any documentation on how sham works?

Please add Sham to the official racket package directory

Hi,
Please add Sham to the official racket package directory:

https://pkgd.racket-lang.org/pkgn/search?q=sham

I think this is an exciting project and people will be interested.

Kind regards,

Stephen

Please add link to paper

Please add link to paper
https://arxiv.org/abs/2005.09028

Sham: A DSL for Fast DSLs

Rajan Walia, Chung-chieh Shan, Sam Tobin-Hochstadt
Domain-specific languages (DSLs) are touted as both easy to embed in programs and easy to optimize. Yet these goals are often in tension. Embedded or internal DSLs fit naturally with a host language, while inheriting the host's performance characteristics. External DSLs can use external optimizers and languages but sit apart from the host.
We present Sham, a toolkit designed to enable internal DSLs with high performance. Sham is itself a DSL embedded in Racket, but compiles transparently to LLVM at runtime. Sham is designed to be well suited as both a compilation target for other DSLs embedded in Racket as well as a language for transparently replacing DSL support code with faster versions. Sham programs interoperate seamlessly with Racket programs, and so no additional effort is required to use a DSL implemented with Sham. Finally, Sham comes with a framework for defining DSL compilers and transformations, which is also used in the implementation of Sham itself.
We validate Sham's design on a series of case studies, ranging from Krishnamurthi's classic automata DSL to a sound synthesis DSL and a probabilistic programming language. All of these are existing DSLs where we replaced the backend using Sham, resulting in major performance gains.

Figure out the loop hoisting slowdown in naive bayes