gilch / hissp Goto Github PK
View Code? Open in Web Editor NEWIt's Python with a Lissp.
Home Page: https://gitter.im/hissp-lang/community
License: Apache License 2.0
It's Python with a Lissp.
Home Page: https://gitter.im/hissp-lang/community
License: Apache License 2.0
It is at least documented. There might be a way to avoid it. It adds a little more magic, but this is probably the lesser of evils.
Basically, the templates would have to always qualify the invocation position as a macro, but the compiler will fall back to the global of the same name if the macro is absent.
It would probably work.
When Initialized like lissp -i mylib.lissp (and suppose there is mistake in mylib) the following happens:
...
...
# define
__import__('operator').setitem(
__import__('builtins').globals(),
'seperator',
('```'))
QzSLASH_QzSTAR_
# Traceback (most recent call last):
# File "/home/mmm/.local/lib/python3.8/site-packages/hissp/compiler.py", line 491, in eval
# exec(compile(form, "<Hissp>", "exec"), self.ns)
# File "<Hissp>", line 1, in <module>
# NameError: name 'QzSLASH_QzSTAR_' is not defined
#
Python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(LisspREPL)
#>
It boots, that's ok. Now do whatever proper operation and while the Lissp interpreter seems to work the code won't get into python and throws exceptions:
#> 7
(7)
Traceback (most recent call last):
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/__main__.py", line 48, in _interact
repl.lissp.compile(code)
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/reader.py", line 384, in compile
return self.compiler.compile(hissp)
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/compiler.py", line 111, in compile
sys.exit(1)
SystemExit: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/repl.py", line 34, in runsource
source = self.lissp.compile(source)
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/reader.py", line 384, in compile
return self.compiler.compile(hissp)
File "/home/mmm/.local/lib/python3.8/site-packages/hissp/compiler.py", line 111, in compile
sys.exit(1)
SystemExit: 1
#>
The misbehaviour could be reproduced and repeated.
Expected behaviour would be one of the following:
Docs seem to have stabilized. I'd like to cut a release before making any significant changes.
Rough outline of the process:
Consider adding a wheel from the same tagged version. This can be done after the sdist. I'm not sure when compilation of the basic macros file should happen. One could make a reasonable case for doing it every time Hissp is imported, or for distributing the compiled version only. The setup.py
builds seem kinda deprecated, but I don't know how to use the alternatives yet. Maybe this will all have to wait for an 0.3.1.
What's the Hissp equivalent of this Scheme?
(define foo 'bar)
(list 1 foo 2)
In Python it's just (1, foo, 2)
... but I don't know how to get that out of Hissp's reader.
Displaying the inner workings of the compiler just adds noise for the end user.
Feels like it's about time again. Improvements since last time #144 are significant enough to warrant a new release.
Hissp has mostly stabilized. I rarely need to touch the .py files in the Hissp package proper. The bundled macros have had some recent volatility, but #154 and the associated wiki page has helped me to select and implement a set balanced between minimalism and utility.
Most of the recent volatility has been in the docs. Rather than adding anything new right now, I want what's already there to be as accurate as possible before the release, which means a lot of proofreading. That gets to be tedious, so it's taking time.
I'm not totally satisfied with how reader macros handle extras, but don't have a better idea right now. The release will have to happen before any deeper changes there. I might want to address some of the parsing quirks #149 though, at least where I have a clear idea of how it should work.
I don't think the next release can be the 1.0, although it feels like that's getting close now. Maybe the one after that. I may need to write some bigger applications in Hissp 0.4 to see where the pain points are. This would, of course, take time. More examples from the community would also be helpful here.
The alias
macro lets us decouple namespacing from package structure a bit. This kind of thing is natural in Clojure, where files don't have to match namespaces at all, but awkward in Python where namespacing is the package structure. You can sort of work around this by using "private" modules for implementation and importing things into a structured public interface.
The alias
macro works well enough on symbols used at the Hissp level, but it can't abbreviate another reader macro. You can't tag a tag. You can certainly put reader macros in another module to give them shorter names, but I'd like abbreviations to work internally without messing with other packages. This is easy enough for individual macros, but not for a whole package of them.
These were never a planned feature. They just happened to work accidentally, and were useful, so I kept them. But they do make Lissp that much harder to learn. They take up a section in the tutorial and quickstart. They're kind of redundant and have weird restrictions: terminating characters must be escaped and no runtime interpolations.
They were never strictly necessary, since we can do the same thing with an injected string. Now that reader macros can take extra arguments and we have the en- group in the prelude, I think we've lost the strongest arguments for keeping them at all.
On the other hand, Hebigo's bracketed expressions show that fixing the escaping problem is maybe not that difficult. We can recognize the left bracket and then keep consuming tokens (atoms and the occasional continue and error) until we have enough to satisfy the Python parser. We can continue to do a literal eval instead of a full eval like Hebigo does. But this still seems redundant given injected strings, and I'd probably have to update the Pygments lexer and REPL to deal with it. I'm also not sure how lisp-mode would handle it. It also feels like feature creep. It's probably better to get rid of them and free up the brackets for symbols that can do other things.
Reader macros originally required fully-qualified names. I've since added unqualified names ending in a hash in the _macro_
, and seem to be using those a lot more.
It's a design goal of Hissp that everything be available in-line without advance imports. This is important for lissp -c
usage, and any other embedding that just does short snippets. Maybe I should add that to the README. Unfortunately, this isn't possible for the prelude definitions, but lissp -c
does imply the prelude. Fully-qualified names work for all importable runtime objects, and for compiler macros. The original fully-qualified reader macros also fit this requirement.
But, it's pretty awkward that using a reader macro that already ends in a hash requires an escape of that hash. E.g.
hissp.._macro_.b\##"Fully qualified b# macro at read time."
It would be nicer if
hissp.._macro_.b#"Fully qualified b# macro at read time."
worked, perhaps by assuming the hash is part of the name. Maybe we don't have to require it to live in a _macro_
namespace anymore, just end in #
. But consider
builtins..ord#Q
Now this won't work, because it's ord
, not ord#
in builtins. But,
.#(ord 'Q)
still would. Maybe we can stop here.
This is both better and worse. It's nice that we don't have to fully qualify it (although we can), but it's too bad we have to wrap it in (), which could force a line break in standard style. ord
was never meant to be a reader macro, we're just invoking it that way, so maybe an inject makes more sense.
But consider the alias macro
(hissp.._macro_.alias M: hissp.._macro_)
M:#!b"Read-time b# via alias."
(alias B builtins.)
B#!ord Q
also from the quick start. As a proof of concept, a custom reader macro could be made to work the same way, on any symbol resolvable at read time:
as-reader-macro#!builtins..ord Q
as-reader-macro#!ord Q
But, we'd have to "import" as-reader-macro
somehow. The whole point of this form was to make everything available inline without advance imports.
One solution might be to extend the built-in inject macro .#
to accept optional extra arguments, as aliases do.
.#!builtins..ord Q
.#!ord Q
This would allow any read-time resolvable callable to be used as a reader macro, but wouldn't necessarily require a fully-qualified name. This frees up the fully-qualified reader macro syntax to assume the name ends in #
, so we don't have to repeat it with an escape, with an overhead of only a few characters: .
, !
, and maybe a space. (Not that fully-qualified reader macros were ever short.)
The current prelude exec's the following Python code
from functools import partial,reduce
from itertools import *;from operator import *
def entuple(*xs):return xs
def enlist(*xs):return[*xs]
def enset(*xs):return{*xs}
def enfrost(*xs):return __import__('builtins').frozenset(xs)
def endict(*kvs):return{k:i.__next__()for i in[kvs.__iter__()]for k in i}
def enstr(*xs):return''.join(''.__class__(x)for x in xs)
def engarde(xs,f,*a,**kw):
try:return f(*a,**kw)
except xs as e:return e
_macro_=__import__('types').SimpleNamespace()
try:exec('from hissp.basic._macro_ import *',vars(_macro_))
except ModuleNotFoundError:pass
Most of the en- group could be replaced with a single reader macro:
(defmacro en\# (f)
`(lambda (: :* $#xs)
(,f $#xs)))
Now you can use en#tuple en#list en#set en#frozenset
, and apply en#
to anything else with a single iterable argument, i.e. anything you can use direct genexpr in without the extra ()
, which is quite a lot. It does add a bit of overhead, but this seems useful. It could also maybe be enhanced to accept kwargs without ambiguity.
I kind of wondered if the prelude could be eliminated with that. It seems like kind of a hack. But, en#frozenset
is much longer than enfrost
, en#dict
is not very helpful, en#str
doesn't work, and neither does en#"".join
nor (en#.join "" ...
, but (.join "" (en#list ...
would. frozenset
was kind of a questionable addition to begin with, and .format
still works for strings, so whatever, but engarde
is a completely different animal, and still seems important. It pretty much can't be implemented as a direct macro, short of inlining an exec
. The prelude seems less bad here. There is the contextlib version in the FAQ,
(hissp.basic.._macro_.prelude)
(deftype Except (contextlib..ContextDecorator)
__init__ (lambda (self catch handler)
(attach self catch handler)
None)
__enter__ (lambda (self))
__exit__ (lambda (self exc_type exception traceback)
(when (isinstance exception self.catch)
(self.handler exception)
True)))
(define bad_idea
(-> (lambda (x)
(operator..truediv 1 x))
((Except ZeroDivisionError
(lambda (e)
(print "oops"))))
((Except `(,TypeError ,ValueError)
(lambda (e)
(print e))))))
But inlining this much also seems bad. The result seems a bit more usable than engarde
though, which pretty much requires an enlosing let
and cond
or something. engarde
seems incomplete compared to the Hebigo, Drython, and toolz versions.
The surrounding template quote qualifies the symbol. This leads to confusing error messages.
Recursive macros currently require an explicit forward declaration. The template quote has to know at read time if an unqualified symbol should be qualified as a global or as a macro. In principle, defmacro shouldn't need this, because the macro name is provided. But by the time defmacro knows the symbol for the new macro, read time has passed and the template is already qualified.
Lisp code can be instrumented by rewriting it with a macro. I think CIDER does something similar for Clojure. It has always been the plan to demonstrate how to do code-walking macros in the Hissp documentation; the TODO has been in there for a while now. Ease of writing code-walking macros is also one of the stated benefits of Hissp having only two special forms.
The bundled macros do currently include spy#
for debugging, and, of course, the Python debugger works fine on the compiled Python. Like the function literal macro, a debugging macro would probably be too complex to bundle, but a simple one probably wouldn't be too complex for a walkthrough. I think we could at least step through expressions and inspect locals.
#173 would have to happen first.
cascade
is from a Smalltalk notation. doto
is from Clojure. They're very similar, too similar to want both in a minimal macro set, but differ in what is returned: cascade
returns the result of the last "method" call, and doto
returns the self.
Regardless of which you want, either one can be used either way, but one is easier. In Smalltalk, the base object has a yourself
method that all other objects inherit, which is frequently used in the end of the cascade. Python doesn't have that, but in Lissp's cascade
, you can use an identity function, or something with similar behavior (like a progn
). For doto
, you can get the result of the final method call by wrapping that call around the doto
instead of putting it at the end:
;; evaluates to the result of (final spam)
(cascade spam (foo) (bar) (final)) ; cascade is easier
(final (doto spam (foo) (bar))
;; evaluates to spam
(doto spam (foo) (bar)) ; doto is easier
(cascade spam (foo) (bar) (progn))
So which case is more common? For the mutable objects common in Python, I'm starting to think doto
is better, but Smalltalk also had mutable objects. In Clojure, doto
is usually for configuring mutable objects from the Java side, so they seem to have similar use cases.
The original object in a cascade
is already available in a gensym, so doto
is very easy to implement by modifying cascade
.
Given the new bundled macros #154 #155, a lot of the FAQ is looking pretty dated. It's also not from actually-asked questions, but anticipated ones. Maybe it's time to get rid of it. A few of them did end up getting asked in the chat room though. It was nice to be able to link to them. The old doc versions are still available though.
Given GitHub's Discussions feature, we probably don't need to keep this part in the Sphinx docs, although those do have the advantage (and disadvantage) of Sybil testing.
There are some valuable tips in there that aren't anywhere else. Those can maybe be converted to their own section or moved to the wiki.
Up and down errors work properly when lissp is started without args.
But as soon it is started with a library arrow keys stop working.
[lily] lissp -i biblioteka.lissp 15:43:04
Python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(LisspREPL)
#> 6
>>> (6)
6
#> 8
>>> (8)
8
#> ^[[A^[[B^C
Version is hissp-0.3.0a0
Manually following the quickstart in the REPL has been valuable, but it takes a while. Automating it like a doctest would help keep the doc itself up to date.
Sybil might be able to do this. If the quickstart were formatted like a REPL session, it would pretty much work already. But displaying all the outputs would make it much longer. I also like that the whole-file document kind of tests the Pygments lexer.
Maybe I can display select outputs in special comments or something.
"Reader macros" run in the Lissp reader, while "compiler macros" run in the Hissp compiler. Simple, right? Except both of these terms mean something completely different in Common Lisp, so using them might cause confusion.
Common Lisp's "reader macros" get the raw character stream, while Lissp's act during parsing (after lexing), which is more similar to Clojure's tagged literals.
Also, Common Lisp's "compiler macros" are not the same as Common Lisp's "macros". Hissp's "compiler macros" are much like Common Lisp's "macros".
But then what should we do instead? Simply make a note of this in the docs? Name them something else? What? Tags and macros? Parse-time macros and compile-time macros? Lissp macros and Hissp macros? Atom macros and AST macros? That one doesn't seem quite right.
Too vague to really be actionable, I know, but first impressions matter, and I feel like it could be better than it is.
GitHub markdown is pretty limiting. Most interactivity is not allowed. Animated GIFs are possible, and I'm considering adding some. It does grab one's attention, and can show things off without too much effort from the watcher. I could probably put the whole quickstart[whirlwind tour] in there with one GIF per section. But making them is a significant effort and the docs are still in flux. There's probably a way to automate this, but I might have to write most of that myself.
I could also possibly link them to a "static" page that loads a Lissp REPL via pyodide. It's kind of slow loading on the first try though. And it would be a nontrivial effort to get it working. I've considered using Brython in the docs, but it had bugs at the time. That would at least load faster, but probably also a nontrival effort.
Hissp keeps gradually accumulating stars, but so far, not many users are doing anything visible with it. I'm considering adding social media buttons to the top of the readme, but since I try not to use those myself, they might be hard for me to test.
I could certainly walk through re-implementations of some of the basic macros. Should probably include nested templates, like defmacro/g
/defmacro!
. See if we can make the Lisp-2 macro work.
This will automatically support metaclasses correctly. The docs should also be updated.
The compiled output is never going to be Pythonic, but I'd at least like it to be human-readable where possible. Long tuples currently end up on one line, which is not readable. pprint.pformat
should be able to do a better job. The width
parameter may need adjustment depending on the current indent, but it shouldn't be allowed to become so narrow as to be unreadable, even if that means making some lines wider.
Some links:
Originally posted by @brandonwillard in #25 (comment)
Currently the compiler only adds a __name__
, but top-level relative imports would require a __package__
to work properly. See PEP 336.
Qualified symbols should be absolute, but import macros might need this.
I thought an sdist would suffice because the Hissp package is pure-Python, but Pyodide really wants a wheel.
Technically, part of the source code is not in Python, but in Lissp, and has to be compiled to Python. Should the distribution include that source? Probably, because it's the more readable form. When should it be compiled? I originally thought on the user's machine, and an sdist can do that. The .lissp file is the source of truth, and its Python compilation is only duplication. But now I don't think the output depends on the user's machine, so probably in advance is better, although the Python version might possibly affect the output.
#42 broke docs. rtfd.io is still using Python 3.7. Hissp now requires 3.8, so autodoc broke. (At least I think it's autodoc. I don't think it's running the doctests or anything.)
Maybe there's some configuration to force 3.8? I'll have to look into this. Or maybe we don't get doc updates until ReadTheDocs updates. Or I can special case some compatibility logic for 3.7. I did not want to have to do that, so I can keep the compiler simple. And there may be more 3.8 features I'll end up using.
We should probably set up an argparse
interface (with help strings) to the existing entry point, hissp
, that starts the REPL, compiles files, etc.
As part of Hissp's commitment to standalone output, from-require
has got to go. If used as directed in the examples, it puts functions from the hissp
package into the current module, which means the module will crash when run on another Python without Hissp installed. Arguably, the standalone promise doesn't apply to anything the user explicitly chooses, but there's no point in having standalone macros if you have to have Hissp installed anyway to "import" them.
Now that we have prelude
and alias
we don't need from-require
for the docs and tests. prelude
would theoretically have the same problem, except I noticed this issue when I originally wrote it, so it silently skip its macro import at runtime if Hissp isn't installed.
Docs and tests will have to be updated as well.
require-as
arguably has the same problem. I'd like to get rid of it as well, but it's currently used in the expansion of defmacro
. I could probably inline it, but that would complicate defmacro
even more.
Lissp's string literals compile to something like ('quote', 'foo', {'str': True})
, but in Hebigo, it's '("foo")'
.
I'm not sure how I feel about quote
ignoring arguments. It seems like it could hide errors. I didn't originally allow this, but couldn't think of another way of representing strings at the time. Representing strings in different ways like this may make macros written in Hebigo incompatible with Lissp.
The current recommendation to invoke the macro using method syntax works, and is maybe adequate for debugging purposes, when you already know the thing you have is a macro (or can easily check). It's also nice that this doesn't require any additional helper functions, like macroexpand
, macroexpand-1
or macroexpand-all
.
However, that's not good enough for metaprogramming purposes. Advanced macros may need to (perhaps conditionally) pre-expand forms in their body, and there's no easy way to tell if any given form is a macro. You'd have to repeat the same checks the compiler is using:
_macro_
namespace?
QzMaybe_
?_macro_
namespace consider that an attr?Then get the macro function, invoke it, and return the result, or don't and return the original form, depending on the answer. Also this should be done in the compiler's macroexpansion context so NS is set properly, just in case the macro function is using that. That last one is already in the right context if you're expanding a macro inside another macro, but maybe not if you're debugging with method syntax.
As one of the design goals of Hissp, compiled Hissp/Lissp code is not supposed to have any run-time dependency on the hissp
package, which means the package can't provide any bundled functions for use with Lissp code, only macros. However, helper functions that are only meant to be used at compile time in a macro technically don't break this rule, because they don't appear in the compiled expansion, and as long as the body is never used at run time, Python doesn't care that lookups wouldn't resolve. The bundled macros already do this with helpers in the reader module, like is_hissp_string()
.
The macroexpand family could be done the same way in the compiler module. There's little run-time use for these, except, perhaps, when debugging at the REPL (if you have a REPL, you have the hissp
package too), so it wouldn't violate the design rule. The checks would have to be factored out of the Compiler class to avoid duplicating logic, but I think that's doable.
I'm not seeing very good alternatives. Maybe a helper macro could expand to an inline function definition, or it could be added to the prelude. Maybe macro recognition could be simplified. Other Lisp-1s would put the macros in the same global namespace as the functions, and just mark them somehow. Python function objects can take attrs and annotations, so this would be easy. Having a separate _macro_
namespace for them has certain tradeoffs that I didn't fully understand at the time I designed it that way. I've added workarounds, but it's certainly not the only way it could be done. Changing that now would deeply alter the character of Hissp. It's not something I'd take lightly, but perhaps there is a middle ground? This will take some careful thought.
.\
eval at read time_\
drop form`
template quote,
unquote,@
splice#\
gensymThe !
built-in reader macro creates an Extra object of length 1,
#> builtins..print# !1 !2 0
0 1 2
>>> None
but they can be longer and still work.
#> builtins..print# hissp.reader..Extra#(1 2) 0
0 1 2
>>> None
I think this should be allowed, but it isn't:
#> builtins..print# ! ! 1 2 0
File "<string>", line None
SyntaxError: Extra for '!' reader macro.
If a !
has an additional !
passed to it, it should just concatenate them. But should that change the order?
I'm also wondering about the tokenizing here. foo#!1!2 X
maybe feels better than foo# !1 !2 X
? But currently the 1!2
is a single atom. I think foo#!! 1 2 X
might work though? Should !1 !2 foo# X
work? !1!2 foo#X
? !!1 2 foo#X?
I'm not sure.
The two main use cases so far are comment strings and aliased reader macros. The strings seem to work fine. I'm not sure about the reader macro aliases though.
Suppose we had (alias M hissp..basic._macro_)
. Then (M#!en tuple 1 2 3)
is the qualified version of (en#tuple 1 2 3)
, but with an aliased qualifier. I think it would look better as (M!en#tuple 1 2 3)
or (M#en!tuple 1 2 3)
, but I'm not sure that can make sense. Maybe (M#en!tuple 1 2 3)
could work if the M#
macro splits the enQzBANG_tuple
symbol on the QzBANG_
. That's just special-cased handling for symbol primaries. For that, we'd have to allow the internal bangs. Or allow the extras to follow the primary, which is the order they're passed in anyway. That might require a lookahead though. Not sure if that's feasible.
I am kind of questioning the whole Extra system. Do we need extras at all when we can tag a tuple? Or inject one? We sort of do, but maybe something else would do better. Some of Clojure's reader macros take multiple arguments, but the extensible ones, the reader tagged literals, only take one.
Given a reader macro with a side effect like
(defmacro p\# (x) (print x))
then
#> p# "foo" _#
foo
#..
foo
#..
foo
#..?
foo
>>> None
This should only print one, but it gets repeated for every line.
Lissp's REPL is based on code.InteractiveConsole
, which buffers one input line at a time until there's a complete form to compile. But the only way to tell if a form is complete yet is to try to read it. If it's not complete, the reader raises a SoftSyntaxError
, and the REPL aborts tries again with another line in the buffer. But this repeats any read-time side effects in the code so far. This could come up a lot with #80.
Because a reader macro may drop the next parsed object, or not, there is no way, in principle, to tell if a form is complete without running the reader on it. We could write a reader macro that only drops on a full moon. A stricter reader could disallow such cases. Clojure's tagged literals, for example, get the next parsed object, and must produce an object. But Clojure's built-in reader macros can have higher arity.
Common Lisp's read macros, on the other hand, get the raw character stream and may well decide when to return based on the phase of the moon, so they can't be analyzed statically either. But, they're not supposed to have side-effects for exactly this reason:
The reader macro function must not have any side effects other than on the input stream; because of backtracking and restarting of the read operation, front ends to the Lisp reader (e.g., ``editors'' and ``rubout handlers'') may cause the reader macro function to be called repeatedly during the reading of a single expression in which x only appears once.
โhttp://www.lispworks.com/documentation/HyperSpec/Body/02_b.htm
While I could probably fix the REPL so it continues rather than restarts, other tooling for Lissp could still run into similar issues. This suggests that I don't need to fix this, and should just document that reader macros are not to have side effects in Lissp either. This all makes the idea of using a stack to implement higher-arity reader macros as in #80 seem like a bad one. I will have to rethink this.
I felt pretty strongly that hyphens shouldn't simply munge to underscores, but recognized that kebab-style
is common in Lisps, and even used this style in the if-else
macro name. I compromised by giving hyphen a single-character munged name, the shortest possible. I still feel that these should be separate, but xH_
doesn't seem that readable in practice. I could see a syntax highlighter helping here, but in most places you see the munged names, you don't have one. A longer name might actually be better.
Currently, red-green-blue
munges to redxH_greenxH_blue
. It's hard to visually separate the xH_
from the word before it.
Perhaps a longer word would suffice?
redxDASH_greenxDASH_blue
_H_
would be much easier to parse visually: red_H_green_H_blue
, but a leading underscore has special meaning in Python, and now it collides with the UPPER_CASE_WITH_UNDERSCORES
style, so this isn't a good idea.
But perhaps a quoting scheme more symmetrical than x
and _
would be more readable? Perhaps x_
and _x
? This doubles the number of characters required, but the point is readability, not length, or we'd just be using ordinals. We'd then use x_H_x
:
redx_H_xgreenx_H_xblue
This seems better. Anything asymmetrical like (x
/x_
) as in redxHx_greenxHx_blue
isn't as nice.
redxH_greenxH_blue ; xH_ status quo.
redxDASH_greenxDASH_blue ; xDASH_
redx_Hx_greenx_Hx_blue ; x_Hx_
redx_H_xgreenx_H_xblue ; x_H_x
redx_DASH_xgreenx_DASH_xblue ; x_DASH_x
\\.
should be an alias of QzBSOL_.
, i.e. a module literal. Instead it's read as QzFULLxSTOP_
. Not sure how that happened, but full stops are handled separately, and this seems wrong.
It is currently possible for a reader macro to start with :
e.g. (defmacro :QzHASH_ (...
, which doesn't seem like the ideal way to say it. This ends up adding a non-identifier string to the _macro_
namespace, which prevents attribute-access symbols like _macro_.:QzHASH_
from working. I'm not sure what \:\#
should be. Making it :QzHASH_
would work for the defmacro, but it doesn't make a lot of sense. Maybe the \:
should suppress the control word interpretation, making it QzCOLON_QzHASH_
? But then the reader macro usage would have to be spelled \:#foo
, which isn't as nice. Should control words be disallowed as reader macro names? Should they always be treated like the first character is escaped? Maybe just colons? I will have to think about this some more.
Reader macros with extras could be more compact if symbols weren't allowed to contain !
. Then you could say foo#!1!2!3 bar
instead of foo# !1 !2 !3 bar
or foo#!!! 1 2 3 bar
. You could still escape them, like any other character. But there are conventions where mutating/dangerous functions end in a !
, and now they'd have to end in a \!
, which isn't as nice. Maybe there's some way to avoid that. Extra could maybe use some other character instead of !
. Lisp's symbols can contain more characters than Python, but that also means you need spaces to separate things in situations Python wouldn't. I'm not really satisfied with the whole Extra system, but I don't have a better idea yet.
The current unqualified reader macros are special-cased in the reader. This is similar to having special forms in the compiler that can't be overridden with a custom macro. But compiler macros do not always have to be qualified. Custom reader macros do. I implemented it this way to make it similar to how Clojure does it.
But now that Lissp has got a more incremental reader, I think the reader could check for the existence of a local reader macro when it finds an unqualified reader macro. We can distinguish reader macro from compiler macro by how it's used, so they could have the same unqualified name, the way variables and compiler macros can now. But I'm not sure if this would be useful for a reader macro.
If we put local reader macros in the _macro_
space, you could create a reader macro using defmacro, and it would behave similarly when invoked as a compiler macro e.g. foo#x
and (foo x)
would be very similar. And you could also invoke a unary compiler macro using the reader syntax, which seems convenient.
Is separating these more useful than combining them? I'm not sure. Similar behavior seems more convenient, and even if you really wanted separate behavior, it would be possible to write a macro that checks if it's expanding during read time or compile time. Another way to distinguish them might be special naming in the _macro_
space, like ending in xHASH_
. The reader could maybe prioritize that one if there's already a macro of the same name without the suffix.
A possible use for reader macros like this would be explicit, but abbreviated qualification. Maybe something like
(defmacro B (sym)
(.format "{}{}" 'hissp.basic.._macro_. sym))
would let you write e.g. B#define
in place of hissp.basic.._macro_.define
, because they would read in the same way. This seems a little more convenient if they're kept separate, but it's workable either way.
Another possible use for short reader macros is to add new types of atoms. This was one of my primary intended uses for reader macros, but forcing them to be qualified can make this inconvenient. For this use case, it might be better to have them combined with the compiler macros.
It is a goal of the project to allow a more Clojure-like reader and a complete function/macro library. But while this informs the design of the compiler, it is beyond the scope of Hissp proper, and does not belong in the Hissp repository.
Does the Clojure macro library currently exist?
Hissp's repository seems to be several projects rolled into one:
Some of these components seem a lot more stable than others, and that's an argument for separating them. The stable parts could get a 1.0 release, without the unstable parts holding them up. I'm obviously not going to break up all of these pieces. Some really do belong together.
Some of the documentation/testing machinery, the Sybil parser, Lissp ReST directive, and the Lissp lexer, at least, would be useful in other Lissp projects. That's an argument for adding them to the hissp
package itself, but they have dependencies (Sybil, Sphinx), and I don't want hissp
to have any dependencies. I could maybe make those dependencies optional, or I could split them off into their own package, which still seems a bit premature until Lissp stabilizes.
Hissp and Lissp maybe don't need to be together. Readerless mode is usable in its own right. Hebigo is already separate. I feel like I'm close to being able to implement an EDN-based Hissp as well. Lissp turned out to be a bit more complex than I wanted, mostly around the "reader macros", which is an argument for simplifying them, and munging. But currently, the docs and tests for Hissp proper depend on Lissp being there. Separating them seems premature, but it's something I'm considering.
That leaves the bundled macros. Having them implemented in Lissp complicates packaging a bit. Hebigo's macros are in readerless mode, so that's an option, but it's dogfooding Lissp/Hissp at least a little, which I feel is important. That said, this is one of the least stable areas, and one most in danger of bloat. Every addition needs tests and docs, which also have to change with changes. Those are strong arguments for separating them, but Lissp feels kind of crippled without them, so I'm reluctant to give them up. On the other hand, most of the early tutorials get by without them, and macros can really only expand to code that you could write yourself, but some of them seem to really help a lot.
Therefore, the bundled macros should be very minimal, but where do we draw the line? It's been my own case-by-case judgement so far, without too much thought for the whole. I've changed my mind several times. I've removed things and tried different approaches. That's why I want to develop some more objective criteria for what gets included and what doesn't.
They have to be macros to avoid adding a dependency to Hissp, but they should really be functions. They also don't handle nil cases properly. It's better to not have them at all if they're going to be that confusing.
I should be able to inline these.
Not a new issue, but I'm writing it down so I don't forget to address it. I knew this was going to happen when I wrote the REPL code, but couldn't see a good way to avoid it at the time. Python's REPL prompts are stored in sys.ps1
and sys.ps2
. Lissp's REPL is based on code.InteractiveConsole
and overwrites these to be the Lissp prompts, because the superclass reads them from there. An unfortunate design flaw.
That means that if you start a code.interact()
session from within the Lissp REPL, you keep getting the Lissp prompts, which is confusing. It's nice to be able to drop into Python from the same environment, so it could come up. The resulting Python console is perfectly functional, so perhaps this is a minor issue.
The methods are also too large to override easily, short of copy/pasting library code. The best available approach may be monkeypatching with unittest.mock.patch
as briefly and as close to use as possible. It's times like this when I wish Python had dynamic variables like Lisp. Contextvars come close, but they're not a drop-in replacement for a global, since you still have to call the .get()
method to dereference. Monkeypatching like this is somewhat brittle, but using the prompts from sys
is documented behavior in the code
module, so this case should work.
The macro tutorial is rough. Things could be reworded for clarity. Some language is too casual to respect. Changes have been made since the docs were written. The quick start could be more minimal. Don't forget to edit the readme as well, remember that shows up on both GitHub and PyPI. Read through the entire generated docs page, including the auto API docs.
Before #46, we should get to full coverage. Coverage is currently approaching 90%, so I think this is very doable. Most of what's missing tests is error handling code. Regressions in this kind of thing are the hardest to notice, because they aren't used as often, so automating tests here will be worthwhile.
Sphinx puts these in the docs as well. And they're useful for cross references. Doctest where applicable.
What do you think about using code
and codeop
to implement the basic REPL mechanics?
The links lead to blank pages. Is it Furo? Isolate and fix or work around and maybe report upstream.
Currently, .lissp
files must be part of a package to compile properly. The macro system relies on absolute imports via qualified symbols, which means it must know how to import each module.
We could disallow top-level modules except for __main__
(and special case that), or we could just special case the empty-string package to mean the top level. So if you have a top-level module foo
, the fully-qualified symbol for foo.bar
might be .foo..bar
.
One of the primary goals of Hissp is to have good error messages.
There are multiple stages to compilation and errors could happen in any of them, which makes debugging more difficult unless you really know what you are doing.
Currently the compile time error messages are pretty long. I've found them to be usable, but as the author of the compiler, I'm also the world's foremost expert on Hissp. My experience may not be typical of users, who may find it difficult to find the relevant information in all that noise.
Hissp has no line numbers to speak of. There's nowhere we could put the metadata. Line numbers wouldn't even make sense in readerless mode but the error message still needs to be good enough to point to the problem.
So if there's an error compiling a form, the error message will point to the exact subexpression that failed to compile by listing all surrounding forms. Really, we only need to print off the top level form and highlight the subform somehow. The rest is pretty much useless noise.
Once we've got the Python output, normal Python debugging and stack traces work. This part isn't too bad because Python has very good error messages, but it will point to the very unpythonic compiled code. This part could perhaps be improved with source maps, but it may take some compiler enhancement to implement them, perhaps by outputting special comments. Then do we map to just the Hissp tuples, or to the skin on top of that, like Lissp? Maybe not a high priority.
Currently the most difficult part is if there is an error during incremental execution (2.c). The error happens in the compiled output, but it hasn't actually been saved yet. Perhaps the output file could be opened in append mode and incrementally updated with each form before its compile-time execution?
Travis perhaps?
100% test coverage doesn't mean that everything is actually tested. I'm not even sure if I've covered all my lines yet. I think the compiler could use a little refactoring, but I'm not quite confident enough in my tests to try it. I'm not sure which mutation testing package to use yet, or how well Travis or the like can use them.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.