monome/druid

terminal interface for crow

druid's Issues

feature-request/discussion: tmux-like window interface. pane for script selection. etc.

I think it could be really cool to have an additional pane when running druid for script selection upload. Some of the functionality I am envisioning:

• move the repl to a pane, and add another with a file tree pointing to a specific location (maybe a druid/scripts folder, which we could encourage people to use as the place to store their scripts in the readme)
• macro to switch focus between the panes
• some way to navigate the file-tree with the arrow keys and some sort of macro for uploading "currently highlighted" script

See here for some idea of the "pane-windowing" I am thinking about. https://github.com/yaronn/blessed-contrib

Note I am not familiar with best-practices or libs as they relate to terminal display. I imagine there would need to be considerations for:

• platform differences for terminals (like bash/zsh), terminal clients (like iterm2 or the mac terminal app), and OSs (mac, linux, etc.)
• different color settings/configurations people have setup their terminal to have
• different terminal sizes. It'd be cool if there was something that could handle these in a way similar to responsive web design.

I figure I'd get the ball rolling about this discussion. Curious what the creators of druid think, as well as the community at large (maybe someone has experience building cool terminal apps like this and has some insight?)

Add command-line option to show version

Quitting druid results in traceback

druid version (latest devel 0a34aa3)

python version 3.7.4 (osx)
asyncio version:

  Downloading https://files.pythonhosted.org/packages/22/74/07679c5b9f98a7cb0fc147b1ef1cc1853bc07a4eb9cb5731e24732c5f773/asyncio-3.4.3-py3-none-any.whl (101kB)```

after typing q to exit the repl:

druid ➤ python3 druid.py git:master*
bye.
Traceback (most recent call last):
File "druid.py", line 196, in
main()
File "druid.py", line 190, in main
loop.run_until_complete(background_task)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
return future.result()
concurrent.futures._base.CancelledError

Add a way to browse the filesystem for upload and run commands

For the u (upload) and r (run) commands it would be nice if browsing of directories would be possible within druid.

This is a follow-up from #35

upload of some scripts cause crash

note, this could be a crossover issue between the crow firmware and druid.

reports of this on the forum, but my own case is this:

i can run the attached script on linux.

on macos, druid freezes immediately on pushing enter after u new.lua or r new.lua

the only fix is to pull the USB cable.

it seems that the crow is possibly crashing. as when i pull the cable i see a flood of "disconnected" messages in druid.

here's the script.

local stage = 1
local expectations =
{ { in_chan = 1, out_chan = 1, expect = -5.0 } ,
{ in_chan = 1, out_chan = 2, expect = -1.0 } ,
{ in_chan = 1, out_chan = 3, expect = 3.0 } ,
{ in_chan = 1, out_chan = 4, expect = 9.0 } ,
{ in_chan = 2, out_chan = 4, expect = 7.0 } }

function test_case( e )
  print('in[' .. e.in_chan .. '] -> out[' .. e.out_chan .. ']')
  for i=1,2 do
    if i == e.in_chan then
      input[i].mode('stream',0.1)
    else
      input[i].mode('none')
    end
  end
  for i=1,4 do
    if i == e.out_chan then
      output[i].volts = e.expect
    else
      output[i].volts = 0.0
    end
  end
end

local threshold = 0.1 -- should be much tighter tolerance
function is_in_range( value, expectation )
  if value > (expectation - threshold)
   and value < (expectation + threshold) then
    return true
  end
end

input[1].stream = function(value)
  if is_in_range(value, expectations[stage].expect) then
    print('\tok!')
    stage = stage + 1
    test_case(expectations[stage])
  end
end

input[2].stream = function(value)
  if is_in_range(value, expectations[stage].expect) then
    print('TEST SUCCEEDED')
  end
end

function init()
  test_case(expectations[stage])
end

No newline in sublime on Windows10 causes upload failure

[string "eval"]:13: 'end' expected (to close 'function' at line 11) near <eof>

See report here: https://llllllll.co/t/crow-druid-scripts/25974/36

update setup details in `README`

notably, this advice appears stale:

source Scripts/activate

and maybe setup.sh too?

perhaps just fly in the setup notes from https://monome.org/docs/crow/scripting/ ?

/cc @trentgill @dndrks

ignore ^^pub and ^^ready messages

clutters the druid experience. these messages weren't meant to be human-read

Uploading and downloading via the command-line

Was wondering how we want to handle uploading and downloading of scripts via the command-line.
Currently there are two simple scripts that do this, that need to be called like python3 ./upload.py, not entirely ideal.

Do we want to roll this into the main druid script so it could become something like

$ druid upload scriptname.lua

Or do we want to keep these as separate tools (druid-upload and druid-download maybe)?

druid --version shows wrong version when in a git repository directory

~/s/monome ❯ druid --version
druid, version 0.2.0
~/s/monome ❯ cd bowery 
~/s/m/bowery:master ❯ git remote -v
origin	[email protected]:monome/bowery.git (fetch)
origin	[email protected]:monome/bowery.git (push)
~/s/m/bowery:master ❯ git rev-parse --short HEAD  
ca0859e
~/s/m/bowery:master ❯ druid --version
druid, version 0.1.dev18+gca0859e

Well, thank you druid, but that isn't really the druid version now, is it? :P

I'm starting to think this whole dynamic version thing is too much effort to get working properly.
Will probably drop it in favor of letting setuptools scm write a file with the version in it so it's still enough to just git tag but we lose the dev version info.

When disconnected druid continually fills the console with messages

currently <lost connection> is printed every time druid tries to reconnect. instead we should keep a global var and store the currently connected status, and only print <lost connection> on the first attempt to reconnect.

macro scripts

some method of sending short little code snippets to crow.
perhaps an in-druid micro-script editor, or it could be a special macros.lua file that druid knows about (eg: F1 key sends the contents of function F1() and then calls F1() to execute it.

windows always shows connected

regardless of whether a crow is attached, windows druid always shows 'crow connected'.

when i attached a crow to an open druid session, it printed some startup info, but did not allow me to speak with crow (or at least, i didn't see responses to print msgs).

perhaps the serial port matching is just grabbing any open COM port (i had an FTDI debugger attached)? unsure, but it requires a little work to test on windows.

Refactor REPL and crowlib code

There are a couple of issues with the current REPL and crowlib code that we should address:

• REPL: Use of globals/no proper place to store state
• Communication internals are implemented in both the REPL and the CLI as well as in crowlib, mainly because we're directly exposing the serial port object (ie we need to do .write(bytes()) or .write(something.encode()).
• Where to write output to from crowlib is passed in from the REPL. Ideally the output from crowlib wouldn't care about this and use normal logging which would just bubble up to consumers of crowlib like the REPL that could then show the logged output.
• Things like uploads are implemented multiple times (once in crowlib and once more with a bit more logic in the CLI) and their behavior is different

I've been trying to get something working for a bit (for example in this branch https://github.com/simonvanderveldt/druid/tree/cleanup-crowlib where I started with refactoring crowlib) but it's been difficult to keep the changes small, mainly because everything is a bit entangled with eachother. It tends to end up in basically changing everything 😬
I don't really know how to solve that yet, but I would like to suggest to start with adding a class for the REPL. This should contain the REPLs state and hopefully be a decent start for the other changes as well.

This might need a separate issue but putting it here for now:
We might also need to change the serial implementation a bit, especially the reading of the serial port. Unfortunately asyncio's add_reader won't work cross platform (mainly issues on Windows, serial isn't an fd on Windows which means it won't work with SelectorEventLoop and ProactorEventLoop doesn't support add_reader at all. See https://docs.python.org/3/library/asyncio-platforms.html#windows for more detail.
So we'll either need to fall back to using pyserial-asyncio, which also has issues on Windows which for now have been solved by polling (see pyserial/pyserial-asyncio#8) or we could also simply split the serial reading off into a thread.

Also see #38

r <file> broken on osx

freezes druid altogether?

`^^help` does not work

> ^^help
repl:1: unexpected symbol near '^'

i tried to dig into this but quickly was in over my head. i'm on v3.0.1.

Fresh install of druid breaks with error ModuleNotFoundError: No module named 'prompt_toolkit.eventloop.defaults'

Hi,

I just got my Crow and tried to install druid on my Mac and got this error message :

$ druid
Traceback (most recent call last):
  File "/usr/local/bin/druid", line 6, in <module>
    from druid.cli import cli
  File "/usr/local/lib/python3.7/site-packages/druid/cli.py", line 9, in <module>
    from druid import repl as druid_repl
  File "/usr/local/lib/python3.7/site-packages/druid/repl.py", line 8, in <module>
    from prompt_toolkit.eventloop.defaults import use_asyncio_event_loop
ModuleNotFoundError: No module named 'prompt_toolkit.eventloop.defaults'

Here is the installation log returned by pip3 :

Requirement already satisfied: monome-druid in /usr/local/lib/python3.7/site-packages (0.2.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from monome-druid) (40.8.0)
Requirement already satisfied: pyserial>=3.4 in /usr/local/lib/python3.7/site-packages (from monome-druid) (3.4)
Requirement already satisfied: setuptools-scm-git-archive in /usr/local/lib/python3.7/site-packages (from monome-druid) (1.1)
Requirement already satisfied: prompt-toolkit>=2.0.10 in /usr/local/lib/python3.7/site-packages (from monome-druid) (3.0.0)
Requirement already satisfied: setuptools-scm in /usr/local/lib/python3.7/site-packages (from monome-druid) (3.3.3)
Requirement already satisfied: Click>=7.0 in /usr/local/lib/python3.7/site-packages (from monome-druid) (7.0)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/site-packages (from prompt-toolkit>=2.0.10->monome-druid) (0.1.7)

Could the problem come from prompt-toolkit 3.0 being installed instead of version 2.0 ? While looking into the error, I read version 3 has some change in the use of asyncio.

Thanks,
Julien Jassaud

tab translation

ii.jf.help() sends back ^I which should be translated as tab

Layout changes

We currently have a layout that is built up like this (this is the actual layout, I've just made the separate parts more visible):

I was wondering if we could make it a bit more standard and also make room for possible tab-completion/help texts.

My suggestion would be to move the ////druid bar to the bottom and treat it as a proper statusbar. This statusbar could then show things like the connectivity status (connected/disconnected, also see #43) as well as the the values of the two input ports. By moving it to the bottom we create room for auto-completion/help popups + this is the normal spot to have a statusbar in.
See below for a quick prototype of this idea.

What do you think @trentgill @tehn ? Do you have any other ideas/suggestions/plans regarding the layout?

P.S. The screenshot also contains a tab-completion example with help text, I'll create a separate issue for that.

firmware version decoding failure

druid fails to decode a git string like 4.0.1-14-g7721314 resulting in the script crashing.

the result, is druid firmware won't work if you have a firmware installed on your crow that fails the version string.

our last update distributed a firmware with such a string.

solution: if the version decode fails, just flash the firmware of the newest. assume it's old.

REPL input not visible with light theme

The text input in the druid REPL is unreadable on a light background, which makes the whole REPL difficult to use.

(image shows issue on kitty terminal; there is text in the input)

It looks like the REPL has some colours hardcoded in, maybe a possible solution is to support an environment variable or flag to use the terminal colour scheme?

pip3 -> pipx

recent linux install requires python-pipx rather than python-pip3 --- checking macos now

simple doc update

druid.log config

druid.log gets dropped in the working folder, so, druid.log files sortof end up everywhere.

i propose setting up some default folders and perhaps a config

input values not at top of the repl

the readme says:

input values are printed on the top line

but I'm not seeing those

cli command to manage imports in lua code

I've been playing with making different types of sequencers for the crow and have found myself repeating quite a bit of code. I was hoping there could be a function that would allow for better code modularity, rounding up all the imports from the file you're trying to upload and constructing a single composite file that can be fed to the crow.

I have been getting familiar with the codebase and I think I could make something that works, but I would like to ask first if this would be a welcome feature or if it's something that was thought of already and discarded for some specific reason.

Design question: Fit and purpose of `u` and `r` commands in REPL

We have two commands in the REPL that to me feel a bit off, u (upload) and r (run).

According to the help they operate on files and allow one to either run the contents of this file or write them to flash and then run them.

IMHO there are a couple of issues with these commands:

• First of all: Assuming the sole purpose of these commands is uploading/running files: It operates on files/data outside of the scope of the REPL unlike all the other things that one can do which are all interactive and stay within the REPL. Is this something we want? Does this make sense?
  To me it feels like the command-line (or a GUI program for that matter) offers easy enough means to browse the filesystem and upload/run scripts so I'm not sure there's much benefit of including it in the REPL.

Now if we do want to keep this functionality there are some other issues:

• If no argument is passed we fall back to a hardcoded sketch.lua as path.

  druid/src/druid/repl.py

  Line 55 in 079c922

  crowlib.execute(writer, myprint, "./sketch.lua")

  IMHO we should only take filenames/paths and that's it, no magic defaults.
• Then if more than one argument is given we send it directly to crow

  druid/src/druid/repl.py

  Line 59 in 079c922

  writer(bytes(cmd + "\r\n", 'utf-8'))

  This seems like a bad idea because it's inconsistent/kinda polymorphic. This also means it's difficult to do any useful validations on the input.

@trentgill @tehn What do you think?

Add development readme

install instructions

setup requires pip: apt install python-pip
readline requires ncurses libs: apt-get install libncurses5-dev

Automatically publish releases to pypi

Would it be an idea to publish druid to pypi? That way it would just be a pip install away instead of needing more complex steps to install it.
If we do, we should make sure this happens automatically for every release we make.

This would mean the following things need to happen:

• Create a proper Python package (fixed by #25 )
• (Manually) publish an initial release to Pypi (done as part of #25)
• Automate the publishing process

For those who have never done this, some usable info can be found here https://realpython.com/pypi-publish-python-package/

remove duplication present with druid/examples and bowery

Currently there are (basically) the same scripts in two locations:

• https://github.com/monome/druid/tree/master/examples
• https://github.com/monome/bowery

My personal vote would be to update references of druid/examples to link to bowery (which I feel is more consistent with the way norns collects scripts, with https://github.com/tehn/ash and previously, monome/dust). I think it could possibly include some quick instruction about git cloneing bowery in the readme.

FR: print results by default

Druid doesn't print results of a line by default, does it? (or something was wrong with my terminal today). May I propose that it would?

This would improve the feedback loop between human and corvid for the advantage of the former, and avoiding repeatedly wrapping simple each line with print( and ) and then editing them out when the line is ready to be built into something.

I think this is the interactive REPL way, and how norns Maiden and interactive vanilla Lua works too. What downside do I fail to imagine?

Clearly output permission errors

Currently when running druid and the user doesn't have the correct permissions druid will say

can't open serial port

If possible we should be more clear/explicit and inform the user that we don't have permissions to open the device because of permissions.

new cli upload doesn't seem to work

I was trying to run

druid upload ../bowery/shiftregister.lua

and it opened the druid repl (but didn't seem to upload the script). Note I was using the latest druid (master), build using the dev readme instructions.

I git checkout'd back to the commit 2d496f1, and ran the old way:

python3 upload.py ../bowery/shiftregister.lua

 uploading ../bowery/shiftregister.lua

lua bootstrapped
input loaded
asl loaded
asllib loaded
metro loaded
ii loaded
crowlib loaded
^^change(2,1)
^^change(2,1)

 file uploaded:
jf loaded
--- digital analog shift register
-- four stages of delay for an incoming cv
-- input 1: cv to delay
-- input 2: trigger to capture input & shift
-- output 1-4: newest to oldest output
-- ii: 6 stages of ASR via just friends

reg = {}
reg_LEN = 4

q = {}
function init()
    for n=1,4 do
        output[n].slew = 0
    end
    input[1].mode('none')
    input[2]{ mode      = 'change'
            , direction = 'rising'
            }
    for i=1,reg_LEN do
        reg[i] = input[1].volts
    end
    ii.pullup(true)
    ii.jf.mode(1)
    metro[1].event = iiseq
    metro[1].time  = 0.005
    metro[1]:start()
end

-- hack to allow 6note polyphony by staggering with metro
function iiseq()
    if #q > 0 then
        ii.jf.play_note(table.remove(q),2)
    end
end

function update(r)
    for n=1,4 do
        output[n].volts = r[n]
    end
    for n=1,6 do
        table.insert(q, r[n]) -- queue a note for i2c transmission
    end
end

input[2].change = function()
    capture = input[1].volts
    table.remove(reg)
    table.insert(reg, 1, input[1].volts)
    update(reg)
end

which works. Not sure what's up with the new CLI (or maybe I am running things incorrectly?)

Can't select text

Not sure why but I'm unable to select any text in the druid REPL.

I'm not sure it matters, but I'm on Linux, specifically using GNOME terminal.

run druid without active connection

it seems strange that druid can handle reconnects gracefully, but can't start up without an active connection.

seems it'd be better to start despite no connection--- just indicate the no connection state.

related: #14

Allow running scripts from the command line

Currently druid can be used to upload a script onto norns from a Lua file from the command line, matching the REPL command ^^upload. It will get stored in memory, and will run when crow is restarted either with ^^reset or power flip.

It would be useful, perhaps, to be able to run a script from a Lua file, like the REPL command ^^run does. This does not store the script on crow, and I have also understood it is a bit faster to run than upload a program.

caw!

Only allow a single instance of druid to attach to a given crow

When multiple druids are open connected to the same port there is a race condition which makes it ambiguous where the returned values will appear. This is very confusing for new users when they don't realize they have druid open in a different window.

I've tried to simply set the exclusive = True param in the serial.Serial call in crowlib, but this is even more confusing. The <disconnected> message never appears in druid, and I'm not sure if it's possible to check why the serial connection failed. If we can test if the port exists but is locked by a different process, then it would be great to print a message "another instance of druid is already open".

roll bootloader into druid

is it possible to add functionality to druid such that it can:

1. install dfu-util
2. fetch the current latest-release from github.com/monome/crow
3. execute the flash.sh script
4. reconnect to crow in druid & query ^^i to show the version is updated

in doing this, rather than going via <lost connection>, it should print <entering bootloader> then <crow updated to version ____>

Remove use of implicit namespace packages

Mainly because it means that for a lot of people a newer setuptools needs to be installed making the installation more complex than it needs to be.
Also we don't really have a reason to use implicit namespace packages.