sdether/josh.js

Toolkit for building a bash-like shell in the browser, including full readline support

josh.js 0.2

http://sdether.github.io/josh.js/

Javascript Online SHell provides a toolkit for building bash-like command line consoles for web pages. JOSH enables the visitor who prefers the bash-like command to maneuver through web content using a console rather than clicking with a mouse. This toolkit is most useful for people who like to use command line because it is faster and more effective than using a mouse. It is easier and convienent to access hierarchical information using JOSH command history, change or display current directory. It consists of the following components:

• readline.js - full readline support for ctrl sequences, tab, history, etc.
• shell.js - visual presentation of the shell and command handling
• pathhandler.js - provide cd, ls, pwd and path completion toolikit
• history.js - localStorage backed command history
• killring.js - killring for kill & yank handling in readline

What to use Josh for and when to use it

Josh allows developers to build their own command line interface to any sites. It supports full CLI Readline in the browser like TAB completion, emacs-style line editing, killring and history with reverse search. When you are tired of clicking your way through a hierachy tree, Josh will come in handy. It will helps you browse or navigate text files quickly and minimal the using of mouse click.

Tutorials

• Hello world - put a console on a web page and add a new custom command with completion
• Quake Console - Create a quake-style console with ls, cd, pwd and bash filename tab-completion
• GitHub Console - Extend the Quake Console to talk to GitHub's REST API to navigate repositories, their branches and file system

Articles

• CLI all the things: Introducing Josh.js Article about the origins of Josh.js with an example console for wordpress sites.

License

josh.js is licensed under the Apache 2.0 License

Status

• code is ready for experimental use
  • Tested under Chrome, Firefox, Safari and IE9
  • API may not yet be stable
• needs minified versions of complete toolkit and just readline.js
• needs code documentation and documentation site
• would like to add AMD support
• base shell UI should get some basic behaviors
  • more-like handling for output that exceeds the shell viewport size
  • resizing and close chrome
• Readline has not been tested with non-ascii.

Usage

Until documentation is written, refer to index.html and example.js (Annotated Source) for a sample implementation of a shell with path completion.

Components

josh is built from 5 components and can be used in part or in full.

readline.js

readline.js has no dependencies on any outside libraries, although it requires either history.js and killring.js or objects implementing the same calls.

It implements key trapping to bring GNU Readline like line editing to the browser. It can be used by itself to bring readline support to custom data entry fields or in conjunction with shell.js to create a full console.

Line Editing

In the below C-x refers to the Ctrl-x keystroke, while M-x refers to the Meta-x keystroke which is mapped to Alt, ⌘ and Left Windows.

Movement

C-b or Left Arrow

Move back one character

M-b or Right Arrow

Move back one word

C-f

Move forward one character

M-f

Move forward one word

C-a or Home

Move to the beginning of the line

C-e or End

Move to the end of the line

Edit/Kill

Backspace

Delete one character back

C-d or Delete

Delete character under cursor

C-k

Kill (i.e. put in kill ring) text to the end of the line

M-Backspace

Kill one word back

M-d

Kill word under cursor

C-y

Yank (i.e. pull from kill ring) the most recently killed text

M-y

Rotate to the next item in killring and yank it. Must be preceded by yank

History

C-r

Reverse search through history

C-p or Up Arrow

Previous entry in history

C-n or Down Arrow

Next entry in history

Page Up

Top of history

Page Down

Bottom of history

Misc

C-l

refresh line (clear screen in shell)

Tab

Invoke completion handler for text under cursor

Esc in reverse search

Cancel search

C-c

call onCancel handler

C-d on empty line

call onCancel handler

shell.js

shell.js has external dependencies of jQuery, Underscore and internal dependencies of readline.js and history.js.

It provides a simple console UI, using a panel for the console viewport and an auto-scrolling view inside the panel. It uses Underscore templates for generating the view html, although any template generator can be substituted as long as it can be expressed in the form of a function that takes a JSON object of arguments and returns an html string.

It also implements command handling so that new commands can be added by name with execution and completion handlers. Out of the box, shell.js provides the following commands:

• help - list all known commands (including user added)
• clear - clear the "screen" i.e. viewport
• history - show the command history captured by readline.js in history.js

pathhandler.js

pathhandler.js is a mix in to easily add the cd, ls and pwd commands as well as path completion. It has the same external dependencies of jQuery, Underscore as shell.js and also uses Underscore templating.

By implementing the functions getNode and getChildNodes, this library adds path traversal, discovery and completion just like a bash shell.

history.js

history.js implements a localStorage back command history storage that persists over page changes and reloads. It is used by the shell.js history command to list all executed commands, and by readline.js for up/down arrow and reverse search capabilities.

killring.js

killring.js implements the kill and yank behavior as well as state tracking, i.e. multiple consecutive kills are combined as a single kill and killring rotation tracks the previous yank, so that the readline.js can remove the previous yank and replace it with the rotated text.

Changelog

0.2.10 -- 2014/04/03

• Added bower support (pr#19 - @bricef)
• Code clean-up for closure compiler issues (pr#20 - @aaronmars)

0.2.9 -- 2013/08/31

• Added ability to bind ReadLine/Shell to an element.
• Added ability to bind/unbind keys (could be used to replace emacs bindings of readline, but primarily added to unbind some keys for using readline on input elements.
• Created input.js for easy binding of readline to either an input element or a span behaving like an input.

0.2.8 -- 2013/03/13

• Added handling of . and .. in Josh.PathHandler.pathcompletionhandler, so that a trailing .. completes to ../ and . to ./
• Removed the hardcoded strong in the input template, making it a span instead so it can be styled via css instead.
• The prompt value itself is now assumed to be html instead of plain text, allowing for richer formatting without changing the input template.

0.2.7 -- 2013/02/13

• Removed all html used for Shell UI generation from config to Shell.templates, so that they can easily be customized (see: Issue 11)
• Removed PathHandler.templates. PathHandler now attches its templates to Shell.templates as well

0.2.6 -- 2013/01/21

• Removed Activation/Deactivation keybindings from Readline, making it an outside concern (see: Issue 2)
• Fixed Backspace regression introduced by 0.2.5
• Fixed M-d not deleting last character of line
• Example shell can now be resized (via jquery-ui.resizable)

0.2.5 -- 2013/01/14

• Implemented missing Readline behavior (see: Issue 1)
• Added scrollbar to sample implemenation (also adds scrollwheel support)

0.2.4 -- 2013/01/14

• fixed path completion handling for scenarios of two possible completions where one is the root of the other
• noted that spaces in paths are completely unsupported right now.. they will complete, but the exec handler will see the space as a separator between arguments

0.2.3 -- 2013/01/13

• changed internal handling of the default command handler (i.e. when no named command is defined).
• removed the pathhandler commandhandlers from the public object, since they should be accessed via shell.getCommandHandler if required
• some readline.js property naming cleanup to make closure compiler happy

0.2.2 -- 2013/01/09

• changed rendering of completion to be more bash-like, i.e. now renders completion and then re-renders prompt with completed text, rather than as a pop-under that disappears.

0.2.1 -- 2013/01/08

• fixed activation/deactivation propagation through the shell, which caused issues with first time activation via activation key instead of method call to fail.

0.2 -- 2013/01/07

• console wrapper to allow debug logging to be turned on and off
• refactored how pathhandler attaches to shell because it needs to keep a reference to the shell
• refactored how prompts are set. now uses dedicated callback rather than returning the prompt in the onEnter callback
• tested and made fixes to ensure compatibility with major modern browsers

0.1 -- 2013/01/04

• Initial code-complete release

josh.js's Issues

Josh JS localStorage Issue when using Safari in Private Browsing Mode

See issue here: http://stackoverflow.com/questions/14555347/html5-localstorage-error-with-safari-quota-exceeded-err-dom-exception-22-an

[Error] QuotaExceededError: DOM Exception 22: An attempt was made to add something to storage that exceeded the quota.
save (history.js, line 41)
History (history.js, line 36)

Any call to local storage will need to get wrapped in a try/catch to catch this error.

Arguments for commands need to understand quoting and space escaping

Current arguments to commands are just determined on whitespace separation. Should respect quoting and {space} to treat an arg with a space as a single argument. Should really expose argument parsing logic as a handler, so that custom logic can be added for deciding what is a single argument.

add to npm

any interest in adding this to NPM so it can get bundled for client side code via browserify or similar libs.

Readline keyhandling needs ability to bind to an element, rather than the root

Close Quake-style console with Esc

You should be able to close it with the Escape key in addition to Ctrl+C (or Ctrl+D on an empty line)

Remove Activation and deactivation keyhandling from readline

This should be controlled from the outside, otherwise handling loss of focus vs. hiding becomes ugly. And instead of Ctrl-C, Ctrl-D and Esc deactivating readline, it should instead be possible to subscribe to key-events not handled by readline, so that they can be externally wired up as deactivation triggers and still be marked as handled without users of readline having to resort to external keyhandling. That also means that commands could be wired up to readline ctrl sequences.

Non-fatal error with closure compiler

Closure gives the following error in readline.js. I'm not sure what the expected behavior is, but the line in question looks like it does nothing.

js/readline.js:165: WARNING - Suspicious code. The result of the 'getelem' operator is not being used.
        _keyMap[k.modifier][k.code];
        ^

copy/paste

scenario: in another browsertab I have an url..I select the url, rightclick copy. I try to paste it into josh.js but it doesnt work on chrome 41.0

any ideas?

I tried:

  $("span.cursor").bind('paste', function(f) {
     console.dir(f);
  }); 

Which got triggered, but I was unsuccesful in getting the data.
Googled a bit, and decided to ask here before getting messy with hidden textareas :)

Mimic standard piping behavior found in Bash

Based onthe work of #14 and #16, implement standard piping and input redirection from Bash in Josh.Shell

cannot create multiple shells in the same page? [solved]

I seem to not be able to render a shell and parse input if another shell variable is created. see http://jsfiddle.net/LHJLd/

basically I do this...

var shell = Josh.Shell({ history: history, shell_panel_id: "shell-panel-0", shell_view_id: "shell-view-0" });
shell.activate();
var shell1 = Josh.Shell({ history: history, shell_panel_id: "shell-panel-1", shell_view_id: "shell-view-1"});

and then in the html body:

shell 0

<div id="shell-panel-1">
  <div>shell 1</div>
  <div id="shell-view-1"></div>
</div>

Inconsistent UI

The user interface and the user experience of your website differs for the zip download or the github download. When a user downloads the zip or from GitHib, they ought to experience a fullscreen terminal. I'm not quite sure why there are two text fields and a terminal overlay.

Replace callback with filehandles object.

I've been giving some thought to how one could support piping, etc. It seem that you can lay the groundwork by replacing the current "callback" mechanism with something more similar to file handles.

So, for instance, the current "callback" function parameter to the command would be instead a "filehandles" object, with two properties, "stdin" and "stdout". The "stdin" object would have two function properties, "onRead" and "onEnd", which would be set by the command to capture input, and the "stdout" property would have two functions "write" and "end". This would allow the chaining of commands. You could probably add a "stderr" property similar to the "stdout" property for more UNIX-like behavior. At that point, the piping problem simply becomes one of parsing the command line and wiring up the handlers - you'd probablywant to start with the rightmost one in order to make sure the input handlers are set up before data is sent to them, or alternatively have an explicit initialization call to each command.

Example does not work properly

Hey

So, I've downloaded the repo, but the example isn't working properly. I can't type in any commands, just can open the shell with ~, but then nothing. No errors.

Any suggestions?

Need documentation hooks in command handlers for help system

The command handler should have an optional description key to be shown in the command list and a optional man key specifying a callback when someone calls help <command> [<args>]

cmds typing before the cmd $

Hi Josh,
Having a play with your cmd in the browser, and loving it. Just one issue, when I type in the cmd, what I type appears before the $ sign. See image. I am pretty sure I have everything included correctly, it might be a CSS issue. Could you suggest anything?
Thanks
Alex

I have included:

<script src="/public/CMD-js/history.js"></script>
<script src="/public/CMD-js/shell.js"></script>
<script src="/public/CMD-js/readline.js"></script>
<script src="/public/CMD-js/killring.js"></script>
<script src="/public/CMD-js/input.js"></script>

and see here where it is typing

Chrome rings bell for all Alt- keycombos it does handle even after javascript traps them

This issue is filed mostly for reference. It doesn't break Readline.js but means there is an annoying ding for every Alt- combo even though Readline.js traps and handles it.

Quake and GitHub demo doesn't work on scandinavian keyboard

On scandinavian keyboards we have to press a key combination to trigger a ~. Because of this we cannot trigger the console in the Quake and GitHub demos.
A solution could be to include a button or another key for triggering the console.

Is possible to execute js code in the shell?

Hey,
I'd like to use this shell to create a tutorial on basic js command, is somehow possible to execute js code in the console?

Thanks

Shell.bestMatch should have case-insensitive option

For completions that are case-insensitve, the bestMatch method should have be able to find matches case insensitively, and PathHandler should expose a config flag to enable this for path completions

Redirect to another page with command | Overwrite Existing Command with defined action

• Please add a feature of redirect to another page, by using some specific command / Custom commands.
• Can we Overwrite some existing commands, to perform the defined action? [Like Custom Commands]

mobile compatibility

Currently on mobile (Android Marshmellow using Chrome or Firefox) the javascript seems to run fine (there is a blinking cursor), but the field is not accessible to input and the keyboard won't appear for input.

Templates used by shell should be exposed as a property and templates on pathhandler should attach there instead of itself

PathHandler has a .templates property, while Shell takes some in as html in the config and some are hardcoded. Instead there should be Shell.templates for all html generated by Shell and PathHandler should attach its own templates there instead of keeping them on its own instance.

Towards curses

Currently if the output returned by a command is unbounded, freeform html. This makes paging support hard. Instead the shell should be created as a terminal windows with with a resolution of rows,cols. Command output would support a single string (which would be wrapped), an array of strings (which would not be wrapped) or freeform HTML (which would be put into a div that can be capped to the current terminal size).

Given output of known lines, implementing --MORE-- behavior could be supported and the grid nature of the screen could become the basis for implementing curses

Mixed content on Github.io

Sources from Cloudfare, Google, and others are pulled in as http and not https in the github.io site. This makes the console unusable when accessing the site as https. These resources should be loaded as https for security and usability purposes.

Allow for all entered text to be handled by client

I may be missing something, but it doesn't appear that the API for the shell allows you to simply apply a command handler to any text entered; rather, you must specify command handlers for each possible command the user can run. In my case, I'm trying to use the shell as a front-end to commands that are run via remote SSH on a real Linux system. As a result, I want the remote system to handle the command and simply use the shell to display the result. What would it take to do that? Is that feasible now without a code change? I haven't discovered a way yet.

Allow Ctrl-C trap to be disabled to allow Copy

Currently Ctrl-C is trapped (as it should for shells). But it would be nice it that trap could be optionally defeated so Copy can continue to work.

Bonus: When that trap is disabled, Josh should instead trap (but still bubble) Ctrl-X and Ctrl-V so that text on the command line can participate in regular Cut/Paste operations

Add a command/completion pre-processor

Currently any input line is considered to be a single command, which makes it impossible to implement piping. Once #14 is implemented and we have input and output streams, we need a command & completion pre-processor hook that could go in and recognize command separators

Missing Readline commands

Missing capabilities:

• C-l needs hook so shell can execute the clear command
• Word commands should use any non-alphanumeric as a word boundary
• Apple Command Key should work as Meta
• Kill buffer is a single field, should be a kill ring

Missing commands:

• M-Backspace kill word back
• M-D kill word under cursor
• M-y cycle through kill ring

Not rendering

I have followed your tutorials and have also downloaded the example you have given, however the console is not rendering

Remove jQuery Dependency

Is it a possibility to remove jQuery in shell.js and pathhandler.js? It seems a bit overkill to me to add such a heavy dependency.

nodejs support

I went looking for GNU readline support in nodejs and found this project. It does most things I want, but is coupled to the DOM. Would you consider factoring out DOM-specific stuff and leaving a core that implements in a UI-agnostic way?

Support !{num} to execute history entries

In bash you can do !{num} with the line number of a command in the history to re-run it.