glavin001/mrshell

Map-Reduce inspired Shell.

MrShell


Map-Reduce inspired Shell.

---

Features

• Extreme Piping syntax
• Tab Completion
• Intelligent Colourization
• Command History

Authors

• Glavin001
• Frozenfire92
• DylanYoung
• Rayls

Building

Run the following:

make

Usage

To run MrShell, execute:

./build/mrsh

mrshell's Issues

Project Report 2

Discuss Feature List

For the end of the week we need to solidify our high level overview of the project including our feature list

• Extreme Piping, see #4
• Map Reduce (under investigation)
• Tab completion, see #9
• Intelligent colorization, see #8
• History, see #10

Improve build process

See https://github.com/jengelsma/lex-tutorial

Command History

Pick a better name

MRShell (Map-Reduce Shell) is not catchy. I suck at naming. Let's discuss names :).

Create ASCII art for shell startup screen

Standard console size (80 characters, 24 lines)

Basic Built-In Shell Functions/Features

• Show the current working directory in shell prompt
  • http://stackoverflow.com/a/298518/2578205
  • Environment variables in general: http://www.gnu.org/software/libc/manual/html_node/Environment-Access.html#Environment-Access
• Show username in shell prompt
  • http://www.unix.com/81721-post4.html (from http://www.unix.com/programming/21041-getting-username-c-program-unix.html)
• cd
  • http://stackoverflow.com/a/9859965/2578205
  • http://linux.die.net/man/2/chdir
• Basic redirection with |
• File redirection support, >>, >
• Backslash escaping characters
  • For instance, a\ b should be a single string a b and not two a and b
• ~ support for $HOME in path args

Extreme Piping™ syntax

Support running tasks on multiple machines on cluster over network

What if the shell also takes (optional) remote servers and processes
remotely whenever possible? Allowing for massive parallelism. Just a
thought...

from @DylanYoung

table command

table will be very commonly used for the joining of multiple streams of extreme piping output data.

So that users of MrShell do not have to implement their own table command for joining and displaying extreme piping output, we can bundle MrShell with a pre-written table command.

Intelligent Colourization

See color text in terminal aplications in unix.

---

@DylanYoung, in #7 :

Basic colouring would be nice. Like colour input, commands (binaries),
flags, and shell characters (like |) differently. Could also indicate an
invalid command without needing to return first.

Shell Core

• Take a line of user input
• Parse line of input into args
• Execute as a single command (ls, whoami, etc)
  • Execute with command args (ls -la is ls with arg -la)

First Report

See https://github.com/smuos/lectures/blob/master/00-Info/project.md#first-report

Due Friday November 7th at 11:59pm
Summary of the features planned and strategies.
Detail work accomplished to date.
Around two pages double spaced.

Command/Tab Completion

• autocomplete with spaces do not work right now
  • Must escape spaces with backslash, \

Handle Quoted Arguments Correctly

Try echo "blah blah" or git commit -m "blah blah" for example.

Map-Reduce syntax for language

I recommend skimming MapReduce on Wikipedia and Map-Reduce examples for MongoDB to get a feel of what it is usually used for and how.

Goals with this shell:

• Multi-process (fork(), exec(), etc) Map-Reduce style queries
  • Implicitly Multi-core scripting, awesome!
• Map function could be n # of pipes to other commands
  • Map(k1,v1) → list(k2,v2)
• Reduce function should take a list of inputs that are piped to it and aggregate results
  • Reduce(k2, list (v2)) → list(v3)
• Syntax for auto-piping to file to be used by Map sub-function
  • Some programs only access files and do not work with stdin, so there should be a special accessor syntax for stdin and for as file, without requiring to use another pipe to another shell script
  • File should be temporary and stored in the Operating System's respective temporary file directory
• Pipes within Map and Reduce functions
  • Map function could have 3 parallel calls, however, those 3 calls could be a grouped representation of sequential (series) or calls to other functions.
  • For instance, one of the Map sub-calls could be to get the last name given the input, and then pipe that to another step where it will ALL-CAPS that outputted last name.

Brew package

See http://userprimary.net/posts/2009/09/28/writing-packages-for-homebrew-on-os-x/

Extreme piping example

For the presentation an awesome example is key. I was thinking an intelligent one liner that moved, manipulated, and processed data into a cool d3 visualization website (python script) would net maximum wow

Potential commands to use? I was thinking

ls
cat
grep
history (or Mr Shell's equiv)
wc
sort
ps
datedash
tar
top / htop
uname