swcarpentry/shell-novice

The Unix Shell

shell-novice's Issues

Spaces in filenames

Per PR #23 , maybe add a note in 01 about spaces in filenames - how to deal with them if they occur, alternatives to spaces, etc.

Remove auto-generated files from version control and find sources for downloaded data.

These should be added by build tooling. Looking through the history:

$ git glog --find-copies --stat=160
…
* c08de6f Example files for intro to shell
|  01-filedir.md                                          |   15 +-
|  01-filesystem/data/access.log                          |    1 +
|  01-filesystem/data/hardware.cfg                        |    2 +
|  01-filesystem/data/network.cfg                         |    1 +
|  01-filesystem/users/vlad/data/amino-acids.txt          |   20 +
|  01-filesystem/users/vlad/data/elements/Ac.xml          |    8 +
|  01-filesystem/users/vlad/data/elements/Ag.xml          |    9 +
…
|  01-filesystem/users/vlad/data/morse.txt                |   50 ++
|  01-filesystem/users/vlad/data/pdb/aldrin.pdb           |   30 +
…
|  01-filesystem/users/vlad/data/planets.txt              |  230 ++++++++
|  01-filesystem/users/vlad/data/sunspot.txt              | 3080 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
…
| img/filesystem-challenge.png                           |  Bin 0 -> 103988 bytes
| img/filesystem.png                                     |  Bin 0 -> 13194 bytes

sunspot.txt looked like it should be easier to track down, so I looked through all my SWC-related commits, and found it in swcarpentry/DEPRECATED-bc@800fa5f54 (Added a bunch of small data files for use in exercises, 2013-08-02) as data/astro/sunspot.txt. The header for that file mentions http://sidc.oma.be/html/sunspot.html, and I poked around there but couldn't find anything that looked similar. I haven't tried to track the other data exoplanets.csv, *.pdb, …. back to its sources yet. If you have any pointers, @gvwilson, I'd appreciate them ;).

Improve paragraph about control keys

In 02-create.md we

• should mention that ^ means the control key before using "Control-O".
• should mention that in some keyboards the control key is the "Ctrl".
• should use only "Control", not "control".
• on Mac OS X, control really means control and not command.

Improve "Relative path resolution"

At 01-filedir.md the exercise "Relative path resolution" aren't in conformance our example filesystem. E.g. there is no /Users/thing. I think that the exercises must be in conformance with our example filesystem because will be easy for students to check their answers.

Issues at 04-loop.md

From check.py:

• Heading at line 270 should be level 2
• Document contains heading not specified in the template: Nelle's Pipeline: Processing Files
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Missing heading in instructors.md

From check.py:

• In /tmp/shell/instructors.md: Document is missing expected heading: Legend
• In /tmp/shell/instructors.md: Document is missing expected heading: Overall

Representations avaliable for questions.

Hi,

In my first lesson I put my questions in a R-presentation using Rstudio, and thought it was a useful way to share the questions I used in my segment.

The slides are here http://rpubs.com/smilefreak/59680, would this or even gists be useful for individual instructors to share their questions. The reason I suggest this is that scrolling to the challenges section on the webpage seems awkward, and sharing and presentation tools could really be leverage here I feel.

Thanks for your time.

Recommendation to Instructors about man because it doesn't work in Windows

Windows doesn't have man installed. To improve the experience of our students that use Windows machines we could make a few changes in the write version of our shell lessons.

• Add call out to mention man and warning Windows users, see #55 (comment).
• Add note at instructors.md, see #55 (comment).

small change between files/directories and creating things

Per PR #82, any strong opinions about moving the note about ~ from 02-creating things to 01-files and dirs? In my mind, that's the logical place for it since that's where we discuss paths. On the other, 01 already has a lot going on and the ~ note might just be distracting. Thoughts?

gh-pages vs master

This repository has two branches: gh-pages and master (96 commits behind gh-pages). This can be confusing to someone new to Software Carpentry (e.g. #28).

@gdevenyi, @ChristinaLK and @wking, Could we remove the master branch?

Issues at 06-find.md

From check.py:

• Heading at line 383 should be level 2
• Document contains heading not specified in the template: Conclusion
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Improve site navigation

It would be nice to have next, previous, and home links at every page to improve the navigation.

Attach shell/novice history from before swcarpentry/bc@b8c7dc3

Using filter-branch:

$ git checkout -b shell-novice b8c7dc3^
$ git filter-branch --subdirectory-filter shell/novice --prune-empty

which gives me the 12 commits touching that stuff. Now go back to my novice-shell branch (which is my local name for this repository's master branch), and graft on everything except the orphan version of swcarpentry/DEPRECATED-bc@b8c7dc3 (Reorganizing material to put novice + intermediate at top level, 2014-02-16):

$ git checkout novice-shell
$ ORPHAN_COPY_COMMIT=$(git rev-parse 'HEAD^{/Reorganizing material}')
$ git rebase -p --onto shell-novice "${ORPHAN_COPY_COMMIT}"

Lets double-check that graft-point:

$ git log --graph --topo-order --oneline --decorate | grep -3 shell-novice
* | 9510798 *   Getting rid of old cheat sheets *   Indenting error output cells …
|/  
* ba251ee Fixing formatting of pipe-and-filter challenges
* 1f11afd (shell-novice) 1.  Making filenames in `find` example match filenames in diagram. 2.  Using `-print` with `find`.
* d405257 Slight edit to @BernhardKonrad's chnages
* ed1a14a Specify key points
*   72ad088 Merge branch 'v5alpha' of https://github.com/gvwilson/bc into v5alpha

That looks reasonable (although I think we need to trim the commit message summary for 9510798 (swcarpentry/DEPRECATED-bc@5705090, Getting rid of old cheat sheets …, 2014-02-27). So remove my temporary local branch:

$ git branch -d shell-novice

and push to this repository's master (forcing, because I rebased the old master when making the graft):

$ git push -f modular-shell novice-shell:master

Now our oldest commit is the filter-branched version of swcarpentry/DEPRECATED-bc@3a38907 (shell: Rename the new-style lessons from 'bash' to 'shell', 2014-01-09, swcarpentry/DEPRECATED-bc#222), so we'll need another filter-branch graft to pick up the bash/novice stuff.

Restore gen-*.py script history

These were dropped by one of the earlier filter-branch clones (see #4). Just to be clear about which repositories I'm referring to, here are my remote names for this section:

$ git remote -v
bc      ssh://[email protected]/swcarpentry/bc.git (fetch)
bc      ssh://[email protected]/swcarpentry/bc.git (push)
modular-shell   git://github.com:wking/swc-modular-shell.git (fetch)
modular-shell   ssh://[email protected]/wking/swc-modular-shell.git (push)

What's the history for gen-sequence.py in bc?

$ git log --follow --oneline bc/master -- novice/shell/gen-sequence.py
b8c7dc3 Reorganizing material to put novice + intermediate at top level
3a38907 shell: Rename the new-style lessons from 'bash' to 'shell'
f4cc356 Updating index file and moving utility scripts to the right place
a41a27d Renaming directories with example files
03b26ac Example files for intro to shell

Which of those commits are in modular-shell?

$ git log --oneline modular-shell/master | grep 'Reorganizing material\|shell: Rename the new-style\|Updating index file and moving\|Renaming directories with\|Example files for'
c6ac923 Updating index file and moving utility scripts to the right place
09fb561 Renaming directories with example files
c08de6f Example files for intro to shell

Taking a closer look at our breaking commit:

$ git show --oneline --stat c6ac923
c6ac923 Updating index file and moving utility scripts to the right place
 creatures/gen-sequence.py | 10 ----------
 index.md                  | 24 ++++++++++++++++--------
 2 files changed, 16 insertions(+), 18 deletions(-)

And the bc commit it's based on:

$ git show --oneline --stat --find-copies f4cc356
f4cc356 Updating index file and moving utility scripts to the right place
 bash/novice/index.md                            | 24 ++++++++++++++++--------
 bash/util/gen-nene.py                           |  2 ++
 bash/{novice/creatures => util}/gen-sequence.py |  2 ++
 3 files changed, 20 insertions(+), 8 deletions(-)

Hmm, so this commit is bumping gen-nene.py, and we'll have to look at that too. What's its history in bc?

$ git log --follow --oneline bc/master -- novice/shell/gen-nene.py
b8c7dc3 Reorganizing material to put novice + intermediate at top level
3a38907 shell: Rename the new-style lessons from 'bash' to 'shell'
f4cc356 Updating index file and moving utility scripts to the right place
090b60b Put randomly generated data in files

Do we have that first commit in modular-shell?

$ git log --oneline modular-shell/master | grep 'Put randomly generated'

What's it touching there?

$ git show --stat 66cf53e
commit 66cf53e794d8cbbb6be291131045b105d751d39e
Author: Damien Irving <[email protected]>
Date:   Fri Nov 29 07:44:39 2013 +1100

    Put randomly generated data in files

 .../north-pacific-gyre/2012-07-03/NENE01729A.txt   | 600 ++++++++++-----------
 .../north-pacific-gyre/2012-07-03/NENE01729B.txt   | 600 ++++++++++-----------
…
 .../north-pacific-gyre/2012-07-03/NENE02043B.txt   | 600 ++++++++++-----------
 05-scripting/backup/chloratin.dat                  | 178 +++---
 05-scripting/backup/sphag-merged.dat               | 344 ++++++------
 05-scripting/chloratin.dat                         |  58 +-
 05-scripting/girmanis.dat                          | 364 ++++++-------
 05-scripting/sphag2.dat                            | 312 +++++------
 05-scripting/sphagnoi.dat                          | 182 +++----
 23 files changed, 5759 insertions(+), 5759 deletions(-)

So our filter-branched commit doesn't add gen-nene.py, because swcarpentry/DEPRECATED-bc@090b60b had added it to a different directory (bash/util/gen-nene.py). I don't think we want auto-generated content in the repository anyway, so I might try and refactor this bit of history entirely…

Need to update LICENSE.md

From check.py:

• The provided license file should not be modified.

.tmp and .my.log hidden files extraneous?

There's an emtpy file called .tmp in data/Users/larry, and there are .tmp files in other directories as well. Likewise, there are empty files .my.log in some directories. find data -name '.*' -print prints a full list of dotfiles.

My impression is that only data/Users/nelle/.bash_profile could serve any didactic purpose (other than perhaps some "easter egg hunt" type challenge). Is this correct, and if so, should we get rid of these .tmp and .my.log files?

Remove mentions of man pages

man pages aren't installed on all systems, we can't teach them in the main material

Replace with google instructions?

Issues at 03-pipefilter.md

From check.py:

• Heading at line 288 should be level 2
• Document contains heading not specified in the template: Nelle's Pipeline: Checking Files
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Add a dotfile to the image related to ls -F -a command

dotfiles are a concept that trip up new users of the command line and shells. In file 01-filedir.md, the description of the .. directory is great. Although . is mentioned related to files, it would be helpful to learners to see a dotfile in the output image or to have a sentence explaining dotfiles for configuration information.

Drop or clarify the purpose for the '$how ...' challenge from 04-loop.md

As I mentioned in #33 and #95, I don't understand what this challenge is trying to show, or why it belongs in the loop lesson. Quoting from my #33 commit:

Naming the last challenge is difficult, because it's not really about
loops. That challenge dates back to 8e77786 (Importing shell lesson
for novices, 2013-11-11), which doesn't explain the pedagogical
motivation for the challenge. Maybe it's just a trick to keep
students from being to focused on the purported topic when crafting
their answers? Maybe it's intentionally a throwback to more basic
shell syntax couched in the decoration of the current topic? Maybe
it's a hint that you can do more with loops than vary arguments to a
single command? In any event, I'd prefer removing this challenge
entirely, or reworking the content as an example (instead of a
challenge).

If anyone does have a clear handle on why this challenge is useful, please share (and we can mention the motivation in instructors.md. If we don't hear anyone motivating it in the next month, I'd recommend dropping the challenge.

Dead link to discussion.html

The discssion.md source was removed with 3eaab25 (Removing example files (they belong in the gh-pages branch), 2014-12-11) but the link was added to index.md with 59fc96f (Reformatting home page, 2014-12-17). Maybe we should have some stub content in discussion.md? Or leave the link off until we have a browsercast slide deck to put there?

Question regarding the us of 3 minus sign in the text

There are some --- used in the text at some places and, I believe, that should represent a —. Is that correct? If so, should we use some "Markdown flavor/plugin" to take care of that? Or should we fix it using the HTML syntax (—)?

List maintainers for this lesson somewhere in the repository?

My impression with the bc split was that Gabriel Devenyi (@gdevenyi) and Christina Koch (@ChristinaLK) would continue to act as maintainers for this lesson. However, recent activity in the main branch has been by others:

$ git log --date=short --pretty='%aN (%ad) - %s' --first-parent origin/gh-pages
Bernhard Konrad (2015-02-01) - Merge pull request #30 from wking/filedir-challenge-titles
Emily Jane McTavish (2015-02-01) - Merge pull request #35 from wking/find-challenge-titles
adina (2015-01-31) - Merge pull request #32 from wking/pipefilter-challenge-titles
Greg Wilson (2015-01-26) - Merge pull request #58 from standage/html
Greg Wilson (2015-01-26) - Merge pull request #57 from standage/typo
Raniere Silva (2015-01-21) - Merge pull request #45 from gvwilson/gh-pages
Greg Wilson (2015-01-21) - Incorporating change made by @ggorman in #22 manually.
Greg Wilson (2015-01-21) - Merging #8 and fixing challenge titles
Greg Wilson (2015-01-16) - Fixing up title
Greg Wilson (2015-01-16) - Merge pull request #40 from standage/loop
Greg Wilson (2015-01-16) - Little Women
Greg Wilson (2015-01-16) - Merge pull request #38 from halexand/gh-pages
Greg Wilson (2015-01-16) - Merge branch 'fixme' of https://github.com/PBarmby/shell-novice into gh-pages
Greg Wilson (2015-01-16) - Adding a short note about touching files.
Greg Wilson (2015-01-16) - Making sub-title display properly
Greg Wilson (2015-01-16) - Adding mention to discussion of other shells.
Greg Wilson (2015-01-14) - New Makefile and footer
Greg Wilson (2015-01-14) - Merge branch 'core' of github.com:swcarpentry/lesson-template into gh-pages
Greg Wilson (2015-01-05) - Merge branch 'core' of github.com:swcarpentry/lesson-template into gh-pages
Greg Wilson (2015-01-05) - Merge branch 'core' of github.com:swcarpentry/lesson-template into gh-pages
Greg Wilson (2014-12-20) - Bringing titles into line with latest version of template
…

@gvwilson has a lot on his plate, so while I'm happy to have him contributing so much to the lesson, I think we should have at least one person who takes the lesson under their wing and focuses on reviewing pull requests, handling issues, preserving the lesson's focus, and all the other little things that go into maintaining a project. If we actually have an active maintainer (or maintainers) like that, it would be nice to list them in a local CONTRIBUTING.md so the rest of us know who to poke when things go quiet ;). If nobody is interested in assuming that responsibility, the shell lesson is quiet enough that I'm happy to fill in the role (although it would be better if we could find an interested maintainer with more free time than I have).

For my earlier thoughts on avoiding absent maintainers, see my comments in swcarpentry/DEPRECATED-site#603.

Providing a VM for the Shell Novice lessons using Vagrant

Good morning,

In the past academic year I used V5.2 of the Software Carpentry lessons with my students and found the experience to be an extremely positive one. The only snag I encountered were inconsistencies between the filesystem on the VM (username swc rather than nelle, and quite a few others besides).

I was interested to see in the V5.3 lessons that there is no longer mention of a virtual machine image, as I found the VM to be a very useful way to ensure a consistent working environment across a wide range of devices (in my teaching context there is significant student preference over machines, so BYOD is common).

My experience with the V5.2 lessons lead me to wonder whether a Vagrant-based [1] approach to providing a definition of the virtual machine would be of interest to the wider community?

This approach would give us the ability to store in Github a very lightweight description of the virtual machine image, its dependencies, and the file structure, which could more easily be kept consistent and up-to-date with changes to the course materials (with previous snapshots of the Vagrant file meaning we could go back to any previous state of the lesson and reproduce a compatible Virtual Machine).

There would be the possibility for students to download an install Vagrant on their own machine, and to obtain the image themselves (a very simple process taking no more than 5 minutes), or the instructor could do this on their behalf and provide the required VirtualBox files to each student (akin to how things were done in V5.2). The latter would be particularly beneficial in situations where internet connections are slow, alleviating the need for a ~1.7Gig download from a Software Carpentry mirror for each of n students in the class.

I would be very interested to hear everyone's thoughts on this, and this would be an area of development I would be more than happy to take a lead on.

All the best,
Matt

[1] https://www.vagrantup.com/

Issues at 05-script.md

From check.py:

• Heading at line 301 should be level 2
• Document contains heading not specified in the template: Nelle's Pipeline: Creating a Script
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Copy over lesson time estimate from swcarpentry/bc

This landed in swcarpentry/DEPRECATED-bc#789 after the feature freeze. We should cherry-pick it over after we finish reconstructing the history, and then take care if/when we integrate this lesson back into bc.

Unspecific link in 06-find.md

There is a side comment in a callout 'Wildcards' promising a lesson (or at least an example for complex searches: "if you want to do complex searches, please look at the lesson on our website") but the link ("our website") only links to the top-level http://software-carpentry.org. Is there a lesson that shows complex find-grep queries so that the link could be updated? In case such example doesn't yet exist, should it be added (e.g. to discussion.md?) or the link removed?

brief job control commands

Add "a couple of sentences on each of ^C, ^D, and ^Z to
discussion.md in shell-novice."
per carpentries-incubator/shell-extras#1 (later in thread)

Do we create a Nelle user?

From reading the lesson plan and the instructor notes, I am uncertain whether i need to create a Nelle user on my linux laptop for the lesson and import her filesystem. What is the recommended way to prepare my machine for teaching about the filesystem, etc.

About ls output

ls output is "different" depending of the size of the terminal. E.g. for a large terminal I get

$ ls
1.txt  2.txt  3.txt  4.txt  5.txt  6.txt  7.txt  8.txt  9.txt

but for a small one I get

$ ls
1.txt  3.txt  5.txt  7.txt  9.txt
2.txt  4.txt  6.txt  8.txt

Can we always display the output of ls as if we are using -1?

$ ls -1
1.txt
2.txt
3.txt
4.txt
5.txt
6.txt
7.txt
8.txt
9.txt

Page not found - Motivation.html for shell-novice

Error 404 when navigating from http://swcarpentry.github.io/shell-novice/ to http://swcarpentry.github.io/shell-novice/motivation.html by clicking on "Motivation"

Some issues with the shell-novice lesson

I had a look at the lesson and found a few things I need to clarify before I can submit a pull request correcting them. I have decided not to create an issue for each as some are fairly minor issues
(numbers are point numbers, not lesson sections).

1. In the introduction it could be mentioned that automation using scripts also makes research more reproducible (or at least the analysis).
2. The actual commands like cd or ls could be mentioned in the learning outcomes.
3. In the 'Files and Directories' section:
   • 'read-run-print cycle' is used but in the Intro REPL is used (switch to REPL for consistency?)
   • When talking about home directory: could mention ~ and cd (w/o argument) here instead of later in 'Creating Things'
   • pwd is cryptic but can be remembered as 'print working directory' -- so I suggest cutting the lengthy discussion (alternatively we could introduce alias command e.g. $ alias whereami=pwd);
   • there is a mismatch in Figures and text (/users vs /Users), which misleadingly suggests that bash is case insensitive
   • arguments vs parameters: why use inconsistently and not define them and stick to definitions?
4. 'Creating Things':
   • could emphasise more that mv overwrites an existing file if second argument is the same as the name of another exiting file (potential for mistake and loss of a file)
   • perhaps the title should be more general as this part covers deleting and copying/moving too
   • discussion of '~' (home directory) could be moved to 'Files and Directories'
5. 'Pipes and Filters':

   • (redirection) overwrites existing files without a warning

   • Figure has no part for 'wc -l *pdb | sort -n' (slight mismatch to text)
6. 'Loops': could mention automation of repetitive tasks as motivation and add a sentence explaining semicolon as separator (useful in one-liner scripts)
7. 'Finding Things' could be introduced before sections 4 (Loops) and 5 (Scripts) as they fit well with 3 (pipes and filters; as an example). What is the rationale for putting it at the end? (I mentioned unspecific link in another issue).
8. Would any of the sections benefit from additional MCQs?

`filesystem` status?

The filesystem folder looks kind of messy right now - not a clear structure in place. Has there been discussion about what it should look like? If so, can someone point me to it? If not...let's discuss!

New commit to BC which modifies shell stuff.

swcarpentry/DEPRECATED-bc#789

Wonky formatting for a challenge item

Re. page on Finding Things file <06-finding.md>, lines 457-471, The formatting of the section on "Matching ose.dat but not temp" is out of wack with the other challenges (the titles is too large, it is not in a box with a pencil icon like other challenge, etc.) Looks like the code has been interpreted as "block indent" and "heading font size" rather than "challenge." I looked at the Markdown file and can't find where the error is but the resulting HTML page is definitely wrong. Can anyone tell me where the problem is in the code?

Add novice/ref/01-shell.md and novice/teaching/02-shell.md

This novice shell content wasn't in novice/shell or the earlier shell/novice or bash/novice, so we need to graft it in by hand. Looking at its history:

$ git log --stat --oneline --follow bc/master -- novice/ref/01-shell.md
b614a72 Putting a blank line between the header and the four-'#' titles so that the latter will format properly
 novice/ref/01-shell.md | 1 +
 1 file changed, 1 insertion(+)
5705090 *   Getting rid of old cheat sheets *   Indenting error output cells *   Removing explicit 'level' keys from Markdown files *   Storing the generated files so that people who don't use those tools won't have to regenerate them. *   Modifying .gitignore to reflect this. *   Updating the Makefile to run Jekyll exactly once. *   Command to install on the server *   No longer worrying about making a page of images *   Better (more guessable) name for the target that builds the website *   Fixing image paths *   Fixing up glossary entries
 novice/ref/01-shell.md | 1 -
 1 file changed, 1 deletion(-)
94fc345 Removing explicit 'level' keys from Markdown files
 novice/ref/01-shell.md | 1 -
 1 file changed, 1 deletion(-)
b8c7dc3 Reorganizing material to put novice + intermediate at top level
 {ref/novice => novice/ref}/01-shell.md | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
b66851c Preparing material for Version 5 novice release
 shell/novice/reference.md => ref/novice/01-shell.md | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
3a38907 shell: Rename the new-style lessons from 'bash' to 'shell'
 {bash => shell}/novice/reference.md | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
5159f8a Adding shell cheat sheet
 bash/novice/reference.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

Are there side effects to these commits? Let's start with swcarpentry/DEPRECATED-bc@5159f8a:

$ git show --stat 5159f8a
commit 5159f8a2fda85bb0106d825a0fe9f84ac28d8c5d
Author: Greg Wilson <[email protected]>
Date:   Sat Nov 30 12:49:10 2013 -0500

    Adding shell cheat sheet

 bash/novice/index.md     |  1 +
 bash/novice/reference.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

so this will be a series of fixes like #13, where we need to tweak filter-branch commits to preserve the cheat-sheet changes.

Loops: potential pitfall with "measure twice, run once" section

I use the approach described in this section quite frequently. One pitfall I occasionally run into is forgetting to quote the command when it includes a file redirect. What if our example was this.

for filename in *.dat
do
    someCommand $filename >> outfile
done

If we simply add echo to the beginning of the looped command, we could easily corrupt our data file.

There are a couple of potential ways we could address this.

• Add a brief parenthetical right after "This isn't foolproof..." citing this as an example of a case where the described approach could be problematic.
• Add an additional example in that section demonstrating why and how to handle this type of case.
• Add an additional challenge, one loop having echo someCommand $filename >> outfile and the other having echo "someCommand $filename >> outfile" and ask what the difference in outcome would be.

On one hand we don't want to set the students up to fail, but on the other hand we don't want to overload them with details. Any suggestions?

Consider changing /users to /home

Users using modern Linux distributions in their laptops while following the lesson will recognize /home immediately, while /users might bring some confusing.

highlight diagram for exercises in 01-filedir

In 01-filedir.md, the exercise "ls reading comprehension" isn't in conformance with our example filesystem.

Attach shell/novice history from before swcarpentry/bc@3a38907

Along the lines of what I did in #3.

$ git checkout -b bash-novice 3a38907^
$ git filter-branch -f --subdirectory-filter bash/novice --prune-empty
$ git checkout novice-shell
$ ORPHAN_COPY_COMMIT=$(git rev-parse 'HEAD^{/shell: Rename the new-style lessons}')
$ git rebase -p --onto bash-novice "${ORPHAN_COPY_COMMIT}"
$ git log --graph --topo-order --oneline --decorate | grep -3 bash-novice
* 6384334 Minor typo
* 79b05ed Corrects runtime calculation for goostat and goodiff. 300 choose 2 is 300*299/2. Also the conversion from seconds to weeks was off.
* da57d31 Preparing material for Version 5 novice release
*   8c5c84c (bash-novice) Merge pull request #255 from wking/no-shebangs
|\
| * a41bc7b bash/novice/guide.md: Explain why we don't cover shebangs
* | 576a2e7 Append "bash" in front of "goostats" calls

Lets also make sure we still have an exact copy of the current
swcarpentry/bc content (using HEAD^ to ignore the changes from
#1):

$ git ls-tree origin/master -- novice/shell
040000 tree 5579d5a5f799bb8af1f708efc3a770aa12142cf4    novice/shell
$ git cat-file -p HEAD^ | grep tree
tree 98a38300332c3817bdb7f774435381360332900a

Hmm, looks like we have some differences. Drilling down into
those trees:

diff -u <(git ls-tree -r origin/master -- novice/shell | sed 's|novice/shell/||') <(git ls-tree -r HEAD^)
--- /dev/fd/63  2014-10-25 10:29:18.284272335 -0700
+++ /dev/fd/62  2014-10-25 10:29:18.284272335 -0700
@@ -210,8 +210,6 @@
 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   filesystem/users/thing/backup/2012-12-01/backup.log
 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   filesystem/users/thing/backup/2013-01-08/backup.log
 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   filesystem/users/thing/backup/2013-01-27/backup.log
-100644 blob 46f7b0914306f602e25e55fcd58757ec39167afe   gen-nene.py
-100644 blob d3c5d6fa9cf61d8fe1ba3deb65595342a0be0217   gen-sequence.py
 100644 blob c3bf2526efc6bda4b0504400120eb2dac33019f7   img/filesystem-challenge.odg
 100644 blob 9e5f34ab0ea7410f74eb0d6745a686eeb4a5e410   img/filesystem-challenge.png
 100644 blob 9c077f0cea68c310e9c2034319b3dfd1e9135c17   img/filesystem-challenge.svg

Where did bc's gen-nene.py and gen-sequence.py come from?

$ git log --oneline --stat --follow origin/master -- novice/shell/gen-nene.py
b8c7dc3 Reorganizing material to put novice + intermediate at top level
 {shell/util => novice/shell}/gen-nene.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
3a38907 shell: Rename the new-style lessons from 'bash' to 'shell'
 {bash => shell}/util/gen-nene.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
f4cc356 Updating index file and moving utility scripts to the right place
 bash/util/gen-nene.py | 2 ++
 1 file changed, 2 insertions(+)
090b60b Put randomly generated data in files
 bash/util/gen-nene.py | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
$ git log --oneline --stat --follow origin/master -- novice/shell/gen-sequence.py
b8c7dc3 Reorganizing material to put novice + intermediate at top level
 {shell/util => novice/shell}/gen-sequence.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
3a38907 shell: Rename the new-style lessons from 'bash' to 'shell'
 {bash => shell}/util/gen-sequence.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
f4cc356 Updating index file and moving utility scripts to the right place
 bash/{novice/creatures => util}/gen-sequence.py | 2 ++
 1 file changed, 2 insertions(+)
a41a27d Renaming directories with example files
 bash/novice/{04-creatures => creatures}/gen-sequence.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
03b26ac Example files for intro to shell
 bash/novice/04-creatures/gen-sequence.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

So it looks like what we want is a history where our version of
swcarpentry/DEPRECATED-bc@b8c7dc3 is an merge pulling in information from a
shell-util branch as well as the shell-novice branch:

* Reorganizing material to put novice + intermediate at top level
|\
| * Updating index file and moving utility scripts to the right place (2013-11-29)
| * Put randomly generated data in files (2013-11-29)
| * Example files for intro to shell (2013-11-20)
*   Stuff from shell/novice

Unfortunately, swcarpentry/DEPRECATED-bc@03b26ac (Example files for intro to
shell, 2013-11-20) is already in our shell/novice history (from
the bash/novice filter branch above). In fact, we have
gen-sequence.py in the rebased history, we just drop it with the
filter-branched c6ac923 (Updating index file and moving utility
scripts to the right place, 2013-11-29). Fixing this will
require a bit more care, but I'll just push what I have for now.

Avoid HTML img tag

Will be nice to avoid HTML img tag. For example, instead of

<img src="fig/foo.svg" alt="Foo" />

we can use

![Foo](fig/foo.svg)

The list of img elements to replace is

$ grep img *.md      
01-filedir.md:<img src="fig/filesystem.svg" alt="The Filesystem" />
01-filedir.md:<img src="fig/home-directories.svg" alt="Home Directories" />
01-filedir.md:<img src="fig/homedir.svg" alt="Nelle's Home Directory" />
01-filedir.md:<img src="fig/filesystem-challenge.svg" alt="Filesystem for Challenge Questions" />
02-create.md:<img src="fig/nano-screenshot.png" alt="Nano in Action" />
06-find.md:<img src="fig/find-file-tree.svg" alt="File Tree for Find Example" />

Issues at 01-filedir.md

From check.py:

• Heading at line 353 should be level 2
• Document contains heading not specified in the template: Nelle's Pipeline: Organizing Files
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Add sleep as an example of the Unix philosophy and use it to introduce CRTL+C

When I teach the shell, usually to motivate pipes, I show the sleep command as an example of Unix philosophy of "small programs, loosely coupled" (or whatever goes the saying). sleep is a funny program and appear useless at first sight, kind of shocking compared to Microsoft Office, but very easy to understand and play with.

After running sleep 10 (which sleeps for 10 seconds), I "accidentally" run sleep 100000 or some other ridiculous number. This gives me the chance to introduce CTRL+C as the way to kill a command. This is useful later in the lesson when people gets trapped in loops or other commands that go wrong.

I think a short paragraph, maybe in a call out, would be nice in one of the first 3 parts of this lesson.

What do you think? If people agrees, I could do a PR, but actually I prefer to leave it for a newcomer.

Add mention of explainshell.com to notes

Because http://explainshell.com is cool.

Issues at 00-intro.md

From check.py:

• Heading at line 55 should be level 2
• Document contains heading not specified in the template: What and Why
• The topic page should not have sub-headings outside of special blocks. If a topic needs sub-headings, it should be broken into multiple topics.

Bits of history we've found and grafted on

This is a meta issue for laying out which history we've found for bc's novice/shell, and whether or not it's been grafted into this repository's master branch:

• everything from swcarpentry/bc under the novice/shell name (since swcarpentry/DEPRECATED-bc@b8c7dc3 (Reorganizing material to put novice + intermediate at top level, 2014-02-16), see swcarpentry/DEPRECATED-bc#812 for commands used).
• everything from swcarpentry/bc under the shell/novice name (after swcarpentry/DEPRECATED-bc@3a38907 (shell: Rename the new-style lessons from 'bash' to 'shell', 2014-01-09, swcarpentry/DEPRECATED-bc#222), but before swcarpentry/DEPRECATED-bc@b8c7dc3, see #3 for commands used).
• everything from swcarpentry/bc under the bash/novice name (before swcarpentry/DEPRECATED-bc@3a38907, see #4 for commands used).
• the gen-*.py scripts which were dropped by the filter-branch in #4 (see #13 for commands used).
• any copy-paste commits there, if it wasn't all from scratch. Mention them (or sections of history that you know were from scratch) and I'll add them to this list.
• cleanup commit-message formatting for long commits like swcarpentry/DEPRECATED-bc@5705090 (Getting rid of old cheat sheets …, 2014-02-27, see #9 for commands used).
• remove auto-generated content
• track down sources for downloaded content (see #14).
• Add novice/ref/01-shell.md and novice/teaching/02-shell.md (see #27 for commands used).

Missing files motivation.md and discussion.md

From check.py:

• In /tmp/shell/index.md: The document links to motivation.html, but could not find the expected markdown file /tmp/shell/motivation.md
• In /tmp/shell/index.md: The document links to discussion.html, but could not find the expected markdown file /tmp/shell/discussion.md

Cleanup commit-message

• 39df817 -> 88ee0fb
• 69fe2c3 -> 213e0aa
• 75ebce5 -> c14ecf2
• 3671a8a -> 8fd947c
• 79b05ed -> 6ad2c06

Consider adding an `-exec` with find

Consider given an example of how to use -exec in conjunction with find. Currently, it is suggested to use command expansion -- $() -- to run a command on a file. As suggested by by @wking in #132, it is simpler and much more idiomatic, to use -exec.
Ex.

$ find .. -name '*.pdb' -exec grep FE {} \+

rather than

$ grep FE $(find .. -name '*.pdb')

Output samples

At 01-filedir.md we have

$ ls -F
creatures/  molecules/           pizza.cfg
data/       north-pacific-gyre/  solar.pdf
Desktop/    notes.txt            writing/

where the output is alphabetic sorted by columns and

$ ls -F data
amino-acids.txt   elements/     morse.txt
pdb/              planets.txt   sunspot.txt

where the output is alphabetic sorted by lines.

This is bugging me.

I previous suggested to implicit use flag -1, see #131. I'm OK to not use it but could we agree in how to sort ls output and the numbers of maximum of columns?

@wking I was trying to use shopt to unset checkwinsize and fix the number of columns but didn't work. Any advice?