corporate-runaways/tooloo

Command LookUp

tooloo's Issues

export listing to markdown

Having a lot of helpful scripts is great, but not having a way to share a listing of what you have with others sucks. Clu gives us a listing, but it's not in a particularly shareable way.

Goal: to output the results similar to clu list in such a format that people can easily incorporate into the README of their dotfile repos.

New command

clu export <format>

Initially there will only be one supported format: markdown.
Markdown because that's what almost everyone uses for their Readme.

Since the database contains the source url of each script I think that the format should be as follows (still sorted alphabetically by script name)

| script name                        | description                                     |
|------------------------------------|-------------------------------------------------|
|[script-name](https://example.com)  | description text here which may be notably wide.|
|[next](https://example.com/1)       | another description here                        |     

Rather than taking the width from the terminal output it should just be calculated to fit the longest link in the first column and longest description in the second. This will be ridiculously wide, but that's ok, markdown tables are kinda crap, and it's not actually intended to be read in this format.

Output should be send to standard out. Users can always clu export markdown > my.md

Update License in 2.0 version

Reminder to myself to change the license with the 2.0 version.

Why isn't CLU available in the Raku ecosystem?

Why isn't CLU available in the Raku ecosystem?

Being able to say:

zef install CLU

would be much better, no?

[feature] Internationalization

We've got a database... why not stick the internationalization strings in it? Something like this...

key	lang	value
add	en	add
add	eo	aldoni

[feature] markdown syntax highlighting of description

I've been using markdown in my description blocks in v2, because it looks great when exported to a static blog, and is very readable when you run tooloo show <command>

But it would be much nicer if we had actual syntax highlighting like that provided by the following projects:

• Charm's glamour (go)
• bat (rust)
• termd (typescript)

I don't believe there is currently a Raku library that provides this.

Short term hacky fix

Check if glow is installed (Charm's cli wrapper around glamour), give it a tempfile to render, and insert the output into our output.

Long term fix

Port Glamor (or similar) to Raku, and use that.

[feature] add support for "exportable" flag

There are many little scripts we can write that only apply for the creator. Either they'll only work on the creator's system, or they depend on other tools / creations that only the creator has.

You might also be documenting some common unix tool that has bad "man pages" and want to show up in tooloo.

Regardless of why, there is stuff that you'll document that you don't want to be included in statically generated sites or other future export forms.

This task is to add support for an exportable flag in the TOML

• add to command's DB table
• add to upgrade tooling for people using current version
• add observation of the loop in the export that iterates over commands OR make the query just not include these in results.

clue find stopped working

given

is_brewed : indicates if a package is installed via homebrew

clu find homebrew and clu find is_brewed both return "No matches found"

test failing during install

fix this failing install ASAP. Note that this happens when doing zef install . also. (as you'd expect)

===> Testing: TooLoo:ver<2.0.1>:auth<zef:masukomi>:api<2>
[TooLoo] # Failed test 'testing display-command'
[TooLoo] # at t/TooLoo/Command.rakutest line 51
[TooLoo] # expected: '┌───────────────────┬──────────────────────────────────────────────────────────────────────────────────┐
[TooLoo] # │ Attribute         │ Detail                                                                           │
[TooLoo] # ├───────────────────┼──────────────────────────────────────────────────────────────────────────────────┤
[TooLoo] # │ command           │ ls                                                                               │
[TooLoo] # │ short description │ lists the contents of directories                                                │
[TooLoo] # │                   │                                                                                  │
[TooLoo] # │ full description  │ a detailed description                                                           │
[TooLoo] # │                   │                                                                                  │
[TooLoo] # │ usage             │ LS(1)                        General Commands Manual                       LS(1) │
[TooLoo] # │                   │                                                                                  │
[TooLoo] # │                   │ NAME                                                                             │
[TooLoo] # │                   │      ls – list directory contents                                                │
[TooLoo] # │                   │                                                                                  │
[TooLoo] # │ type              │ executable                                                                       │
[TooLoo] # │ language          │ c                                                                                │
[TooLoo] # │ location          │ /bin/ls                                                                          │
[TooLoo] # │ source repo       │ https://github.com/wertarbyte/coreutils                                          │
[TooLoo] # │ source url        │ https://github.com/wertarbyte/coreutils/blob/master/src/ls.c                     │
[TooLoo] # │ tags              │ cli, utility                                                                     │
[TooLoo] # └───────────────────┴──────────────────────────────────────────────────────────────────────────────────┘
[TooLoo] # '
[TooLoo] #      got: '┌───────────────────┬────────────────────────────────────────────────────────────────────────────────┐
[TooLoo] # │ Attribute         │ Detail                                                                         │
[TooLoo] # ├───────────────────┼────────────────────────────────────────────────────────────────────────────────┤
[TooLoo] # │ command           │ ls                                                                             │
[TooLoo] # │ short description │ lists the contents of directories                                              │
[TooLoo] # │                   │                                                                                │
[TooLoo] # │ full description  │ a detailed description                                                         │
[TooLoo] # │                   │                                                                                │
[TooLoo] # │ usage             │ LS(1)                       General Commands Manual                      LS(1) │
[TooLoo] # │                   │                                                                                │
[TooLoo] # │                   │ NAME                                                                           │
[TooLoo] # │                   │      ls – list directory contents                                              │
[TooLoo] # │                   │                                                                                │
[TooLoo] # │ type              │ executable                                                                     │
[TooLoo] # │ language          │ c                                                                              │
[TooLoo] # │ location          │ /bin/ls                                                                        │
[TooLoo] # │ source repo       │ https://github.com/wertarbyte/coreutils                                        │
[TooLoo] # │ source url        │ https://github.com/wertarbyte/coreutils/blob/master/src/ls.c                   │
[TooLoo] # │ tags              │ cli, utility                                                                   │
[TooLoo] # └───────────────────┴────────────────────────────────────────────────────────────────────────────────┘
[TooLoo] # '
[TooLoo] # You failed 1 test of 1
===> Testing [FAIL]: TooLoo:ver<2.0.1>:auth<zef:masukomi>:api<2>

add ability to find and ingest all .toml files in a directory

When you need to load a bunch in it sucks, and is slower, to have to do this

find . -name '*.meta.toml' -exec clu add '{}' \; -exec sleep 1 \;
# sleep necessary because otherwise locked db complaints

There should be a way to point clu at a directory and have it ingest all the toml it finds in it or in any subdirectory.

There is, of course, the problem that there may be TOML files in there that aren't for clu.

To solve this i think we should:

• add an exported method in Ingester that validates toml files.
  • make the Ingester use that (side-task)
• add an clu ingest command that checks if it's a directory, and mass-ingests if so, but if it's a path, just calls the ingest method like clu add does.
  • have the mass-ingestion code live in Ingester and descend through all directories to find all .toml files
  • have the mass-ingestion code use the toml validator to see if it is a toml for clu, and just do something like note("skipping /foo/bar.toml") for files that aren't valid clu toml.
  • if it is for us, ingest as per usual

[feature] beautify error when can't figure out where to write a template

In addition to just... not good color choice, we don't need a stack trace.
Just tell the user you couldn't find the correct directory and ask for the right one.

add support for demos via asciicasts (asciinema)

asciinema is probably the best way to demonstrate the usage of any given command. it's files are stored in asciicast v2 format

• add support for asciicast_url to the TOML ingestion
  • 100% optional
  • can be a path /foo/bar/baz.cast
  • can be a file protocol path file://foo/bar/baz.cast
    • NO (maybe later) converted to /foo/bar/baz.cast
  • can be an url https://foo/bar/baz.cast
    • expected mime type is application/x-asciicast but we shouldn't enforce that. I'm sure a lot of servers get it wrong.
  • NO: if an asciicast_url is found in a toml it will check if asciinema support has been enabled, and .... ask the user if they want it enabled? ??? or just store the data? just store the data
• add support for adding .cast files separately
  • clu add path/to/my.cast
  • name must match command already in the db
• add playback support
  • NO: only supported if asciinema support has been enabled
• add clu demo <command> functionality (see Playback below)
• add clu list demos functionality (see Playback below)

Adding Demos

For now, clu add will only support local paths /path/to/my.cast

You can only add a web url via the .toml (for now).

Playback support

No clue how this should work since most commands probably won't have casts.

At the very least there should be an line in the clu show <command> indicating that a "demo" is available.

I'm thinking something like clu demo <command> and it invokes asciinema play /path/to/my.cast if there's a demo (asciicast) for the specified command.

Skipped: However, IF the asciicast_url is an http(s) url it'll just callopen <asciicast_url> and load the web page.

👆 we can add that later.

If there isn't one it should say something like

There's no demo available for . Go here https://example.com/creating_demos.html to learn how to create one.

maybe there's a clu demos command (plural demoS) which lists all the commands that have demos.

I'm open to suggestions .

Side Note / Future Thinking

Once we have support for markdown export we'll be able to link to an asciicast to demonstrate the use of the command.

This will likely have to wait until we have the ability to export a "readme" type page for every command.

I think we're going to end up in a situation where we're basically just generating pages for some static site generator. A custom index and then an individual page for every command. ... would be cool.

add clu list command

clu list would list all the command + description pairs.

The output should be columnular (sp? in columns) and sorted alphabetically by command.

then you could grep it or pipe it to other things.

[feature] enable listing of commands by tag

Builds on, and is blocked by Issue #9 (adding initial support for tags)

Not terribly useful on the command line, this could be a nice-to-have for exporting to create web pages. This would enable web users to browse by tag.

Still, we'll start with the command line because it's the easiest way to implement the core functionality we'll need for the export.

I'm thinking clu list <tag> as a way of invoking this, but am open to other suggestions.

add search support for tags

we should allow users to specify an array of tags in the TOML that can be used for searching.

• add new tags table (id, tag)
• add new commands_tags table (command_id, tag_id) compound key
• add handling of the tags array (if present) to Ingester
  • tags table should update before command because of triggers on command tables
• update the fts tables to have a tags column
• update the triggers to have an insert that selects from commands & joins concatenated tags

example SQL (untested of 1 of the 3 triggers)
note that SQLite uses || instead of the traditional CONCAT function and GROUP_CONCAT for concatenating rows

CREATE TRIGGER commands_fts_update AFTER UPDATE ON commands
BEGIN
    INSERT INTO commands_fts
	(commands_fts, rowid, name, description, language, tags)
	VALUES ('delete', old.rowid, old.name, old.description, old.language);

    INSERT INTO commands_fts
	(rowid, name, description, language)
	(
            SELECT GROUP_CONCAT(tag) 
                from tags where  command_id = old.id
            );
END;

💥 when usage's exit code != 0

given a usage_command which returns a non-zero when called

the error at the end of this issue is raised.

Problems:

1. usage returning something other than zero isn't inherently "wrong" so we need to accept that, read the output, and see if it contains "USAGE" (or something similar, case insensitive). If so, treat it as 👍🏻 and don't proceed to fallback.
2. if usage returns a non-zero exit code and it seems to be an error, then check for fallback, and output a helpful message if it isn't specified.

stack trace from commit bc441b4

❯ clu show is_brewed
empty_database.db in resources? Nil
is_brewed : indicates if a package is installed via homebrew

Use of Nil in string context
  in block  at /Users/kayrhodes/workspace/Clu/lib/Clu/Command.rakumod (Clu::Command) line 163
Invalid attribute name 'on_'
  in sub color at /usr/local/Cellar/rakudo-star/2022.07/share/perl6/vendor/sources/11AAE19828E79B2115B29C27C68926D3C4B029FB (Terminal::ANSIColor) line 54
  in sub colored at /usr/local/Cellar/rakudo-star/2022.07/share/perl6/vendor/sources/11AAE19828E79B2115B29C27C68926D3C4B029FB (Terminal::ANSIColor) line 61
  in sub display-fallback-usage at /Users/kayrhodes/workspace/Clu/lib/Clu/Command.rakumod (Clu::Command) line 163
  in sub display-usage at /Users/kayrhodes/workspace/Clu/lib/Clu/Command.rakumod (Clu::Command) line 152
  in sub display-command at /Users/kayrhodes/workspace/Clu/lib/Clu/Command.rakumod (Clu::Command) line 61
  in sub display-command at /Users/kayrhodes/workspace/Clu/lib/Clu/Command.rakumod (Clu::Command) line 54
  in sub MAIN at /Users/kayrhodes/bin/clu line 59
  in block <unit> at /Users/kayrhodes/bin/clu line 53

[feature] add support for setting settings

the v2.0.0 db adds the clu_metadata table which has a key and value column (both TEXT data type)

There should be a command that enables the setting of settings. I'm thinking

clu set <key> <value>

and a corresponding command to display all the metadata

clu metadata

(displays all the key value pairs)

This will be used, for a variety of things, but from a user facing perspective the first thing will be to toggle support for demos clu set demos true