pgilad/leasot

Parse and output TODOs and FIXMEs from comments in your files

Leasot

Intelligently parse and output TODOs and FIXMEs from comments in your files

Easily extract, collect and report TODOs and FIXMEs in your code. This project uses regex in order to extract your todos from comments.

Comment format

TODO: add some info

• Spaces are optional.
• Colon is optional.
• Must be in a comment (line or block) in its own line (some code(); //TODO: do something is not supported).
• Can be prefixed with a @ (i.e. @TODO).
• Spaces are trimmed around comment text.
• Supported default types are TODO and FIXME - case-insensitive.
• Additional types can be added (using tags in cli and customTags in leasot.parse)
• New extensions can be associated with bundled parsers as many languages have overlapping syntax
• Supports both leading and trailing references. Examples:
  • // TODO(tregusti): Make this better
  • // TODO: Text /tregusti

Supported languages

49+ languages are supported, pull requests for additional language support is most welcomed!

Usage in command line

npm install --global leasot

Command line help

$ leasot --help

Usage: leasot [options] <file ...>

Parse and output TODOs and FIXMEs from comments in your files

Options:
  -V, --version                        output the version number
  -A, --associate-parser [ext,parser]  associate unknown extensions with bundled parsers (parser optional / default: defaultParser) (default: {})
  -i, --ignore <patterns>              add ignore patterns (default: [])
  -I, --inline-files                   parse possible inline files (default: false)
  -r, --reporter [reporter]            use reporter (table|json|xml|markdown|vscode|gitlab|raw) (default: table) (default: "table")
  -S, --skip-unsupported               skip unsupported filetypes (default: false)
  -t, --filetype [filetype]            force the filetype to parse. Useful for streams (default: .js)
  -T, --tags <tags>                    add additional comment types to find (alongside todo & fixme) (default: [])
  -x, --exit-nicely                    exit with exit code 0 even if todos/fixmes are found (default: false)
  -h, --help                           output usage information

Examples:
    # Check a specific file
    $ leasot index.js

    # Check php files with glob
    $ leasot '**/*.php'

    # Check multiple different filetypes
    $ leasot 'app/**/*.js' test.rb

    # Use the json reporter
    $ leasot --reporter json index.js

    # Search for REVIEW comments as well
    $ leasot --tags review index.js

    # Add ignore pattern to filter matches
    $ leasot 'app/**/*.js' --ignore '**/custom.js'

    # Search for REVIEW comments as well
    $ leasot --tags review index.js

    # Check a stream specifying the filetype as coffee
    $ cat index.coffee | leasot --filetype .coffee

    # Report from leasot parsing and filter todos using `jq`
    $ leasot 'tests/**/*.styl' --reporter json | jq 'map(select(.tag == "TODO"))' | leasot-reporter

    # Associate a parser for an unknown extension`
    $ leasot -A '.svelte,twigParser' -A '.svelte,defaultParser' 'frontend/*.svelte'

Usage in NPM scripts

Use leasot -x in order to prevent exiting with a non-zero exit code. This is a good solution if you plan to run leasot in a CI tool to generate todos.

{
    "scripts": {
        "todo": "leasot 'src/**/*.js'",
        "todo-ci": "leasot -x --reporter markdown 'src/**/*.js' > TODO.md"
    },
    "devDependencies": {
        "leasot": "^7.0.0"
    }
}

Programmatic Installation

npm install --save-dev leasot

Programmatic Examples

const fs = require('fs');
const leasot = require('leasot');

const contents = fs.readFileSync('./contents.js', 'utf8');
// get the filetype of the file, or force a special parser
const filetype = path.extname('./contents.js');
// add file for better reporting
const file = 'contents.js';
const todos = leasot.parse(contents, { extension: filetype, filename: file });

// -> todos now contains the array of todos/fixme parsed

const output = leasot.report(todos, 'json', { spacing: 2 });

console.log(output);
// -> json output of the todos

Leasot with build tools

• gulp-todo
• broccoli-leasot
• todo-webpack-plugin

API

const leasot = require('leasot');

See main exported functions

Mainly, you should be using 2 functions:

• parse for parsing file contents
• report for reporting the todos

Type documentation

Built-in Reporters

See built-in reporters

License

MIT © Gilad Peleg

leasot's Issues

C# support

Wouldn't the parser for c# be the same as the existing one for c/c++?

No way to warn only when in npm task

If you put this in an npm task it always fails if there are todo's. This is a little opinionated, as I just want to be reminded that I have todo's and what they are, not that it should interrupt my build process. Do you know of any workaround, or would this be a code fix to add a --warn-only option?

Add Recursive Globing

I could not find a way to allow the glob to recursively enter sub-directories.

My use case is a C# project that has ToDos in varying levels of depth in the directory and I have to run the command line tool as many times as levels of depth I have. I could be totally wrong admittedly.

Support Hyperlink for Markdown reporter

It'd be awesome to have ability of generating a hyperlink to a file using some template/base specified in parameters. Right now it's just a file path.

Erb should use the same parser as EJS.

From this commit: 6e3996d .erb files are apparently being parsed by the twigParser.js.

IMHO this is incorrect in the sense that both <!-- comment --> and <%# comment %> patterns are correct for a ruby erb file depending on the context. ejsParser.js is already setup with both these patterns.

However, I'm not sure how you are planning to organize these parsers. Are you using their comment styles or the languages. EJS obviously is not a Ruby based language so should ERB have it's own parser or is it ok in the framework of how you are building things to rely on a parser with the same patterns?

With that said I can make a PR to fix however you feel is appropriate.

Assign task

Having a task assignment column for contributors/developers would be interesting. Any thoughts on that?

Eg:

//TODO: Inject the TODO list into README.md --@name

or

//TODO: Inject the TODO list into README.md --#name

Using with NPM and iojs causes NPM Err but works fine

I am not sure if this is intended or not as the apparent fault is the exit status of 1

Add support for .sql files

Would love to see support for .sql files (-- and /* */).

Or maybe to add a multi parser option to "associateExtWithParser" function.
Example.
'{.sql': {parserName: ['defaultParser', 'haskellParser']}}

Use "table" package instead of "text-table"

https://github.com/gajus/table package is a very modular, well documented and well tested module/function for generating text tables. It allows granular customisation of how table contents is aligned, text wrapping, text truncation, border style and text padding.

If accepted, I will happily create a PR.

Error: EMFILE

On OSX 10.10, Node 0.10.38:

$ leasot '**/*'
(libuv) Failed to create kqueue (24)
{ [Error: EMFILE, open '/Users/dan.kaplun/stuff/slap/node_modules/node-inspector/node_modules/biased-opener/node_modules/browser-launcher2/node_modules/plist/node_modules/xmlbuilder/node_modules/lodash-node/compat/functions/after.js']
  errno: 20,
  code: 'EMFILE',
  path: '/Users/dan.kaplun/stuff/slap/node_modules/node-inspector/node_modules/biased-opener/node_modules/browser-launcher2/node_modules/plist/node_modules/xmlbuilder/node_modules/lodash-node/compat/functions/after.js' }

Allow HTML files

events.js:141
      throw er; // Unhandled 'error' event
            ^
Error: File: /.../videomail-client/examples/bad_browser.html - Extension .html is not supported

You know, you can write inline JS in HTML files and in my case, there are some TODO remarks. So I think, HTML files should be allowed.

Tests for the CLI does not pass on windows

In either cygwin or cmd.exe.

They all fail with the same message.

  7) check cli should get no error exitCode if no todos or fixmes are found:
     Error: spawn UNKNOWN
      at exports._errnoException (util.js:890:11)
      at ChildProcess.spawn (internal/child_process.js:302:11)
      at Object.exports.spawn (child_process.js:367:9)
      at testCli (C:\code\leasot\tests\cli-spec.js:14:27)
      at Context.<anonymous> (C:\code\leasot\tests\cli-spec.js:135:9)
      at callFnAsync (C:\code\leasot\node_modules\mocha\lib\runnable.js:338:8)
      at Test.Runnable.run (C:\code\leasot\node_modules\mocha\lib\runnable.js:290:7)
      at Runner.runTest (C:\code\leasot\node_modules\mocha\lib\runner.js:422:10)
      at C:\code\leasot\node_modules\mocha\lib\runner.js:533:12
      at next (C:\code\leasot\node_modules\mocha\lib\runner.js:342:14)
      at C:\code\leasot\node_modules\mocha\lib\runner.js:352:7
      at next (C:\code\leasot\node_modules\mocha\lib\runner.js:284:14)
      at Immediate._onImmediate (C:\code\leasot\node_modules\mocha\lib\runner.js:320:5)

Create logo for project

How to use with VSCode?

I can see that one can use VSCode as the reporter, but I cant seem to figure out how to use it with vscode. Is it only customizing the link (but still needs an output file)?

I am sure there is something basic I am missing (or am expecting it to do something it is not designed to do)

Assuming I execute from VSCode integrated terminal, what should I expect

> leasot ./server/**/*.*s -r vscode

exclude glob?

I don't see an exclude glob in the CLI so assuming there is none.
In my case I have an "off" subdirectory in a a lib directory that's excluded/ignored by requireall and I want leasot to do that as well.

Vue support issue

First, thanks for this useful software. It's a great way to generate automatic reports.

I'm having problems with VUE files. Given the follow example file, I would suspect it to find 3 TODOs. However, it only finds the <!-- TODO: Vue Template -->

<template type="text/html">
    <!-- TODO: Vue Template -->
    <div class="container">
        <h1>Example</h1>
    </div>
</template>

<style type="text/css" lang="scss">
    // TODO: Vue CSS
</style>

<script type="text/javascript">
    /* TODO: Vue javascript */
    export default {
    };
</script>

If I move the blocks around, it still only sees the <!-- TODO: Vue Template -->. If I add a <!----> style comment to the style or script section, it will find them.

Is there support for multi-styled comments in Vue files? [Or perhaps a problem with my style]

OS: Win10 & Ubuntu 16.04.2

tsx support

Currently leasot doesn't support tsx extension. Is the support for tsx planned for upcoming releases? #93

Support for filters

I think this is a really cool tool. Would be great if we could have a filter option to, for example, only show TODOs or show only NOTE and XXX.

Best,
Max

add cson parser

add cson parser - I'll do this PR when I get a chance just making an issue as a reminder

Skips the part of double-slash inline comments after the first slash

Example:

// HACK(@sompylasar): Nice underline -- @see https://medium.com/designing-medium/crafting-link-underlines-on-medium-7c03a9274f9

Parsed text:

"Nice underline -- @see https:"

Throw TypeError when appropriate

Whenever you throw an error because a value is not of the expected type, it's more specific and therefore more informative to throw a TypeError. Plain Error constructs aren't wrong per se, just more generic.

This already happens in a few places, like lib/reportes.js#L32, but not in other cases, like lib/reporters/markdown.js#26.

Use node-exit in cli to make sure stdio is drained

Verify if that's relevant

Swift support

It will great to have Swift support

extract cli logic and use a bin interface similar to jscs

https://github.com/jscs-dev/node-jscs/blob/master/bin/jscs

Add strict option

Hi there,

I've tried to use the CLI with additionnal tags PROD and STAGING the problem is I have a comment in a PHP file :

// Products list

The reporter outputs this :

{
    "file": "path/file.php",
    "kind": "PROD",
    "line": 32,
    "text": "ucts list",
    "ref": ""
  },

I'd like to know if it would be possible to add a flag or something like that to strictly match the tags ? So if I summarize :

// Products list

Shouldn't be reported but :

// Prod : add this API key

should be

Cheers !

Github reporter

I set out to write a tool that will take any TODOs in a code base and create a GH issue per each one (with options ect., can go into them as needed).
But had a thought, that if it would not be out of the scope of this tool, to just create a reporter that can do it instead of an entirely new tool.

Thoughts and opnions?

Feature Request: Multiple Line Todos

Hello,

It would be quite useful to be able to do multiple line TODOs. This would allow me to have longer todo tasks, but not violate PEP8 or other code readability standards.

Of course, I will defer to you on the syntax, however either of these would be interesting for python:

# TODO: Here is a very very long comment where I'm really effusive on exactly what
# I need to do in the todo.

or

# TODO: Another way of doing multiple line comments could be by allowing a line
# TODO2: parameter after the todo keyword so that you could have multiple
# TODO3: multiple line comments after one another

I can make a pull request, but this seems like this might be an easy fix, so if you know how to quickly do it and have the time, it would be faster than me going through the overhead of forking and learning the codebase. Thanks,

Alex Hagen

Optionally prevent `exit 1` when there are todos (so it doesn't break the build)

Currently leasot will exit 1 if there are TODOs so it will always break the build when running on CI server (see this line) - Yes, I am running leasot during the build process to log TODOs to the project website.

I think that being able to pass a flag to always exit 0 would be ideal.

thanks for the great work by the way! :)

support for Objective-C and Objective-C++

i just added these two lines and it worked fine:

'.m': { parserName: 'defaultParser' },
'.mm': { parserName: 'defaultParser' },

Path ** does not recurse directories

I would expect leasot src/**/*.js to recurse directories within src and parse all js files

Example directory structure:

src
+-- todo-root.js
+-- folderA
       +-- todo-A-1.js
       +-- todo-A-2.js
+-- folderB
       +-- todo-B-1.js
       +-- todo-B-2.js
       +-- folderC
              +-- todo-C-1.js
              +-- todo-C-2.js

Correct Cases
Running leasot src/*.js parses files in src only, i.e. todo-root.js
Running leasot src/*/*/*.js parses files in folderC only

Incorrect Cases
Running leasot src/**/*.js parses files in folderA and the root of folderB only (It is missing todo-root.js as well as the files in folderC

@todo not TODO

It's possible to grab the default JSDoc3 "@todo" instead of "TODO:"?

Thanks!

Multi-line todo with twig parser does not work

Hi,
I love leasot, and thank you so much for working on it. I have however noticed that multi-line comments do now work with the twig parser. For example

<!--
TODO: this is a 
longer TODO description
-->

will not show any todo outputs. That means that todo in block comments in markdown, html and all of its derivations will not work.

Globbing does not work recursively

Not sure if that's expected behavior or not but i expected leasot app/**/*.js to also surface TODOs in /app/sub/folder/deep/index.js

It seems the glob explicitly refers to the given depth (e.g. app/**/* will only dig one folder-level into app and not go any deeper)

is that the correct behavior?

Give more complete example of using leasot from code with glob some directory and get correct report

In Readme.md you have only 1 example of 1 file.
It's nor really very usefull.

What I mean:
In node.js I can copy string from example like "leasot src/**/*.js" - and thats it!
It's already work and I just need small configuration fixes with some parts of it like glob mask or ignoring directories.

On the other hand, we have an example in js code with parsing one file.

var contents = fs.readFileSync('./contents.js', 'utf8');
// get the filetype of the file, or force a special parser
var filetype = path.extname('./contents.js');
// add file for better reporting
var file = 'contents.js';
var todos = leasot.parse({ ext: filetype, content: contents, fileName: file });

So I really need to type some template code to make it works on all my project, like cli did.

I think it will be very usefull to have an example that makes the same result as cli.
With glob and etc.

Thanks for awesome project, btw 👍

why is inline comment skipped?

Support multiple keys in regex (in addition to TODO and FIXME)

Would love to be able to add other keys as arguments when I invoke leasot

I have a repo which includes "HACK" as a flag for temporary workarounds and in which we often add the dev's name to comments so we can look for who should help or work on a to-do.

Would love to be able to do leasot -k hack,kaldrenon app/**/*.js and get output for HACK, TODO, FIXME, and kaldrenon.

Took a look at the code for this and it seems a little tricky because lib/regex.js doesn't provide an opportunity to manipulate the regex before it's required, and resolving that would change the whole require strategy for the parsers.

Thoughts?

Add Filetype: Pug

Support php files

pgilad/gulp-todo#35

Failing tests on Windows at comments line number

Anyone else experience failing tests on Windows with block line comments - wrong line number ? (apart from #45 issue)
Example:

2) parsing stylus parse block line comments:

      AssertionError: expected 9 to be 5
      + expected - actual

      -9
      +5

Dedicated javascript/typescript parser

This is mostly to be able to handle inline comments, by using a specialized parser like esprima/babel/espree, similar to the proposal in issue 62, but in a language-specific way.

I have branch on my fork with a working prototype. I switched the defaultParser for js files to the javascriptParser and the only test that is failing is the vue.vue file related one. I believe this is not really a bug because the test is expected a js parser to handle a whole vue file. It may be necessary to scope the parser to just <script> tags for example but that is quite a big rework I think.

In any case maybe this javascriptParser can be included as an optional one to be used by options.associateParser? Its quite a bit better at javascript/typescript since it is based on the Abstract Syntax Tree, also it can be configured to use a number of extractors including custom ones.

In the process of this I also saw that uniq in parsers.js is incorrect and should be using uniqWith, see this commit.

vscode markdown link line numbers not working

After generating the report using the VSCode reporter it generates lines like so:

| [src\ServerConfig.ts](src\ServerConfig.ts#301) | 301 | compress?

They appear clickable, but the line numbers do not reference properly. I found this comment on a vscode issue which had an 'L' before the number and indeed after adding it the reference works:

| [src\ServerConfig.ts](src\ServerConfig.ts#L301) | 301 | compress?

globbing/multiple files

I see that you have commented in your code a todo to support multiple files in the cli (well, in your code, it's marked as TOOD). I would love to work on this feature if you haven't already started. Is there any reason or complication you ran into that stopped you from working on it?

Skip unsupported file types

Rather than showing an error when globbing:

$ leasot src/**/*
✖ Filetype .mustache is unsupported.

Add support for PHP # comments.

The following are valid in PHP

TODO Add support for this ;)

Not sure where the best place would be. I see that PHP runs via a default config rather than its own, but this updated REGEX will work with multiple PHP scenarios.

var rLineComment = new RegExp('^(?:\\s*<\\?php)?\\s*(?:\\/\\/|#)' + regex + '(?:\\?>)?$', 'mig');

Works with

// TODO
# TODO
<?php // TODO ?>
<?php # TODO ?>
<div>
    <!-- As well as with indents for inline php in templates -->
    <?php # TODO Do we need this? ?>
    <div></div>
</div>

Looks like it may need its own config file though since those are pretty PHP specific.

Could easily be modified to check for <? php shorthand as well.

Extract the common parser patterns in regex

Will allow for more flexible addition of comment types and variations

Some broken edge cases in JavaScript files

A few edge cases in JavaScript seem not to be working as expected:

Input	Expected	Actual
// TODO	TODO found with text as empty string	No TODO found
/* TODO */	TODO found with text as empty string	TODO found with text "*"
/*TODO text */	TODO found with text "text"	No TODO found

How to --ignore multiple paths

Trying to figure out the easiest way to skip multiple paths

I can get one path just fine...

$ leasot "./**/*.js" --ignore "./node_modules/**/*" --tags error,info

Allow HTML tags in PHP Files

is it possible to allow HTML tags in .php files. Often have .php extension files that have a mixture of regular HTML code (using tags) as well as php code.

Can't run from git-hook

If I put this in my .git/hooks/pre-commit:

node_modules/.bin/leasot "path/to/file.js"

I get this when I try to commit:

TypeError: Path must be a string. Received undefined
    at assertPath (path.js:7:11)
    at Object.extname (path.js:1431:5)
    at getFiletype (.../node_modules/leasot/lib/cli.js:13:30)
    at parseContentSync (.../node_modules/leasot/lib/cli.js:19:15)
    at .../node_modules/leasot/lib/cli.js:94:25
    at process._tickCallback (internal/process/next_tick.js:103:7)

The same thing also happens if I install leasot globally and replace the above with leasot "path/to/file.js".

I can still run the script successfully with .git/hooks/pre-commit, which is strange, because git hooks are run as the same user doing the commit/push etc.

Add option to check inline comments?

In issue #49 the reason inline comments are not supported seems to be the risk of false-positives e.g. a string that contains text which looks like a comment.

Would you maybe accept a pull request that adds an option to include possible inline TODOs?
There could be a warning in the documentation about the risk of false-positives.

Alternatively could there be an option to allow an external parser, e.g. passing a function to associateExtWithParser?

var inlineParser = require('my-inline-parser');

[...].associateExtWithParser({
    '.py': {
        parser: inlineParser
    }
})

Which of these two choices would you prefer? Or do you have another possible solution?