mozilla/addons-linter

🔍 Firefox Add-ons linter, written in JavaScript. 👁

addons-linter's Issues

Try htmlparser2 in RDF

htmlparser2 has better test coverage and seems more robust, so let's try to use it for RDF parsing instead of XMLDom.

Setup eslint settings

Detect duplicate files in a zip

It's possible to create a zip with 2 files being the exact same name. We need to detect that.

Get ESLint to ignore /*eslint*/ comments

We need a way to either configure ESLint not to respect comments inside the code (usually in /* eslint nocheckforbadthing */-type comments).

If that's not possible, we'll need to strip out all comments when we parse code or something so the code can't tell the validator to back off 😄

Add text output processing.

We've got a start on outputting JSON - we need to add something to output the rules in a nice way as just text with nice colours assuming boring is false.

Add validation error/warning objects

From discussions we decide for the validation errors and warnings we need an object for each (they'll mostly be the same.

Carry out some basic CSS validation rules with the CSS parser

We're planning on using https://github.com/reworkcss/css for this. This bug is to stand-up some basic css checks with that parser.

Deal with known JS libs

• Check against known library hashes #383
• Build our own hash list #384
• Hook up Dispensary inside the linter

We need a way to skip over validation of known safe libs.

Setup babel for es6 -> es5 transpilation

Move RDF rules into their own files

Related to #78, we should keep our rules in single files (similarly to what we do with ESLint rules) so they are easy to find and we don't have massive validator classes full of methods.

I'll move the RDF ones that exist already.

Define list of layout tests

The list of rules covers a bunch of tests that relate to files and how they're laid-out in the zip (xpi). We should write a list of those and start implementing them.

Defend against zip-bombs

For use via amo it's important we can blow-up gracefully if a zip bomb is submitted for validation.

Work out requirements for parsing install.rdf

Port tests

This will need breaking down further.

Detect addon-type

We need to know what addon-type we are dealing with as some validation tests are type-specific.

Check for use of mozindexdb

Write a first rule for: mozindexdb_removed, mozIndexedDB has been removed

Add code and lib wrapper to unpack an .xpi for purposes of validation

• API for extracting metadata
• Api for extracting individual files and handing them off to the specific parser/validator.

Set-up test infrastructure.

We've discussed this being based around mocha + chai + plus a test runner.

Decide on CSS parser for CSS processing rules.

Let's investigate what's state of the art in CSS parsing for Node. Then the next step will be to prototype out something simple and double check the chosen solutions looks fit for purpose.

Plan error handling

• How should errors be handled?
• How does the current validator expose various classes of error.

Connect up express so we start building out an API.

Add express into the mix so we're starting to look at non-CLI based consumption of our code.

Initial steps are going to be to handle an incoming package and hand that off to the Validator.

Put scanners/validators in their own directory

Just a minor thought, but as we add separate scanners (CSS, JS, RDF), the root of src/ is getting crowded. It would be easier to see what files did what if they were grouped (ie put all scanners in a scanners/ folder.

So instead of:

- src/
  - messages/
  - rules/
  - cli.js
  - css.js
  - javascript.js
  - rdf.js
  [etc.]

we can do:

- src/
  - messages/
  - scanners/
    - css.js
    - javascript.js
    - rdf.js
  - rules/
  - cli.js
  [etc.]

Kick off a basic CLI to pass an .xpi to the unzipping lib.

Handling CSS parser failures

Deal with inability to process CSS due to parser errors.

Add binary file tests

There are a set of tests that read the first few bytes of a file to identify them.

Add a parser + single rule for chrome.manifest.

• Hook up something to read a chome manifest
• Connect a rule so that it's passing messages into the collector.

Add HTML parsing tests

There are existing HTML/Markup tests in the old validator, so we should hook up at least one to see what that looks like.

Test to make sure Message has required opts

It's possible to create a Message with an empty opts object, which we should just make sure to test against in the future when we decide on/add the required opts.

https://github.com/mozilla/addons-validator/pull/35/files#r40915475

Make sure ESLint doesn't fail with invalid JS (syntax_error rule)

This is a test case I hadn't considered in #44; I'll leave it here as a separate bug.

We should be sure bad JS like:

var = foo:

doesn't crash the validator.

Turns out this is the syntax_error rule. This is a good one to implement early I think.

Explore es6 gettext/translating template string.

So that we can do easy substitutions.

Deal with inline JS and script blocks when parsing the HTML

We're going to need to handle any JS we come across whilst processing the HTML files.

Make tests less noisy

Output from the tests is a bit noisy because we have console.log calls inside the code path that aren't silenced for tests.

Realistically, we probably want an validator.config.output option that is nothing or null.

Not sure when the CLI would ever want it, but it's what we want for the tests and we should already have the infra for it via that config.output option.

Other console.log should be silenced by this option. We probably just want to use our own log method everywhere.

Quick review of existing structure before we move onto adding more rules.

Before we go too far and add lots of rules I think it might be worth taking a step back briefly to look at what we have in terms of parsers and how they are integrated and make sure we are being as consistent in our approach as possible for each parser so far.

Kind of questions I have in mind are:

• Use of rule functions vs class methods.
• How are the message objects setup and passed back to the collector.

We can then file issues for any things that need to be changed if necessary.

Make some graphs/flowcharts showing processing architecture.

Document the high-level flow of how the validator processes and code flows through the various scanners.

Spec out output format for validation errors etc.

Presumably we're going to need to stick closely to the way the current validator outputs the results of a scan.

Add message collector

This classes job is to collect up warnings, errors and notice objects and provide an API for doing so.

Hook up rdf parser and an RDF related rule.

• Add the rdf parser along similar lines to the rest
• Add an example rule and pass messages to the collector.

Work out requirements for parsing chrome.manifest

Plan architecture for processing files.

Starting with the xpi:

• How with the various parsers and processors be glued together.
• We should look to ensure the integrity of the scans (making sure all files are processed).

Work out requirements for unzipping xpi

• What does node have for this?
• What API do we want? e.g. processing things as streams as much as poss?
• Any special considerations re: .xpi files?

Does the validator need to be localized?

I see the current validator isn't - but I wondered if that might make sense.

Add JSCS to tests

I've already made a few silly style nitpicks/raised some style questions in #20.

Shall we add some JS Style checks to the tests so we don't quibble over how we do long strings, line lengths, etc.?

It's worked well for localForage. Keeps styles in-line on the more fluid things eslint won't check.

Draft list of features

Test to make sure number of rules match rules that are run in scan()

Write a test that checks the number of rule files in rules/$VALIDATOR_TYPE (eg: all files sans index.js) and checks how many rules are run in $VALIDATOR_TYPE.scan(). We can test this for at least the HTML and RDF checkers now, and could check the length of the ESLint rule list as well.

This makes sure of two things:

1. All rule files contain exactly one rule.
2. All rules are being exported/imported and run properly.

Test for obsfucation/identify obsfucation attempts

Currently we test for identifiers (#44), as the previous validator does. This leaves us open to obsfucation of restricted identifiers, eg:

var m = "m";
var o = "o";
var z = "z";
var idb = "IndexedDB";
var tricksterVariable = m + o + z + idb;
// Bad!
var myDatabase = window[tricksterVariable];

We need to statically analyse variable paths and find out their eventual values if they're used to dynamically call a function.

Failing this: we need to identify heuristics that we can use to say that obfuscation appears likely and alert the reviewer for closer manual inspection.

Allow CLI parser to be run from a (Celery) queue

@andymckay, @mstriemer, and I talked about #49. What we actually need is to use the existing https://github.com/mozilla/olympia Django app as our API server (having multiple front ends is needless) to handle the Validator's API (eg: upload a file for validation -> get an ID -> query said ID for its validation status).

Do need to get the validator to run as a Celery task that can get data from the Django API (put into celery) and then send data back.

The API would return a status of the validation (processing/complete) with a series of messages/warnings/errors (as JSON, which the validator can already easily produce being a JS app).

Refactor HTML/RDF scanner classes

Looking at the HTML and RDF scanner classes, they're nearly the same now that I've started on #79. Might be nice to define a "markup" scanner class and have them simply extend it (essentially the only difference is the parser used and the rules they're using).

Support web extensions

There are going to be similar bugs in bugzilla for the existing validator, but basically whatever we need to do to make https://bugzilla.mozilla.org/show_bug.cgi?id=1210037 happen, we should repeat in this validator.

Deal with inline CSS and style elements when parsing HTML.

We're going to need a way to hand-off in-line CSS and style-blocks to the CSSParser when parsing the HTML

Connect CLI to zip parser