[BUG] HTML/JavaScript recursion

[target]

[BUG] HTML/JavaScript recursion

[ryanohoro]

I identified cases where this recursion was happening by looking at file.depth:15 (default limit). The frequency is extremely low (0.00003%). The attached file, a VIM macro, triggers this bug.

less.vim.txt

[ryanohoro]

Analyzing a large volume of events, it's apparent the mime type matching for text/html is overly zealous.

html_file: 1.05
text/html: 1.6
both: 1

I see two solutions:

1. Remove the text/html mime type from the default ScanHtml configuration.

   While analyzing the data on this problem, it seems most of what text/html catches, but html_file does not is either not HTML or is broken HTML (from split or partial responses). Some exceptions are things like HTML files that start with white space or comments, which can be addressed by improving the html_file Yara.

2. Prevent ScanHtml from being a child (source) of itself.

   This will prevent the recursion problem, and may be applicable in some other situations if implemented as a configuration. Some scanners should normally recurse. However, it won't prevent mostly unhelpful analysis of files that will not yield interesting results.

   e.g.

  'ScanHtml':
    - positive:
        flavors:
          - 'hta_file'
          - 'text/html'
          - 'html_file'
      exclude_sources:
          - ScanHtml
      priority: 5
      options:
        parser: "html5lib"

The attached file triggers the javascript variety of this bug.

search.js.txt