infosec-intern/vscode-yara

VSCode extension for the YARA pattern matching language

YARA for Visual Studio Code

Language support for the YARA pattern matching language

Check out the project wiki for more information

Screenshot

Features

This extension provides many features common to code editors, such as

• Rule and Identifier Definitions
• Rule and Identifier References
• Module Code Completion
• Hovers for Hex Strings

Snippets

This extension provides some text snippets, which allows users to auto-complete certain common YARA rule patterns.

More information can be found on the Snippets wiki page.

Problems?

If you encounter an issue with the syntax, feel free to create an issue or pull request!

Alternatively, check out some of the YARA syntaxes for Sublime and Atom, such as blacktop's excellent language-yara syntax.

YARA Documentation

https://yara.readthedocs.io/

vscode-yara's Issues

Rename repo

I think the repo name can be updated.

String Highlighting Incorrect (Greedy)

Problem
Syntax highlighting for strings is incorrect. It performs greedy search for the closing quote character.
The screenshot below shows the issue on VS Code:

Solution
Change the regular expression for normal strings to perform non-greedy search for the closing quote character.

Comments in grouping

Syntax highlight of comments inside of a grouping doesn't work. It thinks it's part of the grouping instead of a comment. Below it an example rule and what it looks like.

rule test {
    meta:
        author = "Author"
        description = "Description"

    strings:
        $instructions = {
            72 ?? ?? ?? ??  // IL_0000: ldstr
            80 ?? ?? 00 04  // IL_0005: stsfld
            (
                72 ?? ?? ?? ?? // Comment inside of a grouping

            )
        }
        
    condition:
        all of them
        
}

Add code completion for modules

YARA's module system should have some kind of completion system for helping a user figure out what attributes are available

Examples of available attributes

pe.entry_point == 0x1000
cuckoo.http_request(/someregexp/)

Documentation:
https://code.visualstudio.com/docs/extensionAPI/language-support#_show-code-completion-proposals

Hex string highlights invalid non-hex characters

Current basic hex string regex highlights any characters between = { }

Expected: only highlight valid hex string chars a-fA-F0-9?
This will help with identifying invalid strings in rules

Automatically publish packages to the VSCode and OpenVSX marketplace

Once the CI tests are worked out, and this new version of the YARA extension is ready for release, use GitHub Actions to auto-publish updates to both marketplaces

Invalid regex in grammar: lookbehind assertion is not fixed length

👋 I'm the lead maintainer of the https://github.com/github/linguist library which is used for language detection and providing the syntax highlighting for languages on GitHub.com, and we use this grammar.

Our grammar compiler has found a problem with your grammar which I thought I'd let you know about.

This regex is invalid as the lookbehind assertion is not a fixed length (offset 25):

vscode-yara/yara/syntaxes/yara.tmLanguage.json

Line 375 in abd7b64

"match": "(?<=(^|[\\)]|\\b(?:them)\\b))(?:\\s*):(?=\\s*(\\(|$))"

https://regex101.com/r/v6bHBI/1

This is the error our compiler reported:

Invalid regex in grammar: source.yara (in yara/syntaxes/yara.tmLanguage.json) contains a malformed regex (regex "(?<=(^|[\)]|\b(?:them)\b))(?:\s*...": lookbehind assertion is not fixed length (at offset 25))

Unittests: First test called before UpdateSettings()

Need to either manually call updateSettings(), which requires an extensioncontext, or some way of having it called. This isn't an issue when manually running tests. Previously, the Yara object took care of calling updateSettings() itself

Configuration-based metadata

Idea time: Translate configuration entries into rule metadata

Possible entries and translations:

"metadata": {
  "author": "infosec-intern",
  "confidence": 100,
  "reference": "",
  "created": "$now",          // maybe I'll calculate and add this regardless when a new .yara file is created
  "modified": "$now"         // updated on every save
}

Return Promises for Diagnostics

Need this because we're spawning asyn processes which may not return immediately

Hopefully this helps the unit testing as well

Syntax Bugs

Escape Sequences

The invalid.illegal.missing.escape.yara regex is incorrectly matching against escape sequences (\n, \t, etc.) and not identifying invalid escapes when placed next to a correctly escaped slash.

Note: ASCII Escape Sequences

Current Logic

{
    "name": "invalid.illegal.missing.escape.yara",
    "match": "[^\\\\]\\\\[^\\\\\"]"
}

Example rule from signature-base

Test strings from invalid.yar

Hex Jumps

The constant.numeric.jump.range.yara matcher doesn't identify constant jumps (e.g. [4]) as numeric due to the dash requirement

Current Logic

{
    "name": "constant.numeric.jump.range.yara",
    "begin": "\\[",
    "end": "\\]",
    "match": "[0-9]*-[0-9]*"
}

Example rule from signature-base

Reference Provider has trouble with certain string variables

Reference Provider looks like it has trouble with certain string variable lengths.

Example:

strings: $x1 = "Error = Process.Create(\"powershell -nop cmd.exe /c" fullword ascii

"x1" will not engage the reference provider and display any references.

private rules do not support goto definition, peek definition, ctrl+click

Goto definition, peek definition, ctrl+click etc do not work with private rules.

Example below: ctrl+click on the condition my_private_rule will not take you to the definition of my_private_rule

private rule my_private_rule
{
    condition:
        true
}

rule my_public_rule
{
    condition:
        my_private_rule
}

Provide Diagnostics data

compileRule attempts to run at every file save - not just .yara files. Observed when editing the settings.json files

Research and add FileSystemProvider support if needed

Version 1.23.0 of the API added FileSystem Providers, which allow us to treat any filesystem source as if it were on-disk

Improved ExecuteRule output

An information messagebox isn't good enough

YARA output on a successful rule detection is like this:
${Rulename} ${Target filename}

Better platform independence

Filepaths need to be corrected and OS detection needs to be put in place

Meta keyword loses coloring

"meta:" will lose its coloring if anything but a line break follows it

Add YARA command-line options to Settings

YARA Help Menu

YARA 3.5.0, the pattern matching swiss army knife.
Usage: yara [OPTION]... RULES_FILE FILE | DIR | PID

Mandatory arguments to long options are mandatory for short options too.

  -t,  --tag=TAG                   print only rules tagged as TAG
  -i,  --identifier=IDENTIFIER     print only rules named IDENTIFIER
  -n,  --negate                    print only not satisfied rules (negate)
  -D,  --print-module-data         print module data
  -g,  --print-tags                print tags
  -m,  --print-meta                print metadata
  -s,  --print-strings             print matching strings
  -e,  --print-namespace           print rules' namespace
  -p,  --threads=NUMBER            use the specified NUMBER of threads to scan a directory
  -l,  --max-rules=NUMBER          abort scanning after matching a NUMBER of rules
  -d VAR=VALUE                     define external variable
  -x MODULE=FILE                   pass FILE's content as extra data to MODULE
  -a,  --timeout=SECONDS           abort scanning after the given number of SECONDS
  -k,  --stack-size=SLOTS          set maximum stack size (default=16384)
  -r,  --recursive                 recursively search directories
  -f,  --fast-scan                 fast matching mode
  -w,  --no-warnings               disable warnings
  -v,  --version                   show version information
  -h,  --help                      show this help and exit

Send bug reports and suggestions to: [email protected].

YARAC Help Menu

Usage: yarac [OPTION]... [NAMESPACE:]SOURCE_FILE... OUTPUT_FILE

  -d VAR=VALUE           define external variable
  -w,  --no-warnings     disable warnings
  -v,  --version         show version information
  -h,  --help            show this help and exit

Send bug reports and suggestions to: [email protected].

Highlight invalid/missing escapes as errors

Would be helpful to highlight invalid sequences, e.g. $ = "C:\Users" as invalid/error syntax .
including #26

thanks!

'Or' does not get colored as keyword.other.yara

Add extension output channel

So user can has more visibility into what's happening, and I can ask for logs if something is going wrong

Add parsing for rule references

http://yara.readthedocs.io/en/v3.7.1/writingrules.html#referencing-other-rules

Goto Definition for variables

Currently I'm only providing definitions for rules. If a symbol starts with one of 4 characters, then treat it as a variable and look for the line $variable =

$, #, @, !

Reference: http://yara.readthedocs.io/en/v3.7.0/writingrules.html#conditions

Make some decent images

The extension's undergone a lot of changes over the last few months, but it's still using many of the same images I created years ago. Gotta spice up the wiki and add some pizazz with fancy animated GIFs or at least up to date syntax highlighting

Web Extension shows syntax, but no language features like Definitons, References, etc.

Testing on Firefox v97.0 Linux & Chrome v98.0 macOS

Worked in development, so I'm not sure why the features aren't registering. There might be a coverage gap in my web extension tests

Greedy regex

Similarly to issue #13, the regex regex is too greedy with single-line comments

Support Web Extensions

https://code.visualstudio.com/api/extension-guides/web-extensions

Install YARA binaries?

Install them along with the extension? Or just don't register commands when YARA isn't installed?

ExecuteRule fails silently

Instead of detecting errors like CompileRule does, this just straight up fails with no good message

Operators aren't colored

Includes:
Logical : and | or | not
Relational : > | < | == | !=
Probably more

Broken Syntax Highlighting

The syntax highlighting is broken for me and others since a few days.
Could you please check?

it broke with version 1.6.2

Additional Syntax Opportunities

Modules

Probably something along the lines of support.class or support.function according to this textmate documentation

Off the top of my head, there are a few parts that need matching:

• Module name (pe, elf, cuckoo, etc.)
• Constants (pe.DLL)
• Functions (pe.imports(''))
• Arrays/Dictionaries (pe.version_info[''])

And any of the entries that need strings (e.g. dictionaries) should mark those as string.quoted.double just like regular strings

Rule Tags

Should be pretty straightforward - any words after a colon (:) on the same line as a rule

rule Test : Foo Bar Baz

Open VSX Listing: Signing the Publisher Agreement

Thank you for being part of the Open VSX community by adding your extensions to the Open VSX Registry. Please note that the service was recently transferred to the Eclipse Foundation and urgent action on your part is needed so we can continue to list your extensions. To ensure uninterrupted service, please sign the Eclipse Publisher Agreement on or before January 8, 2021. If not signed by that date, your extensions will be delisted and will no longer appear on the site nor be available via the API. If you sign at a later date, your extensions will then be re-activated. The signing process is explained in the Wiki (steps 1 and 2).

Please also note that all extensions MUST have a license in order to be listed.

More details are in these recent blog posts:
https://blogs.eclipse.org/post/brian-king/open-vsx-registry-under-new-management
https://blogs.eclipse.org/post/brian-king/new-era-open-vsx-registry

Today, there’s growing momentum around open source tools and technologies that support Visual Studio (VS) Code extensions. Leading global organizations are adopting these tools and technologies. This momentum has spurred demand for a marketplace without restrictions and limitations. Thanks for joining us on this journey as we continue to build the Open VSX community. We look forward to continued innovation from you in 2021!

Incorrect highlighting for boolean values

Regexp for (true|false) still applies when any word contains (true|false), so 'sfalse' or 'trues' will highlight (true|false) inside the word

Fix "Publish to VSCode" action

The publish action seems to deploy to OpenVSX just fine, but keeps getting a 401 error on VSCode's marketplace. Probably just an outdated token, but I need to address that at some point

https://github.com/infosec-intern/vscode-yara/runs/3110708909?check_suite_focus=true

Please remove "rule" snippet

Hey,

I love the YARA syntax highlighting library you provide and thanks for coding it!

In more recent versions you have added a snippet for a rule like this:

{
"Rule": {
"prefix": "rule",
"body": [
"/",
"\tRULE DESCRIPTION HERE",
"/",
"rule ${name}",
"{",
"\tcondition:",
"\t\t$1",
"}"
],
"description": "Generate a basic skeleton"
},
"Meta": {
"prefix": "meta",
"body": [
"meta:",
"\t${metadata}"
],
"description": "Generate a 'meta' section"
},
"Strings": {
"prefix": "strings",
"body": [
"strings:",
"\t${string(s)}"
],
"description": "Generate a 'strings' section"
},
"Import": {
"prefix": "import",
"body": "import "${module}"",
"description": "Import a YARA module"
}
}

Whenever VSCode updates, this overrides the default snippet I had created myself for rules which prepopulates metadata sections etc.

Would you consider removing the rule snippet, and instead leave it to users to define their own snippets?

Cheers,
Tom

References for wildcard identifiers

The following rule should return references for $create0 and $create1 if $create* is used. I'll probably only limit this to the local rule scope though so it doesn't go too crazy

rule suspicious_creation : PDF raw
{
	meta:
		author = "Glenn Edwards (@hiddenillusion)"
		version = "0.1"
		weight = 2

	strings:
		$magic = { 25 50 44 46 }
		$header = /%PDF-1\.(3|4|6)/

		$create0 = /CreationDate \(D:20101015142358\)/
		$create1 = /CreationDate \(2008312053854\)/
	condition:
		$magic at 0 and $header and 1 of ($create*)
}

Avast YLS

Avast released a Language Server for Yara: https://github.com/avast/yls
Do you think we should implement it? It seems to be actively developed which this extension could profit from.

Implementation doesn't seem to hard. Avast shows a example in their extension: https://github.com/avast/yls/tree/master/editors/vscode

The first step of installing the Server could be done in an command at startup, which also checks if its up to date. The python extension does that in a similar way.