fxb-cocacoding/yara-signator

Automatic YARA rule generation for Malpedia

yara-signator's Issues

[Bug] - Some rules writing false conditions

Several rules that exist in the malpedia generated ruleset will always evaluate to false.

Consider the following rule. 5 strings exist, but the condition calls for 7 of them.

rule win_samsam_auto
{
        meta:
                author = "Felix Bilstein - yara-signator at cocacoding dot com"
                date = "2020-04-21"
                version = "1"
                description = "autogenerated rule brought to you by yara-signator"
                tool = "yara-signator v0.3.1"
                malpedia_reference = "https://malpedia.caad.fkie.fraunhofer.de/details/win.samsam"
                malpedia_version = "20200421"
                malpedia_license = "CC BY-SA 4.0"
                malpedia_sharing = "TLP:WHITE"
                impact = "100"

        strings:
                $sequence_0 = { 082b c883e10f 03c1 1bc9 0bc1 59 e9???????? }
                $sequence_1 = { ec 8b4508 56 33f6 3bc6 751d e8???????? }
                $sequence_2 = { ec 83ec10 53 ff7510 8d4df0 e8???????? }
                $sequence_3 = { ec 83ec10 53 33db 56 57 }
                $sequence_4 = { ec 6a0a 6a00 ff7508 e8???????? }

        condition:
                7 of them and 
                filesize <483328
}

The following rules seem to have this bug

• win_zerot_w0
• win_zhmimikatz_auto
• win_samsam_auto

Error installing new version

This is what I got when I tried to update to the new version:

Results :

Failed tests:   testInsertIntoBlacklist(tests.postgres_test.BlacklistTest): expected:<0> but was:<4>

Tests in error: 
  tests.postgres_test.BlacklistTest: ERROR: current transaction is aborted, commands ignored until end of transaction block

Tests run: 3, Failures: 1, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.985 s
[INFO] Finished at: 2020-10-21T11:32:42+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project yara-signator: There are test failures.
[ERROR] 
[ERROR] Please refer to /{hidden}/yara-signator/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Prior to this I successfully updated the other components.

Compilation fails

I'm trying to build yara signator and I'm getting the following error:

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project yara-signator: Compilation failure
[ERROR] /home/seifreed/yara-signator/src/main/java/mongodb/MongoHandler.java:[15,22] package javax.xml.bind does not exist

Could you please help me?

Collations in Postgresql

If you run this version in a recent Ubuntu (20.04) in PostgreSQL, the range partition of the blacklist crashes:

[main] INFO postgres.HandleStructures - DROP TABLE IF EXISTS blacklist_list_rest;
[main] INFO postgres.HandleStructures - CREATE TABLE blacklist (concat TEXT UNIQUE NOT NULL) PARTITION BY RANGE (concat);
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_0 PARTITION OF blacklist FOR VALUES FROM ('0') TO ('0ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_0 ADD CONSTRAINT blacklist_list_0 CHECK (concat >= '0' AND concat <= '0ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_1 PARTITION OF blacklist FOR VALUES FROM ('1') TO ('1ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_1 ADD CONSTRAINT blacklist_list_1 CHECK (concat >= '1' AND concat <= '1ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_2 PARTITION OF blacklist FOR VALUES FROM ('2') TO ('2ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_2 ADD CONSTRAINT blacklist_list_2 CHECK (concat >= '2' AND concat <= '2ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_3 PARTITION OF blacklist FOR VALUES FROM ('3') TO ('3ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_3 ADD CONSTRAINT blacklist_list_3 CHECK (concat >= '3' AND concat <= '3ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_4 PARTITION OF blacklist FOR VALUES FROM ('4') TO ('4ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_4 ADD CONSTRAINT blacklist_list_4 CHECK (concat >= '4' AND concat <= '4ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_5 PARTITION OF blacklist FOR VALUES FROM ('5') TO ('5ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_5 ADD CONSTRAINT blacklist_list_5 CHECK (concat >= '5' AND concat <= '5ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_6 PARTITION OF blacklist FOR VALUES FROM ('6') TO ('6ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_6 ADD CONSTRAINT blacklist_list_6 CHECK (concat >= '6' AND concat <= '6ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_7 PARTITION OF blacklist FOR VALUES FROM ('7') TO ('7ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_7 ADD CONSTRAINT blacklist_list_7 CHECK (concat >= '7' AND concat <= '7ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_8 PARTITION OF blacklist FOR VALUES FROM ('8') TO ('8ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_8 ADD CONSTRAINT blacklist_list_8 CHECK (concat >= '8' AND concat <= '8ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_9 PARTITION OF blacklist FOR VALUES FROM ('9') TO ('9ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_9 ADD CONSTRAINT blacklist_list_9 CHECK (concat >= '9' AND concat <= '9ffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_a PARTITION OF blacklist FOR VALUES FROM ('a') TO ('affffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_a ADD CONSTRAINT blacklist_list_a CHECK (concat >= 'a' AND concat <= 'affffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_b PARTITION OF blacklist FOR VALUES FROM ('b') TO ('bffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_b ADD CONSTRAINT blacklist_list_b CHECK (concat >= 'b' AND concat <= 'bffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_c PARTITION OF blacklist FOR VALUES FROM ('c') TO ('cffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_c ADD CONSTRAINT blacklist_list_c CHECK (concat >= 'c' AND concat <= 'cffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_d PARTITION OF blacklist FOR VALUES FROM ('d') TO ('dffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_d ADD CONSTRAINT blacklist_list_d CHECK (concat >= 'd' AND concat <= 'dffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_e PARTITION OF blacklist FOR VALUES FROM ('e') TO ('effffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_e ADD CONSTRAINT blacklist_list_e CHECK (concat >= 'e' AND concat <= 'effffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_f PARTITION OF blacklist FOR VALUES FROM ('f') TO ('fffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - ALTER TABLE blacklist_list_f ADD CONSTRAINT blacklist_list_f CHECK (concat >= 'f' AND concat <= 'fffffffffffffffffffffffffffffffffffffffffffffffffffffffff' );
[main] INFO postgres.HandleStructures - CREATE TABLE IF NOT EXISTS blacklist_list_rest PARTITION OF blacklist FOR VALUES FROM ('?') TO ('?ffffffffffffffffffffffffffffffffffffffffffffffffffffffff');
org.postgresql.util.PSQLException: ERROR: partition "blacklist_list_rest" would overlap partition "blacklist_list_0"

Fix:
Add COLLATE "C" to the CREATE TABLE statement to enable ascii only comparison:

CREATE TABLE blacklist (concat TEXT UNIQUE NOT NULL collate "C") PARTITION BY RANGE (concat);

Will be fixed in the new version. Posted because missing collations may result in strange bugs on different platforms in future releases.

False positive

hi,

your rule elf_blackcat_auto hits on this one version of bat (https://github.com/sharkdp/bat):

https://www.virustotal.com/gui/file/c97ba484cc9a235a899808c32cf9549872437d4f0a27fa99e511d65fa1b9d054/detection

should be from this .deb: https://ubuntu.pkgs.org/22.04/ubuntu-updates-universe-amd64/bat_0.19.0-1ubuntu0.1_amd64.deb.html

doesn't hit on other bat binaries. is that false positive or trojanized?

regards
arnim

Filter sequences that will likely yield YARA atoms of low quality

Rules generated with yara-signator may trigger a warning by YARA that certain sequences are of inferior quality:

$ yarac test.yar test.yac
test.yar(3): warning: $sequence_0 is slowing down scanning
$ grep sequence_0 test.yar
  $sequence_0 = { 68???????? e8???????? 68???????? e8???????? bb???????? 53 }

This warning is triggered when sequences do not fulfill the minimum quality for atoms (i.e. matching terms used in YARA).
The default minimum atom length according to the YARA source code is 4.
I think the warning message could be avoided if only such sequences are selected by yara-signator that have a minimum sequence of 3 or 4 concrete byte values as a continuous sequence.
For the above example, instead going for a sequence like

$sequence_0 = { 68???????? e8???????? bb???????? 53 52 51}

already does the trick.
This requirement could be easily added as a further filter stage.

malpediaEvalScriptOutput file not being written to, causing null pointer exceptions

Hi, I keep getting null pointer exceptions from the function getNgramsForFamily_NextGen_CandidateOne, in NgramCreator. I believe this is due to data not being written to the malpediaEvalScriptOutput file that I have specified in yarasignator.conf. I checked the permissions for this file and they are not an issue. If you have run into this problem before and know how to fix it, or if you could point me in the right direction to check (for example, which class or function is responsible for writing to this file), it would be greatly appreciated.

Thank you