crimson-unicorn/parsers

Unicorn Parsers

This repository contains a number of parsers for different provenance capture systems. Please go to their respective directories for detailed documentation.

⚠️ cadets and cdm parsers are not up-to-date. Use with caution.

parsers's Issues

the difference between a stream graph and a base graph

What is the difference between a stream graph and a base graph? Are they all provenance graph?

how to get the benign or attack files of the cadets

In Makefile：

number=0 ; while [ $$number -le 8 ] ; do
cd ../../../data/cadets-e3/benign && mkdir cadets-e3-benign-$$number && tar zxvf cadets-e3-benign-$$number.gz.tar -C cadets-e3-benign-$$number && mv cadets-e3-benign-$$number/cadets-e3-benign.txt.* ../edgelists_benign ;
cd ../../../data/cadets-e3/benign && rm -f cadets-e3-benign-$$number.gz.tar && rm -rf cadets-e3-benign-$$number ;
number=expr $$number + 1 ;
done

How to get benign or attack files of the cadets(e.g. cadets-e3-benign-0.gz.tar)?

What should I do if there are no exceptions when creating the model? The dataset is the one given to the camflow

Traceback (most recent call last):
File "D:\modeler-master\model.py", line 256, in
precision, recall, accuracy, f_measure, printout = test_graphs(test_files, submodels, tm, ns)
File "D:\modeler-master\model.py", line 135, in test_graphs
test_info)
File "D:\modeler-master\helper\profile.py", line 257, in test_single_graph
max_abnormal_point = max(abnormal_point)
ValueError: max() arg is an empty sequence

Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__'

Hi! I have encountered the following error when trying to run provparser on a clean installation of Ubuntu 18.04 (run in a virtual machine).

[WARNING] Use '-t fivedirections' only for ta1-fivedirections-e3-official data. Exit now otherwise...
[+] initiating parser...
[i] multiprocessing support is on for processing but not scanning
[+] processing regular JSON files in directory data/...
[i] multiprocesses processing regular JSON files
[+] setting up database python1.log.db in current directory...
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/provparser/provparser", line 96, in process
    with open(os.path.join(args.input, fn), 'r') as fileobj:
IOError: [Errno 13] Permission denied: 'data/python1.log'
AttributeError: 'list' object has no attribute 'clear'
Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' ignored
[+] node parsing is done
[+] parsing files again to output final results.
[i] multiprocessing support is on...
[+] processing regular JSON files in directory data/...
[i] opening output file camflow-out.txt for writing...
Traceback (most recent call last):
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/testenv/bin/provparser", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/provparser/provparser", line 371, in <module>
    fileobj = open(os.path.join(args.input, sortedfilenames[0]), 'r')
IOError: [Errno 13] Permission denied: 'data/python1.log'
AttributeError: 'list' object has no attribute 'clear'
Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' ignored

Do you happen to have any suggestions on how to fix this, please?

It seems like somewhere in rocksdb they use a python3 function which is not compatible with the python2. In another VM I have also tried to make python3 the default version, and installed everything, but when I would try to run provparser I would get an error along the lines of prepare module is not found (I don't have the exact error text but I can redo the installation and paste the error code here)

How does unicorn handle the DARPA TC dataset?

I'm trying to process DARPA TC dataset now, but the text doesn't mention how the initial processing is done, can you give an idea?