maecproject/threatexpert-to-maec

Generate MAEC XML from ThreatExpert XML output

threatexpert-to-maec's Issues

Update to use argparse

We should update the script to use the Python argparse library for parsing of command line arguments.

Add Used Options as XML Comment

We should add the options used in the execution of the script (e.g. deduplication, dereferencing, etc.) as an XML comment at the top of the MAEC output document.

Update ID generation based on latest python-maec updates

Based on the latest python-maec updates, we'll need to update the following places to use "id_" instead of "id":

https://github.com/MAECProject/threatexpert-to-maec/blob/master/threatexpert_parser.py#L133

Add Deduplicator Support

We should the option to natively deduplicate the output MAEC Package, probably disabled by default?

REAME file missing from installation

An attempt to install the latest package using setup.py generates the error "IOError: [Errno 2] No such file or directory: 'README'". This is because the README file is missing from the distribution. Instead there is a README.rst file.

Fix namespace declaration

To retain compatibility with the recent python-maec changes, we need to update the namespace declaration for the TE -> MAEC script so that it follows the syntax of (alias,namespace) rather than (namespace,alias).

Using context to better define URIObj

In looking through the generated output the various uses of URIObj, it appears that the type of the URIObj is not set. Might I suggest that the context in which occurs could be used to determine the type, where if it can't be determined the behavior is as it is today. For example, if URIObj is being generated as a result of an action such as 'connect to url', the URIObj type should be able to be set to 'URL' where as in the case of an action 'get host by name' its not as clear what the value is.

Another possibility would be to test if the value is a URL of some kind. Perhaps something like rfc3987.parse( value ) without validation rules and then checking to see if there is a scheme provided would provide sufficient hints.

Enable optional file-submit behavior

The module's generate_package_from_binary_filepath method should support an option to automatically submit the binary to ThreatExpert when it does not exist in ThreatExpert's records.

Unfortunately, there doesn't seem to be a way to use that submission result immediately, because submissions are processed asynchronously. (We could poll to see if it's done yet, but that forces repeated pings against the service, and processing could take a long time.) Even so, this option would still be useful as an auto-submit feature.