Hi, I noticed that some events (specifically from hailataxii) are st

They weren't just array values They were <code cla

I also encountered this when testing with STIX samples from <a href="https://stix.mitr

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

events with No attributes,about misp/misp-taxii-server

Comments (36)

FloatingGhost commented on June 23, 2024 3

They weren't just array values

They were

STRINGS

CONTAINING AN ARRAY REPRESENTATION

Haha kill me

from misp-taxii-server.

FloatingGhost commented on June 23, 2024 2

Ohhhhh, I hate STIX so much.

They're array values to indicators.

ARRAY VALUES.

I want to die.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

https://github.com/MISP/MISP-Taxii-Server/blob/master/misp_taxii_hooks/hooks.py#L66

We only add the event if it has more than 1 attrib. Already implemented. Have not been able to replicate.

from misp-taxii-server.

mastarux commented on June 23, 2024

to replicate, you can try to pull the hailataxii feed (cybertracker) out of the 3.7K events it would create few without an attribute.

from misp-taxii-server.

combobulator commented on June 23, 2024

I also encountered this when testing with STIX samples from https://stix.mitre.org/language/version1.1.1/samples.html
Specifically, the Domain Watchlist sample.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

MISP rejects them as being invalid, so you get the event with 0 attribs

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Ok that should fix it.

Pushed changed to PyMISP, MISP-STIX-Converter and this repo

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

5 days and nothing else posted, assuming fixed

Just tell me if it isn't

it should be

from misp-taxii-server.

Danko90 commented on June 23, 2024

Hi @FloatingGhost,
Sorry I just updated and now I receive this error while trying to pull from the repo

{"logger": "opentaxii.middleware", "exception": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 104, in load_stix\n    stix_package = STIXPackage.from_json(data)\n  File \"/usr/local/lib/python3.4/dist-packages/mixbox/entities.py\", line 486, in from_json\n    d = json.loads(json_doc)\n  File \"/usr/lib/python3.4/json/__init__.py\", line 318, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib/python3.4/json/decoder.py\", line 343, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib/python3.4/json/decoder.py\", line 361, in raw_decode\n    raise ValueError(errmsg(\"Expecting value\", s, err.value)) from None\nValueError: Expecting value: line 1 column 1 (char 0)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 110, in load_stix\n    stix_package = STIXPackage.from_xml(stix)\n  File \"/usr/local/lib/python3.4/dist-packages/stix/core/stix_package.py\", line 249, in from_xml\n    return entity_parser.parse_xml(xml_file, encoding=encoding)\n  File \"/usr/local/lib/python3.4/dist-packages/mixbox/parser.py\", line 179, in parse_xml\n    xml_etree = get_etree(xml_file, encoding=encoding)\n  File \"/usr/local/lib/python3.4/dist-packages/mixbox/xml.py\", line 55, in get_etree\n    return etree.parse(doc, parser=parser)\n  File \"src/lxml/lxml.etree.pyx\", line 3442, in lxml.etree.parse (src/lxml/lxml.etree.c:81716)\n  File \"src/lxml/parser.pxi\", line 1832, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:118903)\n  File \"src/lxml/parser.pxi\", line 1852, in lxml.etree._parseFilelikeDocument (src/lxml/lxml.etree.c:119186)\n  File \"src/lxml/parser.pxi\", line 1747, in lxml.etree._parseDocFromFilelike (src/lxml/lxml.etree.c:117974)\n  File \"src/lxml/parser.pxi\", line 1162, in lxml.etree._BaseParser._parseDocFromFilelike (src/lxml/lxml.etree.c:112701)\n  File \"src/lxml/parser.pxi\", line 595, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:105896)\n  File \"src/lxml/parser.pxi\", line 706, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:107604)\n  File \"src/lxml/parser.pxi\", line 635, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:106458)\n  File \"<string>\", line 5\nlxml.etree.XMLSyntaxError: Opening and ending tag mismatch: Handling line 3 and Handling, line 5, column 34\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/abstract.py\", line 83, in process\n    response_message = handler.handle_message(self, message)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py\", line 126, in handle_message\n    return InboxMessage11Handler.handle_message(service, request)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py\", line 65, in handle_message\n    inbox_message_id=inbox_message.id if inbox_message else None)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/persistence/manager.py\", line 164, in create_content\n    collection_ids=collection_ids, service_id=service_id)\n  File \"/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py\", line 267, in send\n    for receiver in self.receivers_for(sender)]\n  File \"/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py\", line 267, in <listcomp>\n    for receiver in self.receivers_for(sender)]\n  File \"/usr/local/lib/python3.4/dist-packages/misp_taxii_hooks-0.2-py3.4.egg/misp_taxii_hooks/hooks.py\", line 62, in post_stix\n    package = pymisp.tools.stix.load_stix(content_block.content)\n  File \"/usr/local/lib/python3.4/dist-packages/pymisp/tools/stix.py\", line 17, in load_stix\n    stix = convert.load_stix(stix)\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 136, in load_stix\n    return load_stix(f)\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 113, in load_stix\n    raise STIXLoadError(\"Could not load stix file. {}\".format(ex))\nmisp_stix_converter.errors.STIXLoadError: Could not load stix file. Opening and ending tag mismatch: Handling line 3 and Handling, line 5, column 34 (<string>, line 5)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.4/dist-packages/flask/app.py\", line 1612, in full_dispatch_request\n    rv = self.dispatch_request()\n  File \"/usr/local/lib/python3.4/dist-packages/flask/app.py\", line 1598, in dispatch_request\n    return self.view_functions[rule.endpoint](**req.view_args)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/middleware.py\", line 76, in wrapper\n    return _process_with_service(service)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/middleware.py\", line 154, in _process_with_service\n    response_message = service.process(request.headers, taxii_message)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/abstract.py\", line 89, in process\n    in_response_to=message.message_id)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/exceptions.py\", line 48, in raise_failure\n    tb=tb)\n  File \"/usr/local/lib/python3.4/dist-packages/six.py\", line 685, in reraise\n    raise value.with_traceback(tb)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/abstract.py\", line 83, in process\n    response_message = handler.handle_message(self, message)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py\", line 126, in handle_message\n    return InboxMessage11Handler.handle_message(service, request)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py\", line 65, in handle_message\n    inbox_message_id=inbox_message.id if inbox_message else None)\n  File \"/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/persistence/manager.py\", line 164, in create_content\n    collection_ids=collection_ids, service_id=service_id)\n  File \"/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py\", line 267, in send\n    for receiver in self.receivers_for(sender)]\n  File \"/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py\", line 267, in <listcomp>\n    for receiver in self.receivers_for(sender)]\n  File \"/usr/local/lib/python3.4/dist-packages/misp_taxii_hooks-0.2-py3.4.egg/misp_taxii_hooks/hooks.py\", line 62, in post_stix\n    package = pymisp.tools.stix.load_stix(content_block.content)\n  File \"/usr/local/lib/python3.4/dist-packages/pymisp/tools/stix.py\", line 17, in load_stix\n    stix = convert.load_stix(stix)\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 136, in load_stix\n    return load_stix(f)\n  File \"/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py\", line 113, in load_stix\n    raise STIXLoadError(\"Could not load stix file. {}\".format(ex))\nopentaxii.taxii.exceptions.FailureStatus: Could not load stix file. Opening and ending tag mismatch: Handling line 3 and Handling, line 5, column 34 (<string>, line 5)", "event": "Status exception", "timestamp": "2017-06-28T13:49:46.116519Z", "level": "warning"}

[FloatingGhost Edit]
Extracted error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/middleware.py", line 76, in wrapper
    return _process_with_service(service)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/middleware.py", line 154, in _process_with_service
    response_message = service.process(request.headers, taxii_message)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/abstract.py", line 89, in process
    in_response_to=message.message_id)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/exceptions.py", line 48, in raise_failure
    tb=tb)
  File "/usr/local/lib/python3.4/dist-packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/abstract.py", line 83, in process
    response_message = handler.handle_message(self, message)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py", line 126, in handle_message
    return InboxMessage11Handler.handle_message(service, request)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/taxii/services/handlers/inbox_message_handlers.py", line 65, in handle_message
    inbox_message_id=inbox_message.id if inbox_message else None)
  File "/usr/local/lib/python3.4/dist-packages/opentaxii-0.1.10a1-py3.4.egg/opentaxii/persistence/manager.py", line 164, in create_content
    collection_ids=collection_ids, service_id=service_id)
  File "/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py", line 267, in send
    for receiver in self.receivers_for(sender)]
  File "/usr/local/lib/python3.4/dist-packages/blinker-1.4-py3.4.egg/blinker/base.py", line 267, in <listcomp>
    for receiver in self.receivers_for(sender)]
  File "/usr/local/lib/python3.4/dist-packages/misp_taxii_hooks-0.2-py3.4.egg/misp_taxii_hooks/hooks.py", line 62, in post_stix
    package = pymisp.tools.stix.load_stix(content_block.content)
  File "/usr/local/lib/python3.4/dist-packages/pymisp/tools/stix.py", line 17, in load_stix
    stix = convert.load_stix(stix)
  File "/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py", line 136, in load_stix
    return load_stix(f)
  File "/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py", line 113, in load_stix
    raise STIXLoadError("Could not load stix file. {}".format(ex))
opentaxii.taxii.exceptions.FailureStatus: Could not load stix file. Opening and ending tag mismatch: Handling line 3 and Handling, line 5, column 34 (<string>, line 5)

This is the XML block which causes the error

<stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-886c7ae0-16d6-46ff-ba61-8f0733cb893b" version="1.1.1" timestamp="2017-06-28T13:49:39.694821+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
                <marking:Marking_Structure xsi:type="tlpMarking:TLPMarkingStructureType" color="AMBER"/>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Indicators>
        <stix:Indicator id="fsisac:indicator-bfade6ee-f12f-4082-af80-8427b2bb923d" timestamp="2015-04-02T23:38:12.625608+00:00" xsi:type="indicator:IndicatorType">
            <indicator:Title>"UK Fuels ebill for ISO Week 201512" Phishing E-mail with 22328_201512.doc</indicator:Title>
            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">Malicious E-mail</indicator:Type>
            <indicator:Description>UK Fuels ebill for ISO Week 201512 22328_201512.doc emails with an attached word document or Excel XLS spreadsheet containing a macro.

Email Subject:
UK Fuels ebill for ISO Week 201512

ebillinvoice.com or UKL Fuels Ltd have not been hacked or had their email or other servers compromised.</indicator:Description>
            <indicator:Short_Description>UK Fuels ebill for ISO Week contains Word doc or Excel XLS spreadsheet containing a macro</indicator:Short_Description>
            <indicator:Observable idref="fsisac:observable-0275497a-a873-4771-89d3-fc8749a70d15">
            </indicator:Observable>
            <indicator:Handling>
                <marking:Marking>
                    <marking:Controlled_Structure>../../../descendant-or-self::node()</marking:Controlled_Structure>
                    <marking:Marking_Structure xsi:type="tlpMarking:TLPMarkingStructureType" color="AMBER"/>
                </marking:Marking>
            </indicator:Handling>
            <indicator:Confidence timestamp="2015-04-02T23:38:12.625640+00:00">
                <stixCommon:Value xsi:type="stixVocabs:HighMediumLowVocab-1.0">Medium</stixCommon:Value>
            </indicator:Confidence>
        </stix:Indicator>
    </stix:Indicators>
</stix:STIX_Package>

I updated MISP-TAXII-Server, Stix-Converter, PyMISP. The XML seems to be valid.

Thanks

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Seems like it all works, no attrs besides the original document, but that's expected behaviour

I may have edited over your edit whilst extracting the error :P

from misp-taxii-server.

Danko90 commented on June 23, 2024

UPDATE: Update another time this repo, now it doesn't crash but any event is being created, got the same error plus this one:

invalid syntax (<unknown>, line 1)
'cm9vdDpyb290' 12
{"level": "debug", "message_type": "Inbox_Message", "event": "Processing message", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "timestamp": "2017-06-28T14:49:39.839406Z", "message_id": "3abbdc3b-73d2-4869-bcef-3c35b42498cf", "logger": "opentaxii.taxii.services.inbox.InboxService", "service_id": "inbox"}
{"event": "Content block added to collections", "content_block": 348916, "timestamp": "2017-06-28T14:49:39.853363Z", "logger": "opentaxii.persistence.sqldb.api", "level": "debug", "collections": 1}
Building Event...
STIX Import
invalid syntax (<unknown>, line 1)

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

heh, seems my regex was a little hungry. Lemme satiate it a bit.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Try that! Pushed an update to the converter

from misp-taxii-server.

Danko90 commented on June 23, 2024

Hi!
Tried but it doesn't work yet.. Same errors :(

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Then I cannot replicate.

It works here and passes all tests.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Tests

Your XML sample from above was used in a test.

It passes just fine.

from misp-taxii-server.

Danko90 commented on June 23, 2024

This is what I see

'cm9vdDpyb290' 12
{"service_id": "inbox", "message_type": "Inbox_Message", "message_id": "c105dea2-6f9a-4395-8f92-2aca061ca5d4", "timestamp": "2017-06-29T08:33:11.321614Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "event": "Processing message"}
{"content_block": 349028, "logger": "opentaxii.persistence.sqldb.api", "event": "Content block added to collections", "timestamp": "2017-06-29T08:33:11.336926Z", "level": "debug", "collections": 1}
Building Event...
STIX Import
unexpected EOF while parsing (<unknown>, line 1)
'cm9vdDpyb290' 12
{"service_id": "inbox", "message_type": "Inbox_Message", "message_id": "1cbea9a1-fef6-4fae-a204-96489249b07f", "timestamp": "2017-06-29T08:33:11.429857Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "event": "Processing message"}
{"content_block": 349029, "logger": "opentaxii.persistence.sqldb.api", "event": "Content block added to collections", "timestamp": "2017-06-29T08:33:11.446218Z", "level": "debug", "collections": 1}
Building Event...
STIX Import
invalid syntax (<unknown>, line 1)
'cm9vdDpyb290' 12
{"service_id": "inbox", "message_type": "Inbox_Message", "message_id": "dd9304cd-12db-4c97-9e60-0760dd8708cd", "timestamp": "2017-06-29T08:33:11.524451Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "event": "Processing message"}
{"content_block": 349030, "logger": "opentaxii.persistence.sqldb.api", "event": "Content block added to collections", "timestamp": "2017-06-29T08:33:11.537768Z", "level": "debug", "collections": 1}
Building Event...
STIX Import
unexpected EOF while parsing (<unknown>, line 1)
'cm9vdDpyb290' 12
{"service_id": "inbox", "message_type": "Inbox_Message", "message_id": "8f167576-23c2-48cb-93ab-9420562fe6dc", "timestamp": "2017-06-29T08:33:11.616901Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "event": "Processing message"}
{"content_block": 349031, "logger": "opentaxii.persistence.sqldb.api", "event": "Content block added to collections", "timestamp": "2017-06-29T08:33:11.632920Z", "level": "debug", "collections": 1}
Building Event...
STIX Import
invalid syntax (<unknown>, line 1)
'cm9vdDpyb290' 12
{"service_id": "inbox", "message_type": "Inbox_Message", "message_id": "e46c8faa-53f5-406b-8540-6ff15865dd12", "timestamp": "2017-06-29T08:33:11.713389Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "event": "Processing message"}
{"content_block": 349032, "logger": "opentaxii.persistence.sqldb.api", "event": "Content block added to collections", "timestamp": "2017-06-29T08:33:11.729020Z", "level": "debug", "collections": 1}
Building Event...
STIX Import
invalid syntax (<unknown>, line 1)
CHECKING foo.doc
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 13440

I'm going to print the content block so I can paste it here

from misp-taxii-server.

Danko90 commented on June 23, 2024

This are the XML Blocks

Building Event...
STIX Import
'cm9vdDpyb290' 12
{"message_type": "Inbox_Message", "service_id": "inbox", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "message_id": "1bd44093-b101-4e2e-80bd-7c79faaff703", "logger": "opentaxii.taxii.services.inbox.InboxService", "event": "Processing message", "timestamp": "2017-06-29T09:01:33.608727Z"}
{"logger": "opentaxii.persistence.sqldb.api", "content_block": 349064, "level": "debug", "timestamp": "2017-06-29T09:01:33.622836Z", "collections": 1, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:URIObj="http://cybox.mitre.org/objects#URIObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-c3f80a56-00ad-4889-b963-d3eb93f83242" version="1.1.1" timestamp="2017-06-29T09:01:26.479445+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-7ee30a06-ba8e-424b-8964-e9fb986ce57c">
            <cybox:Title>URI : boysclub.web.fc2.com/mono/11.exe</cybox:Title>
            <cybox:Description>Payload attempt / Malicious vba macro content connects to the following</cybox:Description>
            <cybox:Object id="fsisac:uri-fec5cecd-9ac7-473f-be11-0c2767c7008b">
                <cybox:Properties xsi:type="URIObj:URIObjectType">
                    <URIObj:Value>boysclub.web.fc2.com/mono/11.exe</URIObj:Value>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Building Event...
STIX Import
unexpected EOF while parsing (<unknown>, line 1)
'cm9vdDpyb290' 12
{"message_type": "Inbox_Message", "service_id": "inbox", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "message_id": "9f3fb95f-1ba5-4ddb-b22d-cfd3244b92ea", "logger": "opentaxii.taxii.services.inbox.InboxService", "event": "Processing message", "timestamp": "2017-06-29T09:01:33.701862Z"}
{"logger": "opentaxii.persistence.sqldb.api", "content_block": 349065, "level": "debug", "timestamp": "2017-06-29T09:01:33.715246Z", "collections": 1, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:URIObj="http://cybox.mitre.org/objects#URIObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-4c936e21-aff3-4ea2-a9ca-af37eaa2d34e" version="1.1.1" timestamp="2017-06-29T09:01:26.502980+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-c2a56cf0-6441-4ac0-a2e7-9f0a083fcd50">
            <cybox:Title>URI : stream1.sexrura.pl/rtd/43.exe </cybox:Title>
            <cybox:Description>Payload attempt / Malicious vba macro content connects to the following</cybox:Description>
            <cybox:Object id="fsisac:uri-4a373496-68e6-4307-9366-0641478c6b9e">
                <cybox:Properties xsi:type="URIObj:URIObjectType">
                    <URIObj:Value>stream1.sexrura.pl/rtd/43.exe </URIObj:Value>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Building Event...
STIX Import
invalid syntax (<unknown>, line 1)
'cm9vdDpyb290' 12
{"message_type": "Inbox_Message", "service_id": "inbox", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "message_id": "e51b695a-251a-463e-8f65-c6bfc98ddb29", "logger": "opentaxii.taxii.services.inbox.InboxService", "event": "Processing message", "timestamp": "2017-06-29T09:01:33.792371Z"}
{"logger": "opentaxii.persistence.sqldb.api", "content_block": 349066, "level": "debug", "timestamp": "2017-06-29T09:01:33.805736Z", "collections": 1, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:URIObj="http://cybox.mitre.org/objects#URIObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-0fb7236a-5e02-47d6-9af8-0fdf64ad067c" version="1.1.1" timestamp="2017-06-29T09:01:26.530283+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-51e45aa2-9df5-45c4-9b83-2229855ac4fa">
            <cybox:Title>URI : w47e4q423.homepage.t-online.de/joshua/74.exe</cybox:Title>
            <cybox:Description>Payload attempt / Malicious vba macro content connects to the following</cybox:Description>
            <cybox:Object id="fsisac:uri-8065df68-e813-4b1a-bbdf-dbd59c5a8150">
                <cybox:Properties xsi:type="URIObj:URIObjectType">
                    <URIObj:Value>w47e4q423.homepage.t-online.de/joshua/74.exe</URIObj:Value>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Building Event...
STIX Import
unexpected EOF while parsing (<unknown>, line 1)
'cm9vdDpyb290' 12
{"message_type": "Inbox_Message", "service_id": "inbox", "level": "debug", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "message_id": "20e26b28-7a9c-485d-972f-50f0288185b5", "logger": "opentaxii.taxii.services.inbox.InboxService", "event": "Processing message", "timestamp": "2017-06-29T09:01:33.884587Z"}
{"logger": "opentaxii.persistence.sqldb.api", "content_block": 349067, "level": "debug", "timestamp": "2017-06-29T09:01:33.898138Z", "collections": 1, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-fccfeb6f-37a9-481b-a317-e092cfee58d2" version="1.1.1" timestamp="2017-06-29T09:01:26.552255+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-06f96633-27cc-4896-b0e4-5b88d2314285">
            <cybox:Title>File : 22328_201512.doc</cybox:Title>
            <cybox:Description>Word doc Attachment</cybox:Description>
            <cybox:Object id="fsisac:file-6da7d272-12af-45e3-b279-baa4645ff19f">
                <cybox:Properties xsi:type="FileObj:FileObjectType">
                    <FileObj:File_Name>22328_201512.doc</FileObj:File_Name>
                    <FileObj:Device_Path/>
                    <FileObj:Full_Path/>
                    <FileObj:File_Extension>.doc</FileObj:File_Extension>
                    <FileObj:Size_In_Bytes>75776</FileObj:Size_In_Bytes>
                    <FileObj:File_Format> MS Word Document </FileObj:File_Format>
                    <FileObj:Hashes>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SHA256</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>a934018b9b6ff900b391d18b4e9432b1d1322f6ca3bf08ca152472cc144560db</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                    </FileObj:Hashes>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Building Event...
STIX Import
invalid syntax (<unknown>, line 1)
.
.
.

I'm trying to find if I did something wrong with the DB or I didn't update everything

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Ok, try again.

It should log more this time, and I think I fixed your issue along the way

from misp-taxii-server.

Danko90 commented on June 23, 2024

Tried, now a few of events are being created, for example 10/250, I think there is some problem yet. This is the output

'cm9vdDpyb290' 12
{"level": "debug", "timestamp": "2017-06-29T12:06:30.851428Z", "service_id": "inbox", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "logger": "opentaxii.taxii.services.inbox.InboxService", "message_id": "edac2dd7-1ebf-49d0-a677-3f83e1cb3987", "event": "Processing message", "message_type": "Inbox_Message"}
{"level": "debug", "collections": 1, "timestamp": "2017-06-29T12:06:30.866630Z", "logger": "opentaxii.persistence.sqldb.api", "content_block": 349378, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-9e69547c-a0c4-4a5c-a6f0-47b074c6f57a" version="1.1.1" timestamp="2017-06-29T12:05:48.412040+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-c8ef86b8-6433-4d08-b98c-95c91fb14e54">
            <cybox:Observable_Composition operator="AND">
                <cybox:Observable idref="fsisac:observable-b47a516d-2f4a-40e4-90df-33f05b537efe">
                </cybox:Observable>
                <cybox:Observable idref="fsisac:observable-1d46a9ed-33a0-427d-a0cf-94fd1641108d">
                </cybox:Observable>
                <cybox:Observable idref="fsisac:observable-28e0a742-6e66-4391-8954-a38e98d02760">
                </cybox:Observable>
            </cybox:Observable_Composition>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Loading STIX...
Loading STIX...
Argument has 'read' attribute, assuming file-like.
Read file, type <class 'bytes'>.
Attempting to load from JSON...
Attempting to load from XML...
Removing Marking elements...
Writing cleaned XML to Tempfile
Attempting to read clean XML into STIX...
Building Event...
Using title STIX Import
Seting up MISPEvent...
Beginning to Lint_roll...
Processing 2 object...
Working on <cybox.core.observable.Observable object at 0x7f3d924175f8>...
Working on <cybox.core.observable.Observable object at 0x7f3d9241f7b8>...
Making sure we only have Unique attributes...
Finished parsing attributes.
'cm9vdDpyb290' 12
{"level": "debug", "timestamp": "2017-06-29T12:06:30.952318Z", "service_id": "inbox", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "logger": "opentaxii.taxii.services.inbox.InboxService", "message_id": "dc6d960c-42e5-4d97-8161-2871ad8fabe0", "event": "Processing message", "message_type": "Inbox_Message"}
{"level": "debug", "collections": 1, "timestamp": "2017-06-29T12:06:30.967164Z", "logger": "opentaxii.persistence.sqldb.api", "content_block": 349379, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-863d2223-51a3-4611-9ae2-5ec8fadf76c5" version="1.1.1" timestamp="2017-06-29T12:05:48.761186+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-1d46a9ed-33a0-427d-a0cf-94fd1641108d">
            <cybox:Title>File : Payment Slip pdf.7z</cybox:Title>
            <cybox:Description>File Attached</cybox:Description>
            <cybox:Object id="fsisac:file-109f9dfe-adb2-470b-894b-3e4c3bc876dd">
                <cybox:Properties xsi:type="FileObj:FileObjectType">
                    <FileObj:File_Name>Payment Slip pdf.7z</FileObj:File_Name>
                    <FileObj:Device_Path/>
                    <FileObj:Full_Path/>
                    <FileObj:File_Extension/>
                    <FileObj:File_Format>7-zip</FileObj:File_Format>
                    <FileObj:Hashes>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">MD5</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>e8d7a6c77e2156f782e7702a9e0abc40</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                    </FileObj:Hashes>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Loading STIX...
Loading STIX...
Argument has 'read' attribute, assuming file-like.
Read file, type <class 'bytes'>.
Attempting to load from JSON...
Attempting to load from XML...
Removing Marking elements...
Writing cleaned XML to Tempfile
Attempting to read clean XML into STIX...
Building Event...
Using title STIX Import
Seting up MISPEvent...
Beginning to Lint_roll...
Processing 1 object...
Working on <cybox.core.observable.Observable object at 0x7f3d9235cda0>...
Making sure we only have Unique attributes...
Finished parsing attributes.
CHECKING Payment Slip pdf.7z
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10528
CHECKING e8d7a6c77e2156f782e7702a9e0abc40
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10672
'cm9vdDpyb290' 12
{"level": "debug", "timestamp": "2017-06-29T12:06:31.713553Z", "service_id": "inbox", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "logger": "opentaxii.taxii.services.inbox.InboxService", "message_id": "9789a30f-62f8-4edd-8a62-4a22aa4be522", "event": "Processing message", "message_type": "Inbox_Message"}
{"level": "debug", "collections": 1, "timestamp": "2017-06-29T12:06:31.726565Z", "logger": "opentaxii.persistence.sqldb.api", "content_block": 349380, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:FileObj="http://cybox.mitre.org/objects#FileObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-3dc89f42-9e0d-4a34-b508-75a8e4969149" version="1.1.1" timestamp="2017-06-29T12:05:48.893954+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-28e0a742-6e66-4391-8954-a38e98d02760">
            <cybox:Title>File : Payment Slip pdf.exe</cybox:Title>
            <cybox:Description>Compressed file</cybox:Description>
            <cybox:Object id="fsisac:file-9bd19157-0313-43d5-9a56-fd08b9036bb1">
                <cybox:Properties xsi:type="FileObj:FileObjectType">
                    <FileObj:File_Name>Payment Slip pdf.exe</FileObj:File_Name>
                    <FileObj:Device_Path/>
                    <FileObj:Full_Path/>
                    <FileObj:File_Extension>.exe</FileObj:File_Extension>
                    <FileObj:Size_In_Bytes>1387008</FileObj:Size_In_Bytes>
                    <FileObj:File_Format>Win32 EXE</FileObj:File_Format>
                    <FileObj:Hashes>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">MD5</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>907eb352886f7323b9d561b924d61b92</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SHA1</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>1cb0ea821efdb945630c98aecd96cb3cfcda54ba</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SHA256</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>93fab59aca42da7eb15ae85284cd5fd137fab3e47430dea02afabad8c9e9084d</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                        <cyboxCommon:Hash>
                            <cyboxCommon:Type xsi:type="cyboxVocabs:HashNameVocab-1.0">SSDeep</cyboxCommon:Type>
                            <cyboxCommon:Simple_Hash_Value>24576:ScTIsuqnMKWVQuai+Irx2OMvhlqXcf/XNXHr9sis3Df3poC6qrHwA9kd0:rTcOVpBlIEOMJlqXUds3mC6qrH7j</cyboxCommon:Simple_Hash_Value>
                        </cyboxCommon:Hash>
                    </FileObj:Hashes>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Loading STIX...
Loading STIX...
Argument has 'read' attribute, assuming file-like.
Read file, type <class 'bytes'>.
Attempting to load from JSON...
Attempting to load from XML...
Removing Marking elements...
Writing cleaned XML to Tempfile
Attempting to read clean XML into STIX...
Building Event...
Using title STIX Import
Seting up MISPEvent...
Beginning to Lint_roll...
Processing 1 object...
Working on <cybox.core.observable.Observable object at 0x7f3d92417160>...
Making sure we only have Unique attributes...
Finished parsing attributes.
CHECKING Payment Slip pdf.exe
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10564
CHECKING .exe
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 1789910
CHECKING 1387008
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10240
CHECKING 907eb352886f7323b9d561b924d61b92
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10690
CHECKING 1cb0ea821efdb945630c98aecd96cb3cfcda54ba
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 10852
CHECKING 93fab59aca42da7eb15ae85284cd5fd137fab3e47430dea02afabad8c9e9084d
Starting new HTTPS connection (1): 192.168.56.50
/usr/local/lib/python3.4/dist-packages/urllib3/connection.py:344: SubjectAltNameWarning: Certificate for 192.168.56.50 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
https://192.168.56.50:443 "POST /attributes/restSearch/download HTTP/1.1" 200 11320
'cm9vdDpyb290' 12
{"level": "debug", "timestamp": "2017-06-29T12:06:33.800134Z", "service_id": "inbox", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "logger": "opentaxii.taxii.services.inbox.InboxService", "message_id": "0e71b182-7120-42da-9b1f-5cad24bb058a", "event": "Processing message", "message_type": "Inbox_Message"}
{"level": "debug", "collections": 1, "timestamp": "2017-06-29T12:06:33.813825Z", "logger": "opentaxii.persistence.sqldb.api", "content_block": 349381, "event": "Content block added to collections"}
CONTENT BLOCK : <stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:AddressObj="http://cybox.mitre.org/objects#AddressObject-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:fsisac="http://fsisac.com/" xmlns:edge="http://soltra.com/" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-79efe7f6-348c-49e1-96ec-5f56ede5736a" version="1.1.1" timestamp="2017-06-29T12:05:48.923276+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node() | ../../../../descendant-or-self::node()/@*</marking:Controlled_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Observables cybox_major_version="2" cybox_minor_version="1" cybox_update_version="0">
        <cybox:Observable id="fsisac:observable-b47a516d-2f4a-40e4-90df-33f05b537efe">
            <cybox:Title>Address : [email protected]</cybox:Title>
            <cybox:Description>Sending Email Address</cybox:Description>
            <cybox:Object id="fsisac:address-d7a0bd4b-8bcc-4a8d-a172-0738082f2835">
                <cybox:Properties xsi:type="AddressObj:AddressObjectType" category="e-mail" is_source="true">
                    <AddressObj:Address_Value>[email protected]</AddressObj:Address_Value>
                </cybox:Properties>
            </cybox:Object>
        </cybox:Observable>
    </stix:Observables>
</stix:STIX_Package>

Loading STIX...
Loading STIX...
Argument has 'read' attribute, assuming file-like.
Read file, type <class 'bytes'>.
Attempting to load from JSON...
Attempting to load from XML...
Removing Marking elements...
Writing cleaned XML to Tempfile
Attempting to read clean XML into STIX...
Building Event...
Using title STIX Import
Seting up MISPEvent...
Beginning to Lint_roll...
Processing 1 object...
Working on <cybox.core.observable.Observable object at 0x7f3d924b4b38>...
Making sure we only have Unique attributes...
Finished parsing attributes.
CHECKING [email protected]

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

There we go, that'll update will log EVERYTHING and explain it

from misp-taxii-server.

Danko90 commented on June 23, 2024

Also, I can notice many events created without attributes

from misp-taxii-server.

Danko90 commented on June 23, 2024

Ok, I have another example..

'cm9vdDpyb290' 12
{"message_version": "urn:taxii.mitre.org:message:xml:1.1", "level": "debug", "service_id": "inbox", "message_id": "fa93849c-8037-414d-874c-a0630eb7dac5", "event": "Processing message", "timestamp": "2017-06-29T14:46:38.818885Z", "logger": "opentaxii.taxii.services.inbox.InboxService", "message_type": "Inbox_Message"}
{"level": "debug", "content_block": 350243, "event": "Content block added to collections", "collections": 1, "timestamp": "2017-06-29T14:46:38.832620Z", "logger": "opentaxii.persistence.sqldb.api"}
Posting STIX...
Loading STIX...
Loading STIX...
Argument has 'read' attribute, assuming file-like.
Read file, type <class 'bytes'>.
Attempting to load from JSON...
Attempting to load from XML...
Removing Marking elements...
Writing cleaned XML to Tempfile
Attempting to read clean XML into STIX...
Building Event...
Using title STIX Import
Seting up MISPEvent...
Beginning to Lint_roll...
Processing 1 object...
Working on <cybox.core.observable.Observable object at 0x7f2cb8f59438>...
Making sure we only have Unique attributes...
Finished parsing attributes.
STIX loaded succesfully.
Extracted ['billyjoseph123.no-ip.biz']
Checking for existence of billyjoseph123.no-ip.biz

I checked for that value on MISP and it's not present. If you want I can print the XML blocks again.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

That log is incomplete.

After we print "Checking for existence..." it'll either say if it's unique or a duplicate

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Like so

from misp-taxii-server.

Danko90 commented on June 23, 2024

I attached a screenshot, this is what I see

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Working as intended. No issues to see there.

from misp-taxii-server.

Danko90 commented on June 23, 2024

Yes, but if I search for the value "updateceb.zapto.org" on MISP I won't find anything, is this normal?

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

You might have updateceb.zapto.org/some_subpath

MISP has no way to do exact search, it'll so substring though.

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

If you run a pymisp pymisp.search("attributes", "updateceb.zapto.org") you'll see what the duplicate it

from misp-taxii-server.

Danko90 commented on June 23, 2024

Well, I have only 14 events and I searched for it manually one by one and it's not present

from misp-taxii-server.

Danko90 commented on June 23, 2024

from misp-taxii-server.

FloatingGhost commented on June 23, 2024

Run the pymisp search. That'll tell you what's up

from misp-taxii-server.

Danko90 commented on June 23, 2024

Found it, thanks! It's not shown probably because under the column there is this error:

Notice (8): Undefined index: Orgc [APP/View/Attributes/index.ctp, line 80]

from misp-taxii-server.

Danko90 commented on June 23, 2024

Hi @FloatingGhost,
I fixed the error above, now it's working, but I'm still having events with no attributes. Below a screenshot.

At the moment I'm trying to understand which of those events are being inserted in MISP without attributes.

from misp-taxii-server.

Danko90 commented on June 23, 2024

Ok here we go.. I looked at the logs and I found an example of error I get

Uploading event to MISP with attributes ['[email protected]']
JSON FULL : {'Event': {'distribution': '3', 'date': '2017-07-18', 'analysis': '0', 'info': 'STIX Import', 'published': False, 'threat_level_id': '2', 'Attribute': [{'distribution': '5', 'value': 'handyma
[email protected]', 'disable_correlation': False, 'to_ids': True, 'category': 'Network activity', 'comment': 'Address : [email protected]', 'type': 'ip-src'}]}}
Starting new HTTPS connection (1): 192.168.56.50
https://192.168.56.50:443 "POST /events HTTP/1.1" 200 1178
{
    "errors": [
        {
            "Attribute": [
                {
                    "value": [
                        "IP address has an invalid format."
                    ]
                }
            ]
        },
        "Error in Attribute: IP address has an invalid format."
    ],
    "Event": {
        "ShadowAttribute": [],
        "published": false,
        "disable_correlation": false,
        "info": "STIX Import",
        "orgc_id": "1",
        "distribution": "3",
        "locked": false,
        "publish_timestamp": "0",
        "RelatedEvent": [],
        "uuid": "596ddcfa-6658-4b22-ac43-629ec0a83832",
        "Attribute": [],
        "event_creator_email": "[email protected]",
        "attribute_count": "0",
        "Orgc": {
            "uuid": "5924041f-ec94-440c-8d68-07b1c0a83832",
            "name": "MISP",
            "id": "1"
        },
        "date": "2017-07-18",
        "analysis": "0",
        "org_id": "1",
        "Galaxy": [],
        "threat_level_id": "2",
        "id": "84834",
        "proposal_email_lock": false,
        "timestamp": "1500372218",
        "sharing_group_id": "0",
        "Org": {
            "uuid": "5924041f-ec94-440c-8d68-07b1c0a83832",
            "name": "MISP",
            "id": "1"
        }
    }
}

EDIT:
I didn't understand why it recognizes an email address as ip-src yet.. but I was wondering if it's possible to avoid the event creation if something goes wrong, please let me know what you think.

from misp-taxii-server.

events with No attributes about misp-taxii-server HOT 36 CLOSED

Comments (36)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent