misp/misp-book

User guide of MISP

MISP - Threat Intelligence Sharing Platform

MISP is an open source software solution for collecting, storing, distributing and sharing cyber security indicators and threats about cyber security incidents analysis and malware analysis. MISP is designed by and for incident analysts, security and ICT professionals or malware reversers to support their day-to-day operations to share structured information efficiently.

The objective of MISP is to foster the sharing of structured information within the security community and abroad. MISP provides functionalities to support the exchange of information but also the consumption of said information by Network Intrusion Detection Systems (NIDS), LIDS but also log analysis tools, SIEMs.

  ●  Core functions   ●  Website / Support   ●  PHP and MISP
  ●  Installation   ●  Documentation   ●  Contributing
  ●  License

Latest Release
CI
Gitter
Mastodon
Twitter
Localization
Contributors
License

Core functions

• An efficient IOC and indicators database, allowing to store technical and non-technical information about malware samples, incidents, attackers and intelligence.
• Automatic correlation finding relationships between attributes and indicators from malware, attack campaigns or analysis. The correlation engine includes correlation between attributes and more advanced correlations like Fuzzy hashing correlation (e.g. ssdeep) or CIDR block matching. Correlation can also be enabled or event disabled per attribute.
• A flexible data model where complex objects can be expressed and linked together to express threat intelligence, incidents or connected elements.
• Built-in sharing functionality to ease data sharing using different model of distributions. MISP can automatically synchronize events and attributes among different MISP instances. Advanced filtering functionalities can be used to meet each organization's sharing policy including a flexible sharing group capacity and an attribute level distribution mechanisms.
• An intuitive user-interface for end-users to create, update and collaborate on events and attributes/indicators. A graphical interface to navigate seamlessly between events and their correlations. An event graph functionality to create and view relationships between objects and attributes. Advanced filtering functionalities and warning lists to help the analysts to contribute events and attributes and limit the risk of false-positives.
• storing data in a structured format (allowing automated use of the database for various purposes) with an extensive support of cyber security indicators along fraud indicators as in the financial sector.
• export: generating IDS, OpenIOC, plain text, CSV, MISP XML or JSON output to integrate with other systems (network IDS, host IDS, custom tools), Cache format (used for forensic tools), STIX (XML and JSON) 1 and 2, NIDS export (Suricata, Snort and Bro/Zeek) or RPZ zone. Many other formats can be easily added via the misp-modules.
• import: bulk-import, batch-import, import from OpenIOC, GFI sandbox, ThreatConnect CSV, MISP standard format or STIX 1.1/2.0. Many other formats easily added via the misp-modules.
• Flexible free text import tool to ease the integration of unstructured reports into MISP.
• A user-friendly system to collaborate on events and attributes allowing MISP users to propose changes or updates to attributes/indicators.
• data-sharing: automatically exchange and synchronize with other parties and trust-groups using MISP.
• delegating of sharing: allows for a simple, pseudo-anonymous mechanism to delegate publication of event/indicators to another organization.
• Flexible API to integrate MISP with your own solutions. MISP is bundled with PyMISP which is a flexible Python Library to fetch, add or update events attributes, handle malware samples or search for attributes. An exhaustive restSearch API to easily search for indicators in MISP and exports those in all the format supported by MISP.
• Adjustable taxonomy to classify and tag events following your own classification schemes or existing classification. The taxonomy can be local to your MISP but also shareable among MISP instances.
• Intelligence vocabularies called MISP galaxy and bundled with existing threat actors, malware, RAT, ransomware or MITRE ATT&CK which can be easily linked with events and attributes in MISP.
• Expansion modules in Python to expand MISP with your own services or activate already available misp-modules.
• Sighting support to get observations from organizations concerning shared indicators and attributes. Sighting can be contributed via MISP user-interface, API as MISP document or STIX sighting documents.
• STIX support: import and export data in the STIX version 1 and version 2 format.
• Integrated encryption and signing of the notifications via GnuPG and/or S/MIME depending on the user's preferences.
• Real-time publish-subscribe channel within MISP to automatically get all changes (e.g. new events, indicators, sightings or tagging) in ZMQ (e.g. misp-dashboard) or Kafka publishing.

Exchanging info results in faster detection of targeted attacks and improves the detection ratio while reducing the false positives. We also avoid reversing similar malware as we know very fast that other teams or organizations have already analyzed a specific malware.

A sample event encoded in MISP:

Website / Support

Checkout the website for more information about MISP software, standards, tools and communities.

Information, news and updates are also regularly posted on the MISP project Mastodon account, twitter account and news page.

PHP and MISP

MISP currently requires PHP 7.4, an end-of-life version of PHP. Because of this it is recommended that you only run MISP on distributions or PHP installs that you know will get security fixes backported, like Red Hat or Debian and derratives.

MISP 3.x, currently in development will support PHP 8.x.

Installation

For test- og production installations we recommend you check out the possible options on misp-project.org/download.

Documentation

MISP user-guide (MISP-book) is available online or as PDF or as EPUB or as MOBI/Kindle.

It is also recommended to read the FAQ

Contributing

If you are interested to contribute to the MISP project, review our contributing page. There are many ways to contribute and participate to the project.

Please see our Code of conduct.

Feel free to fork the code, play with it, make some patches and send us the pull requests via the issues.

Feel free to contact us, create issues, if you have questions, remarks or bug reports.

There is one main branch:

• 2.4 (current stable version): what we consider as stable with frequent updates as hot-fixes.

and features are developed in separated branches and then regularly merged into the 2.4 stable branch.

License

This software is licensed under GNU Affero General Public License version 3

• Copyright (C) 2012-2023 Christophe Vandeplas
• Copyright (C) 2012 Belgian Defence
• Copyright (C) 2012 NATO / NCIRC
• Copyright (C) 2013-2023 Andras Iklody
• Copyright (C) 2015-2023 CIRCL - Computer Incident Response Center Luxembourg
• Copyright (C) 2016 Andreas Ziegler
• Copyright (C) 2018-2023 Sami Mokaddem
• Copyright (C) 2018-2023 Christian Studer
• Copyright (C) 2015-2023 Alexandre Dulaunoy
• Copyright (C) 2018-2022 Steve Clement
• Copyright (C) 2020-2023 Jakub Onderka

For more information, the list of authors and contributors is available.

misp-book's Issues

Broken images links

There are some broken links for images into the quick start (the last one, "") and in the General Layout (in event actions, "").

Describe setup/requirement recommendations on misp-project

It's a recurring question, the idea isn't to set some hard minimums, but rather to describe usual / recommended requirements and guidance.

Document ZeroMQ plugin

Document base-URL

Depending on your configuration,you might need to change the base URL in MISP server settings.
This could be documented on the VM part

Visualise data-flow of data within MISP (from whitelist to warning lists)

MISP/MISP#2734

Events, Attributes, Indicators, Observable

Events, Attributes, Indicators, Observable definition and description how this is used in MISP 2.4

How to contribute to MISP development - new chapter

Some topics to add

• Addition of new attributes not currently supported in MISP (like MISP/MISP#800)
• Proposing tools using PyMISP
• Updating documentation

Error 500 after fresh Centos7 Install

I followed the install guide according, and when I start the process everything appears to start but I am not able to access http://misp/user/login url. I am getting an error 500.
sudo su -s /bin/bash apache -c 'scl enable rh-php56 /var/www/MISP/app/Console/worker/start.sh'
[sudo] password for sjohn345:
Stopping workers
Stopping 12950 ... Done
Stopping the Scheduler Worker ... Done
Stopping 12968 ... Done
Stopping 12986 ... Done
Stopping 13004 ... Done

Creating workers
Starting worker ... Done

Creating workers
Starting worker ... Done

Creating workers
Starting worker ... Done

Creating workers
Starting worker ... Done

Creating the scheduler workers
Starting scheduler worker ... Done
! more misp.local_error.log
[Wed Jul 05 21:28:56.478562 2017] [:error] [pid 11684] [client ::1:51188] PHP Fatal error: Can't use function return value in write context in /var/www/MISP/app/Controller/UsersController.php on line 97
[Wed Jul 05 21:28:56.479803 2017] [:error] [pid 11684] [client ::1:51188] PHP Fatal error: Can't use function return value in write context in /var/www/MISP/app/Controller/AppController.php on line 305
[Wed Jul 05 21:29:02.058256 2017] [:error] [pid 2369] [client ::1:51190] PHP Fatal error: Can't use function return value in write context in /var/www/MISP/app/Controller/AppController.php on line 305

Remote Server description

As far as I can see, there is no part that covers how to deal with servers / sharing etc.

Shall I write it?

Defanging and un-defanging rules in MISP

We should document all the rules in MISP to un-defang URLs and other artifacts.

Document usage of data filtering strategy

Quote from a mail, this but better explained:

"our mantra is, keep your data for correlation and exclude it from the exports. What I'd suggest:

a. Set an automatic tag for your feed (such as "expireMeInAMonth") - these tags will be automatically applied to all events coming from the feed hereafter
b. When exporting data from MISP, for example for your SIEM/NIDS/etc use the following rules:

• 1x full data set, but exclude everything tagged "expireMeInAMonth"
• 1x the data set carrying the "expireMeInAMonth" tag, but with the "last":"30d" parameter set
  c. Feed both data sets to your tools

This will get you all your regular data + the past 30 day's worth of data from the feed."

new strict JSON schema to add

new strict JSON schema to add https://github.com/MISP/PyMISP/blob/master/pymisp/data/schema.json

RESTful API - Specify dates

I am using the api endpoint:

https://<server-name>/events/

and trying to get all the events. The issue I'm having is that this endpoint returns too many events at once. Is there a way to specify start / end in the RESTful api? I saw you could with the /events/xml/download endpoint, but I just want to get the json response rather than a file.

I've tried:

https://<server-name>/events?from=2016-01-01
https://<server-name>/events/null/null/null/2016-01-01

Using MISP with other Threat sharing solutions

Is MISP compatible with other Threat sharing solutions such as CIFS?

Requirements section

We got a lot of questions regarding the MISP hardware requirements. Maybe we should add a specific section about this.

A note about the PHP session and the recommended Redis session

A note about the PHP session and the recommended Redis session and the config
aspect too:

https://www.digitalocean.com/community/tutorials/how-to-set-up-a-redis-server-as-a-session-handler-for-php-on-ubuntu-14-04

Document enrichment plugin & enrichment module format

Feed field explanations are missing

Explain fields such as delta merge

Publish event to ZMQ does not include Objects information

Dears,

Since MISP 2.4.80, we have the possibility to define objects. If you download a sample, the object is automatically created with subsequent attributes (filename, md5, sha1, sha256 and filesize).
However, it seems that this information is not included when publishing the event to ZMQ.

Am I missing something or is it not supported? I am running 2.4.80 on Ubuntu.

Thanks a lot in advance for your help!

BR's
Irving

Mentioning double-click to edit an event

Mentioning double-click to edit an event

example - convention with example organisations

example - convention with example organisations

MISP provisioning descriptions

1. Create a use-case based description of how to deploy MISP in the sense of requirements, topologies, etc
2. Write a tool that calculates optimal provisioning based on desired data/community sizes.

Documenting and explaining threat id

As requested in MISP/MISP#729 , we should better document the threat id.

1 = high
2 = medium
3 = low
4 = undefined

The practical examples of the tool-tip could be used to show the potential interpretation of the threat id.

Include the default example of lifetime of an event

Include the default example of lifetime of an event (based on the current xmind)

Broken anchor (interface from scheduled task)

Broken anchor (interface from scheduled task)

via @adulau

Sharing groups explanation and usage to add

Sharing groups explanation and usage to add

Document export plugin & export modules format

Describe role of each worker

Describe role of each worker and their respective interdependencies.

New API call to get the JSON with all attributes and categories

https://misppriv.circl.lu/attributes/describeTypes.json

to add in the doc

Back-up of MISP to be documented

Back-up of MISP to be documented

Rework API Documentation

This is to document the open points in the API documentation:

• Describe Error Codes

• make CURL examples for every Endpoint

• document searchtimestamp (see #32)

• POST /events more fields explanation

• remove API keys from sample data

Misp-book: general-layout =>drop-down menus explanations

I up to date misp-book/general-layout/

In the menu bar "layout Filter", it seems to be a new tab "Warning List", who is not documented. What is it and for what can I use that? (Yeah, I know that it will shows a list of warnings, but I don't know why it is usefull).

Lifecycle documentation of event workflow, inter-analyst workflows etc..

During Hackathon & Training days in Luxembourg in March 2018, we discussed with Andras and Alexandre Dulaunoy about having Sequence Diagrams or other visualization to explain how people work and interact with MISP.

A suggestion is to use Web Sequence Diagrams like markup text to graph / svg / png:

Title: Adding new MISP events

Alice (Org A) -> MISP unpublished: Add event

Note: 
**Any edit/modification on exiting event** puts back the Event in _unpublished mode_.

Alice (Org A) -->> Bob (Org A, Publisher): _Out of band Notification (voice, sms, ...): Please verrify and publish my Event_

MISP unpublished -> Bob (Org A, Publisher): Publish

Bob (Org A, Publisher) -> MISP published: Publish

Note: See **[MISP guide](https://github.com/MISP/MISP/)** for more details

Many tools:
Commercial: https://swimlanes.io/#dVBNa4NAEL3vr3hHI0bvOQgWeughTSC9lSLqjjpUd4M72ubfd42Shn7Asiwz72vfC0tHO2Ras2lg6AP7p9MRNJERp1TWcUUIDkODbINtumxHcx7Ljl1L+kpd4Eo9W/FaKgwzcwFplqS3mmuuCmFr4A99ssxGV0IY4jyKQ1lU75CW8DhPwQb5nQO8BuXxryzbNMWDLddBhONKGDY75IdRYGuvbDR8qu8MwWS9SgTXuwhxHHvwsaPCESYaBq4vmCmrOfrLkilX6ufH5zL+s1/fSv0NuPV41+KNsnR4IkIYvl5Rzcia3oJW5Ox2SdKwtGMZV7ZP5vVybXyXtR18VwNBkxTcOfUF

or Free: https://bramp.github.io/js-sequence-diagrams/

PyMISP documentation in misp-book

PyMISP documentation in misp-book

add visual for explaining the distribution levels including misp-training slides (usage)

add visual for explaining the distribution levels including misp-training slides (usage)

Write documentation about the MISP sync protocol

Write a simple documentation about the MISP protocol use for synchronisation along with the distribution of events within the MISP instances. (2.4 protocol)

Usage instructions quite limited

I tried getting the MISP Book running locally but failed to do so only relying on the notes in the USAGE file.
I'm on Ubuntu (15.10 in this case) and finally got it working after doing these steps on a clean installation:

Install node and npm

sudo apt-get install nodejs nodejs-legacy npm 

Install further requirements

sudo apt-get install libcairo2-dev libjpeg-dev
sudo apt-get install calibre  # for generating PDFs

Might save other people a few minutes if you could include these instructions, what do you think?

Confusing snippet in Taxonomy section

https://github.com/MISP/misp-book/blame/master/taxonomy/README.md#L86

It's not clear to me what "Her, we classified Tags." is supposed to mean.

use-case chapter - how to deal with false-positive

use-case chapter - how to deal with false-positive

Chapters which needs improvements

Might be incomplete or having outdated screens

• Administration
• Misp Objects
• Quick Start
• General layout
• Managing feeds
• Sightings
• Taxonomies (lists)
• Galaxy (lists)
• Using the system

(Non exaustive list)

License of MISP-Book

Hi

Since misp-book has been exported from main MISP archive, license of miss-book should be GNU Affero GPL v3.
However, since it is a documentation, and I would believe that it is in the interest of the MISP project for such documentation to be licensed under a more "academic" framework, I would suggest the following:

a - IF you can get the formal agreement of all authors (i.e., contributors to the documentation and - if applicable and required, so check with contributors - their employers), relicense the content under a simpler Creative common CC-BY 4.0 license (which in version 4 is compatible with the GNU Affero GPL v3 License). Other option is dual license of documentation (again, if agreement is granted by all contributors or reap. right holders), CC BY 4.0 AND Affero GNU GPL v3.
Attribution-only licenses will allow third parties to use the content of documentation as they please (as long as authors are acknowledged), which shouldn't be an issue since this is a supportive content to the benefit of MISP.
b - IF NOT, license for documentation is solely GNU Affero GPL v3.

Matthieu

Add best-practices for tagging (especially when to tag at event or attribute level)

Add best-practices for tagging (especially when to tag at event or attribute level)

OSINT feed creation and usage to add (based on Koen blog post)

OSINT feed creation and usage to add (based on Koen blog post)

https://www.vanimpe.eu/2016/03/23/using-open-source-intelligence-osint-with-misp/

Use GPG consistently

Okay, this is super-nitpicky... :-)

The book uses "GPG", "Gpg", "GnuPG", and "PGP". Suggest to use only one term to simplify searching...

Document import/export plugin & import/export modules format

Pymisp issue

Good morning ,

I have an issue with Pymisp.

When I try to change/add/delete the tag of any event via web, the event
goes to unpublished status. But,changing/adding/removing the tag via API
(pymisp) does not change the publishing status of that event.

Is that an error?
What would be the method to make these massive label changes through
Pymisp and change the publication status of each event?

Thank you very much in advance.

I look forward to hearing from you.

Greetings.

Documentation about "Hover" and "Expansion"

There is no documentation about "hover" and "expansion" that is supposed to come with the misp-modules.

Clarification on use of "thumbs-up" and "thumbs-down" sightings buttons

Some of our users have pointed out that the thumbs-up/down interface for sightings may be confusing. They understood that the "thumbs-up" button was to be used to signal any kind of sighting (true or false positive), and then the thumbs-down was to be click additionally in case of false positive.

Even though that was not my understanding, by reading the documentation, together with the icon tooltips, I think the wording may be a bit ambiguous.

I open this issue to get confirmation from the developers/designers that the "canonical use" is:

• Thumbs-up for signalling only a true positive.
• Thumbs-down for signalling a false positive.

If confirmed, I can propose a pull-request to clarify this point in the documentation.

Also, a change in the icon tooltips could be considered to clarify this use. I will do a PR for that in the corresponding repo.

Distribution model

Following this discussion MISP/MISP#1015 we should update the documentation about the distribution and what is pulled or not following the distribution level.

Feed section needs to be updated following the new separation in MISP (caching/import)

Feed section needs to be updated following the new separation (caching versus import)