anssi-fr/tabi

BGP Hijack Detection

TaBi - Track BGP Hijacks

Developed since 2011 for the needs of the French Internet Resilience Observatory, TaBi is a framework that ease the detection of BGP IP prefixes conflicts, and their classification into BGP hijacking events. The term prefix hijacking refers to an event when an AS, called an hijacking AS, advertises illegitimately a prefix equal or more specific to a prefix delegated to another AS, called the hijacked AS.

Usually, TaBi processes BGP messages that are archived in MRT files. Then, in order to use it, you will then need to install a MRT parser. Its favorite companion is MaBo, but it is also compatible with CAIDA's bgpreader. Internally, TaBi translates BGP messages into its own representation. Therefore, its is possible to implement new inputs depending on your needs.

Authors

• Nicolas Vivet [email protected]
• Guillaume Valadon [email protected]
• Julie Rossi [email protected]
• François Contat [email protected]

Building TaBi

TaBi depends on two external Python modules. The easiest method to install them is to use virtualenv and pip.

If you use a Debian-like system you can install these dependencies using:

apt-get install python-dev python-pip python-virtualenv

Then install TaBi in a virtual environment:

virtualenv ve_tabi
source ve_tabi/bin/activate
pip install py-radix python-dateutil
python setup.py install

Removing TaBi and its dependencies is therefore as simple as removing the ve_tabi directory ans the cloned repository.

Usage

Historically TaBi was designed to process MRT dump files from the collectors of the RIPE RIS.

Grabbing MRT dumps

You will then need to retrieve some MRT dumps. Copying and pasting the following commands in a terminal will grab a full BGP view and some updates.

wget -c http://data.ris.ripe.net/rrc01/2016.01/bview.20160101.0000.gz
wget -c http://data.ris.ripe.net/rrc01/2016.01/updates.20160101.0000.gz

tabi - the command line tool

The tabi command is the legacy tool that uses TaBi to build technical indicators for the Observatory reports. It uses mabo to parse MRT dumps.

Given the name of the BGP collector, an output directory and MRT dumps using the RIS naming convention, tabi will follow the evolution of routes seen in MRT dumps (or provided with the --ases option), and detect BGP IP prefixes conflicts.

Several options can be used to control tabi behavior:

$ tabi --help
Usage: tabi [options] collector_id output_directory filenames*

Options:
  -h, --help            show this help message and exit
  -f, --file            files content comes from mabo
  -p PIPE, --pipe=PIPE  Read the MRT filenames used as input from this pipe
  -d, --disable         disable checks of the filenames RIS format
  -j JOBS, --jobs=JOBS  Number of jobs that will process the files
  -a ASES, --ases=ASES  File containing the ASes to monitor
  -s, --stats           Enable code profiling
  -m OUTPUT_MODE, --mode=OUTPUT_MODE
                        Select the output mode: legacy, combined or live
  -v, --verbose         Turn on verbose output
  -l, --log             Messages are written to a log file.

Among this options, two are very interesting:

• -j that forks several tabi processes to process the MRT dumps faster
• -a that can be used to limit the output to a limited list of ASes

Note that the legacy output mode will likely consume all file descriptors as it creates two files per processed AS (i.e. around 100k opened files). The default is the combined output mode.

Here is an example call to tabi:

tabi -j 8 rrc01 results/ bview.20160101.0000.gz updates.20160101.0000.gz

After around 5 minutes of processing, you will find the following files in results/2016.01/:

• all.defaults.json.gz that contains all default routes seen by TaBi
• all.routes.json.gz that contains all routes monitored
• all.hijacks.json.gz that contains all BGP prefix conflicts

Using TaBi as a Python module

TaBi could also be used as a regular Python module in order to use it in your own tool.

The example provided in this repository enhance BGP prefix conflicts detection, with possible hijacks classification. To do so, it relies on external data sources such as RPKI ROA, route objects and other IRR objects.

tabi's Issues

Dealing with false positives

Hi @nv-anssi ,

I have analysed the results of detection tool with our older RIB files and I suspect there false positives.

{"timestamp": 1489445634.0, "collector": "rrc01", "peer_as": 4755, "peer_ip": "121.244.206.224", "announce": {"type": "F", "prefix": "1.12.0.0/16", "asn": 4847, "as_path": "4755 6453 4134 4847"}, "conflict_with": {"prefix": "1.12.0.0/14", "asn": 18245}, "asn": 18245, "type": "ABNORMAL"}

Here prefix -"1.12.0.0/16" is said to be Abnormal for "asn": 4847, "as_path": "4755 6453 4134 4847

But the history of the RIB files shows that the ASN 4847 has been consistently advertising the prefix 1.12.0.0/16 and seems to be maintaining its stability.

So according to some papers/algoithms they are not abnormal. How are you trying to deal with such false posivities.
Please help me to understand If I am wrong here

How to generate dynamic CSV files with realtime data?

May I know how you are able to generate & use the dynamic CSV files with real time data instead of static files as shown in repo.

@nv-anssi @guedou @ol-anssi @pcapillon @gdedrouas

Description of the "all.hijacks.json" fields

Hello,

I have unzipped "all.hijacks.json.gz" file and I am not sure about the signification of each json fields. There are some informations about some fields and I tried to answer :

• collector : BGP collector which has been hijacked ?

• conflict_with :
  prefix : address block of the hijacker ?
  asn : ASN of the hijacker ?

• timestamp : when the announce was received (UTC timestamp)

• peer_ip : IP of the BGP peer (neighbor) which received the announce (so we talk about the hijacker ?)

• peer_as : AS of the BGP peer (neighbor) which received the announce (so we talk about the hijacker ?)

• announce :
  prefix : address block of the hijacked ?
  asn : ASN of the hijacked ?
  as_path : List of ASN used to reach the hijacked ASN ?

• type : either "U" if the announce was received from a BGP update or "F" if it was from a BGP full view

• asn : ASN of the hijacked or the hijacker ?

TypeError: can only compare to a set

python detect_hijacks.py -c rrc01 -i mabo \
    --rpki-roa-file roa.csv \
    --irr-ro-file routes.csv \
    --irr-mnt-file maintainers.csv \
    --irr-org-file organisations.csv \
    ../../{bview,updates}.*.gz

Running hijack detection tool is throwing the error :

Traceback (most recent call last):

File "detect_hijacks.py", line 81, in <module>
  for conflict in detect_hijacks(**kwargs):
File "build/bdist.linux-x86_64/egg/tabi/emulator.py", line 146, in detect_hijacks
File "build/bdist.linux-x86_64/egg/tabi/annotate.py", line 163, in annotate_if_roa
File "build/bdist.linux-x86_64/egg/tabi/annotate.py", line 222, in annotate_roa_announce
TypeError: can only compare to a set 

feature request: add support for monitoring prefixes

I guess most tabi users care about entire ASes and use the -a parameter to specify the list of ASes they care about.
It would be great if there were an option to monitor specific prefixes only as well for those that only care about the prefixes they have services in.

thanks!
nusenu

and thanks for making me aware of this tool with your tweet :)
https://twitter.com/guedou/status/997010949335928832

Reporting intra-RIB conflicts

I have a RIB file with exactly two records, that are in conflict with each other. See pull request link below. In this case, I found out that the detect_conflicts function returns two conflicts. Is this an expected behavior?

https://github.com/ANSSI-FR/tabi/pull/3/files#diff-2f062b81eb0021f04e0686060432a769R76