emileaben/django-openipmap

Django app for OpenIPMap

django-openipmap's Issues

Accuracy levels for user input and guesses

It would be useful to put in states, countries or continents. without having to provide the full city, state, country location.
This might also be useful for guesses

Functionality: user comments

It would be helpfull to add user comments, I have seen hops being incorrectly corrected because the PTR name suggestio a location which is incorrect.

Small text comments would help that a hop is in a location and the PRT is just wrong.

Location suggestion based on previous and next known hop

To improve accuracy of the suggestions the code could look at the previous and next hop location, when these are the same and have a confidence above -test say- 50. It's very likely that this current hop is in the same location.

hop4 Amsterdam
hop5 ??
hop6 Amsterdam

Hop5 is likely Amsterdam.

This would mostlikely happen in template/oim/tracemap.html

Atlas Probe locations as data source

Use locations in atlas probe database as suggestion for ip address

undefined locations map random to cities (DNSLOC / blue nodes)

Have seen several traces in which there are several "?? undefined" locations that get mapped to a city and maybe receive a high confidence as the route is marked in blue.

Add RTT constraints to crowdsourced info that is returned

system guesses are already RTT constrained, due to frontend now calling this new URL:
/openipmap/iprtt?q=|||
What isn't done is to make the crowdsourced info returned by the per-ip call info also geo-constrained.
ie. if somebody put in something that doesn't make sense RTT wise, then don't return it.

work harder to get to interesting parts of hostnames/resolve ips

sometimes errors/config make hostnames hard to parse, or even to get to.

Examples of things to correct for before analysing host/ip are:

kartal-t3-2-acibadem-t2-1.turktelekom.com.tr.208.212.81.in-addr.arpa (remove in-addr-arpa)

$host ::ffff:130.117.50.133
Host 5.8.2.3.5.7.2.8.f.f.f.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa not found: 3(NXDOMAIN)
$host 130.117.50.133
133.50.117.130.in-addr.arpa domain name pointer be2108.agr22.par01.atlas.cogentco.com.

bulk upload not working

invalid literal for int() with base 10: '"georesult": "?? London'
Request Method: POST
Request URL: http://localhost:8000/openipmap/bulk-upload/
Django Version: 1.6.6
Exception Type: ValueError
Exception Value:
invalid literal for int() with base 10: '"georesult": "?? London'
Exception Location: /home/vagrant/openipmap/models.py in from_file, line 105

Crowdsourced negative assertions

Like storing positive assertions about hostname to loc. mapping it would be useful to capture the same information about wrong guesses. In the UI this should be a separate button, next to OK. All negative suggestions should be ignored on next guess.

Draw lines across dateline

See: Leaflet/Leaflet#82
In leaflet it should be possible to draw lines across the dateline (ie. shortest distance between the points). This should be implemented for openipmap traceroute viz. instead of the current default behaviour for leaflet.

Best available data export facility

Current data exports are raw data of all available data in openipmap, but these are not immediately useful for people wanting to download openipmap data for offline use, because they contain all assertions anybody ever made in the openipmap system about ip to location and hostname to location. Furthermore for ip->location mapping you'd need ip->hostname mapping which is cached in openipmap but not exposed.

A single datafile that has ip->location mappings would solve this.

Steps to do this:

• Define data format to export to ( csv? one entry per IP, support for IP ranges? ). Think about extensibility (ability to add a confidence factor)
• Write django command to export the best. For now pick the 'latest' if multiple assertions exist, but allow for extensibility in more elaborate selection of the best mapping of IP->location (confidence-level/reputation-based).
• Think about precedence (if a hostname -> loc exists and an ip->loc exists in openipmap, which one to pick? latest data, or always prefer the hostname-based mapping, even if it's older?). Most important here is to document the default, and potentially have the flexibility in the django export command to play around with different settings.
• Expose this dataset (or multiple datasets) in openipmap interface

add max_probes parameter to UI

tracemap allows max_probes parameter. Add that to UI

file name incorrect

When starting a local vagrant installation with openIPmap an error appears.

Django tried loading these templates, in this order:
Using loader django.template.loaders.filesystem.Loader:
/home/vagrant/djangoproject/templates/index.html (File does not exist)

shouldn't /home/vagrant/openipmap/templates/idx.html be index.html ?

Allow users to make IP->location mappings via user-interface (tracemap)

User interface currently only allows mappings of hostname->location, users should also be able to create IP->location mappings via the UI.

Add https access to map tiles by removing http: string from sourcecode

Presently map tiles are accessed via http vs https and yet the actual webpage is always https based.

function init_map() {
    map = L.map('map');

    var osmUrl='http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png';
    var osmAttrib='Map data &copy; <a href="http://openstreetmap.org">OpenStreetMap</a> contributors';

    var mapqUrl='http://otile1.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.jpg';

    //var osm = new L.TileLayer(osmUrl, {attribution: osmAttrib});
    var osm = new L.TileLayer(mapqUrl, {attribution: osmAttrib});
    map.setView([20, 0], 3);
    map.addLayer(osm);
}

The following would be required in order to remove the browser errors when accessing the webpage [1].

    var osmUrl='https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png';

    var mapqUrl='https://otile1-s.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.jpg';

Alternately; there could be additional code to select the URL for MapQuest (OpenStreetMap doesn't need code; it just needs less in the url string).

    var osmUrl='//{s}.tile.openstreetmap.org/{z}/{x}/{y}.png';

    var mapqUrl = (document.location.protocol == 'https:' ?
            ''//otile1-s.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.jpg''
        :
            ''//otile1.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.jpg'';

There are references to https within the code.

 <script type='text/javascript' src='https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.10.4/jquery-ui.min.js'></script>

These could also be removed and just // used instead.

Finally (and this is being pedantic), the only other reference to http is in a comment:

// sprintf (from http://stackoverflow.com/questions/610406/javascript-equivalent-to-printf-string-format )

Which does not need editing; however also works with https. :)

[1] Found on http://developer.mapquest.com/widget/web/products/open/forums/-/message_boards/view_message/361015

take encoding expressions

current openipmap prototype doesn't take expression->hostname mappings (ie. regex). Big problem with this is that there is a large potential for large scale error.
i've been thinking of a process like this, to

• user uploads encodings to openipmap (and/or work with ddec.caida.org)
• a reverse DNS scan ( for instance from scans.io ) finds ips used in these mappings
• we look at the last X days of traceroutes in ripe atlas to see if we actually see these
• we put these traceroutes on maps, and have people verify the individual hostname->geoloc