biodiv/anycluster

Server-side clustering of map markers for (Geo)Django

anycluster's Issues

Application integration with pip install

Hello,

Maybe you can integrate your application with pip? Or add setup.py file.
In this way people could install like
pip install git+https://github.com/biodiv/anycluster.git

Thank You

Possible Bug in kmeansCluster(self, clustercells, filters)

Hi,
I am really getting some inconsistent behavior with anycluster and I am wondering if you could help me understand what is wrong. I am getting different results when I use anycluster from a web request and when I run it on the console. My theory is that the kmeansCluster function either has a bug with it or is not playing nice with my setup. My issue is that the map never shows a point on zoom level 3 and at other zoom levels, not all of the clusters always show. Here are my findings:

Running from the shell things look fine:

Python 2.7.5 (default, Aug 25 2013, 00:04:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from anycluster.MapClusterer import MapClusterer
>>> viewport6 = {'left':133.82065082500003, 'top':66.88222838558252, 'right':28.87924457500003, 'bottom':-8.875300215087336}
>>> c = MapClusterer()
>>> c.zoom = 3
>>> c.gridSize = 128
>>> cells = c.getClusterCells(viewport6);
VIEWPORT(wgs84datum, 4326, longlat): {'top': 66.88222838558252, 'right': 28.87924457500003, 'bottom': -8.875300215087336, 'left': 133.82065082500003}
>>> print cells
['9,7', '9,8', '9,9', '9,10', '9,11', '9,12', '10,7', '10,8', '10,9', '10,10', '10,11', '10,12', '11,7', '11,8', '11,9', '11,10', '11,11', '11,12', '12,7', '12,8', '12,9', '12,10', '12,11', '12,12', '13,7', '13,8', '13,9', '13,10', '13,11', '13,12']
>>>

However, when I run this query to get back the json cells from my browser:

http://localhost:8080/anycluster/kmeans/3/128/?left=133.82065082500003&top=66.88222838558252&right=28.87924457500003&bottom=-8.875300215087336&cache=load

I get the empty set back:

[]

My theory is that the webquery is going through kmeansCluster and things are getting messed up there. I have tried this with a bunch of different google maps version. Here are the settings I am using:

var anyclusterSettings = {
    mapType: "google", // "google" or "osm"
    gridSize: 128, //integer
    zoom: 3, //initial zoom
    center: [37.9, -98.6500523], //initial center in lng lat
    MapTypeId: "TERRAIN", 
    clusterMethod: "kmeans", //"grid" or "kmeans" or "centroid"
    iconType: "exact", //"exact" (with exact cluster counts) or "simple" (with rounded counts)
    singlePinImages: {
//        'pinimg': 'icons/ppl_locations.png' //optional, use in conjunction with django settings: 'ANYCLUSTER_PINCOLUMN'
    }
}

I'm at a loss for what the cause of this is. Any ideas?

Thanks,
Larry

Join on field 'location' not permitted.

I am getting this exception:

FieldError at /anycluster/grid/2/64/
Join on field 'location' not permitted. Did you misspell 'within' for the lookup type?

at this point:

File "/srv/www/cwu/anycluster/clusterer.py" in gridCluster
  697.                     pin_count = Gis.objects.filter(Q_filters).count()

Here are local vars at that point (idk if useful):

gridSize    
64
filters 
[]
self    
<anycluster.clusterer.MapClusterer instance at 0x7fc24834bc68>
Q_filters   
<django.db.models.query_utils.Q object at 0x7fc24830a510>
zoom    
2
poly    
'POLYGON((-0.000000 -21.943046, -0.000000 -40.979898, -22.500000 -40.979898, -22.500000 -21.943046, -0.000000 -21.943046))'
clustercells    
set([10,10', '10,11',  '10,12', ...])
cell    
'7,6'
cell_topright   
<Point object at 0x7fc2482d7370>
lookup  
'location__within'
gridCells   
[]
kwargs  
{}
srid_db 
4326
cell_bottomleft 
<Point object at 0x7fc2482d7250>

TypeError

Hi,
I have been trying to get anycluster working for a project I am doing. The error I keep receiving is:
TypeError: set(['1,2', '0,2', '3,2', '2,2']) is not JSON serializable

I'm running Django 1.6.1 Python 2.7.5
I have a psql database with all the relevant extensions installed. The model/field I am using to cluster is defined as follows:

class PlaceMark(models.Model):
name = models.CharField(max_length=255)
description = models.TextField()
geom = models.PointField(srid=4326)
objects = models.GeoManager()
layer = models.ForeignKey(Layer)
def unicode(self):
return self.name

In settings, I have added:
#anycluster settings
ANYCLUSTER_GEODJANGO_MODEL = "locator.PlaceMark"
ANYCLUSTER_COORDINATES_COLUMN = "geom"

The error originates from my main map page. My console shows that the specific page being requested is:
http://localhost:8080/anycluster/kmeans/2/256/?left=-180&top=30.000000000000018&right=180&bottom=30.000000000000018&cache=load&time=%3E%3D_2012-02-26&is_doubt=%3D_FALSE&_=1391549514849

Any ideas?
Thanks,
Larry

Reporting

Hi,
I am interested in giving my users the ability to download reports in CSV format. I am able to do this with a standard call to a view using Django. I tried to do this via your anycluster.getAreaContent javascript function but am having issues. I would prefer to not have this be an AJAX call and just call the view directly which would initiate the download. The problem is that there are a number of variables that I need for this request such as geoJson, grid, zoom, and filters. I have not discovered an elegant way to pass these variables to my view. My plan was to stuff these variables in a hidden input form field and update them during each loadEnd() event. This seems clunky but doable. Before I go down that road, I am wondering if you have any alternative suggestions.

Thanks,
Larry

Ambiguity with filters, array or dict. Suggestion

Hi, I'm really thankful to this project and all the work you've put in. It's surprising there is no alternative to AnyCluster in server-side clustering area in Python/Django. Kudos!

Regarding filtering, I've noticed some ambiguity in the code (JS vs Python) and obviously the docs.

In the method constructFilterstring in MapCluster.py, filters is treated as a dictionary which is in-line with the docs

for column in filters:
            filterparams = filters[column]
...

However, in anycluster.js, filters is treated as if it was an array

addFilters: function(newfilters){
	
	for (var f=0; f<newfilters.length; f++){
			
		this.filters.push(newfilters[f]);
	}

	this.clearMarkers = true;
		
},
	
removeFilters: function(activefilters){
	
	for (i=0; i<= activefilters.length; i++){
	
		delete this.filters[ activefilters[i] ];
	}
	
	this.clearMarkers = true;
	
},

I believe this is a problem.

I would actually suggest, if I may, the array approach. It'd enable filtering multiple times on the same column. It is less limiting than the alternative and one use case is when you'll like to filter on a range e.g. a date range. With the current implementation it'd be really hard to achieve without modifying AnyCluster anyways. I've been using this approach in my current project.

What do you think?

Viewport as geojson

Hi,
Wouldn't it be better if the getCluster javascript function took a geojson representation of the viewport? It already does a conversion before calling the clusterMethod view. This would add the ability to for us to pass in a drawn polygon. I need this functionality for users who want to see clusters/placemarks for a specific area such as along a river. I am adding the Google Maps drawing library so a user can trace out a specific area of the map and only get clusters from that area. What do you think?
Larry

Demo doesn't work

Hi,
I am interested in your solution but the demo doesn't work for me:

csrf

I am using the latest version. Once I cleared my browser cache, the map stopped rendering points. I've traced it to a csrf issue. If I remove the following middleware line from settings then the app works fine:

'django.middleware.csrf.CsrfViewMiddleware',

It seems like the issue originates from anycluster.js

        xhr.open("POST",url,true);

        var csrftoken = getCookieValue('csrftoken');
            xhr.setRequestHeader("X-CSRFToken", csrftoken);

        xhr.send(JSON.stringify(postParams));

I tried messing with the django_ajax_csrf.js but couldn't make much progress. Any ideas? is this a bug?
Thanks,
Larry

(JS) Marker with label broken with latest google maps api update

The MarkerWithLabel (external code) in the javascript is broken and has to be replaced. It massively slowed down the clusterer anyway. If you need a replacement JS, take the one from https://www.anymals.org/static/anycluster/gmap_clusterer.js until I upload a new and cleaned up version.

clusterPopup.html override

Hi,
Is there an easy way to override this file? I really would prefer to not modify files inside anycluster. If I could specify my own template, that would be nice.
Larry

Change separator in parseFilters method

What about to change separator in parseFilters from "_" to less common symbol, for example "__" or something else.

Expose anycluster-client filter types

Using typescript it might be useful to export more of the types in the anycluster-client. Specifically Filter and NestedFilter help when constructing filter and modulation requests.

Question - Markers in Viewport

Hi,
Do you have a suggested way to get a list of all markers currently visible in the viewport? Not clusters but the actual markers themselves. I need to generate a report of visible markers.
Thanks,
Larry

number of database queries

There is a significant performance hit for making SELECT for each cell in gridCluster and kmeansCluster. Have you thought about reducing it to just one (or a few) queries? I am not completely sure that it is possible but I feel it should be and it would improve performance greatly (especially if you have lots of cells). Have you thought about it? I am looking for a way to do it but I would like to hear from you first what you think.

Filter docs

Hi,
Can you write some docs about using filters?
Thanks.

request: Include demo app in repo

Any chance you would include the demo app in the repo so I can see a working example?

thanks

nuids

Hi,
I am not sure this is the best spot to ask this question but I don't have a direct email for you. I am leveraging a number of elements from your anymals site and I had a question about the way you are keeping track of the placemarks that are contained in a cluster that is clicked. I saw this line in your .js:

    var params = {'lat':position.lat(),'lng':position.lng(), 'nuids':googleMap.filters['nuid']};

I am unclear on how you are using a filter to track these. What is the general concept here? I need to be able to pass a list of placemark identifiers to my Django view on click so I can show the popup.

Thanks!
Larry

markerFinalClickFunction on single marker

Hi,
With the latest version from master, I've started having an issue occur on markerFinalClickFunction with a single marker. It appears that the longitude and latitude are not getting passed or the ID is not being sent for lookup. When I click a cluster that is in the final marker stage, things behave normally. However, when I click it for a single marker, I get the following:

Failed to load resource: the server responded with a status of 500 (INTERNAL SERVER ERROR) http://gouda:8080/anycluster/getClusterContent/13/256/?x=undefined&y=undefined&id=0

I've traced this down to getKmeansClusterEntries inside scripts.py but can't make much sense of the root cause. Any chance this is a new bug?
Thanks,
Larry

"Unsupported SRS" exception

I am trying to get this app (which looks promising btw) working but now I am stuck on SRSException at /anycluster/grid/2/64/,
Getting this at:

        /srv/www/cwu/anycluster/clusterer.py in point_ToMercator
        line 161. mercator = SpatialReference("900913")

Any idea how to solve it?

AttributeError at /anycluster/kmeans/2/128/: 'str' object has no attribute 'x'

Getting the exception for kmeans method:

/srv/www/cwu/anycluster/clusterer.py in points_calcPixelDistance
        distance_m = math.sqrt( (pointA.x - pointB.x)**2 + (pointA.y - pointB.y)**2 ) ...
▼ Local vars
pointA  
'0101000020E6100000A0FD481119B760C06E8B321B64165140'
pointB  
'0101000020E6100000079964E42CC65CC0AD6EF59CF4F45040'
zoom    
2
self    
<anycluster.clusterer.MapTools instance at 0x7f2bd02437e8>

Pin Images

Can you elaborate on this new setting?
singlePinImages: {
'dbvalue':'/static/path/to/image.png' //optional, use in conjunction with django settings: 'ANYCLUSTER_PINCOLUMN'

I have a number of layers each with their own icon for a single point. Does your project support this or do I need to use the same marker icon for all markers? I have the relevant setting in my settings file.
ANYCLUSTER_PINCOLUMN = 'pin'

ST_WITHIN

Hi,
I am trying to get getViewportContent working properly for some reporting I am doing. I can get back markers and I can customize the template. That all works fine. The problem for me appears to be with the database call. It doesn't return an accurate set of data based on the polygon I am passing. Have you ever had any experiences like this? It seems as though the SELECT statement inside views.loadAreaContent returns unreliable data. I think this may be a database issue on my end but before I dig too deep, I wanted to see if you have tested this function and if you had any suggestions.
Thanks,
Larry

get_map_content_counts requires nested sql statement

get_map_content_counts returns false count if a nested modulation is used on top of a nested filter in some cases. The raw sql statement should be altered to use nested sql.

Compatibilty with mysql

Hello,
P
We want run Your project on MySQL, but there are a lot of PostgreSQL-specific SQL in the code. Is there any chance to run this project on mysql too? :)

Panning

Hi,
I noticed both in my tests and on your demo at https://www.anymals.org/nx/bigmap/ that the certain clusters do not get rendered when you pan. Here's how you can reproduce on your demo:

• Make your browser window small enough so that when you refresh your map, you only see Europe/Africa. Then pan the window over to where the US is centered. You will notice that not all the clusters show for the US.

• Now refresh the window again and move over so the US is in the bottom left portion of the window. The clusters will render.

I am attaching my screenshot. Not sure what is going on here. I tried turning debugging on in clusterer.py but it started complaining about line 804
print('pins after phase2: %s' %cellpins)

Viewport and Pincount

My goal is to be able to get a list of all the markers in the current viewport. My starting point for this has been pincount. My theory is that pincount needs to be accurate first before I can expect to receive a valid list of markers. What I am finding is that the pincount does not behave as I would expect. If I load my site, I get n number of pins. If I pan around the map, I continue to get n or a little more than n. I would expect the pincount to decrease as I go from the full view of everything down to a smaller sampling. I have also experienced instances where the pincount will double and continue to double as I move around the map. I am leveraging loadStart and loadEnd to see what the value of pincount is.

            window.onload = function () {
                var googlemap = new Anycluster("gmap", anyclusterSettings);

                googlemap.loadStart = function () {
                    console.log("Loading started->" + googlemap.pincount);
                };

                googlemap.loadEnd = function () {
                    console.log("Loading ended->" + googlemap.pincount);

                };
            }

Is this a bug or me?
Thanks!
Larry

Demo is not running

Files missing:

• OpenLayers.js
• infobox_packed.js
• markerwithlabel_packed.js