ctwardy/mapscore

Probability Map Scoring Portal (SARBayes, George Mason, BYU)

mapscore's Issues

Anti-Reverse Engineering clause

add an anti-reverse engineering page to the new user registration.

Increase size limit for model name

Lanny: I thought the textbox size limit for model name is too short (or maybe I just have really long names for my models).

Add option to download test case??

Check with Bob Koester on this.

Lanny: It might also be nice to be able to download the test case scenario with a button. I know I can copy and paste, but I am lazy.

Automatic Simple Models

Have the site run some simple models on each case when it's uploaded:

• Straight distance model using overall stats from LPB
• Straight distance model using the category

More ambitious (for another task):

• Syrotuck model
• ESRI model

Profiles

implement profiles

Look into adding a Captcha to registration

Add some sort of captcha to registration.

Fix Usability Test Issues

Switch to permanent webserver

We're running on the python server now, which is meant for debug/development, not production.
See also the issue "speed".

public test data

Make some tests fully available as public tests, so people can see just how the metrics etc work. I guess this would
mean the test case parameters, and both pixel and geo coordinates for IPP and find location.

95% circles

Add 25%, 50%, 75%, and 95% circles on the test image, just for show.

First version: use "Hiker" or overall average distances.
Second version: use the ones for the correct class.
??Third version: use a Bayes net model??

Make grid size more prominent

Lanny: With the “Grid Alignment Verification”, it helps to know what is the size of the grid you are using, because mine might be different. (I just noticed the 5001x5001 in test case. Maybe include this in the instructions for the “Grid Alignment Verification” page?)

password reminder feature

Once passwords are secure, have a reminder feature? We can just skype each other for now.

error - loading imgs after save

When you have an image already saved in the server, the website often alters this image as part of a url requests and then renders the altered image as part of a template. However, about 30 - 40% of the time, the template will render the previously saved image version without the alterations. you then refresh the page, and the altered image is loaded. It does not seem that a sleep time delay after saving corrects this problem. Thus, we need to find out how to alter, save, and load an image consistently without error.

Modify score so uniform gray gets 0 not -1

Bob notes that a uniform gray should get the same score as random, not a -1.
To do this, count all pixels with the same score as the true pixel, and add half above and half below.

Document the change (can be a separate ticket).

Validate input

Ensure all forms are validated.
(Add "Security" label to the issue tracker here.)

Delete models

Allow users to delete models.

Add New Cases (9 Nov)

Add Bob Koester's new cases -- the ones with good coordinates.

no issue just a milestone test

Sort test cases by category

Categorize test cases, and in the leaderboard, allow sorting / selecting by category.

header menu: link to leaderboard

Add a quick Leaderboard link to the top menu.
If you happen to find a more pleasing color (white or Mason green perhaps), that's fine too. :-)

Leaderboard: s/category/scenario/g

Substitute "Scenario" for "Category" on the leaderboard.

Prepare google email regarding rings

Draft an Email for Google regarding % probability rings on satellite images.

Beta-test mode

Not sure what we meant, but it was in my notes from the 9 July meeting that we should have a beta-test mode.

password security

We shouldn't send the password in cleartext. Find the standard widget/protocol for password handling.

Repair Formatting problem on GridSync pages

the right hand image formatting on the grid sync pages is off. Repair the html code that dictates how a browser displays the images.

Filter by Country/State

Allow participants to filter cases by country or state -- they may only have data for their own region. Doing this will mean that different groups can challenge different test cases.

Leaderboard sorting

Add column sort to leaderboard.
Change "sort" text to "select" or "filter".
Fix typo in "Categroy"

Color Blind

Modify image feedback to use patterns as well as colors.
Use other colors, not red-green, as this is the most common form of color-blindness.

Recruit Users (Charles)

BYU
Paul Dougherty (mobility model)
Naval Postgrad School
Liz @ ESRI
Tom Patterson @ ESRI
Don Ferguson?
MAPSAR group at National Park Service

Scripting

Consider how to support a scripting interface for batch submission and testing.

use case: Team1 develops a new model and wants it tested on all the cases. Team1 script connects to the site, logs in, registers the new model, and requests to run that model on all test cases. Site does protocol tests and begins sending test cases, one at a time.

Report Scale with Metric

When reporting the result, eg .9965, remind people the scale is -1..1, where 0 is random and 1 is perfect.

Instructions: file formats

Site needs text describing the file formats

Home page

Site home page should include a short description, then leaderboard, then funding info. Longer description can be below, or via click.

database error

Tried to get the "sarbayes" profile and got a "MultipleObjectsReturned" error. Guess it didn't like both "crt" and "ctwardy".

Environment:

Request Method: GET
Request URL: http://daggre.org:8000/Account_Profile/?Account=SARBayes

Django Version: 1.3
Python Version: 2.6.6
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'MapRateWeb.framework',
'django.contrib.admin']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

Traceback:
File "/home/sarbayes/Python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/handlers/base.py" in get_response

1.                     response = callback(request, _callback_args, *_callback_kwargs)
   

   File "/home/sarbayes/git/MapRateWeb/../MapRateWeb/Views.py" in Account_Profile

2. Active_account = Account.objects.get(institution_name = Account_in)
   

   File "/home/sarbayes/Python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/manager.py" in get

3.     return self.get_query_set().get(_args, *_kwargs)
   

   File "/home/sarbayes/Python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/query.py" in get

4.             % (self.model._meta.object_name, num, kwargs))
   

Exception Type: MultipleObjectsReturned at /Account_Profile/
Exception Value: get() returned more than one Account -- it returned 2! Lookup parameters were {'institution_name': u'SARBayes'}

encrypt passwords!

Passwords and other possibly sensitive items are sent cleartext in the url!

profile picture

My profile picture (SARBayes) changed to a giant question mark. Uploading a new image just changed the size of the question mark. But on some screen the old sarbayes logo appears. Are there separate user and organization photos?

New Cases and New Fields

We need new test cases, and we need these fields:

• Country
• State
• (anything else?)

Confidence / StDev / #cases

The leaderboard should indicate # of test cases for a score (I think it does), alert people to scores based on <10 cases, and possibly estimate the standard deviation or confidence interval for each score.

Suppose #scores = n. One way to estimate is to take other scores based on many cases, create random subsets of size n, and track the spread over those subsets. For N >> n:

Repeat 1000 times:

1. Shuffle the N scores
2. Partition the scores into m subsets of size n. (where m = N % n)
3. Record the m averages
4. From the dataset of 1000*m averages, get the 95% bounds and/or the stddev

It may be that 1,000 times is too many, but use it for now.

Use .csv format for case data

Use the CSV module to allow reading and writing of .csv files instead of having to delimit them with '|' first.

speed

When user1 is uploading a rating image, the site hangs for user2. It should have better threading, or a pre-emptive "Wait a minute, uploading a rating image for another user..." prompt. Better to provide some sort of concurrency.

email regexp

add (...)+ to your trailing regexp for the email, or look up standard regexps, to allow c4i.gmu.edu as well as .gmu.edu.

Model description

Add a description field for the models. Editable.

Rename "Submit Query" buttons

Lanny: I see many “Submit Query” buttons. Please rename those accordingly (e.g., Login).

Implement GridSync once per model

For the Gridsync, implement once per model rather than once per test case.

Video tutorial

Create a video tutorial for using the system.
Add "documentation" label to the issue tracker.

Test Cases

Create test images for checking the scores. Uniform, circle in the right area, exact point, etc.

Add NoRobots to site

Look into adding NoRobots to site to prevent indexing of database elements.

Describe metrics

"Documentation": describe the metrics being used.

Leaderboard

Implement

Unit tests

Big task -- implement a unit testing framework and write unit tests for key routines.
Add "Unit testing" label to the issue tracker.