geobin-io/geobin.io

[DEPRECATED] Inspect HTTP requests with Geographic Data.

geobin.io's Issues

path to object is too shallow for feature collections

When posting a large feature collection (example: www.traileditor.org/trailheads.json), path to each point is [], so the content in each popup ends up being the entire feature collection. @aranasaurus is there a way to traverse the feature collection for the sake of showing the specific feature that a point on the map represents?

Custom slugs for geobins

Add "Create this bin" option to bins that are 404.

I should be able to type a URL like http://geobin.io/test and see "Create this Geobin" instead of "Create a new Geobin". This makes it more consistent with sites like talky.io or wikis.

Smaller point and radius things should be added to the map on top of larger ones.

Currently I think we're adding the geos in the payload to the map in the order they are listed. We should either change the server sort the found geos by radius (if they have one), or make the client do it as it adds things to the map.

Specific use case is a Geotrigger callback will generally have two geos found in the payload, one for the trigger itself, and one for the location update that fired it. The location update will always be smaller than the trigger, and will likely be completely overlapped by it and therefore unclickable on the map.

Add more example code on new geobin page

For now we only have a curl example:

curl -X POST \
-H "Content-Type: application/json" \
-d '{"lat":45.5218,"lng":-122.6773,"rad":1000}' \
geobin.io/NUMGz0br0Y

As suggested by @jyaganeh in #89, it would be nice to have samples for other languages, e.g. Ruby, Python, JavaScript, Go.

visibility toggle resets when toggling between 'list' and 'details' view

cool project!

steps to repro:

1. create a new bin, fire the example cURL request and zoom in to flannel town.
2. toggle the eyeball icon to hide the graphic
3. select 'details' and notice that the visibility toggle has been reset even though the graphic is still hidden.

server todos

• use headers as part of ZADD member
• on create, put in an empty value as a first entry in the set, and set expiration for 48 hours once
• give back 404 when someone hits a URL that we don't have an entry for in redis
• add index route with create button
• serialize request information as a struct before putting into redis, and deserialize on the way back out to make templating straight forward

Log received requests that we find no geo data in somewhere

This will provide us a way to check for geo data that we aren't automatically detecting and improve the geo detection functionality.

change bin expiration to be based on time since last access

Currently there's a hard expiration on bins, which is less than ideal if you spend time setting up a bin and want to keep using it. We should change bin expiration to keep bins around until they are unused for some amount of time.

This could potentially be worked on leading up to or as a follow on for #82

Show connected/disconnected status somewhere

If the browser loses the websockets connection to the server, it should show some sort of indicator as such. I think I've seen this done somewhere else with a little green/red circle in the header bar.

Detect Esri JSON in payload

Improve code coverage for client-side tests

Current status:

File	% Stmts	% Branches	% Funcs	% Lines
js/	60	32.35	66.2	59.92
app.js	28.57	100	50	28.57
controllers.js	54.84	7.14	47.06	54.84
directives.js	33.73	11.54	45	33.73
filters.js	91.43	92.86	88.24	93.75
services.js	90.57	35.71	93.33	90.57
---------------------	-----------	-----------	-----------	-----------
All files	60	32.35	66.2	59.92

We need to make those numbers bigger!

Benchmarking notes

Steps to reproduce:

1. create a file called geobin.json containing some json to post: {"geo":{"latitude":"45.5165","longitude":"-122.6764"}}
2. create a geobin
3. test once: ab -n 1 -p geobin.json -T "application/json" -v4 "http://127.0.0.1:8080/{BIN_ID}
4. make 1000 requests with concurrency = 10: ab -n 1000 -c 10 -p geobin.json -T "application/json" "http://127.0.0.1:8080/{BIN_ID}"

Results:

Server Software:
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /w4Qa9N2tGr
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   0.248 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      116000 bytes
Total POSTed:           200000
HTML transferred:       0 bytes
Requests per second:    4038.66 [#/sec] (mean)
Time per request:       2.476 [ms] (mean)
Time per request:       0.248 [ms] (mean, across all concurrent requests)
Transfer rate:          457.50 [Kbytes/sec] received
                        788.80 kb/s sent
                        1246.30 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     1    2   0.7      2       6
Waiting:        1    2   0.6      2       5
Total:          1    2   0.6      2       6

Percentage of the requests served within a certain time (ms)
  50%      2
  66%      3
  75%      3
  80%      3
  90%      3
  95%      4
  98%      4
  99%      5
 100%      6 (longest request)

The requests complete basically immediately, then it took about ~4sec for all 1000 of them to show up in my geobin. Not too shabby!

502 bad gateway

https://geobin.io is no longer serving the site :(

Wanted to show a good mapping interface to a colleague as an example. Sad to see it appears to be dead.

travis build error on `go get -t`

Getting a build error in travis: https://travis-ci.org/geobin-io/geobin.io/builds/54944184#L108-L112

$ go get -t
package github.com/go-redis/redis
    imports github.com/go-redis/redis
    imports github.com/go-redis/redis: code in directory /home/travis/gopath/src/github.com/go-redis/redis expects import "gopkg.in/redis.v2"
The command "go get -t" failed and exited with 1 during .

Looks like a naming convention changed for redis. /cc @CourtF @jyaganeh @aranasaurus

Render bug in google-chrome-stable on linux

• Disclaimer, I believe this is a bug in the browser or with esri leaflet, not necessarily with geobin, as opening the geobin in other browsers does work.

URL:
http://geobin.io/xCPZsm0y5k

When I have the page open and post to the url:

When I open the same bin in a second window or tab:

Seems like the esri maps just get hung up in some intermediate render step, as the screen does momentarily look just like this in other browsers before cleaning itself up.

Roadmap

Talked with co-conspirators, here are some ideas that came out of it:

decouple web application from server

• continue API improvements on api.geobin.io repo
• continue client improvements on geobin.io repo

This would allow us to improve and deploy both pieces separately, develop in parallel rather than tightly bundle client and API releases, and more easily modularize and generalize logic for potential separate go/node/whatever mini-projects under geobin-io.

There were other things but I forget them now. @CourtF @aranasaurus @jyaganeh please add thoughts, wishes, aspirations. The above is my main wish right now. If that happens methinks tackling issues and deploying small things more regularly would be easier.

whitelist geotriggers IP in the rateLimit middleware

Otherwise trigger callbacks will stop working if geobin gets enough use

Add number of posts received to "Your History" dropdown

The dropdown currently shows the bin name and time remaining for the bin. Would be great to also show the number of requests the bin has received so they can be identified easier.

make a pass for logging

would be nice to use something like what is described here:
http://www.goinggo.net/2013/11/using-log-package-in-go.html

Replace curl sample with embedded curl

https://www.embedcurl.com

provide bounding box for each bin

• It would be nice to make the bounding box of all features in a bin available, both in the client and API.
• It would also be nice to have the bounding box of all features per request made available, both in the client and API.

http://geojson.org/geojson-spec.html#bounding-boxes

expose yourselves

@jyaganeh @CourtF @aranasaurus it's lonely out there 😿

Get go tests running in travis

• mock up redis in our tests
• solution for config.json not existing when tests run (create a temp file and then remove, or have config be a command line argument and provide a test config)

Geobins expiring pre-maturely?

http://geobin.io/xCPZsm0y5k

This no longer exists, but as you can see in #118, it certainly did exist, and was created less than 48 hours ago. Not sure what happened there or if the timeout has been shortened in some PR I didn't see. The language in the History UI does state a minimum 48 hour keep-alive on bins.

UI Goals

• More room in the sidebar
• List/Detail work in sidebar
• Focus on Items in list, zoom map to extents
• Toggles and Layers
• Toggle for incoming data to map or not map on auto

Be able to paste some json in somewhere and have geobin.io generate a request to itself.

WebSocket error (response code: 400)

Expected Behavior

WebSocket connection should work fine.

Observed Behavior

WebSocket handshake fails.

WebSocket connection to 'ws://geobin.io/api/1/ws/PAEBpoqQJt' failed: Error during WebSocket handshake: Unexpected response code: 400

Steps to Reproduce

Create a new geobin, check the JS console. Try using the cURL example. POST is successful but page doesn't update.

finish & ship custom bin IDs

@aranasaurus What's left to finish? Stuff from #82, #101, sanity check from a gopher?

Let's make a list, check it twice, get this bad boy out the door.

Detect street addresses in the payload and geocode them

parse geo objects out of geotrigger callback payloads

This will be challenging because we have two geometry objects in here, one of which is definitely not standard geojson (lat/lng/accuracy) and one of which may be geojson or a geotrigger style point geo object (lat/lng/distance).

Sample payload:

{
  "triggeredAt": {
    "iso8601": "2014-04-26T22:39:40.355+00:00",
    "unix": 1398551980
  },
  "trigger": {
    "triggerId": "Barista",
    "condition": {
      "direction": "enter",
      "geo": {}
    },
    "action": {
      "callbackUrl": "http://geobin.valis.pw/debug",
      "notification": {
        "text": "coffee time"
      }
    },
    "times": 0,
    "rateLimit": 0,
    "tags": [
      {
        "name": "trigger:Barista",
        "deviceTagging": true,
        "deviceLocation": false,
        "deviceToken": false,
        "deviceList": false,
        "triggerApply": true,
        "triggerUpdate": true,
        "triggerDelete": true,
        "triggerList": true,
        "triggerHistory": true,
        "discoverable": false
      },
      {
        "name": "test",
        "deviceTagging": true,
        "deviceLocation": false,
        "deviceToken": false,
        "deviceList": false,
        "triggerApply": true,
        "triggerUpdate": true,
        "triggerDelete": true,
        "triggerList": true,
        "triggerHistory": true,
        "discoverable": false
      }
    ]
  },
  "device": {
    "deviceId": "CzluaPdV41St2o5S",
    "tags": [
      {
        "name": "device:CzluaPdV41St2o5S",
        "deviceTagging": false,
        "deviceLocation": false,
        "deviceToken": false,
        "deviceList": false,
        "triggerApply": true,
        "triggerUpdate": true,
        "triggerDelete": true,
        "triggerList": true,
        "triggerHistory": true,
        "discoverable": false
      },
      {
        "name": "test",
        "deviceTagging": true,
        "deviceLocation": false,
        "deviceToken": false,
        "deviceList": false,
        "triggerApply": true,
        "triggerUpdate": true,
        "triggerDelete": true,
        "triggerList": true,
        "triggerHistory": true,
        "discoverable": false
      }
    ]
  },
  "location": {
    "latitude": 45.51986460661744,
    "longitude": -122.67545711249113,
    "accuracy": 8,
    "trackingProfile": "adaptive",
    "timestamp": "2014-04-26T22:39:26.513+00:00"
  }
}

How should this be parsed and represented? Leaflet can render a circle (point+radius) but geojson does not have a concept of point+radius.

Add google analytics

Add GA so we can get usage statistics

fixing API GET requests broke WebSocket requests :(

@aranasaurus I think making all non-POST requests to /api/ serve the index has broken requests to /api/1/ws/*.

Example error:

WebSocket connection to 'ws://localhost:8080/api/1/ws/dmH2fKPSoc' failed: Error during WebSocket handshake: Unexpected response code: 200

Release Todos

• documentation
  • API
    • review & finalize
    • publish to static pages for site
  • readme
    • consolidate/link server & client
  • style docs
  • provide navigation between docs
• client
  • swap out angular-router for ui-router (#52)
  • use counts route for # of requests on index (#53)
  • implement separate list & detail views
  • finish user interface (#45)
• server
  • fix API GET requests -- GET /api/1/create not redirecting (it's creating!)

Request list looks bad on Safari

In Safari it renders the date with WORDS. In Chrome it's numbers. This causes a width issue.

Handle 404's when bin's die on client-side

Zoom in on last location received when loading an existing bin

From the home page, clicking on a recent bin loads that bin to a zoomed-out map of the world. It should probably zoom in to something sensible for the bin, like the last location received.

Concatenate & minify client assets

Currently we're sending 19 scripts and 8 stylesheets when serving the client assets. This is bad. Ideally should be 1 & 1. I'd also like to get rid of jquery & bootstrap.js and switch to using angular-bootstrap directives, but that's another issue.

I wasn't able to find anything like sprockets in the go package ecosystem that does this well yet, so unless server folks can provide a smart way to wrap this up I'll probably end up writing a script to do this that can be run as npm run build or something along those lines. Might end up moving the contents of static/app to a client directory that builds to static to keep things clean.

I'd rather not keep the built assets in version control as it will just be a bunch of minified noise, so if there any suggestions as to how to do that efficiently with the build process we have in place now please let me know (@aranasaurus @jyaganeh @CourtF).

convert to public archive

Hello fellow geobineers!

I think this project might be ready for the archive.

With your consent, may I go ahead and add a deprecation warning to the readme and archive the repo?

cc @aaronpk @aranasaurus @jyaganeh

move global vars into server struct

var config = &Config{}
var client = &redis.Client{}
var pubsub = &redis.PubSub{}
var socketMap SocketMap

These should be fields on a geobinServer struct that has a ServeHttp method so that it can be passed in to http.Handle

Demo link or screenshot

It'd be nice to be able to show people how the site works at a glance, either with a screenshot or a bin that has some prepopulated data in it.

Example lat/lng or lng/lat?

I was just been bitten by Twitter::Tweet#geo returning lat/lng and Terraformer wanting lng/lat. What's the order here in this example on the homepage?