tracyhenry/kyrix

Interactive details-on-demand data visualizations at scale

kyrix's Issues

new viewport function should get viewport width/height and possibly rendering params

compiler should give warning/error when canvas size is smaller than viewport size

should let renderers directly append stuff under a <g>

rather than creating a new inside the renderer themselves

should allow specification of zooming bbox

right now zooming bbox is using the placement bbox, which might not be visually ideal.

Issue with using port 3306

Error messages:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project tile-server: An exception occured while executing the Java class. Address already in use -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project tile-server: An exception occured while executing the Java class. Address already in use

rendering params don't allow functions or objects

zoom back button breaks when being clicked right after it's enabled

change most REST methods to GET (but not /project for example)

/first, /tile, etc should all be GET method for all the usual reasons:
https://www.google.com/search?q=why+get+post+method

In practice, GET is easier for testing, automation and it's bookmarkable. It's limited to 2K request sizes (URL length) and dangerous for calls that change state, e.g. https://stackoverflow.com/questions/786070/why-should-you-delete-using-an-http-post-or-delete-rather-than-get

easy debug

1. developer shouldn't need to delete project definition before modifying the spec

2. after writing new spec, backend should automatically detect the changes and re-precompute accordingly

something wrong with tile cache

things are broken with lzoomscatterplot.

dbox should use figure out delta queries

Dbconnector statement not closed properly

right now methods like getQueryResult don't close statements, this is harmful

axis should take render data as input.

or maybe predicates

layer constructor should receive a transform object, instead of an id

installation/startup error if project table not created first

Ubuntu 16.04, Postgres 10.0, NodeJS v8.12.0, Maven (mvn) 3.3.9, Java 8,

The setup instructions read: "Go to tile-server/. Run mvn compile to build the server. Run mvn exec:java -Dexec.mainClass="main.Main" to start the server. After the server starts, it will prompt that it did not find the spec of the app and is waiting for it..." and then "cd compiler/examples/nba; node nba.js"

This results in "kyrix@kyrix ERROR: relation "project" does not exist at character 21" (seen in postgres logs). The bad SQL: select content from project where name = 'nba'

Perhaps the best workaround is to manually create the table using this idempotent SQL:
CREATE TABLE IF NOT EXISTS project (name VARCHAR(255), content TEXT, dirty int, CONSTRAINT PK_project PRIMARY KEY (name));

pushing predicate generation into backend (important security issue)

right now the predicate functions are run in the client. The resulting predicates are then passed to the backend. This will invite SQL injections...

Tentative solution is to put the generation back into the backend. Also, it's necessary to write a parser to ensure the predicates generated are in "good form"

complete literal zoom interactions

1. smooth transition
2. literal zoom buttons
3. geometry-based literal zoom (using bounding box info)

cannot modify raw database when the backend is running

the connections to the raw databases are not closed.

config.txt.example has one line that's not documented

the line above the last line

NBA example / box scores: display placement is absolute, causing cropping on small windows

examples/nba/placements.js:14 looks suspicious?
boxscorePlacement.centroid_x = "con:1000";
boxscorePlacement.centroid_y = "con:500";

compiler --> backend server breaks on Windows

[compiler] check #layer > 0

stop sending geom

backend should recompute index when transform-related parameters are changed

clever bbox table field type

set type according to raw data type

and type TEXT is not long enough for geoJSON strings

http://www.peachpit.com/articles/article.aspx?p=1752305&seqNum=2

inconsistent indentation method in backend

should change all to spaces

dbox 'fails' when panning is fast

right now front-end uses pendingBoxRequest to disable concurrent dbox requests.

However, this ignores all subsequent requests while a dbox request is still being processed, and will result in incomplete visualizations when the user already pans outside the returned new box.

should allow newPredicate to be an empty string

[compiler] should check rendering param exists

or make an empty dictionary as a default value

[compiler] placement existence

check that dynamic layers have placement object and that static layers don't.

canvas request can be removed

canvas request is used by the frontend to fetch canvas info and static layer data. However, we can probably assume these static info can fit in frontend's memory, and completely eliminate this kind of requests, which sometimes cause laggy animation.

[backend] spatial index and sorted tuple mapping break for 100m records

wrong 'this' reference in startLiteralZoom

'this' actually is the browser window.

write an indexer that uses built-in postgres gist index

It is confirmed now that postgres has built-in geometry types and GiST indexes. So we should write a new indexer that uses it, test if it works fine, and then possibly get rid of PostGIS, which is a pain to install.

https://www.postgresql.org/docs/9.4/functions-geometry.html
https://www.citusdata.com/blog/2017/10/17/tour-of-postgres-index-types/
https://medium.com/@Alibaba_Cloud/principles-and-applications-of-the-index-types-supported-by-postgresql-481f59bab67d

tile server better error/exception handling

backend should not completely shutdown when there is exception occurring during either precomputation or tile requests

[compiler] postgres password error does not have prompt

make backend stateless

The backend is now stateful because it memorizes the user's history of positions.

This history should be instead maintained by the client, to make the backend stateless and parallelizable

Check that columns in the placements file are in table

*TODO: add more detail

add back SQL predicate specification

should allow transform_func to be an empty string when creating a transform

Also, transform object should be able to be specified as null
Also, get rid of all transform_id stuff.

zoom/panning "state" not stored in URL (e.g. #z=<level>&x=XXX&y=YYY)

Web users expect to be able to reload a page and go back to the current view.
In particular, this means that kyrix views are not shareable to other users and can't be bookmarked.

(to demonstrate this, pick any example, zoom/pan and then hit reload in the browser or try to copy/paste the URL into a new tab/window)

usmap example weird shift after zooming into Hawaii

it works for all other states.

semantic zoom without entering animation

this is needed in many applications, e.g., Ticketmaster seat map, zoomable circle packing

smoother zooming back transition for geometric & geometric semantic zooms

right now it's an abrupt view change

dbox history fails when zooming back from a canvas having only static layers

fix is to send 'hasBox' to backend, similar to how it is done in mgh branch to cope with this statefulness

tiling breaks for non-deterministic rendering function (like random colors)

not sure how to fix this issue of static tiling.

Dynamic box will not have this problem.

nashorn-common-js breaks when requiring d3

This is because nashorn-common-js does not implement native node modules (e.g. http) while d3-request requires http.

The current workaround is to comment out the lines requiring d3-request in node_modules/d3/build/d3-node.js

wrong use of canvas cache in jump.js

https://github.com/tracyhenry/Kyrix/blob/master/front-end/js/jump.js#L380

postData was used before assigned

newScale function for zoom

Letting the zoom specify a function to calculate the scale of the destination canvas.

This is useful for creating ticketmaster-like geometric_semantic_zooms, where the zoomed-in canvas need to be scaled differently for different zooming entities.

And, an initial scale is needed for the initial canvas.

one-command script to load new datasource

so users can quickly move beyond the NBA example.
do this for both docker-compose and kubernetes
follow-on: create instructions on how to format the data (if not already present)
follow-on: create instructions on how to scale up for very large datasets that must be loaded in parallel