paambaati/solonglatlng

A simple web app that lets you search a coordinate on a GeoJSON

solonglatlng

A simple webapp that lets you search a coordinate on a GeoJSON layered on top of a Map view.

📣 Note that this project was purpose-built for a coding challenge (see problem statement).

🚀 Quick-Start

docker pull gpdocksthings/solonglatlng
docker run --init gpdocksthings/solonglatlng

🛠️ Setup

Before you run this app, make sure you have Node.js installed. yarn is recommended, but can be used interchangeably with npm. If you'd prefer running everything inside a Docker container, see the Docker setup section.

git clone https://github.com/paambaati/solonglatlng
cd solonglatlng
yarn install

👩🏻‍💻 Usage

yarn run dev

You can then access the app at http://localhost:3000

This URL should return a valid search with the bundled GeoJSON — http://localhost:3000/?lat=80.1256269&lng=12.9262308

🐳 Docker Setup

docker build -t solonglatlng .
docker run -p 3000:3000 -ti solonglatlng

🧩 Design

The app is built with Next.js, TypeScript, react-leaflet and stream-json.

Next.js is used to build the app on top of React.js and Next.js's built-in SSR-powered API.

🗺️ Map Operations

The Map operations are kept intentionally simple.

1. The original GeoJSON is scanned linearly, so searches are O(n) complexity. For very large datasets, it is recommended to either run the queries on top of a GIS-capable database like PostgreSQL or build a custom index on top of R trees or k-d trees.

2. Instead of writing our own logic to find if a given polygon contains a point, I've used the D3 library's geoContains() method.

#2 now improves search performance dramatically!

🎨 UI

The UI is built on top of the wonderful Tailwind CSS library.

1. Tailwind CSS is a radical rethinking of how we write CSS.

   Instead of using fullblown UI libraries like Bootstrap or Material UI, Tailwind gives you only the building blocks ("utility classes") to build your own UI components, without having to fight overriding opinionated styles. It lets us extract component classes into custom components, giving us a design system from day 1. It also allows for lesser cognitive overload, as scanning the classes gives us a clear picture of what the component does exactly.

🚚 CI/CD

The CI/CD setup is straightforward.

1. On every commit, Github Actions is used to continually build the app, package it into a Docker image and push it to Heroku's container registry.

   <See CI pipeline>

2. On a git tag or a release, the latest image is deployed to Heroku and available at https://solonglatlng.herokuapp.com/

   <See CD pipeline>

On a production grade pipeline, we'll be tagging the Docker image with the git tag for traceability.

🕵 Missing Pieces & Gotchas.

1. Unit tests.

   Though critical, for a lack of time, I did not write unit tests.

2. Search API will timeout in 30 seconds.

   Heroku's free public dyno times out after 30 seconds, so any search API request that takes longer than that will timeout (although that shouldn't really happen after #2).

   For testing, you can use these 2 URLs —

   1. https://solonglatlng.herokuapp.com/?lat=77.2025745&lng=27.7811323 (matches first Feature in the GeoJSON).
   2. https://solonglatlng.herokuapp.com/?lat=73.1991148&lng=23.3492689 (matches after a few hundred Features).

3. The original sample GeoJSON was > 200 MB, but Github only allows checking in files under 100 MB. To work around this, I've downsampled the GeoJSON to 2 locations and checked the file in, but when building the Docker image, I'm downloading the original GeoJSON from a copy on my Dropbox.

solonglatlng's Issues

Lookups are way too slow

Currently, the search is O(n) and takes really long for coordinates farther down the GeoJSON.

A faster search implementation (perhaps based on an implementation of R-trees) should be built.

App crashes on Heroku 😥

On Heroku, the app crashes when it tries to read the GeoJSON file. It works fine on local environments (8 GB RAM).

2020-03-22T06:06:23.033135+00:00 heroku[web.1]: State changed from up to crashed
2020-03-22T06:06:22.917561+00:00 app[web.1]: 
2020-03-22T06:06:22.917579+00:00 app[web.1]: <--- Last few GCs --->
2020-03-22T06:06:22.917580+00:00 app[web.1]: 
2020-03-22T06:06:22.917581+00:00 app[web.1]: [25:0x55f7550d6e80]  1953332 ms: Mark-sweep 194.4 (258.6) -> 194.4 (257.6) MB, 15.3 / 0.0 ms  (average mu = 0.900, current mu = 0.002) last resort GC in old space requested
2020-03-22T06:06:22.917581+00:00 app[web.1]: [25:0x55f7550d6e80]  1953350 ms: Mark-sweep 194.4 (257.6) -> 194.4 (257.6) MB, 17.8 / 0.0 ms  (average mu = 0.799, current mu = 0.001) last resort GC in old space requested
2020-03-22T06:06:22.917582+00:00 app[web.1]: 
2020-03-22T06:06:22.917582+00:00 app[web.1]: 
2020-03-22T06:06:22.917582+00:00 app[web.1]: <--- JS stacktrace --->
2020-03-22T06:06:22.917583+00:00 app[web.1]: 
2020-03-22T06:06:22.917583+00:00 app[web.1]: ==== JS stack trace =========================================
2020-03-22T06:06:22.917584+00:00 app[web.1]: 
2020-03-22T06:06:22.917584+00:00 app[web.1]: Security context: 0x302c539808d1 <JSObject>
2020-03-22T06:06:22.917585+00:00 app[web.1]:     0: builtin exit frame: utf8Slice(this=0x1f86415831b1 <Uint8Array map = 0x36027d241a9>,65536,0,0x1f86415831b1 <Uint8Array map = 0x36027d241a9>)
2020-03-22T06:06:22.917585+00:00 app[web.1]: 
2020-03-22T06:06:22.917586+00:00 app[web.1]:     1: toString [0x11d454faf111] [buffer.js:763] [bytecode=0x2d5e675386f1 offset=31](this=0x1f86415831b1 <Uint8Array map = 0x36027d241a9>,0x037f810404b1 <undefined>,0x037f810404b1 <undefined>,0x037f810404b1 <undefined>)
2020-03-22T06:06:22.917586+00:00 app[web.1]:     2: arguments adaptor frame...
2020-03-22T06:06:22.917586+00:00 app[web.1]: 
2020-03-22T06:06:22.917593+00:00 app[web.1]: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
2020-03-22T06:06:22.955141+00:00 app[web.1]: error Command failed with signal "SIGABRT".

This is most probably due to the very limited RAM (~400 MB) available on the free Heroku dynos. This can be solved by —

1. Pre-building the indexes in batches offline and using something lightweight like RocksSB or SQLite to quickly lookup matched polygons.