philipkd/landemic

Landemic client and data source

Landemic is a virtual land NFT on Ethereum available at https://landemic.io. We've designed it to live in perpetuity, which is why we've open-sourced the web component so that you can serve the Landemic site either locally from your machine or on the web.

The web component has two parts: an HTML client and a data service. You can run the full functions of Landemic from just the HTML client, but if you want the charts feature on the left and the cache of tiles on the maps to stay updated, you'll need to need to run the parallel data service. Since the volume is low, you can get by just running the data service manually, however we've set it up so that it's easy to run practically for free on Heroku.

Quickstart: Run client locally

cd client
yarn global add serve
serve -s build

Maintainer notes

The most likely points of failure for future maintainers, in order of when they could be a problem are:

• Data syncer breaks: this could be due to an API change or rate limit from Alchemy. Trick: to check that the syncer is working, you should see a recent (<24h) commit in this repo. Also, you can open the browser dev console on the client, and there is a log output that says "Updated # hours/minutes ago."
• Client doesn't build: Node 18 is being deprecated, so dependencies may have to be managed or replaced to migrate to Node 20
• Slug doesn't compile on Heroku: see same issue above regarding Node 18
• Client doesn't play well with MetaMask: in the dev console, you can see that there are web3 deprecation warnings, but it may be a while before MetaMask breaks backwards compatibility

The most confusing aspects of the client are how it synchronizes data with the blockchain:

• The Drizzle framework, which may or may not be around in the future, watches the blockchain for updates. Look at REACT_APP_DRIZZLE_FALLBACK_URL in .env.development/production as a possible suspect.
• Check Body.js and see fetchAllOwned as the core engine for syncing. You'll notice this has parallels with the files in the api folder. If you are having issues, it may help to create a copy of api/src/codes.js or events.js as a console so you can directly talk to the blockchain.

Another confusing aspect has to do with using Github Pages for serving and data:

• At the previous hosting provider, Landemic could handle "/"-based routing in URLs. However, Github Pages doesn't have a URL writer. As a result, we use the 404 page to fill that role.
• It's normally not considered best practice to save cached data and build files in a repo, but doing so solves some problems for us:
  1. It saves cycles to start from a previous event history of the contract, rather than rebuilding it every time.
  2. Having the build saved means that if for whatever reason the build fails, you'll always have a last working copy.
• There are other solutions that could make these things seem more logical, but one overriding principle has been to keep the number of moving parts as few as possible to help with futureproofing.

Running the HTML client

Build the client

Set Google Maps API key

In client/.env set REACT_APP_GOOGLE_MAPS_KEY for Google Maps. We currently hard-code a default API key, but it is domain-gated, so it will only run locally. If you build the client and host it somewhere, Google Maps will return an error.

(Optional) Set Alchemy API key

In client/.env set REACT_APP_DRIZZLE_FALLBACK_URL. See .env.example. This allows people who aren't connected to MetaMask to still view live blockchain content.

Build commands

Run these in the landemic folder, not client

nvm install 18
nvm use 18
npm i
npm run build-client

Currently tested on an Apple M1, Sonoma 14.5

Serve from GitHub Pages

1. Fork this repo
2. Go to Settings > Pages
3. Under Build and Deployment select GitHub Actions
4. Then select Actions from the tab bar
5. If serving from a subdomain (default), set "homepage" : "https://YOURUSERNAME.github.io/landemic" in client/package.json
6. Commit your latest build
7. Push your repo
8. Visit Landemic at https://YOURUSERNAME.github.io/landemic/

Syncing data

Landemic gets its data directly from the blockchain, but because there are so many transactions, 99% of that has to be cached. The tiles on the map and the charts on the left draw from this cache. Currently, the cache is served from the same place where we serve the HTML.

You can synchronize with the blockchain manually per the instructions below, and you can set up a cron job to have it run from your computer at night. But if you have a laptop that sleeps at night or for whatever reason want to run it remotely, here are some instructions for running it on Heroku.

Note: There aren't any IaC or Heroku-specific hooks in this repo, so you should be able to run it on any VPC, not just Heroku's.

Running locally

Set your Alchemy key per /.env-example

npm run sync-data

Run git status to see the results of your fresh data.

Run remotely

You can run that data syncer nearly for free on Heroku by using only the Scheduler add-on. The syncer can run however frequently you want (once-a-day, once-an-hour (current)), and Heroku will only charge you one-off dyno fees for the duration of each run.

Getting started

Create an app on Heroku. Then in your repo folder run:

heroku login
heroku git:remote -a NAME_OF_APP
git push heroku main

This should successfully create your repo, install npm modules, and build a slug, which is a compiled image of your app. If it doesn't work, you may have to troubleshoot node and npm issues, such as upgrading to node 20 or fixing compatibility issues with the dependencies in package.json or package-lock.json

Then, turn off the web dyno, since we only need the Scheduler. Click on Resources, then click the pencil icon next to the web dyno, and then flip the switch off. This way you'll only be charged for the one-off dyno.

Set your Github key

The syncer stores its cache on GitHub pages, alongside our repo. See maintainer note above for why. To configure remote updating, go to your app, then Settings > Config Variables

Create a key called GITHUB_KEY. Then, get your private key from GitHub then convert it to be readable by code. Here's a handy trick:

1. Option-Command-i to open your developer console.
2. Type `
3. Paste your private key
4. Type ` again
5. Press Enter
6. Right-click on the key string, then choose "Copy string as JavaScript literal"
7. Remove the ' characters at the start and end

Then paste the key into the value field.

Set your Github remote

Set GITHUB_REMOTE to the [email protected]:YOUR_USERNAME/landemic.git of your new repo

Set your Alchemy key

The service fetches data from the blockchain through Alchemy. The requests are light enough that you can use the free tier.

Create an Alchemy key, this time called ALCHEMY_KEY and add it to Heroku.

Sync the data

heroku run ". prep-remote.sh && npm run sync-data"

Since we've turned off the web dyno, the above command will spin up an instance of your previously compiled slug, fetch the latest data from the blockchain, and then publish an updated client. The slug will then be deleted.

To run this on a schedule, you can then add the job via the Heroku Scheduler add-on. If you run it once an hour, it shouldn't use more than $1/mo. But you can even run it once a day to get it even closer to $0.

The data syncer is designed fails quetly if any of the fetch-data commands fail. To get extra visibility, run: heroku logs --tail --ps scheduler. Normally heroku logs would just work, but we don't have a standing dyno to query.