itsiw/vilf

The Good Taste Guide™ Bay Area

Vegans In Love with Food™

Running locally on linux

1. Activate virtual python environment

If using conda:

# Makes environment if it doesn't exist
conda env list | grep VILF || conda create --name VILF

# Activates the environment
conda activate VILF

If using virtualenv:

# Makes environment if it doesn't exist
[[ -f venv/bin/activate ]] || virtualenv venv

# Activates the environment
source venv/bin/activate

2. Install dependencies

pip install -r requirements.txt

3. Build static files

./vilf build

4. Serve static files locally

python3 -m http.server 8080 --directory build

5. Visit website

Open localhost:8080 (if you open 0.0.0.0:8080 then the map will not render).

Helpful infra commands

1. Push static files to bucket

# Requires roles/storage.objectAdmin on gs://vilf-org
gsutil -m rsync -R build gs://vilf-org

2. Invalidate Cloud CDN cache

# Requires roles/owner on projects/vilf-com
gcloud compute url-maps invalidate-cdn-cache vilf-lb --path /

3. Track SSL propagation status

openssl s_client -showcerts -servername scripts.org -connect $(dig +short A vilf.org):443 -verify 99 -verify_return_error

4. Manage infrastructure

We use OpenTofu to deploy infrastructure as code primarily to GCP. Install the tool and running it in ./infra for more details. This will read the configuration from main.tf.json to determine what to administer. This file is generated from the configuration in default.nix, which requires the Nix package manager to interpret. Once installed and the development shell activated, run vilf tofu to manage further.

5. Nix development shell

We use a Nix development shell to currently to manage infrastructure and autoformat code. It will likely accumulate more functionality too. After installing Nix: the package manager, you can enter the development shell by running nix develop. In this shell you can run pre-commit to run repository commit hooks (autoformatting, linting, etc.), as well as access repository tools through the vilf executable.

Tools for contributors

spatula: Automating the restaurant data collection and markdown file generation

Interactive mode (recommended)

Think of a good search query that will locate your restaurant (e.g. "Lion Dance Cafe"). Run the following from the terminal to see an interactive prompt:

./vilf spatula
# Enter Google Maps search terms (ex: Lion Dance Cafe in Oakland):

Enter the hint:

./vilf spatula
# Enter Google Maps search terms (ex: Lion Dance Cafe in Oakland):
Lion Dance Cafe

The scraper will be able to identify the restaurant automatically and generate a markdown file for you:

./vilf spatula
# Enter Google Maps search terms (ex: Lion Dance Cafe in Oakland):
Lion Dance Cafe

# Waiting for Google Maps page to redirect...

# Using the Google Maps page: https://www.google.com/maps/place/Lion+Dance...
# Name = Lion Dance Café
# Address = 380 17th St
# City = Oakland
# State = CA
# Zip code = 94612
# Phone: None
# Lat, lon = 37.806069, -122.276674

# Successfully wrote markdown to file places/lion-dance-cafe.md

The markdown will contain the fields pre-populated and the relevant values already filled in. The rest is up to you to fill in.

cat lion-dance-cafe.md
# ---
# name: Lion Dance Café
# cuisine:
# address: 380 17th St
# area:
# lat: 37.8060489
# lon: -122.267932
# phone:
# menu:
# drinks:
# visited:
# taste:
# value:
# ---

# <REVIEW>

File name formatting happens automatically. It will safely remove bad characters, use the restaurant name (and possibly street name, see below), and will append an integer to the end of the filename in cases of conflict with a preexisting file. You can also flag to use the city name for the field area (though you may want to be more specific like "Downtown Oakland").

./vilf spatula --city-as-area --street-in-filename
# ...
# Successfully wrote markdown to file lion-dance-cafe-380-17th-st.md
cat lion-dance-cafe-380-17th-st.md
# ...
# address: 380 17th St
# area: Oakland
# lat: 37.8060489
# ...

Sometimes searches are ambiguous. In this case, the scraper will allow you to select one of the top results from a search or try a different search:

./vilf spatula
# Enter Google Maps search terms (ex: Lion Dance Cafe in Oakland):
Bongo Java Nashville

# Waiting for Google Maps page to redirect...

# I found multiple potential locations, collecting top results...
# Gathering search result data: 100%|█████| 5/5 [00:23<00:00,  4.63s/it]

# 0: Try searching again
# 1: Bongo Java at 2007 Belmont Blvd, Nashville, TN
# 2: Bongo Java East at 107 S 11th St, Nashville, TN
# 3: Bongo Java at 364 Rep. John Lewis Way S, Nashville, TN
# 4: Bongo Java Roasting Co. at 372 Herron Dr, Nashville, TN

# Select one of the above choices to proceed (0 - 4):
3

# Waiting for Google Maps page to redirect...

# Using the Google Maps page: https://www.google.com/maps/place/Bongo+Java...

# Name = Bongo Java
# Address = 364 Rep. John Lewis Way S
# City = Nashville
# State = TN
# Zip code = 37203
# Phone: None
# Lat, lon = 36.157151, -86.776074

# Successfully wrote markdown to file places/bongo-java.md

Manual URL mode

Alternatively, you can pass in a URL corresponding to a Google Maps restaurant manually. Be careful to escape characters correctly (most terminals will automatically escape upon pasting).

Ex:

./vilf spatula --url https://www.google.com/maps/place/Lion+Dance+Caf%C3%A9/@37.8060737,
-122.270113,17z/data\=\!3m1\!4b1\!4m5\!3m4\!1s0x808f817f59aa5fa9:0xc6930eb94f2d3188\!8m2\!3d37.8060489\
!4d-122.267932

# Waiting for Google Maps page to redirect...

# Name = Lion Dance Café
# Address = 380 17th St
# City = Oakland
# State = CA
# Zip code = 94612
# Phone: None
# Lat, lon = 37.806074, -122.270113

# Successfully wrote markdown to file places/lion-dance-cafe-0.md

Notice the -0 added to the filename to avoid a collision with the original file we produced.

Shortcuts and extras

1. ./vilf spatula --ask-first prompts the user before writing metadata to markdown.
2. ./vilf spatula --search-query 'lion dance cafe' or ./vilf spatula -s 'lion dance cafe' avoids the search prompt and jumps right to the action
3. ./vilf spatula --directory '/path/to/folder' allows you to specify the directory for the markdown file (directory doesn't have to exist yet)
4. ./vilf spatula --manual-filename '/path/to/folder/filename.md' allows you to manually specify the output file
5. ./vilf spatula --timeout 30.0 let's you set the timeout for the browser (default is 10.0)
6. ./vilf spatula --no-headless let's you see the browser GUI as the searches are being made (kinda fun but not recommended unless debugging)
7. For more details run ./vilf spatula --help.

Checking new file additions against Google Maps

Before committing new markdown files, spatula can be leveraged to check new files against Google Maps scraping. Simply run the following from the repo home directory:

./vilf check $(git diff --staged --name-only places/)

If everything looks as expected, you will see

./vilf check $(git diff --staged --name-only places/)

# Testing files:
# ✔ places/lion-dance-cafe.md
# ✔ places/maya-halal-taqueria.md

# All files look good.

If anything is wrong, the metadata will be displayed:

./vilf check $(git diff --staged --name-only places/)

# Testing files:
# ✘ places/lion-dance-cafe.md
# ✔ places/maya-halal-taqueria.md

# The following files may need inspection:

# places/lion-dance-cafe.md
# Current address: 382 17th St | Determined address: 380 17th St
# Current latitude: 34.8060489 | Determined latitude: 37.8060489
# Current longitude: -120.267932 | Determined longitude: -122.267932

vilf's Issues

Testing for PRs

Simple test: does build.py run, let people see any errors

Floating names on map

Allow multiple reviews per restaurant

This ties in well with #20, where multiple authors can review the same restaurant.
I imagine there won't be many double reviews yet, so let's just list them in order of submission on the restaurant page.

This feature does beg the question what color we display on the map.
I think it makes the most sense to show the average rating on the circle (over food and value, equally weighted?).
The color could just be the typical color of the rating rounded to the nearest whole number.

Another cool feature would be to provide a toggle to change what the number and color represent, like:

1. lowest rating
2. highest rating
3. average rating

Contributor form

Remove SFGI stuff

PRs go live before merge

On my last PR I noticed that VILF displayed my review as soon as it finished running the CI job to test the build. I believe this was changed when I made the CLI interface so that we could test changes to commands used to build VILF. Specifically, I added

  pull_request:
    branches: [ develop ]

to deploy.yaml. Is this the intended outcome? If not, maybe we should change this so that for PRs we just run up to ./vilf build but don't run these lines?

    - uses: google-github-actions/auth@v0
      with:
        credentials_json: "${{ secrets.CLIMAX_VILF_SA_KEY }}"
    - uses: google-github-actions/setup-gcloud@v0
    - run: gsutil -m rsync -R build gs://climax-vilf-bucket

Use a data store instead of putting reviews in git

I think the main motivations for this are:

1. Keeping photos in git creates unnecessary bloat.
2. The git workflow is not ideal for non-technical people to contribute.

I think the form at #11 already addresses a much improved workflow for other people actually.
But with this workflow, it shifts work onto @ItsiW to upload people's reviews when he can into git.
If it's not a problem, then we can keep the reviews in git but have @ItsiW upload photos to the bucket.
This way, we solve the problem with the first motivation too without changing much.

If we want to move reviews out of git too, that requires using some kind of static file store on the bucket.
Maybe this can just be JSON format? We could provide a script for @ItsiW to easily migrate new submissions
to this new data store. Automated triggers might be overkill (and could cost additional money, but maybe not much,
especially if Climax is bankrolling).

Showcase review authors

We could have a page that lists all the different review authors with links to all of their reviews.
There could be an author index page and a page per author, or just one giant page with all links.
All the authors need to provide is a unique identifying string, but we can provide more fields for
them to fill in if they want.