openbrewerydb/openbrewerydb

🍻 An open-source dataset of breweries, cideries, brewpubs, and bottleshops.

🍻 Open Brewery DB Dataset

This is the open-source dataset for the Open Brewery DB API which is served by a REST API built with Ruby on Rails

🎯 Purpose

Provide an approval-based pipeline to update the dataset and API.

🗄 Data Formats

• CSV - Full Dataset
• JSON
• PostgreSQL SQL

API

Access the dataset programmatically via the Open Brewery DB API. Use the following tools to get started without any code:

databar.ai

If you don't know how to use APIs, you can use Brewery DB without code through the databar.ai platform.

Postman

A shared Postman collection containing all the API requests to fetch breweries information from the open-source dataset.

🚀 Getting Started

1. git clone [email protected]:openbrewerydb/openbrewerydb.git
2. cd openbrewerydb && npm install

🤝 Contributing

For information on contributing to this project, please see the contributing guide and our code of conduct.

1. Fork the repository
2. Add or update breweries in the CSV (Excel, Google Sheets)
3. Submit a Pull Request

Tips

First and foremost, don't worry about messing up! 🙂 Thank you so much for contributing! 🙌

• CSVs are organized by data/[country]/[state_province]
• Required fields/columns: name, brewery_type, city, state_province, postal_code, and country
• When adding a brewery, do not include an id. This will be created after review.
• Please either add to breweries.csv (preferred if adding breweries for a new country) or the individual state/province CSV file. Adding to both at the same time may introduce duplicates/errors.

⚙️ Scripts

These are the npm scripts used to maintain this dataset.

• npm run csv:combine - Combine CSVs from country/state-region folders into breweries.csv
• npm run csv:split - Split breweries.csv into country/state-region/city CSVs
• npm run contributors:add - Add contributor (interactive CLI)
• npm run contributors:check - Check if there are any missing contributors
• npm run contributors:generate - Generate contributors into README.md
• npm run generate:ids - Generate unique OBDB IDs based on the brewery name and city and overwrite breweries.csv
• npm run generate:json - Generate JSON from breweries.csv output to breweries.json
• npm run generate:sql - Generate PostgreSQL SQL from breweries.csv output to breweries.sql
• npm run validate - Validate CSVs based on JSON Schema
• npm run workflow:maintain - Combine, generate, split (used when updating individual CSVs)

👾 Community

• Join the Newsletter
• Join the Discord

📫 Feedback

Any feedback, please email me.

Cheers! 🍻

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Mike Putnam 🔣	Andrew A. Barber 🔣	Jason Allen 🔣	Juicob 🔣	Will Karnasiewicz 🔣	Dylan T. Vavra 🔣	Madison Martinez 🔣
Daniel Eremchuk 🔣	Alex Chong 🔣	Matt S 🔣	Samuel Rusher 🔣	Evan Caraway 🔣	Tyler K Kuromiya Parker 🔣	kendellmendoza 🔣
Johnnyk737 🔣	James Schuler 🔣	Creighton Leif 🔣	Vitaly Tomilov 💻	Kyle Scudder 🔣	Chris Mears 💬 💻 🔣 🚧 📆 🔧 ✅	donkeyslaps 🔣
Pranav Davar 🔧	Alexandre Hernandes Barrozo 🔣	Resten 🔣	Matt Higgins 🔣	Alex Justesen 🔣	Craig Kelly 🔣	Krzysztof Rewak 🔣
John Baumert 🔣	Charlie Cox 🔣	Miles Kane 🔣	Anthony Laflamme 💻	Georg Engelsmann 🔣	Clinton Williams 🔣	Brent Busby 🔣
kenster89 🔣	Adilet Sarsembayev 🔣	Pranav Davar 🔣	b-mc2 🔣	Nicole 🔣	Nicholas Hance 🔣	Joachim Nilsson 🔣

This project follows the all-contributors specification. Contributions of any kind welcome!

openbrewerydb's Issues

ArcGIS REST Service

Overview

This could be either an implementation of ArcGIS REST Service (does this cost anything?) or just an endpoint to return the properly formated data.

Per Paul Doherty on Twitter:

"Ideally an ArcGIS REST service, but anything with consistent headers and latitude/longitude (decimal degrees) would work (geojson, .csv). Obviously, with attribution and links to whatever can help your amazing project succeed!"

https://twitter.com/pjdohertygis/status/1374182569936232456

Add US Census data

Use the Geocoding Census data and store block and tract in the DB. This data will allow us to map anything about the American public to the data.

Website: https://geocoding.geo.census.gov/

Examples:

• how many lawyers work near breweries in New York
• the average income of neighborhood around breweries

Replace `frictionless` with `csv-file-validator`

The Python library frictionless keeps breaking. Plus, it's Python in a mostly JS-based repo. Let's replace with csv-file-validator

Add Breweries with missing/not enough data

Is your feature request related to a problem? Please describe.
There may be some breweries where there isn't enough available data (like planning or nano types for example) and currently, these aren't being included. However, that does not make them invalid or breweries that should not be known (my opinion). Additionally, it may be helpful to see a list of breweries that need information, so in the future, when more information about a particular brewery becomes available, it can be updated.

Describe the solution you'd like
It would be great if there was a section where breweries with missing or not enough data can be placed for future use.

Describe alternatives you've considered
At the moment, I've got my own helper repository for this to keep track of the ones that are missing breweries_with_missing_data.json. It would be great to keep this information within the main repo though.

Add Cloudflare Cache Cleaner Github Action

https://github.com/marketplace/actions/cloudflare-cache-cleaner

🇬🇧 UK Schema

Is your feature request related to a problem? Please describe.
It would be great to have UK Breweries on the DB too!

Directory Structure
In the UK we have all the individual countries then individual county areas, so what would be the best directory tree for this?
Would something like:

openbrewerydb/data/uk/scotland/west-dunbartonshire.csv
openbrewerydb/data/uk/scotland/argyll-bute.csv

or

openbrewerydb/data/uk/west-dunbartonshire/west-dunbartonshire.csv   
openbrewerydb/data/uk/argyll-bute/argyll-bute.csv

or

openbrewerydb/data/uk/scotland/west-dunbartonshire/west-dunbartonshire.csv   
openbrewerydb/data/uk/scotland/argyll-bute/argyll-bute.csv

be best?

CSV Structure
Additionally, what would be the best layout for the CSV be? Maybe:
id,name,brewery_type,street,city,**county**,postal_code,website_url,phone,created_at,updated_at,country,longitude,latitude,tags

Info
Unfortunately, SIBA don't provide a great data source, but I'm hoping to start the pull from there, then try and automate some further parts..

Add Brewery Data Change Manager

Continuing the conversation from #12.

Ideally, this would be a UI that opens up the main dataset file, allows you to directly edit/add/delete entries, saves and submits this as a Pull Request to the dataset repository.

Questions we'd like answered

• Where should this UI live?
• Use 3rd party tool to manage VCS or build our own?

Fix how international phone numbers are handled

Describe the bug
Currently, international phone numbers do not seem to be handled properly. If they are too long, they are treated like a long scientific number which ends up looking like this: "phone":"3.53599E+11"

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.com/openbrewerydb/openbrewerydb/blob/master/data/ireland/laois.csv
2. See the phone number on the first line

Expected behavior
We would expect phone numbers to be handled appropriately. This value should be [phone]

OS is irrelevant as this is an issue with the data itself.

Context
For Ireland, the format is country code + region number/mobile prefix + 7 digit number. Example: [phone]. The region number is similar to the US areacode. It can be 3 digits, with a 0 prefix if the country code is not included (local dialing).

This also not limited to Ireland. For example, Germany has phone numbers with a max of 11 digits (not including country code)

CLI tool to create obdb_id

The obdb_id is essentially a kebab case combination of the brewery name and the city. This should handle most of the cases but let's see.

Notes

• Might make use of the notebook
• Perhaps there needs to be more additions to the ID for it to be unique?

International translations

We want to handle international translations in the dataset because not everything is English.

From the Discord thread:

Resten — Today at 2:30 AM
this is my personal opinion.
How about separating English and foreign names by column?
Currently, the method added in Korean is 맥파이(Magpie), so it would be better to manage it separately for future use.

@chrisjm — Today at 8:20 PM
@resten Thank you for the suggestion! This is a great idea and I'm still mulling it over. Perhaps a better solution is to have another linking translations table. I also think the names (and any other field) in the DB is reflective of the native language depending on the country. So in the Korean case, the English names should go in the translation table.

Export: CSV

Export /data to /breweries.csv

Dependencies

• I chose Papaparse because of the small footprint (249kB) compared to csvtojson (8.69MB).

Import: CSV

Import /breweries.csv to /data

Some Bad Characters in Data: © -> � or © -> â€

Describe the bug
There seems to be some bad characters in the dataset. For example, Anheuser-Busch Inc establishments are all across the US. I think the dataset meant to have the © copyright character, but bad decoding changed it to â� or �.

To Reproduce
Steps to reproduce the behavior:

1. you can see results using the api endpoint
2. also visible in the dataset, row 12

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Desktop (please complete the following information):

• OS: macOS 10.15.7 (19H2)
• Browser: Chrome Version 95.0.4638.69 (Official Build) (x86_64)

Smartphone (please complete the following information):

• Device: iPhone 11 Pro Simulator
• OS: iOS 14.3
• Flutter application: Flutter version 2.5.1

Autocomplete endpoint does not match documentation

Describe the bug
The /v1/breweries/autocomplete endpoint specifies:

The maximum number of breweries returned is 15.

This appears not to be the case.

To Reproduce
Steps to reproduce the behavior:

1. Issue a get request to https://api.openbrewerydb.org/breweries/autocomplete?query=brew
2. Observe the >5000 entries generated

Expected behavior
A maximum of 15 entries are generated

Screenshots

System

• Browser: Anything that can send an HTTP GET request

BREAKING CHANGES: Update database schema

Tasks

• Rename obdb_id to slug
• Update id to use UUID

Notes

• This should be done after the versioning is migrated (it might already have)
• The API will also need to be updated

Github Action > Update Open Brewery DB API

Broken search by keyword and autocomplete queries

Describe the bug
The search by keyword and autocomplete queries return an empty array.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://www.openbrewerydb.org/documentation
2. Scroll down to Search Breweries
3. Click submit button for the example queries

Expected behavior
A network response of an array of brewery info objects.

Screenshots

Desktop (please complete the following information):

• OS: iOS
• Browser chrome
• Version 108.0.5359.124

Question: How to handle closed breweries?

Overview

I realized I'm not accounting for closed breweries. I think I'd like to keep them in the dataset for historical and analytical reasons, but I'm curious how best to do this.

Options

1. Add a "closed" brewery type
2. Add a new boolean field/column for "Active" or something similar

Other options?

Add Contributors

Overview

It's long overdue to add the contributors to the README and automate future additions via All Contributors.

Github Action: CSV Schema

Validate CSV file

Discussion: Scraping for brewery data

While we want the community to help to keep brewery data up to date, it might be easier / more efficient to set up some scraping scripts. There are several things we want to take into consideration when doing this:

• What to scrape? (brewery guilds? brewery associations? google places? yelp?)
• Ethical scraping (i.e., getting permission, not overloading the server, using APIs when available, etc.)
• How to automate this process? (Consul? Airflow? other?)

OpenStreetMap database ?

Hi,
in openstreetmap we can find around 8000 craft=brewery (https://taginfo.openstreetmap.org/tags/craft=brewery#map)

but you can't import OSM data in your database in regard of MIT licence (not compatible with OpenDataBaseLicence).

Do you consider to change your licence ?

Add gzipped versions of datasets

Github Action: Validate ESLint

npm run lint