kyroskoh/webuild

:memo: Robot-handpicked list of open events and open source for designers, developers and makers in Singapore


We Build SG automatically curates a list of free public events (Facebook / Meetup / Eventbrite / ICS / manual) and open source projects (Github) for the curious folks who love to make things in a particular city. This repository is an example for Singapore.

Please feel free to fork this for your choice of city/country too 😄

Repositories curated automatically every hour:

1. Github repositories

• user location contains Singapore
• repos with more than 50 watchers
• repos pushed date less than 3 months ago

Events curated automatically every hour:

1. Facebook selected groups

• Meetup.com event category Technology, free, has a valid location
• Eventbrite event category Technology, free, has a valid location
• ICS url
• Manually added events

Who are we? We are geeks - engineers, designers, programmers, hackers or makers. And we want to connect various geeks to come together and connect:

• veteran geeks to get introduced to the community of open events and open source
• beginner geeks to get examples of great open source projects and events to meet mentors/seniors
• traveling geeks to drop by and connect with the local ones
• existing geeks to keep connecting, mentoring and growing the open community

#Websites

• Main
• Production in Heroku
• Staging in Heroku
• Github Repo
• Twitter
• Facebook

#API endpints

The events, repositories and podcasts data feeds are available as JSON.

• https://webuild.sg/api/v1/repos
• https://webuild.sg/api/v1/events
• https://webuild.sg/api/v1/podcasts
• https://webuild.sg/api/v1/check/:checkdate where checkdate is in the format YYYY-MM-DD to check for clashed events with checkdate

#Archived snapshots

A daily snapshot of the repos and events API V1 endpoints are stored in the archives for future data analaysis.

#Install for development

1. Clone the app:

   git clone [email protected]:webuildsg/webuild.git
   cd webuild

2. Setup the necessary environment variables. Refer Environment Variables section for more details.

   cp .env-example .env

3. Install required packages with npm and RubyGems.

   gem install foreman thor tmuxinator
   gem install dotenv -v 0.11.1
   gem install dotenv-deployment -v 0.0.2
   npm install -g bower
   npm install -g grunt-cli
   npm install
   bower install

4. Build frontend css and javascript files, along with other tasks with grunt

   grunt

5. Run in command line ./run.sh to start the app.

6. Open localhost:4000 in your browser.

7. Run the following command in another terminal to update events and repos:

   ./update.sh

#Deploy to Open Shift

We are using Open Shift for production. These are the steps for setting it up for Open Shift:

1. create an application with folder .openshift with various Open Shift related configurations

• install client tools with gem install rhc

• setup the app with rhc setup

• create an app using cartridge - note the GIT_REMOTE_URL

• to ssh into your gear, use rhc ssh {APP_NAME}

• add the cron cratridge with rhc cartridge add cron -a {APP_NAME}

• set environment variables with

  rhc env-set BOT_TOKEN={secret} EVENTBRITE_TOKEN={secret} GITHUB_CLIENT_ID={secret} GITHUB_CLIENT_SECRET={secret} MEETUP_API_KEY={secret} NODE_ENV={APP_NAME} TZ=Asia/Singapore WEBUILD_API_SECRET={secret} WEBUILD_AUTH0_CLIENT_ID={secret} WEBUILD_AUTH0_CLIENT_SECRET={secret} --app {APP_NAME}

• add a git remote to the git config, so you can push your code to the gear

  [remote "{APP_NAME}"]
    url = {GIT_REMOTE_URL}
    fetch = +refs/heads/*:refs/remotes/{APP_NAME}/*

• create a build file in path .openshift/action_hooks/build for your app (if you're forking webuildsg, this is already inside the repo)

• make sure the build file permissions for is executable chmod +x .openshift/action_hooks/build

• push the app git push {APP_NAME} master --force

• check if the app website is up

• if you need to restart the app use rhc app-restart {APP_NAME}

• to see app info use rhc app-show {APP_NAME} -v

• to check out the logs from the app use rhc tail {APP_NAME}

#Deploy to Heroku

Alternatively, we also used Heroku. These are the steps we took to deploy:

1. Install Heroku command line

• Create new Heroku app for NodeJS

• Setup the following environment variables under the Heroku app settings:

   BOT_TOKEN=secret
   EVENTBRITE_TOKEN=secret
   GITHUB_CLIENT_ID=secret
   GITHUB_CLIENT_SECRET=secret
   MEETUP_API_KEY=secret
   NODE_ENV=production
   TZ=Asia/Singapore
   WEBUILD_API_SECRET=secret
   WEBUILD_AUTH0_CLIENT_ID=secret
   WEBUILD_AUTH0_CLIENT_SECRET=secret

• Get Heroku Scheduler add on and add 2 tasks with an hourly frequency:

  • update events every hour

     curl -X POST --data "secret=<WEBUILD_API_SECRET>" <your_production_url>/api/v1/events/update

  • update repos every hour

     curl -X POST --data "secret=<WEBUILD_API_SECRET>" <your_production_url>/api/v1/repos/update

  • store to archives repos and events every day

     curl -X POST --data "secret=<WEBUILD_API_SECRET>" <your_production_url>/api/v1/archives/update

#Environment variables

Set the following environment variables on your system:

• WEBUILD_API_SECRET (required) Used as a password when remotely refreshing the feeds.
• MEETUP_API_KEY (required) Used to list available meetup events in Singapore.
• EVENTBRITE_TOKEN (required) Used to list available eventbrite events in Singapore.
• WEBUILD_AUTH0_CLIENT_ID (required): Used to retrive facebook events in Singapore. Auth0 takes care of OAuth2 social logins.
• WEBUILD_AUTH0_CLIENT_SECRET (required): Same as above.
• PORT (optional, default: 4000) Configures the port used by the web server.
• LOCATION (optional, default: Singapore) The GitHub feed shows only repositories owned by developers in this area. Matches the GitHub "Location" property in user profiles.
• MAX_USERS (optional, default: 1000) Show only repositories belonging to developers in this ranking. Only the last updated repository of a user is shown.
• MAX_REPOS (optional, default: 50) Show up to this many total repositories.
• GITHUB_CLIENT_ID (optional) App OAuth client ID for GitHub.
• GITHUB_CLIENT_SECRET (optional) App OAuth client secret for GitHub.
• NODE_ENV Environment variable. By default it is staging and for production it is production
• BOT_TOKEN This token is used by the We Build SG Bot to store the api endpoint responses for repos and events to the archives every day. Generate a token for the Github user We Build SG Bot.

Use an external "web cron" service to periodically refresh the GitHub data feed. Keep in mind that due to GitHub API rate limiting it may take >15 minutes to retrieve the entire feed. Register a GitHub OAuth application and configure the GITHUB_CLIENT_* environment variables (see above) to increase the rate limit. Do not refresh the feed too often or the rate limit will cause it to take longer.

Create an Auth0 account (you get one free app) and a Facebook app and link them with these instructions. Configure the WEBUILD_AUTH0_CLIENT_* environment variables (see above) and add your callback url in auth0. Run the app and if all is configured well, add your fb aceess token by logging in at <localhost>/admin

Editing events list

1. Add any events manually in file events/whitelistEvents.json

• Remove events already added manually (paid / duplicate ones) in file events/blacklistEvents.json
• Remove Meetup.com group by adding the group_id (found in the /api/v1/events) to the file config/meetupBlacklistGroups.json
• Add a Facebook groups
  1. Go to Lookup ID to find a facebook group id
  • Edit file /events/facebookGroups.json
• Add an *.ics format URL to file events/icsGroups.json
• Cleanup old events manually in files events/whitelistEvents.json and events/blacklistEvents.json with a grunt task: $ grunt cleanup

#Contribute

Please see CONTRIBUTING.md for details.

#Versioning

Every production code has a version following the Semantic Versioning guidelines. Run the grunt bump, grunt bump:minor or grunt bump:major command to bump the version accordingly and then push to production with git push production master.

#License

We Build is released under the MIT License.