vpdb/server

The VPDB RESTful API.

A database for Virtual Pinball tables.

What is it?

VPDB is an open platform where people can share their digital recreations of virtual pinball tables and everything else around virtual pinball. This is the server side of the platform, a.k.a the API. If you're looking for the web application, it's here.

Stack

VPDB runs on a Node.js stack written in TypeScript. The web framework is koa. Data comes from MongoDB and queues and caches are handled by Redis.

"Quick" Install

• Download and install GraphicsMagick, pngquant, OptiPNG, FFmpeg and Unrar and make sure that the binaries are in your PATH.
• Install MongoDB and Redis. Windows: Get binary here, extract it to somewhere, open an admin shell and run redis-server --service-install redis.windows.conf --loglevel verbose --maxheap 500m for a local dev setup.

Install Node.js 8+ and Git, then open a command line and type:

git clone https://github.com/vpdb/server.git vpdb-server
cd vpdb-server
npm install
npm run serve:dev

That's it, the API should now be available. Retrieve current version:

curl http://localhost:3000/api/v1

For more detailed instructions, check out the Installation Guide.

Cool Stuff

• Processors
• Realtime
• Caching
• 3D Export
• Data Structure
• Table Blocks

Tests

There are 1200+ automated integration tests. It's best to use two terminals to run them:

• npm run test:serve - Runs the server in test mode
• npm run test:run - Runs the tests

For more info, see tests.

Code Quality

When running the server in development or test mode, the code is linted using JSHint. If the code doesn't pass, tests fail. You can run the linting manually with the lint task.

There's also continuous integration running on two services:

	Status
Travis CI
Codeship

After every test run, code coverage stats are sent to Coveralls.io.

Code is also run through Code Climate, with the following result:

Credits

• To Mukuste. He gave the community such a nice momentum and I thought I could add a little bit to it.
• To Tom for his support of this project.

Thanks also to JetBrains for their awesome IDE and support of the Open Source Community!

Finally, big shouts to Sqreen for their excellent security services protecting our host at vpdb.io!

License

GPLv2, see LICENSE.

server's Issues

Switch from marked to markdown-it

Marked has as security vulnerability they apparently don't want to fix.

Move tooling deps into devDependencies

Validate pages on mobile

Recycle non-table files

Tables with e.g. external music usually don't update the mp3s across versions, yet they need to be included in the download archive.

Currently, if the uploader doesn't include them for a new version, they won't be included in the download. There should be an option "persist file across releases" where non-table files are retrieved from the latest version that includes them.

Release List: Add flavor filters

Add correct download content to zip file

Still needs README & co filled up correctly.

Test with MongoDB 3.2

Fix date picker style

Fix release sub title

Long primary titles will overlap. This is a solution.

Make buttons 2D

Add parameter to include thumbs for all files (with format restriction) when listing releases

Needed if VPDB Agent shows multiple matches with different thumbs of the same release.

Simulate slow connection and add "loading" labels where necessary

Windows setup

Just some things I ran into during the setup.

Node JS

Redis

Server install command.
<redis-server --service-install redis.windows.conf --loglevel verbose -- service-name redisService>

Fix collapse animations

Or rollback to Angular 1.3 if angular-ui/bootstrap#3676 doesn't get fixed

Check for memory leaks

Good article

Add support for multiple emails when using OAuth

Not sure if we can pull multiple email addresses from GitHub, but Google returns a list of emails and we should try to match any of them in order to avoid duplicate account creation.

Add Raygun crash reporting

https://raygun.io/docs/languages/node-js

Add more pleasant download-release experience when not logged

Currently, it just refuses download and the user needs to click on download again when logged.

It should either

• Automatically open the release download dialog after login, or
• Start with the release download dialog even when not logged, display the login dialog after pressing "download", and serve the zip after login.

Manage expired registrations or email updates

Some cron job should regularly clear them out.

Replace FA of sort icon by CSS

Lock ratings in ui when logged out

Trouble with redis connection.

Is the Redis server started, anyway to test? I have no idea...

'<2015-11-03T01:50:30.846Z - info: [model] Schema "Build" registered.
2015-11-03T01:50:30.860Z - info: [model] Schema "Comment" registered.
2015-11-03T01:50:31.282Z - info: [quota] Initializing quotas...
2015-11-03T01:50:31.283Z - info: [quota] Creating quota for credits per day...
2015-11-03T01:50:31.333Z - info: [model] Schema "File" registered.
2015-11-03T01:50:31.343Z - info: [model] Schema "Game" registered.
2015-11-03T01:50:31.347Z - info: [model] Schema "LogEvent" registered.
2015-11-03T01:50:31.349Z - info: [model] Schema "LogUser" registered.
2015-11-03T01:50:31.350Z - info: [model] Schema "Rating" registered.
2015-11-03T01:50:31.368Z - info: [model] Schema "Release" registered.
2015-11-03T01:50:31.371Z - info: [model] Schema "Rom" registered.
2015-11-03T01:50:31.373Z - info: [model] Schema "Star" registered.
2015-11-03T01:50:31.374Z - info: [model] Schema "Tag" registered.
2015-11-03T01:50:31.377Z - info: [model] Schema "Token" registered.
2015-11-03T01:50:31.384Z - info: [model] Schema "User" registered.
events.js:141
throw er; // Unhandled 'error' event
^

Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at RedisClient.on_error (I:\PinballX\node-vpdb\node_modules\redis\index.js:196:24)
at Socket. (I:\PinballX\node-vpdb\node_modules\redis\index.js:106:14)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at emitErrorNT (net.js:1253:8)
at doNTCallback2 (node.js:450:9)
at process._tickDomainCallback (node.js:405:17)

I:\PinballX\node-vpdb>>'

Fix login/register flow+wording

Nice resource

Fix visibility of IPDB search button when adding a game

Add filter by VP version to release list.

When returning matched releases by size, return matched files instead of latest.

Add support for Direct B2S Backglasses

• [API] Add directb2s MIME type
• [API] Crush down PNG files in directb2s file
• [API] Generate thumbnails from directb2s uploads
• [API] Create entity which links files and authors
• [API] Link uploads automatically to correct game using the ROM name in the file
• [Web] Add single upload to frontend
• [Web] List backglasses under game

Fix upload progress

So far, progress doesn't seem to be working when uploading

• backglass
• logo
• playfield screenshot
• playfield video

Also check rotations, there seems to be a problem rotating the video.

Fix settings migration issue

Plans in settings are defined as props, so when migrating settings, default plans are re-added every time.

Either change plans to an array (duh) or disable automatic settings migration.

Finish user info box

Add action log

Contrarily to the user log, the action log is visible to everybody. It serves a general time line and can be filtered, trigger notifications etc.

Add token timeout box

Add release name generator

Something to help with release naming. See here

Distort landscape screenshots so they look good in portrait

When listing releases, we show them by default with portrait thumbs because they better fill up space and are nicer to look at:

However, some releases obviously are only available in landscape ("desktop"), so they end up looking like this:

The idea here is to take the original landscape screenshot and distort it like this:

Then crop and resize it so it looks like this:

The problem is that the angle of the distortion can vary quite a bit:

So in order to generate the thumbnail automatically, VPDB needs to look at the table params and somehow calculate the right angle in order to distort correctly. There are quite a few parameters to play with:

I'm planning to use ImageMagick's four point distortion feature for this, so I'd need some way to calculate the 4 intersection points between the table and the canvas.

The table params should be easy enough to read from the table file. However, if someone (like a math wizard such as @c-f-h) has an idea how to calculate the distortion params, I'm listening. :)

Add G+ authentification to passport.

Auto-select a flavor in download dialog

Specially when there's only one flavor to download.

Make game editable

API:

• Add update resource
• Copy media when necessary

Web:

• Edit base data
• Edit backglass and logo
• Be able to remove external media

Release List: Finish list views

Should be at least compact and large view. Table view probably not yet.

Add a footer.

Should contain links to:

• About
• GitHub
• Rules
• Sitemap
• FAQ
• Privacy Policy
• Terms and Conditions
• Forum (?)

Replace ui.bootstrap in ng deps with components to avoid loading components we don't need.

Add support for P-ROC games

I think this would be cool.

As far as I understood, P-ROC-based games run without ROM but a set of Python scripts (among other stuff). These files however end up in a separate folder, where P-ROC is installed. The table then needs to adapt the script in order to access it.

@horseyhorsey: is this accurate, and what interesting metadata could be extracted from those games that could be displayed to the user when browsing? Is the scripting something that could be done automatically or is in any case a special .vpx build needed?

See also:

• Thread on VPF
• Wiki

Add "upload moderation"

The goal is to verify that content is owned by the uploader, that it corresponds to VPDB's quality requirements and that it aligns with VPDB's content policy before showing and distributing it. Per default, releases, media and B2S backglasses are verified (ROMs should become upload by moderator only).

Per default, uploaded content needs to be "approved" before it shows up publicly. Once a moderator approves, it becomes visible for everybody. Contributors have auto-accept permissions which skip the whole process.

Unapproved or refused content is invisible in listings and detail views are only visible by moderators and authors of the uploaded content. There is a view that allows moderators to list the queue of pending items. Uploaders are notified by email if their content is accepted or refused (see #67).

Adding updates or new versions to an existing release does not need further verification. However, there should be a way to "unaccept" content that has falsely published.

Data-wise, we'll have the following attributes:

is_approved:    { type: Boolean, required: true, 'default': false },
is_refused:     { type: Boolean, required: true, 'default': false },
auto_approved:  { type: Boolean, required: true, 'default': false },
history: [{
    event:       { type: String, 'enum': ['approved', 'refused', 'pending'], required: true },
    message:     { type: String },
    created_at:  { type: Date },
    _created_by: { type: Schema.ObjectId, ref: 'User' }
}]

A moderate mongoose plugin is used so controller code stays minimal.

Display release panels in game details and add release details page

First release takes all the place. It would be better to only briefly list releases on the game details page and add another release details page.

This comes with a few more advantages:

• More space for everything on the release details page (now it's really crowded)
• Releases can be linked more easily
• It makes more sense navigation-wise. Ending up on the same page when clicking on a game or on a release is weird.
• Add release details page
• Add release views to API and stats
• Change game details page to smaller panels
• Update links: Release lists should point to the details pages

Comment for the record:

I think it might be easier for use to see a screen similar to the Release Tab but filtered only on Mystery Castle, so the see the 2 screenshots of the releases side by side and when they select on, it goes to the detailed page with release details and download option. if you keep the current format, maybe put at the top, how many releases are available, provide sort options, as it was not obvious to me that there was the AUTHOR2 release below. If there are tons of comments on the first release, scrolling to the 2nd author or 3rd author will be cumbersome.

Make media fullscreen on click

Some lightbox style fullscreen mode.

Fix "click or drag and drop here" media uploads

Usually, only drag and drop is working. Add click support.

ROMs get double-posted

When uploading another ROM after having submitted one or more ROMs already, The client seems to multi-post to /api/v1/games/{game_id}/roms resulting in a validation error on server side (client is silent, only console outputs error).

Fix release date field

Either make both empty or fill current date.

Check behavior during upload

Feedback:

While uploading VPT file as ongoing, I typed release name then tried to set flavors and i think the
options could not be selected during the table upload. Once upload was done, I could select the
options.

Add "is moddable" field(s) to release

Migrate to Node v4.x

See Dependency Report