Brainyserver is the server counterpart of Brainydroid, a framework to easily create behavioral experiments running on Android devices. Using Brainydroid, researchers (or anybody else, really) can publish experiments as Smartphone / Tablet ( / Other form-factors) applications and bring behavioral studies into the internet age.
Now when a user completes an experiment, data needs to be sent back to the researcher. Enter Brainyserver: a platform for researchers to collect their data.
THIS DOCUMENTATION IS OUTDATED! WILL BE UPDATED SHORTLY.
You will need a few packages to run the server.
If you're running Ubuntu, the best way to get things running is to first install the python-virtualenv
and virtualenvwrapper
Ubuntu packages. To do that, open up a terminal and run sudo apt-get install python-virtualenv virtualenvwrapper
. These programs will let you easily set up sub-environments to install python packages that are not in the Ubuntu repositories. Finally, you should close your terminal and open it up again to let virtualenvwrapper apply changes.
Now you can create a virtual environment for brainyserver by running mkvirtualenv --system-site-packages brainyserver
. You'll see the text (brainyserver)
appear to the left of your prompt. That means the virtual environment is activated. D'ont hesitate to read more about virtualenvwrapper for better understanding (http://www.doughellmann.com/projects/virtualenvwrapper/).
Here is the list of packages needed to run and test the server. Ubuntu packages can be installed by using sudo apt-get install <packagename>
, and PyPI packages can be installed inside a virtualenv using pip install <packagename>
.
- To run the server:
python
>= 2.6 [ubuntu]mongodb
[ubuntu]uwsgi
[ubuntu] and a web server if you want to run in production environment
- The web framework and related:
python-pymongo
[ubuntu]python-mongoengine
[ubuntu]python-flask
[ubuntu]Flask-Script
[pypi]Flask-Uploads
[pypi]Flask-PyMongo
[pypi]flask-mongoengine
[pypi]WTForms
[pypi]Flask-WTF
[pypi]flask-debugtoolbar
[pypi]
- The digital signing tools:
gnupg
[ubuntu]python-gnupg
[pypi]
- To test the server:
curl
[ubuntu]
Once you have those installed (and using virtualenv
is a good idea for that), you can run a small test server by running python manage.py runserver
in a terminal. This will fire up in debug mode on 127.0.0.1:5000
. You can check the server is working by visiting 127.0.0.1:5000
in your browser; a page (probably blank) should show up.
Right now the software does very little, just enough to start testing uploads from an Android app. There are two basic concepts:
(Would you believe it?)
An Android app represents an experiment published by a researcher (or other), running on Android devices. It usually needs to upload data to a server.
In a first attempt to prevent the server database from being relentlessly spammed, each Android app is assigned an Android app ID and a public/private key pair generated by the experimenter. Let's see how this is useful:
The private key is hardcoded into the app (yes, this has flaws as a security design, but let me test it before flaming), and the public key is published to the brainyserver and gets associated with its Android app ID.
All data uploaded from apps gets stored under an Android app ID. So when an app wants to upload data, it signs it with its hardcoded private key and sends it over along with the Android app ID under which it wants the data to be stored. The server then checks:
- that the signature is valid
- that the public key associated with the private key used for signing is indeed the public key associated with the given Android app ID (on the server). If so, the data is accepted and stored; if not, it is rejected.
So in order to upload data to the server and store it under a given Android app ID, one needs access to the private key associated with the public key that is itself associated, on the brainyserver, with the Android app ID under which we want the data to be stored. This means that as long as the hardcoded private key is not retrievable from the compiled Android app (a major problem to be addressed soon), data accepted by the brainyserver and stored under a given Android app ID will necessarily come from a user of that app (and not from a spammer).
The first steps needed to upload data to the server are a bit cumbersome, but it boils down to this:
- Choose an Android app ID
- Generate a public/private key pair
- Upload the public key to the server, associating it with an Android app ID
- Check your key is up
- Generate data and sign it
- Upload the signed data to the server
For this, you will be using the four urls the server provides:
<servername>:5000/upload/pubkey/<androidapp_id>
: to upload your public key<servername>:5000/upload/data/<androidapp_id>
: to upload signed data<servername>:5000/admin/show_db
: to see what's stored in the server database<servername>:5000/admin/flush_db
: to empty the database and start over
If you're running a test instance on your own computer, you can replace <servername>:5000
with 127.0.0.1:5000
. If a friend has a test instance running for you, you need to put his domain name instead.
Here's how to do it, broken down into little bits:
-
You start by choosing an Android app ID (alphanumeric and a few more characters, but don't push it). For this example, I chose
wehlutyk_exp
(you can replace it with your own Android app ID when completing the steps). -
You then need to generate your public/private key pair. Under GNU/Linux, this can be done with the
gpg
program:Open a terminal and run
gpg --gen-key
. Answer the questions asked (the default answers work well if you have no clue), and be sure to enter an empty passphrase (or else all your Android users will have to enter that passphrase when uploading data!).When
gpg
is done generating the key pair, inspect its output and look for the line saying "gpg: key <key_id> marked as ultimately trusted
". Write down the <key_id> that shows up in that line, it identifies the generated key forgpg
. In my example, it isF44FDE5A
. (That key ID also appears further down, in a line that looks like "pub 1024R/F44FDE5A 2012-06-22
".)Next, you can export your public key to a file. We'll call that file
key.pub
. To do that, rungpg --armor --output key.pub --export <key_id>
(so in my case,gpg --armor --output key.pub --export F44FDE5A
). The--armor
option makes the output readable (i.e. not in binary format). Make sure you put the--export
argument last or some option won't be taken into account. -
You can now upload your public key to the server and associate it with your Android app ID. This is done with
curl
under GNU/Linux:Run
curl -F [email protected] <servername>:5000/upload/pubkey/<androidapp_id>
(so in my case,curl -F [email protected] 127.0.0.1:5000/upload/pubkey/wehlutyk_exp
). This command simulates a POST method uploadingkey.pub
as thepubkeyfile
field in a html form. The server should gratify you with a nice "Key saved.
" message. Careful: your upload file must have a.pub
extension, or else the server will reject it. -
At this stage, you can have a look at
<servername>:5000/admin/show_db
in a browser, which will show you what data has been stored on the server. Your public key fingerprint should show up there, associated with your Android app ID (under theCollection: "androidapps_keyfingerprints"
line). If it doesn't, you can start over by visiting<servername>:5000/admin/flush_db
, which will reset all databases and let you start over on clean ground (when you see "Database flushed.
", it means everything was reset). If that still doesn't work, send me en email :-). -
Now you can generate some test data: data is stored in a MongoDB database, so you need to send it in JSON format, signed with your private key.
Save this:
{
"subject_age": 23,
"date": 20120622154423,
"results": [1, 5, 3, 9, 0, 3, 56, 23]
}to a file called
test.json
.Next, sign the data: in a terminal, run
gpg --clearsign --default-key <key_id> --output test.json_signed --sign test.json
(so in my case,gpg --clearsign --default-key F44FDE5A --output test.json_signed --sign test.json
). Here, again, make sure the--sign
argument comes last, or some options won't be taken into account. The resultingtest.json_signed
file should have two sections: one with the original data, the second with the signature (the seemingly random lines of characters). -
At long last, you can upload the signed data to the server by issuing
curl -F [email protected]_signed <servername>:5000/upload/data/<androidapp_id>
(so in my case,curl -F [email protected]_signed 127.0.0.1:5000/upload/data/wehlutykexp
). Again, the server should thank you with a stern "File uploaded.
" message. Careful: your upload file must have a.json_signed
extension, or else the server will reject it.You can now check that your data was stored in the database by visiting
<servername>:5000/admin/show_db
. (You'll notice the added_id
field, which is internal to MongoDB, andandroidapp_id
field, added by the brainyserver to identify the data.)Finally, if you test this procedure by signing your file with another private key (one for which the public key was not published to the server), claiming another Android app ID, or even not signing your data at all, you should see the server reject it (or more likely crash because of an Exception...).