sirikata/sirikata-cdn

Website and scripts for the content distribution network for Sirikata

sirikata-cdn's Issues

Tags entered when uploading aren't saved

I just uploaded some files and tried to enter tags when I was setting the name and description, but when I visit the page for the model, it says there aren't any labels.

Include subfile hashes with name lookup

We currently get subfiles in headers, e.g.,

Subfile-Name: foo.jpg
Subfile-Path: /user/model/foo.jpg

Most of the time we're just going to download these anyway, so it would be nice to get all the hashes as well, e.g.,

Subfile-Hash: abcd6543

Especially when content is already cached, this could greatly speed up the client as it has to do many thousands of name lookups for dependent textures. Each of these is relatively quick, but we could entirely remove that overhead. It would probably be better for the CDN servers as well since they'll get far fewer http requests.

Add /api/upload method

We need to be able to upload via the API for automatically generated meshes, like aggregates from the space server.

Allow API uploads to specify URLs for assets (e.g. textures)

These could be restricted to URLs on the same CDN, but this is pretty important for making aggregate uploads efficient since textures can be a large fraction of the content and aggregates are mostly just reusing them.

Error setting up Cassandra schema

When doing a fresh install, I run

python sirikata-cdn/manage.py sync_cassandra

to setup the schema and get:

Traceback (most recent call last):
  File "sirikata-cdn/manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/home/ewencp/sirikata-cdn.git/sirikata-cdn/cassandra_storage/management/commands/sync_cassandra.py", line 32, in handle_noargs
    strategy_options=settings.CASSANDRA_STRATEGY_OPTIONS)
TypeError: create_keyspace() got an unexpected keyword argument 'replication_factor'

Looking at pycassa docs, it looks like that should be in an 'strategy_options' parameter, but I don't know pycassa/cassandra well enough to say if that's the right fix & check that it works.

Audio support

This would involve adding audio files into the Names table under type=audio, and then making import_upload and place_upload detect audio files.

Add non-OpenID authentication that doesn't require human intervention

We need to have some sort of upload authentication that doesn't require OpenID -- something like OAuth is probably ideal, but even something more basic to get us started would be fine.

Add flag(s) to make model non-browsable, transient

Content which is dynamically generated, like aggregate meshes from the space server, are likely transient and almost certainly not usable by regular users. We need some flags which disable displaying these models when browsing the CDN (in a browser or by API). This same flag or another might also indicate that the data is transient -- it'll likely become invalid.

Faulty ephemeral uploads staying seemingly forever

Uploaded ephemeral meshes, for which the upload process does not complete successfully, seem to stay in my pending upload queue forever. See, for example, the list of pending uploads under my account on singular.stanford.edu. The process did not complete for these meshes because of the bug in sirikata, where a "Content-Length" header in the response was deemed an error condition.

Add keep-alive call + garbage collection

For transient data, we probably want some sort of keep alive call or a timeout to indicate when the data becomes invalid. As a result, we'll be able to garbage collect that data periodically (or as soon as the individual asset goes past its expiration). This will be necessary as models from the space server can become an extra storage burden.