Giter Site home page Giter Site logo

Introduction

This is a client-server for implementing a web interface to configuring, running, monitoring model runs. The initial target for this web interface is the TRIKE model control system. The idea behind this open source release is to allow other model control systems to be used.

The server component is implemented on top of the Django 1.5 framework, although it is not really an idiomatic Django project: in particular, instead of using the Django ORM we implement a similar interface for simplifying interaction with Trike itself.

The server ideally just provides a simplified interface to Trike itself, and basic management and coordination functionality: the bulk of the application is implemented in javascript.

Project Layout

Project layout and management borrows from the following links:

Server

The project/main app is webtrike/. This just contains settings, and the root URL configuration.

All other apps and helper modules live in apps/webtrike (and should be imported as apps.webtrike.rcf.cmds, etc). Note, most Django documentation and tutorials tend to assume that each app is in the top-level (or a sub-directory of the project directory, if you're reading older docs). This has minor implications for testing; see below.

Global templates, and over-rides, are in the top-level templates/ directory.

Client (Browser)

All browser assets are contained in assets/webtrike. The directory static/ is configured as Django's STATIC_ROOT, which is where the collectstatic command will place everything. Note though if you see static/ in the directory, remember this is non-versioned and should be considered read-only.

This has the following sub-layout:

  • less/: By default, we use LessCSS for all styling. This is actually just Twitter Bootstrap in uncompiled form (so you have access to its variables), plus a custom file for anything we need to style ourselves (marvl.less).
  • css/: This contains the plain CSS for any third-party components included (such as OpenLayers)
  • img/: Any image assets.
  • js/: All javascript.

The javascript sub-directory is further structured:

  • libs/: This contains (mainly third-party) libraries such as requirejs, Backbone, jquery, etc.
  • models/: Contains all Backbone models (including collections).
  • views/: Contains all Backbone views and routers.
  • tpl/: All javascript templates; we are just using Handlebars templates

The top-level file main.js is the top-level file loaded by requirejs. Note, we use a custom build of OpenLayers; the file ol-build.cfg can be used to create another, for example if you need to use another component that isn't already included.

Note that compilation of assets (lesscss, for example) is done automatically by django-pipeline.

Miscellaneous

  • requirements/: Python dependencies, in the format used by pip. You will want to install via pip install -r requirements/dev.txt (or .../prod.txt); note that most requirements are actually specified in common.txt. For convenience there is also a top-level requirements.txt file that installs the production requirements.
  • assets/webtrike/jsdoc/: Documentation relating to the JS interface.
  • bin/: A few archives, scripts, etc.

Technologies and Libraries

WebTRIKE utilises quite a lot of third-party code. This has all been chosen to be, where possible, a modern consensus choice with a strong community in order to minimise risk.

Server

As already mentioned, the server is built on Django. Given that the server is currently fairly light-weight, Django might almost be over-kill. However, django has excellent documentation, and will easily grow if and as necessary.

Django-pipeline is used to manage static assets; this can include compilation of LessCSS files, concatenation and minification of CSS, cache-busting renaming (including updating references), etc.

Client

Two main technologies need to be understood to navigate your way around the interface: Backbone, and RequireJS.

Backbone

Backbone provides a very lightweight and flexible ("un-opinionated") MVC framework for Javascript. It is well-documented, but don't be afraid to dive into the source if you're ever unclear how something works; it's terse, but very readable, and quite small. There is also a wealth of community tutorials, plugins, etc. One slight caution is because it is so flexible it can sometimes get confusing when multiple tutorials all have a different approach. Another issue is that at scale it can get difficult to manage, memory leaks become a problem, etc.

A "view" in Backbone is actually a fairly lightweight concept; it essentially just manages a particular DOM element, and there are a few conventions around implementing them, but ultimately you have as much flexibility as you need. One convention we've adopted to standardize the way in which templates are displayed is the through the use of handlebars templates, as previously mentioned.

RequireJS

RequireJS is an implementation of AMD, or Asynchronous Model Definition. This is a means of defining self-contained javascript modules, in terms of their dependencies. When a module is requested, RequireJS will ensure that its dependencies have first been loaded. Each module is a separate file, which helps keep things manageable in terms of size. If necessary this can also be used with an optimiser (called r.js), which will concatenate files, and ignore an unused files. You will notice that the main page just has a single script tag, which loads requirejs and specifies (via the data-main attribute) the initial file to load. In our case this is main.js (see above), which simply configures requirejs and loads the initial module.

Requirejs is also extendable via plugins; we only make use of tpl.js (which itself uses text.js), which will pre-compile templates when they are specified as dependencies.

Miscellaneous

Backbone depends on underscore.js and jquery (although jquery is not used much directly; please avoid the temptation yourself, as it can easily lead to spaghetti code!). We actually use the lodash implementation of the underscore API.

We use moment.js for date/time handling, and OpenLayers for map rendering and interaction. We also use the bootstrap javascript components and jquery datatables to manage the table of model runs shown in the main view.

Developer Documentation

Developer documentation for the webtrike client interface uses jsdoc3. The documentation can be built using the script builddocs.sh and the jsdoc.conf file in the directory assets/webtrike.

A README.md in the directory assets/webtrike gives and overview of the client interface and more details on the technologies that it uses.

Setup and Installation

Installation is (mostly) done from server/requirements.txt, which sources additional files in server/requirements/ (starting with common.txt).

Note that the default requirements.txt installs the production dependencies; you probably want to install requirements/dev.txt instead for development. Most dependencies will be specified in requirements/common.txt anyway.

Server Integration

WebTRIKE uses the traditional mod_wsgi-and-Apache approach. This is reliable and will work, but if you have the flexibility you might want to investigate using a separate evented WSGI application server, and just use the web server to reverse-proxy to it. The standard application server in this case is gunicorn. The Django docs have sections on deployment in general, specifically via wsgi, and using gunicorn. One advantage of the gunicorn approach is that if you need to restart you don't need to touch your web server; additionally, once you no-longer need mod_wsgi you can use nginx instead of Apache, which is leaner and more efficient, and (personally) easier to configure as well.

Regardless of the approach used, remember to make sure the server is serving all static content, not Django!

Development Requirements

You will need to have installed:

  • Python
  • Virtualenv (for bootstrapping)
  • The lessc compiler for LessCSS, from node.js/npm. Make sure you have a recent version of of node/npm first (your linux distribution may have an old one), as older versions may not install lessc correctly.

Virtualenv

Virtualenv is used for repeatable and testable python-based deployments. In a nutshell, this simply creates a local directory with its own python environment. The advantages of this is you don't need root permissions to install anything, if anything goes wrong you can simply delete it and start again, and of course it makes the installation process much easier to test.

Django Settings

We use a settings module instead of the traditional single file (although, there is a lot of discussion about the best to handle Django settings!). You should use webtrike.settings.dev during development and webtrike.settings.prod for production; both of these actually just import webtrike.settings.common which is where the majority of the configuration lives, but this should never be used directly.

Note that you will need to specify these (or set the DJANGO_SETTINGS_MODULE environment variable) every time you run a manage.py command; for example manage.py runserver_plus 0.0.0.0:8000 --threaded --settings=webtrike.settings.dev (which itself assumes that you have initialised and activated the virtualenv). Setting the variable is probably easier.

Testing

Django-nose is installed in the dev settings, and should be mostly transparent. Because of the project layout though there is one point to be aware of: you need to be a bit more explicit when running just one app's tests. That is, by example, you need to run ./manage.py test --settings=webtrike.settings.dev apps.trike.tests not ./manage.py test --settings=webtrike.settings.dev apps.trike as you ordinarily would. You can however run all apps for example with ./manage.py test --settings=webtrike.settings.dev apps

webtrike's Projects

celledit icon celledit

Allows datatable cells to be edited inline

pointsamplingtool icon pointsamplingtool

QGIS plugin: Samples polygon attributes and raster values from multiple layers at specified sampling feature

rompy icon rompy

Relocatable Ocean Modelling in PYthon (rompy) combines templated cookie-cutter model configuration with various xarray extensions to assist in the setup and evaluation of coastal ocean model

webtrike icon webtrike

web interface for configuring, running and monitoring model runs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.